[Thread Prev][Thread Next][Index]

Re: [ferret_users] OPeNDAP/JGOFS



Hi James,

I would be very interested in making the OPeNDAP/JGOFS/Ferret connection work.

My current environment is:
FERRET v5.80 (installed 05-01-31)
RedHat 9 Linux(g77) 2.4.20 - 01/03/05
netcdf library version 3.5.0 of Oct 10 2003 13:42:35
OPeNDAP DODS server core software: DODS DAP++/3.4.8
Server version: jg-dods/3.4.3

So I guess the first question is whether I must install a more recent NetCDF library: from Unidata web site this appears to be: netCDF-3.6.0-p1
( I tend not to upgrade this library unless forced to - if it isn't broken don't fix it line of thinking. but perhaps v 3.5.0 from 2003 is too out of date to support reading OPeNDAP/JGOFS format data from within Ferret )

I'd be interested in continuing this discussion offline, and subsequently posting any solutions to the user group for all to enjoy :-)

I've included details below of of my investigation so far, if you care to read on ...

THanks, Cyndy


In hopes that I would not be required to upgrade my NetCDF library, I tested out a couple JGOFS format objects with ferret ...

When you said 'nested Sequences' in your email (as in 'if it has them, it won't work') do you mean more than one level of data? Being hierarchical, the JGOFS system supports multiple levels of data (level 0 changing least rapidly ... level N changing most rapidly). However, the OPeNDAP/JGOFS server 'flattens' the data so it all appears at level 0. So, I suspect that even a hierarchical JGOFS data object would be OK.

My suspicion is that Ferret is having trouble with the OPeNDAP/JGOFS format data because the OPeNDAP/JGOFS identifies all fields as type STRING. My limited experience with Ferret makes it difficult for me to do much more than trial and error (sorry to say - equally matched at this point!), but I'm following the idea that string data type is the difficulty.

Here's an example (this is a multi level JGOFS object of Arabian Sea Niskin bottle data):
date,time_begin,lat_begin,lon_begin are all Level 0 Parameters
press and NO3 are at level 1

JGOFS Data Sysytem:
http://usjgofs.whoi.edu/jg/serv/jgofs/arabian/ttn-045/bottle.html0
DODS: http://usjgofs.whoi.edu/dods-bin/nph-dods/jgofs/arabian/ttn-045/bottle.dds
Notice all are identified as string.
http://usjgofs.whoi.edu/dods-bin/nph-dods/jgofs/arabian/ttn-045/bottle.html
http://usjgofs.whoi.edu/dods-bin/nph-dods/jgofs/arabian/ttn-045/bottle.asc?date,time_begin,lat_begin,lon_begin,press,NO3
returns flat, CSV list of strings

FYI: MATLAB loaddods can read this data
no3=loaddods ('-F','http://usjgofs.whoi.edu/dods-bin/nph-dods/jgofs/arabian/ttn-045/bottle?NO3');
>> whos
Name Size Bytes Class
no3 2062x1 16496 double array

>> figure(1);
>> plot(no3,depth,'b*');
>> axis ij;

and the loaddods function supports the full range of JGOFS object projection and selection modifiers as well.


Here's another example (this is a single level JGOFS object):
JGOFS Data Sysytem:
http://usjgofs.whoi.edu/jg/serv/jgofs/arabian/sed_comp.html0
DODS: http://usjgofs.whoi.edu/dods-bin/nph-dods/jgofs/arabian/sed_comp.asc?lon,lat,depth_w,Si_opal

so I tried this Ferret command:
FILE/ORDER=XYZ/VARIABLES="lon,lat,depth_w" "http://usjgofs.whoi.edu/dods-bin/nph-dods/jgofs/arabian/sed_comp.asc?lon,lat,depth_w,Si_opal";
sho data
currently SET data sets:
1> http://usjgofs.whoi.edu/dods-bin/nph-dods/jgofs/arabian/sed_comp.asc?lon,lat,depth_w,Si_opal (default)
name title I J K L
LON LON 1:20480 ... ... ...
LAT LAT 1:20480 ... ... ...
DEPTH_W DEPTH_W 1:20480 ... ... ...

# note that whatever dataset I read via DODS/JG server ends up being size 1:20480

yes? list SI_OPAL
**TMAP ERR: non-existent or not on line
DATAFILE NOT FOUND
Data set: http://usjgofs.whoi.edu/dods-bin/nph-dods/jgofs/arabian/sed_comp.asc?lon,lat,depth_w,Si_opal
Data file: http://usjgofs.whoi.edu/dods-bin/nph-dods/jgofs/arabian/sed_comp.asc?lon,lat,depth_w,Si_opal

# I am unable to list or plot the data in Ferret.

not having much experience with Ferret, I looked at the User Guide and tried some variations ...
try SKIP first 2 records

FILE/VARIABLES="lon,lat,depth_w,Si_opal"/SKIP=2 "http://usjgofs.whoi.edu/dods-bin/nph-dods/jgofs/arabian/sed_comp.asc?lon,lat,depth_w,Si_opal";
yes? sho data
currently SET data sets:
1> http://usjgofs.whoi.edu/dods-bin/nph-dods/jgofs/arabian/sed_comp.asc?lon,lat,depth_w,Si_opal (default)
name title I J K L
LON LON 1:20480 ... ... ...
LAT LAT 1:20480 ... ... ...
DEPTH_W DEPTH_W 1:20480 ... ... ...
SI_OPAL SI_OPAL 1:20480 ... ... ...


Now use the /TYPE qualifier to specify that all columns be treated as numeric.
FILE/TYPE=numeric/VARIABLES="lon,lat,depth_w,Si_opal"/SKIP=2 "http://usjgofs.whoi.edu/dods-bin/nph-dods/jgofs/arabian/sed_comp.asc?lon,lat,depth_w,Si_opal";

FILE/format=DELIM/VARIABLES="lon,lat,depth_w,Si_opal" "http://usjgofs.whoi.edu/dods-bin/nph-dods/jgofs/arabian/sed_comp.asc?lon,lat,depth_w,Si_opal";

causes seg fault - core dump

=============== end of detailed examples ==============




James Gallagher wrote:

Hi Ansley,

I'm coming in on this in the middle, so this may be way off the mark...

If Ferret is built using a newer version of the netcdf client library, it may be able to read this data source. If the JGOFS data source has only one Sequence, then it should work. If it has nested Sequences, then it won't.

James

On Oct 26, 2005, at 1:48 PM, Ansley Manke wrote:

Hi again,
I answered too quickly before -- Note this entry in the Ferret Users
Guide, in the section under DODS , accessing remote data:

"For the time being, NetCDF and HDF files can be read via DODS by
Ferret. As DODS (OPeNDAP) NetCDF libraries become available, other
data types will be made available."

So, Ferret can't be used to access remote ASCII data at this time. If
the dataset is not too large, you could save it to your local system and
then the command that you've used will open the ascii file.

Ansley


Namba Takaya wrote:

Dear Cyndy

I post this to ferret user mailing list.

Thank you very much for your very kind reply.

According to your suggsetion, I tried


FILE/FORMAT=DELIM/ORDER=TXYZ/VARIABLES="date,lon,lat,depth,nitrate"
"http://usjgofs.whoi.edu/dods-bin/nph-dods/jgofs/merged_objects/
US_JGOFS/North_Atlantic/bottle_nabe.asc?date,lon,lat,depth,NO3"

but error comes like

yes? show data
currently SET data sets:
1>
http://usjgofs.whoi.edu/dods-bin/nph-dods/jgofs/merged_objects/ US_JGOFS/North_Atlantic/bottle_nabe.asc?date,lon,lat,depth,NO3
(default)
name title I J
K L
DATE DATE 1:20480 ...
... ...
LON LON 1:20480 ...
... ...
LAT LAT 1:20480 ...
... ...
DEPTH DEPTH 1:20480 ...
... ...
NITRATE NITRATE 1:20480 ...
... ...

yes? list DATE
**TMAP ERR: non-existent or not on line
DATAFILE NOT FOUND
Data set:
http://usjgofs.whoi.edu/dods-bin/nph-dods/jgofs/merged_objects/ US_JGOFS/North_Atlantic/bottle_nabe.asc?date,lon,lat,depth,NO3
Data file:
http://usjgofs.whoi.edu/dods-bin/nph-dods/jgofs/merged_objects/ US_JGOFS/North_Atlantic/bottle_nabe.asc?date,lon,lat,depth,NO3


Could you tell me how to solve this problem.

Best Regards,

Takaya Namba



--------------------------------------
Know more about Breast Cancer
http://pr.mail.yahoo.co.jp/pinkribbon/




--
James Gallagher                jgallagher at opendap.org
OPeNDAP, Inc                   406.723.8663


--
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Cyndy Chandler		| voice: (508) 289-2765
MS #43, WHOI		| Office hrs: M-F ~ 9-4
Woods Hole, MA  02543	| FAX: (508) 457-2161
cchandler@whoi.edu | http://www.whoi.edu/people/cchandler
Ocean Carbon and Biogeochemistry Data Management Office
Department of Marine Chemistry and Geochemistry
Woods Hole Oceanographic Institution


[Thread Prev][Thread Next][Index]

Dept of Commerce / NOAA / OAR / PMEL / TMAP

Contact Us | Privacy Policy | Disclaimer | Accessibility Statement