Re: non COARDS-compliant via DODS

[Joe Sirott <sirott@xxxxxxxxxx>]

Andrew,

Your first problem (out-of-range errors) is due to the fact that there is no metadata in your HDF files for the X (longitude) axis indicating the units for the axis. Ferret also needs another netCDF attribute for the axis named "modulo"; this is detailed in the COARDS conventions.

The solution is to configure LAS with the <init_script> tag. I've attached a script that will work for your dataset. The steps to use this are:

1) configure Ferret to use custom directories (if you haven't already)
2) Save the attached script init_modulo_x.jnl in: lasxml/server/custom.
3) Add/change the following:
   <datasets><d4733_restart1_a_nc_dods_zip name="DODS HDF test" ...>
   <properties>
         <ferret><init_script>init_modulo_x</init_script>
         </ferret>
     </properties>
4) Regenerate the server with genLas2.pl (or make).

You problem with bizarre variable names can be solved as follows:

1) For variables that have names that can't be translated into XML tags, you can create your own XML tag name and use the url attribute; for instance:
     <variables>
      <barotropic_v_velocity name="barotropic_v_velocity"     url="http://www.nerc-essc.ac.uk/cgi-bin/nph-dods/OCCAM/d4733.restart1_a.hdf.gz#barotropic%20v%20velocity">
...
2) Unfortunately, Ferret requires that variables with "unusual" characters be quoted with single quotes. This can be fixed by installing the attached file 'custom.pl' in the lasxml/server/custom directory.

Andrew Woolf wrote:

Thanks Joe and others with help on my question the other day (addXml.pl).

I'm trying to set up LAS for our data, which is currently in non
COARDS-compliant HDF format. I could convert all of these, but there is a
huge amount of data (~150GB compressed), so I'd prefer if I could get it
all working as is.

In order to do this I've currently got a DODS server running and am trying
to read the data in LAS via DODS.

Of course I need to hand edit las.xml for the configuration.

*So*: my data has 2-d variables (lat,lon) and 3-d (lat,lon,depth). I set up
las.xml defining the grids for these variables with the axes in the
correct order, but it produces an error when asking ferret to do the plotting.

An example is the DODS file
http://www.nerc-essc.ac.uk/cgi-bin/nph-dods/OCCAM/d4733.restart1_a.hdf.gz

An ncdump shows salinity as:

        float SALINITY(LATITUDE_T, LONGITUDE_T, DEPTH) ;
                SALINITY:long_name = "Salinity" ;
                SALINITY:units = "(PSU-35.)/1000." ;
                SALINITY:_FillValue = 0.0099999998f ;
                SALINITY:LEVEL = 0 ;
                SALINITY:T_GRID = 1 ;

Coordinate vars are:
 * LATITUDE_T (577 pts equally spaced between -78.125 and 65.875)
 * LONGITUDE_T (1440 equally spaced between 0.125 and 359.875)
 * DEPTH (lets say 1:36 to keep things simple)

(Ignore all variables except salinity, because is another problem with
underscores in other variable names being translated to spaces....).

For a trial I set this up in las.xml as follows:

<d4733_restart1_a_nc_dods_zip name="DODS HDF test"
url="http://www.nerc-essc.ac.uk/cgi-bin/nph-dods/OCCAM/d4733.restart1_a.hdf.gz"
doc="">
 <variables>
  ...
  <salinity units="PSU" name="Salinity">
    <link match="/lasdata/grids/OCCAM_LATITUDE_T_LONGITUDE_T_DEPTH_grid"/>
  </salinity>
  ...
 </variables
</d4733_restart1_a_nc_dods_zip>

...

<grids>
 ...
 <OCCAM_LATITUDE_T_LONGITUDE_T_DEPTH_grid>
  <link match="/lasdata/axes/d4733_restart1_a_nc_LATITUDE_T"/>
  <link match="/lasdata/axes/d4733_restart1_a_nc_LONGITUDE_T"/>
  <link match="/lasdata/axes/d4733_restart1_a_nc_DEPTH"/>
 </OCCAM_LATITUDE_T_LONGITUDE_T_DEPTH_grid>
 ...
</grids>

<axes>
 <d4733_restart1_a_nc_LATITUDE_T units="degrees_north" type="y">
  <arange start="-78.125" step="0.25" size="577"/>
 </d4733_restart1_a_nc_LATITUDE_T>
 <d4733_restart1_a_nc_LONGITUDE_T units="degrees_east" type="x">
  <arange start="0.125" step="0.25" size="1440"/>
 </d4733_restart1_a_nc_LONGITUDE_T>
 <d4733_restart1_a_nc_DEPTH units="layer" type="z">
  <arange start="1" step="1" size="36"/>
 </d4733_restart1_a_nc_DEPTH>
</axes>

However, when I now try to plot this in LAS, the default full-region x-y
plot for salinity, I get the error:

Illegal region

You specified a time or depth value that is outside the region contained by this
dataset.

Checking the LAS error log, I find the command it tried to run in ferret was:

set region/x="180":"540"/y="-78.0":"65.875"/z="1":"1"/l=1

And the ferret error message:
**ERROR: illegal limits: "SALINITY" is not in the range X=180:540
  Axis extremes are X=0.5:36.5

Can anyone suggest a way to fix this so I can access this under LAS?

Thanks very much,
 - Andrew
 
 

-----------------------------------------------------
Andrew Woolf  (awo@mail.nerc-essc.ac.uk)
Environmental Systems Science Centre (ESSC)
Reading University
3 Earley Gate
Whiteknights
Reading RG6 6AL
UNITED KINGDOM
Phone: +44 (0)118 931 8741   Fax: +44 (0)118 931 6413
-----------------------------------------------------

--
Joe Sirott
 

\cancel mode verify
! init_modulo_x.jnl
! 9/00
!
! The optional second variable is included for vector plots.

! Description: open (and initialize) a dataset and set the modulo
! flag on the time axes of the incoming variable(s).

! arguments:                 1        2         3        4 
! Usage: GO init_modulo_x [dset] [var num] [dset num] [var1]
!
! dset -- the name of the dataset
! var num [1/2] -- first var or second (comparison) var
! dset num [1/2] -- first dataset opened or second
!                   (comparison var may be from first dataset)
! var 1 -- name of var 1
! var 2 -- name of var 2 (for vectors) 

! e.g. yes? go init_modulo_x  coads_climatology 1 uwnd

! check the arguments
! (var2 is optional)

! We cannot check argument 1 because absolute pathnames
! are parsed by ferret as command qualifiers.  Checking
! for arguments 2 - 4 should be sufficient.

set mem/size=20
query/ignore $2%<Usage: GO init_modulo_x data_set var_# data_set_# variable_1%
query/ignore $3%<Usage: GO init_modulo_x data_set var_# data_set_# variable_1%
query/ignore $4%<Usage: GO init_modulo_x data_set var_# data_set_# variable_1%

! Hack to get a quoted string for DODS datasets.
def symbol dset = "$1"
set data ($dset)

! Set the x axis "modulo" flag
define axis/x=0:359.75:0.25/units=degrees_east/modulo   `$4,return=xaxis` 

! We must define var$2_grid use in differencing
! This grid must reference a file variable: one that actually
! exists in dataset 'dset' as opposed to a virtual variable
! created with an expression involving other variables.
define grid/like=$4[d=$3] var$2_grid_

! Any extra work: defining virtual variables, regridding, etc
! should be done at this time.

Attachment: custom.pl
Description: Perl program