[Thread Prev][Thread Next][Index]

Re: [ferret_users] ASCII to netCDF via ferret.



Hi Dan,

I'm just tuning in from the wings ... ignore if I'm in left field.

Normally in Ferret you'd read data such as this onto a single variable -- i.e. you would create a time axis for your grid that encompasses all of the years and a 3D grid of shape lon-lat-year.  Then you won't see this artificial limitation in the number of distinct variables that you can read.  It looks like the way the file is organized each year is a separate column.  The year number is in effect the "fastest moving axis" of a grid of shape year-lon-lat.  This is permuted from Ferret's default ordering.

You'll want to
  1. define a grid something like this:
    1. DEFINE AXIS/T=1868:1999:1  year    ! see #3
      def axis/modulo/x=-180:179/npoints=360/units="degrees_east" xax
      def axis/y=-90:89/npoints=180/units="degrees_north" yax
      DEFINE GRID/x=xax/y=yax/t=year  my_grid3d
  2. On your FILE command
    1. /grid=my_grid3d
    2. Use the /COLUMNS qualifier tell Ferret that all of the columns belong to a single variable
    3. use the /ORDER=TXY qualifier to tell Ferret the permutation of your file axes
    4. use /VAR=my_var_name
  3. The way the data are organized the lon lat values lie at the start of each line of data.  They will appear in this method of reading as if they are data for years 1868 and 1869.  All you need do is to make the time axis longer by these two introductory years and then ignore the resulting time values (which will actually contain lat/long positions).  See step #4

    Your time axis can be simplistic ... just numbers 1868 ... 1999   or it can be correct in every detail, incorporating leap year length variations.  To do the latter define an axis that is 1-jan for each year per the instructions found under "DEFINE AXIS/EDGES" at Ferret Users guide section http://ferret.pmel.noaa.gov/Ferret/documentation/users-guide/commands-reference/DEFINE#_VPINDEXENTRY_1299
  4. Best to save the resulting data into a netCDF  file ... and never have to deal with the messy parts again  :-) .
    • SET VARIABLE/TITLE="my title"/UNITS="my_units" my_var_name
    • SAVE/FILE=my_file.nc my_var_name[l=3:<whatever>]     ! start at L=3 to skip the first 2 years, which contain lon-lat values
    - Steve

==========================

On 9/20/2011 6:36 PM, Oostra, Daniel H. (LARC-E302)[SCIENCE SYSTEMS AND APPLICATIONS, INC] wrote:
Hi Russ,

Thanks for your advice.  This was a rookie error on my part, while the code works, I had somehow decided that it was a 360x180 degree grid, when it was actually a 256x128 degree grid.  So my x and y axes were off.  Once I changed these values for xax and yax the plots came out as intended.

The issue I'm having now is that ferret will only allow me to assign 20 variables.  I'm going to try to break up the data into 20 year blocks but this seems clumsy.  Any idea on how to do this without manipulating the file original data file?

Thanks again,

Dan



On Sep 20, 2011, at 8:34 PM, "Russell Fiedler" <russell.fiedler@xxxxxxxx> wrote:

Hi Daniel,

I think the problem is with the way you have defined your time axis. Is it supposed to have values at the
end of the year or should it represent the middle of the year?

-------------
yes? show grid my_grid1
   GRID MY_GRID1
name       axis              # pts   start                end
XAX       LONGITUDE          360mr   180E(-180)           179E
YAX       LATITUDE           180 r   90S                  89N
normal    Z
TAX1      TIME                 2 r   01-JAN-1870 00:00    01-JAN-1871 05:49
--------------

If these values are meant to be in the middle of the year then the time axis should be specified as

define axis/t="01-jan-1870":"01-jan-1871"/npoints=1/edges/units=years tax1

Note that we put the bounds at the extremes of the year of which the end date is the start of january of the next year
and not 31 December.

Another problem might be the length of the lines. You may have to use the /FORMAT="(some_fortran_format)"  
option on the FILE command.

Russ


On Wednesday 21 September 2011 01:11, Oostra, Daniel H. (LARC-E302)[SCIENCE SYSTEMS AND APPLICATIONS, INC] wrote:
Hi Ferret users,

I'm trying to use ferret to convert an ASCII Tabular dataset into a netCDF file that I can use in our Live Access Server.

So far I've been able to successfully create plots but they are not coming out correctly.  It seems the data portion is being displayed in the southern hemisphere only.

I've run into some memory problems and variable assignment issues, but both of those can be solved.

Here's a data sample (first 16 lines):

*******************
Band Name:                    tas
Description:                  (deg. C). NCAR-CCSM3 ensemble average for the IPCC Climate of the 20th Century experiment. This experiment simulates past climate starting from the mid-19th Century and is part of the 4th IPCC Assessment. Data represent averages for 5 ensembles.
Holding name:                 air temperature - Annual
Collection name:              Climate Changes in the 21st Century
Date class:                   time series
Begin date of output          1870-01-01
End date of output            1999-12-31
Time step units:              years
Data units:                   Deg C
Fill value:                   -99

Center Lon,Center Lat,1870,1871,1872,1873,1874,1875,1876,1877,1878,1879,1880,1881,1882,1883,1884,1885,1886,1887,1888,1889,1890,1891,1892,1893,1894,1895,1896,1897,1898,1899,1900,1901,1902,1903,1904,1905,1906,1907,1908,1909,1910,1911,1912,1913,1914,1915,1916,1917,1918,1919,1920,1921,1922,1923,1924,1925,1926,1927,1928,1929,1930,1931,1932,1933,1934,1935,1936,1937,1938,1939,1940,1941,1942,1943,1944,1945,1946,1947,1948,1949,1950,1951,1952,1953,1954,1955,1956,1957,1958,1959,1960,1961,1962,1963,1964,1965,1966,1967,1968,1969,1970,1971,1972,1973,1974,1975,1976,1977,1978,1979,1980,1981,1982,1983,1984,1985,1986,1987,1988,1989,1990,1991,1992,1993,1994,1995,1996,1997,1998,1999
-179.296875,89.296875,-18.2,-17.6,-17.8,-17.5,-17.6,-17.6,-17.8,-17.5,-18.2,-18.0,-17.6,-17.8,-17.9,-18.3,-18.7,-18.8,-18.4,-18.9,-19.1,-19.5,-18.5,-18.9,-19.4,-18.9,-19.1,-19.0,-18.8,-18.7,-18.6,-18.3,-18.0,-18.9,-18.8,-19.5,-19.9,-19.5,-19.6,-19.4,-19.2,-18.5,-19.5,-19.0,-19.2,-19.5,-19.2,-19.0,-18.7,-19.0,-18.6,-18.7,-18.7,-18.4,-18.5,-18.4,-18.3,-18.3,-18.0,-18.8,-18.4,-18.8,-18.9,-18.2,-19.1,-18.2,-18.3,-18.2,-18.5,-17.9,-18.1,-17.9,-18.2,-18.1,-18.2,-18.3,-17.6,-17.8,-18.1,-18.1,-17.4,-17.2,-17.6,-17.6,-17.9,-17.9,-17.9,-17.8,-18.2,-17.0,-17.6,-17.8,-17.4,-17.4,-17.3,-18.0,-17.6,-18.5,-18.0,-17.8,-17.4,-17.1,-16.9,-16.9,-17.1,-17.7,-17.2,-16.9,-17.9,-17.3,-17.1,-17.3,-17.0,-16.8,-16.8,-16.7,-16.7,-16.9,-16.7,-16.4,-16.7,-16.4,-15.6,-16.4,-16.1,-16.4,-15.8,-16.2,-16.4,-15.6,-15.7,-15.3
-177.890625,89.296875,-18.2,-17.7,-17.8,-17.5,-17.6,-17.7,-17.8,-17.5,-18.3,-18.0,-17.6,-17.8,-17.9,-18.3,-18.8,-18.8,-18.4,-18.9,-19.1,-19.5,-18.5,-18.9,-19.4,-18.9,-19.1,-19.0,-18.8,-18.7,-18.6,-18.3,-18.0,-18.9,-18.8,-19.5,-19.9,-19.5,-19.7,-19.4,-19.2,-18.5,-19.5,-19.0,-19.2,-19.5,-19.3,-19.0,-18.8,-19.0,-18.6,-18.7,-18.7,-18.4,-18.5,-18.4,-18.3,-18.3,-18.0,-18.8,-18.4,-18.8,-18.9,-18.2,-19.1,-18.2,-18.3,-18.2,-18.5,-18.0,-18.1,-17.9,-18.2,-18.1,-18.2,-18.3,-17.6,-17.8,-18.1,-18.1,-17.4,-17.2,-17.6,-17.6,-17.9,-18.0,-17.9,-17.8,-18.3,-17.0,-17.6,-17.8,-17.5,-17.4,-17.3,-18.0,-17.6,-18.6,-18.1,-17.8,-17.4,-17.1,-16.9,-16.9,-17.1,-17.8,-17.2,-16.9,-17.9,-17.3,-17.1,-17.3,-17.0,-16.8,-16.8,-16.7,-16.8,-17.0,-16.7,-16.4,-16.7,-16.5,-15.7,-16.4,-16.2,-16.4,-15.8,-16.2,-16.4,-15.6,-15.7,-15.4
-176.484375,89.296875,-18.2,-17.7,-17.9,-17.5,-17.6,-17.7,-17.8,-17.5,-18.3,-18.1,-17.7,-17.8,-18.0,-18.4,-18.8,-18.9,-18.4,-19.0,-19.1,-19.5,-18.5,-18.9,-19.4,-18.9,-19.1,-19.0,-18.8,-18.7,-18.6,-18.4,-18.0,-18.9,-18.8,-19.6,-19.9,-19.5,-19.7,-19.4,-19.2,-18.6,-19.5,-19.0,-19.2,-19.5,-19.3,-19.1,-18.8,-19.0,-18.6,-18.8,-18.8,-18.4,-18.5,-18.4,-18.3,-18.4,-18.1,-18.9,-18.4,-18.9,-18.9,-18.2,-19.1,-18.2,-18.3,-18.2,-18.5,-18.0,-18.1,-17.9,-18.2,-18.1,-18.3,-18.3,-17.7,-17.8,-18.1,-18.1,-17.5,-17.2,-17.6,-17.6,-17.9,-18.0,-17.9,-17.8,-18.3,-17.0,-17.6,-17.8,-17.5,-17.4,-17.3,-18.0,-17.6,-18.6,-18.1,-17.8,-17.5,-17.1,-16.9,-16.9,-17.2,-17.8,-17.2,-16.9,-17.9,-17.3,-17.1,-17.3,-17.0,-16.8,-16.8,-16.7,-16.8,-17.0,-16.7,-16.5,-16.7,-16.5,-15.7,-16.4,-16.2,-16.4,-15.8,-16.2,-16.5,-15.6,-15.7,-15.4
-175.078125,89.296875,-18.2,-17.7,-17.9,-17.5,-17.6,-17.7,-17.8,-17.5,-18.3,-18.1,-17.7,-17.8,-18.0,-18.4,-18.8,-18.9,-18.5,-19.0,-19.1,-19.6,-18.5,-18.9,-19.4,-18.9,-19.1,-19.0,-18.8,-18.7,-18.6,-18.4,-18.0,-18.9,-18.8,-19.6,-19.9,-19.6,-19.7,-19.4,-19.3,-18.6,-19.5,-19.1,-19.2,-19.5,-19.3,-19.1,-18.8,-19.0,-18.6,-18.8,-18.8,-18.4,-18.5,-18.4,-18.3,-18.4,-18.1,-18.9,-18.4,-18.9,-18.9,-18.2,-19.1,-18.2,-18.4,-18.2,-18.6,-18.0,-18.1,-17.9,-18.2,-18.2,-18.3,-18.3,-17.7,-17.8,-18.1,-18.1,-17.5,-17.2,-17.7,-17.6,-17.9,-18.0,-17.9,-17.8,-18.3,-17.0,-17.6,-17.8,-17.5,-17.5,-17.3,-18.0,-17.6,-18.6,-18.1,-17.8,-17.5,-17.1,-17.0,-16.9,-17.2,-17.8,-17.2,-16.9,-18.0,-17.3,-17.2,-17.4,-17.0,-16.9,-16.8,-16.7,-16.8,-17.0,-16.7,-16.5,-16.7,-16.5,-15.7,-16.4,-16.2,-16.5,-15.9,-16.2,-16.5,-15.6,-15.7,-15.4
*************


Here is my ferret script, I'm processing the first 5 years 1870-1874 (of 130 years):


**********************
set memory/size=512

def axis/modulo/x=-180:179/npoints=360/units="degrees_east" xax
def axis/y=-90:89/npoints=180/units="degrees_north" yax

define axis/t="01-jan-1870":"31-dec-1870":1/units=years tax1
def grid/x=xax/y=yax/t=tax1 my_grid1
file/skip=13/var="longitude,latitude,r1870,r1871,r1872,r1873,r1874,r1875,r1876,r1877,r1878,r1879"/grid=my_grid1 "12890ncar_ccm3_tas_EA1-5_20C3M-annual.txt"
let air_temp = r1870
set variable/title="Monthly, Annual Average Air Tempurature" air_temp
set variable/units="Degrees C"/bad=-99 air_temp
save/file=air_temp_1870.nc/clobber air_temp


define axis/t="01-jan-1871":"31-dec-1871":1/units=years tax1
def grid/x=xax/y=yax/t=tax1 my_grid2
file/skip=13/var="longitude,latitude,r1870,r1871,r1872,r1873,r1874,r1875,r1876,r1877,r1878,r1879"/grid=my_grid2 "12890ncar_ccm3_tas_EA1-5_20C3M-annual.txt"
let air_temp = r1871
set variable/title="Monthly, Annual Average Air Tempurature" air_temp
set variable/units="Degrees C"/bad=-99 air_temp
save/file=air_temp_1871.nc/clobber air_temp

define axis/t="01-jan-1872":"31-dec-1872":1/units=years tax1
def grid/x=xax/y=yax/t=tax1 my_grid3
file/skip=13/var="longitude,latitude,r1870,r1871,r1872,r1873,r1874,r1875,r1876,r1877,r1878,r1879"/grid=my_grid3 "12890ncar_ccm3_tas_EA1-5_20C3M-annual.txt"
let air_temp = r1872
set variable/title="Monthly, Annual Average Air Tempurature" air_temp
set variable/units="Degrees C"/bad=-99 air_temp
save/file=air_temp_1872.nc/clobber air_temp


define axis/t="01-jan-1873":"31-dec-1873":1/units=years tax1
def grid/x=xax/y=yax/t=tax1 my_grid4
file/skip=13/var="longitude,latitude,r1870,r1871,r1872,r1873,r1874,r1875,r1876,r1877,r1878,r1879"/grid=my_grid4 "12890ncar_ccm3_tas_EA1-5_20C3M-annual.txt"
let air_temp = r1873
set variable/title="Monthly, Annual Average Air Tempurature" air_temp
set variable/units="Degrees C"/bad=-99 air_temp
save/file=air_temp_1873.nc/clobber air_temp


define axis/t="01-jan-1874":"31-dec-1874":1/units=years tax1
def grid/x=xax/y=yax/t=tax1 my_grid5
file/skip=13/var="longitude,latitude,r1870,r1871,r1872,r1873,r1874,r1875,r1876,r1877,r1878,r1879"/grid=my_grid5 "12890ncar_ccm3_tas_EA1-5_20C3M-annual.txt"
let air_temp = r1874
set variable/title="Monthly, Annual Average Air Tempurature" air_temp
set variable/units="Degrees C"/bad=-99 air_temp
save/file=air_temp_1874.nc/clobber air_temp
***********************

Any insights, advice, criticism is welcome!  Thanks ahead.

Bests,

Daniel Oostra (SSAI)
NASA/LaRC Atmospheric Sciences Data Center
2 South Wright Street
Hampton, VA 23681-2199
Ph: 757.864.6157
daniel.h.oostra@xxxxxxxx









[Thread Prev][Thread Next][Index]
Contact Us
Dept of Commerce / NOAA / OAR / PMEL / Ferret

Privacy Policy | Disclaimer | Accessibility Statement