[Thread Prev][Thread Next][Index]

Re: [ferret_users] scattered ASCII data to NetCDF



Hi Mick

I would like to thank immensely for your elegant solution and well explained answer.

My file is 628 rows (points) by 176 columns (times) with (lon,lat) in the first two columns. I left separated by ";" because there's no data for some times. So, I done:

def axis/x=1:628:1 xax
def axis/t=1:174:1 tax
def grid/x=xax/t=tax grd

def axis/t=-1:174:1 tax2
def grid/x=xax/t=tax2 grd2

file/form=delim/delim=";"/col=176/g=grd2/ord=tx/var=vin chuva08.txt

let lon = vin[l=1]
let lat = vin[l=2]
let pcp = vin[g=grd]

Then, when I 'show data', I get:

yes? show data
     currently SET data sets:
    1> ./chuva08.txt  (default)
 name     title                             I         J         K         L
 VIN      VIN                              1:628     ...       ...       1:176

Everything seems Ok. But, When I try to list variables, I get

yes? list PCP
 **ERROR: illegal limits: T axis of VIN

And then, when I 'show data' again, I get

yes? show data
     currently SET data sets:
    1> ./chuva08.txt  (default)
 name     title                             I         J         K         L
 VIN      VIN                              1:628     ...       ...       1:1

Look, now I have only one "time step". It contains LONs. I tried spaces " " and commas "," using /form=free and bars "|" and semicolons ";" replacing /form=free by respective /form=delim/delim=, without success.

I think it should be working now. Any idea?


2008/6/26 mick spillane <Mick.Spillane@xxxxxxxx>:
Hi Paulo,
  The "/skip=" is intended to skip a number of records at the start of a file.  For problems like yours, which has a mixture of types of variables in a record (here lon,lat followed by a series of data values) I usually use regridding to separate them.
  To make the following work you should first replace the semicolons with blanks or commas as the delimiter so that you can use "file/form=free" to access the file.

Suppose you have 50 records, each of which has the LON, LAT, [V(L),L=1,170], in your file.
Then define a grid to hold ALL the V values:

  def axis/x=1:50:1 xax ; def axis/t=1:170:1 tax ; def grid/x=xax/t=tax grd

Now the data file has two extra columns at the beginning which we treat as "fake" times in another grid:

  def axis/t=-1:50:1 tax2 ; def grid/x=xax/t=tax2 grd2

(If the LON,LAT values were at the end of the record you would use "def axis/t=1:172:1" )
Now read in everything on the grid grd2:

  file/form=free/col=172/g=grd2/ord=tx/var=vin data.file

The variable "vin" has LON in L=1, LAT in L=2, and the V data in the remaining 170 columns. So

  let LON=vin[L=1] ; let LAT=vin[L=2]
  let V=vin[g=grd]

Here is a quick demo where there are only 6 "times" and 4 records in the file test.data
123.4 34.5 1 2 3 4 5 6
124.5 35.6 9 8 7 6 5 4
125.6 36.7 2 4 6 8 6 4
126.7 37.8 1 3 5 7 5 3

def axis/x=1:4:1 xax ; def axis/t=1:6:1 tax ; def grid/x=xax/t=tax grd
def axis/t=-1:6:1 tax2 ; def grid/x=xax/t=tax2 grd2
file/form=free/var=vin/g=grd2/col=8/ord=tx test.data
list/nohead lon,lat

1   / 1:  123.4  34.50
2   / 2:  124.5  35.60
3   / 3:  125.6  36.70
4   / 4:  126.7  37.80

list/nohead/ord=tx v
1   / 1:  1.000  2.000  3.000  4.000  5.000  6.000
2   / 2:  9.000  8.000  7.000  6.000  5.000  4.000
3   / 3:  2.000  4.000  6.000  8.000  6.000  4.000
4   / 4:  1.000  3.000  5.000  7.000  5.000  3.000

Good luck,
  Mick
--------------
Paulo Henrique wrote:
Hi Hanh,

It seems to be skipping initial records in my file.

I'll try to separate data by months. Maybe using bash/awk.

Thanks!

2008/6/26 <nguyen@xxxxxxxxxx <mailto:nguyen@xxxxxxxxxx>>:


   Paulo

   indeed I believe that it's limited to 100 columns so you may have
   to read
   your file twice. First you read the first 100 columns and then you
   use the
   option "/skip=100" to skip the first 100 columns to read the 76
   remaining
   columns.

   Hanh





   > Hi Hanh, it will be useful.
   >
   > My file has 176 rows where the two first are coordinates. I used
   >
   > file/form=delim/delim=";"/col=176 my_ascii_data.txt
   >
   > When I do 'show data' I got 100 variables sequentially named v1,
   v2, ...,
   > v100. Shall it be v1, v2, ..., v176, not? What's happening? Is
   it a Ferret
   > limitation or am I doing something wrong?
   >
   > Bill, your comment is surely valuable. I've used scat2grid
   taking care
   > about
   > errors produced by interpolation, but I hadn't thought any kind of
   > evaluation.
   >
   > Thanks for all attention!
   >
   > 2008/6/26 <nguyen@xxxxxxxxxx <mailto:nguyen@xxxxxxxxxx>>:

   >
   >> oups sory I didn't get your question
   >> well in this case you can check the section "2.5. ASCII data on
   line":
   >>
   >>
   http://ferret.pmel.noaa.gov/Ferret/documentation/users-guide/data-set-basics/ASCII-DATA
   >> especially the option /column=ncol to define the number of
   columns you
   >> have.
   >> Hanh
   >>
   >>
   >> > Hanh,
   >> >
   >> > Thanks for your help.
   >> >
   >> > scat2grid is the following step of my job. This time, I just
   would
   >> like
   >> to
   >> > store my data without any interpolation.
   >> >
   >> > Or, at least, I would like to read these data without
   specifying 170
   >> > variables in my 'file/var="..."' command.
   >> >
   >> > 2008/6/26 <nguyen@xxxxxxxxxx <mailto:nguyen@xxxxxxxxxx>>:

   >> >
   >> >> Hi Paulo
   >> >>
   >> >> there is a function "scat2grid" that grids scattered data.
   Check the
   >> >> documentation for detailed explanations Ch3 Sec2.3:
   >> >>
   >>
   http://ferret.pmel.noaa.gov/Ferret/documentation/users-guide/index/index-s
   >> >>
   >> >> Hanh
   >> >>
   >> >> > Hi Ferreters,
   >> >> >
   >> >> > I have some scattered precipitation data stored in a ASCII
   file
   >> with
   >> >> LON
   >> >> > in
   >> >> > first row, LAT in second row and with third to last rows being
   >> daily
   >> >> > accumulated rainfall relative to that (lon,lat) beginning at
   >> >> 01-jan-2008
   >> >> > separated by semicolons, e.g.:
   >> >> >
   >> >> > -39.05;-7.37;0;0;0;0;0;0;0;0;0;0;0;23;10;5;8 ;...
   >> >> > -40.12;-2.88;0;9.8;0;12.6;17.2;0;0;11;0;0;0;0;0;4.2;0 ;...
   >> >> > -39.58;-6.08;0;0;0;0;0;0;0;0;0;0;0;0;5;20;0 ;...
   >> >> > -40.12;-6.57;0;0;0;3;0;0;0;0;0;0;0;0;0;0;13 ;...
   >> >> > -40.55;-3.58;0;0;0;0;0;0;0;6;0;0;0;0;0;0;4 ;...
   >> >> > -39.73;-7;0;0;0;25.9;0;0;0;0.8;0;0;0;0;2.8;9.4;9.6;0 ;...
   >> >> > -38.25;-5.52;0;0;0;0;0;0;0;0;0;0;0;0;2;0;2.2;...
   >> >> > .
   >> >> > .
   >> >> > .
   >> >> >
   >> >> > The full dataset covers about 170 days and, hence, we may
   expect
   >> about
   >> >> 170
   >> >> > rows. I know this procedure to read ASCII data:
   >> >> >
   >> >> >
   file/form=delim/delim="<delim>"/var="lon,lat,day1,day2,day3,...,dayN"
   >> >> > my_ascii_data.txt
   >> >> >
   >> >> > But it seems to be not really useful in my case.
   >> >> >
   >> >> > Is there a way to produce a NC file with one variable
   ('precip') in
   >> a
   >> >> > regular time axis and irregular XY axis?
   >> >> >
   >> >> > If not, should be useful a solution to read this data without
   >> >> specifying
   >> >> > one
   >> >> > variable for each day (say, using file/var="<list of vars>"
   >> >> my_data.txt).
   >> >> >
   >> >> > Well... to finish, I would assure a nice reward for that
   who first
   >> >> help
   >> >> to
   >> >> > solve this problem!
   >> >> >
   >> >> > Thanks
   >> >> >
   >> >> >
   >> >> > --
   >> >> > Ms. Paulo Henrique Santiago de Maria
   >> >> > Grupo de Modelagem Atmosférica
   >> >> > Departamento de Meteorologia e Oceanografia
   >> >> > Fundação Cearense de Meteorologia e Recursos Hídricos
   >> >> > Av. Rui Barbosa 1246 - CEP 60115-221
   >> >> > Fortaleza, Ceará
   >> >> > Fone: (85) 3101-1106 / 3101-1126
   >> >> >
   >> >>
   >> >>
   >> >
   >> >
   >> > --
   >> > Ms. Paulo Henrique Santiago de Maria
   >> > Grupo de Modelagem Atmosférica
   >> > Departamento de Meteorologia e Oceanografia
   >> > Fundação Cearense de Meteorologia e Recursos Hídricos
   >> > Av. Rui Barbosa 1246 - CEP 60115-221
   >> > Fortaleza, Ceará
   >> > Fone: (85) 3101-1106 / 3101-1126
   >> >
   >>
   >>
   >
   >
   > --
   > Ms. Paulo Henrique Santiago de Maria
   > Grupo de Modelagem Atmosférica
   > Departamento de Meteorologia e Oceanografia
   > Fundação Cearense de Meteorologia e Recursos Hídricos
   > Av. Rui Barbosa 1246 - CEP 60115-221
   > Fortaleza, Ceará
   > Fone: (85) 3101-1106 / 3101-1126
   >




--
Ms. Paulo Henrique Santiago de Maria
Grupo de Modelagem Atmosférica
Departamento de Meteorologia e Oceanografia
Fundação Cearense de Meteorologia e Recursos Hídricos
Av. Rui Barbosa 1246 - CEP 60115-221
Fortaleza, Ceará
Fone: (85) 3101-1106 / 3101-1126




--
Ms. Paulo Henrique Santiago de Maria
Grupo de Modelagem Atmosférica
Departamento de Meteorologia e Oceanografia
Fundação Cearense de Meteorologia e Recursos Hídricos
Av. Rui Barbosa 1246 - CEP 60115-221
Fortaleza, Ceará
Fone: (85) 3101-1106 / 3101-1126
[Thread Prev][Thread Next][Index]

Contact Us
Dept of Commerce / NOAA / OAR / PMEL / TMAP

Privacy Policy | Disclaimer | Accessibility Statement