[Thread Prev][Thread Next][Index]

[ferret_users] Re: How to manipulate data ( irregular ) after reading from file to make grid



Hello,

This looks like it is a set of data at various stations, but are not organized as a grid.  I'll show a few Ferret commands to let you look at what is in the file.

If the station locations were chosen so that they form a grid, then you could define a grid that corresponds to the station locations (or in fact define any grid you want to), and use one of the SCAT2GRID*XY functions to put the temp, salt variables into the grid.  That would mean that you would be ignoring the fact that the measurements represent several year's worth of data.  You could perhaps define an X-Y-T grid and use a SCAT2GRID*XYT function to put the data onto that.  Likely this data would be fairly sparse in such a grid.

Instead you might want to think about the data as non-gridded data, as sets of time series at each station.

Note that if you edit the file to remove the quotes around the date-time field, then you can read the dates and times, by telling Ferret that a blank is another delimiter to use.  (Future versions of Ferret will have options to read time records like this using a single date-time data type, for now read two variables date and time).

yes? columns/skip=1/format=delim\
/delim="\b\,"/type="numeric,numeric,numeric,numeric,eurodate,time,numeric,numeric"\
/var="pkey1,pkey2,latitude,longitude,st_date,st_time,temp,salt" test.csv

yes? sh dat
     currently SET data sets:
    1> ./test.csv  (default)
 name     title                             I         J         K         L
 PKEY1    pkey1                            1:729     ...       ...       ...
 PKEY2    pkey2                            1:729     ...       ...       ...
 LATITUDE latitude                         1:729     ...       ...       ...
 LONGITUDE
          longitude                        1:729     ...       ...       ...
 ST_DATE  st_date                          1:729     ...       ...       ...
       (Julian days since 1-Jan-1900)
 ST_TIME  st_time                          1:729     ...       ...       ...
       (Time of day)
 TEMP     temp                             1:729     ...       ...       ...
 SALT     salt                             1:729     ...       ...       ...

! time in days and fractions of days
yes? let date_and_time = st_date + st_time/24

Now you can investigate the data by making some plots, for instance:

yes? plot longitude
yes? plot latitude
yes? plot/vs/line/sym longitude,latitude
yes? plot date_and_time
yes? plot/i=1:50 date_and_time  ! to zoom in on the details of the time information

You'll see that the longitude/latitudes as they appear in the file not represent a grid; if they did then the one varying fastest would repeat. You'll also see that the times are sequences of times at each station, but they do not represent a single time axis.

-Ansley

On 12/2/2016 4:49 AM, Don DNA wrote:
Hey All,

I got csv file, which I can read easily on ferret with 1 line command, but I don't know how to manipulate further to make grid which I am looking forward, please someone help me, it will be very useful.

Here is my problem

$ wc -l test.csv
730 test.csv

$ head -1 test.csv
"pkey1","pkey2","LATITUDE","LONGITUDE","ST_DATE","TEMP","SALT"

$ cut -d',' -f1,2,3,4 test.csv  | sort -un | wc -l
82

$ cut -d',' -f1,2,3,4 test.csv  | sort -un | head -10
"pkey1","pkey2","LATITUDE","LONGITUDE"
869,1,0,93
869,2,0,93
869,3,0,93
869,4,0,93
869,5,0,93
869,6,0,93
884,1,0,83
884,2,0,83
884,3,0,83

$ ferret
     NOAA/PMEL TMAP
     FERRET v6.96 
     Linux 2.6.32-573.7.1.el6.x86_64 64-bit - 12/02/15
      2-Dec-16 17:52    

yes? columns/skip=1/format=delim/delim="\,"/type="numeric,numeric,numeric,numeric,text,numeric,numeric"/var="pkey1,pkey2,latitude,longitude,st_date,temp,salt" test.csv
yes? sh d
     currently SET data sets:
    1> ./test.csv  (default)
 name     title                             I         J         K         L         M         N
 PKEY1    PKEY1                            1:730     ...       ...       ...       ...       ...
 PKEY2    PKEY2                            1:730     ...       ...       ...       ...       ...
 LATITUDE LATITUDE                         1:730     ...       ...       ...       ...       ...
 LONGITUDE
          LONGITUDE                        1:730     ...       ...       ...       ...       ...
 ST_DATE  ST_DATE                          1:730     ...       ...       ...       ...       ...
 TEMP     TEMP                             1:730     ...       ...       ...       ...       ...
 SALT     SALT                             1:730     ...       ...       ...       ...       ...
 
yes?


I would like to make grid such a way that, when I type "sh d", I want to see like this, since my data is not uniformly spaced, I want to keep in 'M' as string only

yes? sh d
     currently SET data sets:
    1> ./test.csv  (default)
 name     title                             I         J         K         L         M                 N
 PKEY1    PKEY1                            1:82     ...       ...       ...       ...               ...
 PKEY2    PKEY2                            1:82     ...       ...       ...       ...               ...
 LATITUDE LATITUDE                         1:82     ...       ...       ...       ...               ...
 LONGITUDE
          LONGITUDE                        1:82     ...       ...       ...       ...               ...
 ST_DATE  ST_DATE                          1:82     ...       ...       ...       1:maxlength       ...
 TEMP     TEMP                             1:82     ...       ...       ...       1:maxlength       ...
 SALT     SALT                             1:82     ...       ...       ...       1:maxlength       ...


If I list like this, it should show no of records = maxlength, something like below

list/i=1 st_date,temp,salt
             DATA SET: ./test.csv
             X: 0.5 to 8.5
 Column  1: ST_DATE
 Column  2: TEMP
 Column  3: SALT
                      ST_DATE   TEMP   SALT
1           / 1: "2000-02-13 17:00:00"  14.37  35.14
2           / 2: "2000-02-13 18:00:00"  15.25  35.09
3           / 3: "2000-02-13 19:00:00"  14.42  35.10
4           / 4: "2000-02-13 20:00:00"  14.37  35.14
5           / 5: "2000-02-13 21:00:00"  14.25  35.12
6           / 6: "2000-02-13 22:00:00"  14.32  35.19
7           / 7: "2000-02-13 23:00:00"  14.37  35.14
maxlength   / 8: "2000-02-14 00:00:00"  14.30  35.14

Please someone help me.


Thanks in advance.
















[Thread Prev][Thread Next][Index]
Contact Us
Dept of Commerce / NOAA / OAR / PMEL / Ferret

Privacy Policy | Disclaimer | Accessibility Statement