[Thread Prev][Thread Next][Index]

Re: [ferret_users] Reading ASCII data including formatted dates



Hi Billy,
Ferret does not read dates such as  "05-Sep-2007".  The months are numeric for these formats - I've added a note in the "set data" documentation on that detail.

The separators should be able to be dashes or slashes, however slashes seem to be what works. I will look into that and at least update the documentation.

I converted your example to this file:

 #sec  central_date  Miss ID  corr   Pmax    Q(0-500m)   Q(0-700m)
  1   05/09/2007   078006a     1   500        18.57       25.19        
  2   06/12/2007   07B018a     1   600         5.78        9.63        
  3   15/01/2008   07B018b     1   600         3.80        9.62        
  4   08/04/2008   082001a     1   600        -3.71        4.87        
  5   16/06/2008   082001b     1   600        16.43       16.63        
  6   21/07/2008   087006a     1   700        28.64       35.86        
  7   16/09/2008   087006b     1   700        14.80       15.31        
  8   01/12/2008   08B018a     1   700        16.57       22.62        
  9   23/01/2009   08B018b     1   700         4.22        6.70        
 10   02/08/2009   097006a     1   700        25.98       28.79        
 11   03/08/2009   097001a     1   700        24.83       28.62        

And then it can be read with this command.  These commands work with Ferret back to v6.72; I think they should work with the versions you have. 

By the way,  the name of this type is "EURODATE", for just a date 23/01/2009 or "EDATIME" for a combined euro-date-and-time (available in v7.1 and higher), such as 22/01/2014T01:00:00.  The documentation has an example where it names a variable edate, but its data type is eurodate.


yes? file/form=delim/delim=" "/skip=1/type="num,eurodate,text,num,num,num,num" dates.dat
yes? show data
     currently SET data sets:
    1> ./dates.dat  (default)
 name     title                             I         J         K         L
 V1       V1                               1:20480   ...       ...       ...
 V2       V2                               1:20480   ...       ...       ...
       (Julian days since 1-Jan-1900)
 V3       V3                               1:20480   ...       ...       ...
 V4       V4                               1:20480   ...       ...       ...
 V5       V5                               1:20480   ...       ...       ...
 V6       V6                               1:20480   ...       ...       ...
 V7       V7                               1:20480   ...       ...       ...
 
yes? list v2
             VARIABLE : V2 (days)
                        Julian days since 1-Jan-1900
             FILENAME : dates.dat
             SUBSET   : 11 points (X)
 1    /  1:  39328.
 2    /  2:  39420.
 3    /  3:  39460.
 4    /  4:  39544.
 5    /  5:  39613.
 6    /  6:  39648.
 7    /  7:  39705.
 8    /  8:  39781.
 9    /  9:  39834.
 10   / 10:  40025.
 11   / 11:  40026.

yes?  define axis/t/t0=1-jan-1900/units=days taxis = v2
yes? show axis taxis
 name       axis              # pts   start                end
 TAXIS     TIME                11 i   05-SEP-2007 00:00    03-AUG-2009 00:00
T0 = 1-JAN-1900
   Axis span (to cell edges) = 744.5




On 3/19/2018 4:07 PM, William S. Kessler wrote:
I'm trying to read a file consisting of a list of values (first few lines listed below). I've tried with both Ferret 6.94 and PyFerret 7.1 (on a Mac). I want to turn the list into a netcdf file (on a time axis) that I can do subsequent work on.

The hard part is the formatted dates; I am unable to do this with SET DAT/FORMAT=DELIMITED

First there's the problem with the alphabetical months, but even after laboriously changing all of those to numerical, and the separating dashes to slashes, EDATE failed.

I finally kluged this by editing the text to enforce comma-separated values, but this seems like unnecessary work. This is especially true since after having read the edited file on the default X-axis, using the numerical day/month/year to define a time axis, RESHAPEing all variables to that, then writing as cdf, Ferret's listing of the first few values is remarkably like the original!

Namely, I did a bunch of manual editing to make a file that Ferret could read and convert back to much like what I had to start with.

Was there a more straightforward way to do this? Can Ferret make sense of ASCII dates like "05-Sep-2007"? Should have asked the list before I started? 

Billy K

PS - sorry for my mailtool ruining the nice columns in both the original file and the Ferret listing.
-------------------

Original file (first 11 of hundreds of lines):

#sec  central_date  Miss ID  corr   Pmax    Q(0-500m)   Q(0-700m) 
  1   05-Sep-2007   078006a     1   500        18.57       25.19         
  2   06-Dec-2007   07B018a     1   600         5.78        9.63         
  3   15-Jan-2008   07B018b     1   600         3.80        9.62         
  4   08-Apr-2008   082001a     1   600        -3.71        4.87         
  5   16-Jun-2008   082001b     1   600        16.43       16.63         
  6   21-Jul-2008   087006a     1   700        28.64       35.86         
  7   16-Sep-2008   087006b     1   700        14.80       15.31         
  8   01-Dec-2008   08B018a     1   700        16.57       22.62         
  9   23-Jan-2009   08B018b     1   700         4.22        6.70         
 10   02-Aug-2009   097006a     1   700        25.98       28.79         
 11   03-Aug-2009   097001a     1   700        24.83       28.62         

Ferret listing of resulting cdf:

yes? list/l=1:11 secnumt,midt,bott,t500t,t700t
             DATA SET: ./solsea103-tr_v2b.cdf
             TIME: 21-JUL-2007 00:00 to 27-AUG-2009 00:00
 Column  1: SECNUMT is RESHAPE(SECNUM,TTHH)
 Column  2: MIDT is RESHAPE(MID,TTHH)
 Column  3: BOTT is RESHAPE(BOT,TTHH)
 Column  4: T500T is RESHAPE(T500,TTHH)
 Column  5: T700T is RESHAPE(T700,TTHH)
                              SECNUMT     MIDT    BOTT   T500T  T700T
05-SEP-2007 00 /  1:    1.00 "078006a"  500.0  18.57  25.19
06-DEC-2007 00 /  2:    2.00 "07B018a"  600.0   5.78   9.63
15-JAN-2008 00 /  3:    3.00 "07B018b"  600.0   3.80   9.62
08-APR-2008 00 /  4:    4.00 "082001a"  600.0  -3.71   4.87
16-JUN-2008 00 /  5:    5.00 "082001b"  600.0  16.43  16.63
21-JUL-2008 00 /  6:    6.00 "087006a"  700.0  28.64  35.86
16-SEP-2008 00 /  7:    7.00 "087006b"  700.0  14.80  15.31
01-DEC-2008 00 /  8:    8.00 "08B018a"  700.0  16.57  22.62
23-JAN-2009 00 /  9:    9.00 "08B018b"  700.0   4.22   6.70
02-AUG-2009 00 / 10:   10.00 "097006a"  700.0  25.98  28.79
03-AUG-2009 00 / 11:   11.00 "097001a"  700.0  24.83  28.62




[Thread Prev][Thread Next][Index]
Contact Us
Dept of Commerce / NOAA / OAR / PMEL / Ferret

Privacy Policy | Disclaimer | Accessibility Statement