Re: [ferret_users] How to merge daily and monthly data file in Ferret

[Ansley Manke <ansley.b.manke@xxxxxxxx>]

Hi,
To answer the question about the syntax

let var1_with_T = t[g=gg]*0 + wnd10[d=1]

If a variable does not have any time axis at all, this is a way to give it a time axis.  Your data has an axis in the time dimension, but it doesn't follow the standard for time attributes, so it won't be defined as a calendar axis, days in the year 1985, for instance; it'll just say t=1, 2, 3.

In Ferret you will see error messages at different times depending on the type of error. Syntax errors, like the wrong qualifier on a command, or mismatched parentheses, are given when you enter the command. Errors having to do with the grids of variables, or various kinds of operations with variables, aren't detected until Ferret tries to load data and resolve the sets of definitions that have been made.  I think that when you tried to use var1_with_T, you'd see an error about the time axes.  So a LIST, or SAVE command using the variable would trigger an error.

Try things with just one or two datasets to see what it looks like:

use file1.nc
define axis/t ...

let var1_with_t = wnd10[gt=t_day@ASN]

list/L=1:10/x=180/y=0 var1_with_t

save/clobber/file=test.nc var1_with_t

and look at the ncdump header for test1.nc

cancel data/all

use file2.nc
define axis/t  ...  ! the new definition for the time range of this dataset

let var1_with_t = wnd10[gt=t_day@ASN]

list/L=1:10/x=180/y=0 var1_with_t

save/append/file=test.nc var1_with_t

and check that. 

For your question about saving data, you can do a list command as I did above, but specify a file name LIST/file=test.dat, for instance, to write an ascii text file; or you can use SAVE/file=test.nc (which is an alias for LIST/format=cdf) to write a netCDF file.  Once you have a time axis defined, the time axis information is automatically part of the file.

Ansley

On 4/5/2013 2:52 AM, Don DNA wrote:

Thank you Ansley, but I am facing one problem with @asn transformation while using multiple datasets

say

use file1.nc
use file2.nc

variable names in both files are same, say wnd10, then following I can't use with @asn

let var1_with_T =  wnd10[d=1][gt=t_day@ASN];\
let var1_with_T =  wnd10[d=2][gt=t_day@ASN];\

but following giving no error, could you please explain difference between these two commands (given by you and Akshay)

let var1_with_T = t[g=gg]*0 + wnd10[d=1]
let var1_with_T = t[g=gg]*0 + wnd10[d=1]

and one more thing I want to save  show axis / T taxisname   command output  in variable say tax_out, and then I want to list with some other variable available in grid

whether command result can be saved ? 

-Dona Clara

---

From: Ansley Manke <ansley.b.manke@xxxxxxxx>
To: Don DNA <don_vbe@xxxxxxxxx>
Cc: Ferret users mailing list <ferret_users@xxxxxxxx>
Sent: Wednesday, April 3, 2013 10:38 PM
Subject: Re: [ferret_users] How to merge daily and monthly data file in Ferret

Hi Dona,
So, you want to merge a set of daily data into one file, and a set of monthly data into another, is that correct?

For the set of files in units of days, I think  Akshay's idea should work  - define a time axis for each file, representing the days in that file, and then put the data onto that time axis.  The crucial thing there is to use the same common time origin (Ferret sets a time origin if you don't give one in the DEFINE AXIS command), and the same time-axis name for all of the input data files. Then the data from each file can be appended to the big dataset. 

However I think that instead of these commands,

let var1_with_T = t[g=gg]*0 + wnd10;\
let var2_with_T = t[g=gg]*0 + err;\

you would need to define the variables like this, to transfer the data from the incoming time axis to the time axis you're going to write the data on.

let var1_with_T =  wnd10[gt=t_day@ASN];\
let var2_with_T =  err[gt=t_day@ASN];\


I have something else to point out here. The files you show don't have the time-axis attributes that Ferret, and the CF standard uses to recognize calendar time axes.  The attributes must contain information in standard attributes that spell out the time origin. Here is the standard,
http://cf-pcmdi.llnl.gov/documents/cf-conventions/latest-cf-conventions-document-1

and the time-axis portion of the standard is here,
http://cf-pcmdi.llnl.gov/documents/cf-conventions/1.6/ch04s04.html

The time coordinates should have a time origin and units, which for your data should probably be,

netcdf ws_oaflux_1985 {
dimensions:
    time = UNLIMITED ; // (365 currently)
    lat = 180 ;
    lon = 360 ;
variables:

...

    int time(time) ;
        time:units = "days since 1-jan-1985" ;
...

If you wanted to do something along the lines that Hein suggests, defining a new time axis and using Ferret transformations to work with the time axes, these individual yearly files would need to be first fixed with a units attribute that specifies the time origin.

Ansley

On 4/3/2013 7:42 AM, Akshay Hegde wrote:

Hi ! Dona Clara,

try following may help you, if you modify as per dataset, either monthly or daily

let ii=0
let yr=1984
let model={spawn:'ls *.nc '};
repeat/range=1:27 (let ii=`ii+1`; let yr=`yr+1`;let a=model[i=`ii`];use `a`;\
let npts = `wnd10,RETURN=lsize`;\
def axis/t="01-jan-`yr`":"31-dec-`yr`"/npoint=`npts`/units=days t_day;\
def grid/t=t_day gg;\      
let var1_with_T = t[g=gg]*0 + wnd10;\
let var2_with_T = t[g=gg]*0 + err;\
LET/units="`wnd10,RETURN=units`"/TITLE="Daily mean neutral wind speed at 10m" Wind_Speed = var1_with_T;\
LET/units="`err,RETURN=units`"/TITLE="Estimated error of analysed neutral wind speed at 10m" Error = var2_with_T;\
save/append/file=output_filename.nc/format=cdf Wind_Speed[l=1:`npts`],error[l=1:`npts`];\
canc data 1)

Akshay Hegde,

National Institute of Oceanography,

Goa, India.

On Wed, Apr 3, 2013 at 4:49 PM, Don DNA <don_vbe@xxxxxxxxx> wrote:

I don't want to do either averaging or daily to monthly or monthly to yearly

see first daily file has l length is 365(days), second may contains 366 days,

I want to simply merge like this 365+366....so on upto 2012, for daily data 365*27 files around 9855 l length should be,

have you seen ICOADS monthly data ?

like that I want to create grid for monthly data, if I plot by setting

set r/i=1:360/j=1:180/l=1
should result 1958 January 1st

set r/i=1:360/j=1:180/l=2
should result 1958 February 1st ...so on upto 2012

those who have done before please help.

-Dona Clara

----- Original Message -----
From: Hein Zelle <hein.zelle@xxxxxxxxxxxxx>
To: Don DNA <don_vbe@xxxxxxxxx>
Cc: Ferret users mailing list <ferret_users@xxxxxxxx>

Sent: Wednesday, April 3, 2013 3:45 PM
Subject: Re: [ferret_users] How to merge daily and monthly data file in Ferret

Dear Dona Clara,

> Daily data I am having from 1985 to 2012 total 27 files, and monthly
> data I am having from 1958 to 2012 total 54 files, finally I want to
> create single file,

what do you want to do with the overlapping period?

> please explain how to define time axis for daily
> and monthly data and moving data into grid.

As I said, please read up on the "define axis" command in the manual.
You can find it here:

http://ferret.pmel.noaa.gov/Ferret/documentation/users-guide/commands-reference/DEFINE

You'll need to decide if you want all data on a daily axis, or all
data on a monthly axis.  Once you decide that,

1)    define your new axis, covering 1958-2012
2)    define a new variable on the new axis, e.g.

      let new_montly_data = old_daily_data[gt=my_monthly_time_axis@ave]

3)    decide how to deal with duplicate data.  Do you want to average
      them, or choose daily over monthly?

4)    Assuming you want daily data where it is available, combine the
      two variables with something like

      let my_combined_data = if (new_monthly_data) then \
               new_monthly_data else old_monthly_data

Kind regards,
     Hein Zelle


--

Dr. Hein Zelle
Senior consultant meteorology & oceanography
BMT ARGOSS

Tel:        +31 (0)527-242299
Fax:        +31 (0)527-242016
E-mail:    hein.zelle@xxxxxxxxxxxxx
Website:    www.bmtargoss.com
       
BMT ARGOSS b.v.
Voorsterweg 28, 8316 PT Marknesse, the Netherlands
Postal address: P.O. Box 61, 8325 ZH Vollenhove, the Netherlands

Registered in The Netherlands, Registered no. 39060160.

Unless otherwise agreed by BMT ARGOSS in writing, all work,
services, goods or products supplied by BMT ARGOSS shall be subject
to and governed by BMT ARGOSS' own terms and conditions which are
available for inspection from BMT ARGOSS on request.

E-mail confidentiality notice and disclaimer:
The contents of this e-mail and any attachments are intended for the
use of the mail addressee(s) shown. If you are not that person, you
are not allowed to read it, to take any action based upon it or to
copy it, forward, distribute or disclose the contents of it and you
should please delete it from your system. BMT ARGOSS does not accept
liability for any errors or omissions in the context of this e-mail or
its attachments which arise as a result of internet transmission, nor
accept liability for statements which are those of the author and
clearly not made on behalf of BMT ARGOSS.