Re: [ferret_users] insert data into variable at certain time step

[Fabian Lienert <cccma-student-003@xxxxxxxx>]

Hi Ferret crowd,

The script below works fine if there are no missing values. Since I started
computing lagged correlations for SST, a netcdf dataset with missing values,
the troubles started. Ferret crashes when reading an ASCII file with missing
values. Adding different format options when writing and reading text file like 
list/form=comma didn't help, Fferret either crashes or complains about the missing
values ***** in the text file.

So I tried writing the 2d field 'correl' in the repeat loop into a netcdf file
using save/append/file="temp.nc"/clobber correl[g=sst[d=2]] without success.
Correl does not have a time axis, so correl is not appended but overwritten.
Using just save/append/file="temp.nc" or adding and increasing /l=1 does not help either.
I also tried to define different time axis for correl like the following, without success.
def axis/t=1:3:1 lagx; def grid/like=sst[d=2,l=1]/t=lagx glag; correl[g=glag]

The only fix I could come up with was the following:
let max_lag = 1
repeat/range=`-1*max_lag`:`max_lag`:1/name=lag (let q = some_ts[l=@shf:`lag`]; let p = sst[d=2]; go variance;\
            let lagcorr = correl + 0*sst[d=2,l=`lag+max_lag+1`];\
	     save/append/file="temp.nc"/clobber lagcorr;\
	     cancel var correl; cancel var q; cancel var p; cancel memory )

Cheers,
Fabian


Fabian Lienert wrote:
> Thanks very much Mick,
>
> I tested the commands. There should be an append option while writing
> the ASCII file in order not to overwrite the previous one.
> So for people finding this useful, below is the corrected version.
>
> Cheers,
> Fabian
>
>
> use my_data  !(the original xyt dataset containing "myvar")
> spawn touch temp.file
> repeat/range=1:5/name=lag ( ... define and compute the lagged 
> correlation variable "correl" from "myvar"  ;\
>    list/nohead/file=temp.file/form=(f12.5)/append correl )
> def axis/t=1:5:1 lagx
> def grid/like=myvar[d=1]/t=lagx glag
> file/form=free/g=glag/ord=xyt/var=mylagvar temp.file
> set var/title="Lagged Correlations" mylagvar
> ! now you can plot or use them ... or use ferret to write them to a 
> netCDF file then discard temp.file
> save/file=MyLaggedResults.nc mylagvar
>
>
> mick spillane wrote:
> > Hi Fabian,
> >    Unlike some other languages, what you are doing in Ferret is 
> > setting up the algebraic quantities to compute something, until you 
> > use or write the variable you define, it doesn't exist as a number. 
> > Your situation is a case like this -- for each lag the definition is 
> > changing so it is best to write out the results for each lag to a 
> > file, then read them back later.
> >    The most elegant way would be to write to a netCDF file, one time 
> > slice (lag) at a time.  But, unless the x-y grid is huge, I typically 
> > write to an ASCII file (one value per record) then define the 
> > appropriate grid and read it back. It would go something like this:
> >
> > use my_data  !(the original xyt dataset containing "myvar")
> > spawn touch temp.file
> > repeat/range=1:5/name=lag ( ... define and compute the lagged 
> > correlation variable "correl" from "myvar"  ;\
> >     list/nohead/file=temp.file/form=(f12.5) correl )
> > def axis/t=1:5:1 lagx
> > def grid/like=myvar[d=1]/t=lagx glag
> > file/form=free/g=glag/ord=xyt/var=mylagvar temp.file
> > set var/title="Lagged Correlations" mylagvar
> > ! now you can plot or use them ... or use ferret to write them to a 
> > netCDF file then discard temp.file
> > save/file=MyLaggedResults.nc mylagvar
> >
> > Things to note:
> > 1) ";\" continues the repeat statement to the next line
> > 2)  the build up lag-by-lag of the results file
> > 3) I just typed this in without doing a demo -- there may be typos but 
> > I hope the gist is clear
> >
> > Hope it helps,
> > Mick
> > PS Look in the user guide for the slice-by-slice construction of a 
> > netCDF file if you like to be more elegant
> > -------
> > Fabian Lienert wrote:
> > > Hi Ferreters,
> > >
> > > Simple question, couldn't find an answer so far.
> > > In a repeat loop, I am calculating lagged cross correlations:
> > >
> > > yes? repeat/range = 1:5:1/Name=lag (let q = 
> > > timeseries[d=1,i=1,l=@shf:`lag`]; \
> > >          let p = 3dvariable[d=2]; go variance;\
> > > All I want to do is to insert a 2-d field (correl) into a 3-d 
> > > variable (x,y,t) at each time step inside the repeat loop, something 
> > > like the following.
> > >          let lag_corr/l=`lag` = correl)
> > >
> > > In other programming languages this is easily done by filling an array:
> > > for (t in 1:5) {
> > >     lag_corr[,,t] = correl(t)
> > > }
> > >
> > > I am using FERRET v6.1 on Ubuntu Linux.
> > >
> > > Any help appreciated,
> > > Fabian


--
Fabian Lienert                  |  Canadian Centre for Climate Modelling and Analysis
PhD Student                     |  Meteorological Service of Canada
phone: +1(250)721-7211          |  University of Victoria
cccma-student-003@xxxxxxxx      |  P.O. Box 1700
http://www.cccma.bc.ec.gc.ca    |  Victoria, BC,  V8W 2Y2, Canada