[ferret_users] Bug report: listings multiple variables to ascii file

["Ansley C. Manke" <ansley.b.manke@xxxxxxxx>]

Hi all,

We've discovered a Ferret/PyFerret bug which, though I hope it's not too common a case, can result in incorrect data being listed to the screen or to ascii (text) files.  It does not affect netCDf output.  So this applies to the simple LIST command, or LIST/FORMAT=comma, LIST/FORMAT=tab but not SAVE.  This is not a new bug; it's been present for quite a few years.

We've identified the cause of this and the bug will be fixed in an upcoming release, but we thought we'd let you know about it and ways that it can be worked around in current releases using command syntax.

First a bit of explanation. In Ferret, variables that are being combined in expressions or in output listings must have "conformable" grids, that is grids that share their coordinates and can be automatically combined by Ferret.  If a variable has just an X axis, then it can be combined with a multi-dimensional variable that has the same X coordinates.  This lets you do something like subtract a time average from a variable, even though the variable is in XYT and the average of course is in XY, as in  "LET anomaly = temperature - temperature[T=@AVE]"

The bug came up in a listing of a data variable, say sst on an XY grid, along with the coordinates, so something like this:

yes? list/format=(F8.2)/file=mydata.dat  y[gy=sst], x[gx=sst], sst

The coordinates y[gy=sst], x[gx=sst] are of course simple 1-D  lists in Y and X.  They will be automatically replicated in a listing like this, so if this is a simple test variable

yes? list testvar
             VARIABLE : example
             SUBSET   : 2 by 3 points (X-Y)
             1      2   
             1      2
 1   / 1:  100.0  200.0
 2   / 2:  300.0  400.0
 3   / 3:  500.0  600.0

and it's listed in comma-delimited form, along with its coordinates, the first two columns will be the x's then the y's and finally the variable.  This listing with x varying fastest, is correct.

yes? list/form=comma/norow x[gx=testvar], y[gy=testvar], testvar
             X: 0.5 to 2.5
             Y: 0.5 to 3.5
 Column  1: X is X (axis XAXIS)   BAD FLAG : -1.E+34
 Column  2: Y is Y (axis YAXIS)   BAD FLAG : -1.E+34
 Column  3: TESTVAR is example    BAD FLAG : -1.E+34
X,Y,TESTVAR
 ---- J:1 Y:   1
1,1,100 
2,1,200 
 ---- J:2 Y:   2
1,2,300 
2,2,400 
 ---- J:3 Y:   3
1,3,500 
2,3,600 
 
 

The bug is seen if the Y coordinates are first in the list of variables to write.  This puts the listing into a state where Y is varying fastest, but the 2-D variable isn't permuted so it's just listed in such a way that it no longer matches the coordinates listed.

 yes? list/form=comma/norow y[gy=testvar], x[gx=testvar], testvar
             Y: 0.5 to 3.5
             X: 0.5 to 2.5
 Column  1: Y is Y (axis YAXIS)   BAD FLAG : -1.E+34
 Column  2: X is X (axis XAXIS)   BAD FLAG : -1.E+34
 Column  3: TESTVAR is example    BAD FLAG : -1.E+34
Y,X,TESTVAR
 ---- I:1 X:   1
1,1,100 
2,1,200 
3,1,300 
 ---- I:2 X:   2
1,2,400 
2,2,500 
3,2,600  

One solution is to put a permutation in the list command.  Either of these forms would give a correct listing; varying fastest in y or varying fastest in x,

yes? list/order=yx/form=comma/norow y[gy=testvar], x[gx=testvar], testvar
yes? list/order=xy/form=comma/norow y[gy=testvar], x[gx=testvar], testvar

Or, it's often nice to write data out in a cleaner listing, without those extra lines labeling the I/X or J/Y information.  Expand the coordinates explicitly into 2-D variables, and then turn them along with the data variable into simple lists with XSEQUENCE.  These listings will be correct using any version of Ferret/PyFerret. 

yes? let xvar = x[gx=testvar] + 0*y[gy=testvar]
yes? let yvar = 0*x[gx=testvar] + y[gy=testvar]
yes? list yvar, xvar, testvar

yes? let/title="Longitude" xlon= xsequence(xvar)
yes? let/title="Latitude" ylat = xsequence(yvar)
yes? let/title="Example data" values = xsequence(testvar)

yes? list/format=comma/norow xlon,  ylat, values

yes? ! list out to a file
yes? list/format=comma/clobber/file=yxdat.txt xlon,  ylat, values