Re: [ferret_users] MEMORY/SIZE= changes the result of SAVE

[Ryo Furue <furue@xxxxxxxxxx>]

Hi Martin,

Thank you for your help.

First, I've found that if I trim my data a little bit, the problem goes away.  There seems to be a clear threshold.  When i=1:3600, the problem occurs.  If I save the i=31:3600 portion or the i=1:3500 portion of the data, the new datafile doesn't cause the problem.  Something like that.

That means that I have to debug the problem with this largish dataset.

On Mon, Dec 16, 2019 at 3:57 PM Martin Schmidt <martin.schmidt@xxxxxxxxxxxxxxxxx> wrote:

Generate the axes and than the data like
   let myvar = 0*(x[gx=xax] + y[gy=yax} ......) + 1
   save /file=   ... myvar

That's brilliant!  Unfortunately, I haven't been able to generate a variable like that which reproduces my problem.  I confirm

1) "set data the_new_dummy_data.nc" ---The problem doesn't occur.

2) "set data my_problematic_data.nc" --- The problem occurs.

To repeat, the problem is

set data my_problematic_data.nc ! contains variable "usss"

set memory/size=300 !<--

let u = usss[d=1]

save . . . u[m=1:10@AVE]

save . . . usss[m=1:10@AVE]

The two datasets contains different values when the SET MEMORY command exists.

When I generated a new netCDF file using a dummy variable as Martin suggested and used it instead of my original data file, this problem doesn't show up.

Below is the script.  The range of the dummy variable is ~1 and the maximum error is ~1E-17 .

In contrast, my original data file, which contains a velocity dataset in cm/s so its range is ~1--10, the maximum difference is ~10.  When I remove the "SET MEMORY" statement, the difference goes down to ~1E-14 .

Tested on PyFerret 7.5 on Linux.

Regards,

Ryo

===============

! Instead of using the original data file, generate a dummy data only from
! axis information.

define axis/x=0:360:0.1/units=degrees_east xax
define axis/y=-90:90:0.1/units=degrees_north yax
define axis/t="1-JAN-2020 00:00:00":"1-JAN-2020 00:00:00":1/units=days tax

let ens = _e[e=1:10]
let usss = sin(x[gx=xax]/500 + ens) * sin(ens*y[gy=yax]/200) \
          *sin(t[gt=tax]) * (ens^0.5)
set memory/size=300
SPAWN rm tmp.nc
repeat/m=1:10 ( save/append/mlimits=1:10/file=tmp.nc usss )
cancel var/all

set data tmp.nc ! <- Dummy data
set memory/size=300 ! <<==
let u = usss[d=1]
save/clobber/file=tmp-u.nc     u[   m=1:10@AVE]
save/clobber/file=tmp-usss.nc  usss[m=1:10@AVE]


cancel var/all
cancel data/all
set data tmp-u.nc
set data tmp-usss.nc
let diff = u[d=1] - usss[d=2]
plot diff[x=@MAX], diff[x=@MIN]