Re: [ferret_users] data size estimation is not unique

["Ansley C. Manke" <ansley.b.manke@xxxxxxxx>]

Hi Martin,

Thank you for the report.  In very old versions of Ferret, the reported size was the size of the abstract axis used to read ascii data with SET DATA/EZ

        NOAA/PMEL TMAP
        FERRET v6.72 
        Linux 2.6.32-131.12.1.el6.x86_64 64-bit - 09/13/11
        25-Oct-17 09:49    

yes?  file/var="t1,t2"/skip=1 test.data
yes? say `t1,return=iend`
 !-> MESSAGE/CONTINUE 20480
20480
yes? load t1
yes? say `t1,return=iend`
 !-> MESSAGE/CONTINUE 3
3

We've improved upon that, but it seems Ferret is counting the number of header lines.  If there are no header lines, the value is correct; if I add more lines, the reported size increases accordingly. The `var,return=`  commands give information about variables and grids as stored by Ferret, generally without reading data.  When the data has been loaded, information about the grid has been updated.  This should be simple to track down and fix.

Ansley

On 10/24/2017 8:29 AM, Martin Schmidt wrote:

Hi,
reading ascii data, I encountered the following problem. The estimated length of an input field depends on the history of operations.

My data file test.data looks like this

header
1,   1
2,   2
3,   3

The script to read the data and to estimate the length of the field is

set data/ez/skip=1/var=t1,t2/format=delim/type=numeric  test.data
let imax=`t1,return=iend`
list imax

ferret
        NOAA/PMEL TMAP
        PyFerret v7.22 (optimized)
        Linux 4.4.87-25-default - 10/05/17
        24-Oct-17 17:12

CAN MODE LOGO
SET MEM /SIZE=25.6
SET MODE VERIFY
yes? set data/ez/skip=1/var=t1,t2/format=delim/type=numeric test.data
yes? let imax=`t1,return=iend`
 !-> DEFINE VARIABLE imax=4
yes? list imax
             VARIABLE : 4
          4.000
yes?
I would say, the result for imax is wrong.

Modifying the script, just by  adding a 'list' command gives the correct (or a more likely) result:
yes? set data/ez/skip=1/var=t1,t2/format=delim/type=numeric test.data
yes? list t1
             VARIABLE : t1
             FILENAME : test.data
             SUBSET   : 3 points (X)
 1   / 1:  1.000
 2   / 2:  2.000
 3   / 3:  3.000
yes? let imax=`t1,return=iend`
 !-> DEFINE VARIABLE imax=3
yes? list imax
             VARIABLE : 3
          3.000
yes?

This dependence of the variable size estimate on the history of operations seems to be inherited over generations of ferret. I have very old scripts, where some 'list' or 'shade' was added to get them running.

May be, the report helps.

Best,
Martin