[Thread Prev][Thread Next][Index]

Re: [ferret_users] Double precision calculations in ferret




Hi,

Yes, there's definitely a bug in the embedded _expression_ evaluation. Even using the PREC keyword fails to return the correct precision.

yes? list a
             VARIABLE : 1.000000001
          1.000
yes? list/prec=12 a
             VARIABLE : 1.000000001
          1.00000000100
yes? list/prec=12 `a`
 !-> list/prec=12 1
             VARIABLE : constant
          1.00000000000
yes? list/prec=12 `a,prec=12`
 !-> list/prec=12 1
             VARIABLE : constant
          1.00000000000

Under section 3.3.1 of the user guide we have:

PRECISION=#digits

can be used to control the number of significant digits displayed, up to a maximum of 16. (Prior to Ferret v6.8 this limit was 10 digits but actually at most 7 digits were significant since Ferret calculations were performed in single precision.)


So it is definitely not working correctly.

Russ



On 19/02/17 23:41, Martin Schmidt wrote:
Hi,
I did not check the manual, but I remember a statement that ferret is doing calculations double precision now.

Under this point of view, I cannot understand the following example:
ferret
        NOAA/PMEL TMAP
        PyFERRET v7 (opt)
        Linux 3.19.0-64-generic - 07/01/16
        19-Feb-17 13:28
yes? let a=1+1e-12
yes? list a-1
             VARIABLE : A-1
          1.000E-12
is correct.

yes? let a={`1+1e-1`,`1+1e-8`}
 !-> DEFINE VARIABLE a={1.1,1}
yes? list/prec=12 (a-1)
             VARIABLE : (A-1)
             SUBSET   : 2 points (X)
 1   / 1:  0.100000000000
 2   / 2:  0.000000000000

is not correct. There may be examples, where double precision is needed. Could it be that the `a` operator to evaluate reals still gives single precision results?

Best,
Martin



[Thread Prev][Thread Next][Index]
Contact Us
Dept of Commerce / NOAA / OAR / PMEL / Ferret

Privacy Policy | Disclaimer | Accessibility Statement