[Thread Prev][Thread Next][Index]

# Re: [ferret_users] Double precision calculations in ferret

 Hi, Yes, there's definitely a bug in the embedded _expression_ evaluation. Even using the PREC keyword fails to return the correct precision. yes? list a              VARIABLE : 1.000000001           1.000 yes? list/prec=12 a              VARIABLE : 1.000000001           1.00000000100 yes? list/prec=12 `a`  !-> list/prec=12 1              VARIABLE : constant           1.00000000000 yes? list/prec=12 `a,prec=12`  !-> list/prec=12 1              VARIABLE : constant           1.00000000000 Under section 3.3.1 of the user guide we have: PRECISION=#digits can be used to control the number of significant digits displayed, up to a maximum of 16. (Prior to Ferret v6.8 this limit was 10 digits but actually at most 7 digits were significant since Ferret calculations were performed in single precision.) So it is definitely not working correctly. Russ On 19/02/17 23:41, Martin Schmidt wrote: Hi, I did not check the manual, but I remember a statement that ferret is doing calculations double precision now. Under this point of view, I cannot understand the following example: ferret         NOAA/PMEL TMAP         PyFERRET v7 (opt)         Linux 3.19.0-64-generic - 07/01/16         19-Feb-17 13:28 yes? let a=1+1e-12 yes? list a-1              VARIABLE : A-1           1.000E-12 is correct. yes? let a={`1+1e-1`,`1+1e-8`}  !-> DEFINE VARIABLE a={1.1,1} yes? list/prec=12 (a-1)              VARIABLE : (A-1)              SUBSET   : 2 points (X)  1   / 1:  0.100000000000  2   / 2:  0.000000000000 is not correct. There may be examples, where double precision is needed. Could it be that the `a` operator to evaluate reals still gives single precision results? Best, Martin

[Thread Prev][Thread Next][Index]
Contact Us
Dept of Commerce / NOAA / OAR / PMEL / Ferret

Privacy Policy | Disclaimer | Accessibility Statement