Re: [ferret_users] precision issue ?

[Ansley Manke <ansley.b.manke@xxxxxxxx>]

Hi again,
The issue of the accuracy of numerical calculations has come up before. 
There's a nice, short discussion of this topic in the first chapter of 
Numerical Recipes (or any good text on numerical calculation).  It's 
good to have a basic understanding of what one can expect from any 
computational tool or language and also the approximations that come 
along with doing discrete mathematics.

Ansley

Alexa Griesel wrote:

> Dear ferret users,
> I have variables defined on a lon,lat,depth grid.  I am calculating 
> cross (uteddyc, vteddyc) and along (uteddya, vteddya) temperature 
> contours components of eddy heat fluxes. As a test,  
> uteddyc*uteddya+vteddyc*vteddya  should be zero.
> but what I am getting is 2^n in a lot of places (i.e. 2, 8, 16, 32, 
> 64,128..etc.), e.g. see below:
>
> yes? list UTEDDYC[k=1,j=500,i=205]*UTEDDYA[k=1,j=500,i=205]
>                    -1.458E+09
> yes? list VTEDDYC[k=1,j=500,i=205]*VTEDDYA[k=1,j=500,i=205]
>                     1.458E+09
> yes? list 
> UTEDDYC[k=1,j=500,i=205]*UTEDDYA[k=1,j=500,i=205]+VTEDDYC[k=1,j=500,i=205]*VTEDDYA[k=1,j=500,i=205] 
>                    -128.0
>
> so listing the terms individually, they should add up to zero. Is this 
> a consequence of ferret operating in single precision ? This makes
> me not trust my ferret calculations any more..
> thanks for any hints! Alexa