[Thread Prev][Thread Next][Index]

Re: odd numbers larger than 2^24



Hi Hyoun-Woo,
This is because all data in Ferret (other than coordinate axes) is
stored as single-precision floating point variables. The /PREC=
qualifier on the LIST command sets the output precision, but this
does not affect the internal precision. Thevalue of any variable as
stored internally preserves at most only about 7 significant digits,
which is why you're seeing these numbers being rounded off.

You can demonstrate this for yourself in a simple Fortran program
(outside of Ferret)

program test
real r
real*8 d

50 format (2f12.1)

r = 16777216.
d = 16777216.

print 50, r, d
r = r + 1
d = d + 1

print 50, r, d
r = r + 1.5
d = d + 1.5
print 50, r, d

stop
end

The output shows the single precision real number being rounded,
and the double precision number keeping enough accuracy:
16777216.0 16777216.0
16777216.0 16777217.0
16777218.0 16777218.5

We do not have a way to force Ferret to run in double precision.

Ansley

Hyoun-Woo Kang wrote:

>Dear ferreters,
>
>There is an odd behavior on simple calculation.
>Here are my results.
>--------------------------
>NOAA/PMEL TMAP
>FERRET v5.70
>Linux(g77) 2.4.20 - 06/28/04
>2-Feb-05 15:21
>
>yes? list/p=9 2^24
>VARIABLE : 2^24
>16777216.0 -> Correct !
>yes? list/p=9 2^24-1
>VARIABLE : 2^24-1
>16777215.0 -> Correct !
>yes? list/p=9 2^24+1
>VARIABLE : 2^24+1
>16777216.0 -> Wrong !
>yes? list/p=9 2^24+9
>VARIABLE : 2^24+9
>16777224.0 -> Wrong !
>-------------------------------------------
>
>Some more tests show that odd numbers bigger than 2^24 is not
>represented correctly.
>It is always represented as an even number, sometimes smaller value by 1
>or sometimes bigger by 1.
>
>How about others?
>Is this a bug?
>
>Sincerely,
>
>Hyoun-Woo
>
>--
>Hyoun-Woo Kang
>
>Korea Ocean Research and Development Institute,
>P.O.Box 29, Ansan, 425-600, Korea
>Phone : 82-31-400-7801
>Fax   : 82-31-408-5823
>Email : hwkang@kordi.re.kr
>
>
>
>  
>


[Thread Prev][Thread Next][Index]

Dept of Commerce / NOAA / OAR / PMEL / TMAP

Contact Us | Privacy Policy | Disclaimer | Accessibility Statement