[Thread Prev][Thread Next][Index]

Re: [ferret_users] accuracy with axis in seconds

I am aware that when setting up time axes limits in Ferret using real numbers (usually as a result of some computation), one comes up against significant figure loss.  In response to Patrick Brockmann's post, I have thought about this a little.  What I present is not complete, but perhaps it will get others thinking.  

I wanted to figure out, for instance, how long an axis one can define and still resolve times down to 1 second.  Real*8 floating point numbers are usually reckoned to have 7 significant figures, so I used that value.  The attached table (in png and Excel form) shows the results.  An example of how to use this is as follows:  from row 16 one can see that with 7 sig. figs., one can define an axis of up to 0.32 years long in units of seconds and still resolve 1 second  time increments.  Similarly from row 18, one can define an axis 2.48 x 10**8 hours long and still resolve time increments down to 1 day.  This assumes that the T0 one defines is hard up against the beginning of the the axis.  If your T0 is 1-Jan-1900, your axis time increment is 1 second, and your data begin on 1-jan-2009, then the first data point is at about 63,355,824,000 seconds which would require 11 sig. figs. to resolve.  

I hope this helps!  

P.S. I am not a Ferret developer, but simply a user.

PNG image

Attachment: sig_figs.xls
Description: Binary data

Edward D. (Ned) Cokelet, Ph.D. Oceanographer
NOAA/PMEL                         off:  (206) 526-6820
7600 Sand Point Way NE     fax: (206) 526-6485
Seattle, WA 98115-6349

The contents of this message are mine personally and do not necessarily reflect any position of the Government or the National Oceanic and Atmospheric Administration.

[Thread Prev][Thread Next][Index]

Contact Us
Dept of Commerce / NOAA / OAR / PMEL / TMAP

Privacy Policy | Disclaimer | Accessibility Statement