[Thread Prev][Thread Next][Index]

floating point precision e.g. spatial ave


I find that my spatial averages are not very accurate. In searching for
the problem, I've wanted to consider round off error. What is the
floating point precision of ferret calculations? Can it be varied to test
the sensitivity of the results? How accurately does it calculated the area
of a grid box? 

Does anyone have experience with checking the accuracy of the spatial
average transform that would be informative?

Thanks for your help!
Rob Scott

[Thread Prev][Thread Next][Index]

Dept of Commerce / NOAA / OAR / PMEL / TMAP

Contact Us | Privacy Policy | Disclaimer | Accessibility Statement