[Thread Prev][Thread Next][Index]

Re: [ferret_users] accuracy of axis in ferret

Hi Ryo,
You're right about this being a matter of the evaluation of the variable in the command. Good catch. I agree that being able to set a default precision for immediate expressions would be useful; this would be similar to the "SET LIST/prec=" that we already have. I've added it to our list of tasks.

Ryo Furue wrote:
Hi Ferreter,

| Axis coordinates in Ferret are stored and used internally as double
| precision values, but any variables -including numbers used in
| expressions as you're defining axes- are single precision. So long
| axes in and of themselves are not a problem, but defining them
| accurately can be.

While there is no question about this statement, I think Martin's
particular problem involves the issue of the precision of back-quote

  yes? let del = 0.00833333333333333
  yes? define axis/x=-60:50:`del`/edges xax
   !-> define axis/x=-60:50:0.0083333/edges xax
  yes? say `x[gx=xax],return=iend`
  yes? define axis/x=-60:50:`del,p=7`/edges xax
   !-> define axis/x=-60:50:0.008333334/edges xax
  Replacing definition of axis XAX
  yes? say `x[gx=xax],return=iend`

In short, when you evaluate an expression using a pair of backqoutes,
you lose a bit of precision unless you specify a ",p=".

In this particular case, ",p=" gives an okay result.
But, this should probably be regarded as fortuitous.
In general, the "/npoints" solution would be better
because it's robust.  (Notice, for example,
`del,p=7` = 0.008333334, whereas "let del = 0.00833333333333333".)

In passing, it would be nice to be able to specify a default precision
for backquote evaluations.  Five digits for single precision is losing
precision in most cases.


[Thread Prev][Thread Next][Index]

Contact Us
Dept of Commerce / NOAA / OAR / PMEL / TMAP

Privacy Policy | Disclaimer | Accessibility Statement