Re: [ferret_users] accuracy of axis in ferret

[Ryo Furue <furue@xxxxxxxxxx>]

Hi Ferreter,

| Axis coordinates in Ferret are stored and used internally as double
| precision values, but any variables -including numbers used in
| expressions as you're defining axes- are single precision. So long
| axes in and of themselves are not a problem, but defining them
| accurately can be.

While there is no question about this statement, I think Martin's
particular problem involves the issue of the precision of back-quote
evaluation:

  yes? let del = 0.00833333333333333
  yes? define axis/x=-60:50:`del`/edges xax
   !-> define axis/x=-60:50:0.0083333/edges xax
  yes? say `x[gx=xax],return=iend`
   !-> MESSAGE/CONTINUE 13201
  13201
  yes? define axis/x=-60:50:`del,p=7`/edges xax
   !-> define axis/x=-60:50:0.008333334/edges xax
  Replacing definition of axis XAX
  yes? say `x[gx=xax],return=iend`
   !-> MESSAGE/CONTINUE 13200
  13200

In short, when you evaluate an expression using a pair of backqoutes,
you lose a bit of precision unless you specify a ",p=".

In this particular case, ",p=" gives an okay result.
But, this should probably be regarded as fortuitous.
In general, the "/npoints" solution would be better
because it's robust.  (Notice, for example,
`del,p=7` = 0.008333334, whereas "let del = 0.00833333333333333".)

In passing, it would be nice to be able to specify a default precision
for backquote evaluations.  Five digits for single precision is losing
precision in most cases.

Regards,
Ryo