[Thread Prev][Thread Next][Index]

Re: [ferret_users] glitch in automatic shade color levels



Thanks Hella for the example and your ideas Russ.

The automatically-generated symbols, LEV_MIN, LEV_MAX etc. are set using some formatting that isn't capturing the values used. For generating the levels themselves, I think Russ's guess that there is a precision round-off issue is correct. We should be able to clear this up.

Ansley



On 7/3/2017 6:23 PM, Russ Fiedler wrote:

This looks like a rounding error in the calculation of the bounds. If you change the number of levels to 50 a suitable upper bound is found.

shade/lev=50 co_emac

works as expected. The maximum level appears to be calculated as 1.85E-7 rather than 1.90E-7. The symbol LEV_MAX should report this but I see it as "0" which is rather useless.


LEV_MIN = "0"
LEV_MAX = "0"
LEV_NUM = "29"
LEV_DEL = "0"

My guess is that there is a problem with the number of significant figures required and the precision available in the 1E-7 neighbourhood.

Russ


On 04/07/17 05:30, Riede, Hella wrote:
shade co_emac





[Thread Prev][Thread Next][Index]
Contact Us
Dept of Commerce / NOAA / OAR / PMEL / Ferret

Privacy Policy | Disclaimer | Accessibility Statement