[Thread Prev][Thread Next][Index]

Re: [ferret_users] missing value for binary data and single to double conversion



Hi,

If you're converting this data to netCDF, another workaround would be to write the data as single-precision.  Then the resulting data is not changed from the original binary form.

   save/outtype=float/file=mynetcdf.nc myvar

And then the missing-data flags and the flagged values within the variable should match. 


-Ansley


On 8/24/2016 9:46 AM, Ansley C. Manke wrote:

hi Ryo,

Yes, good discussion.  The issue of precision conversions is tricky.  Within netCDF files, I think most of this is handled consistently, but for binary data, or ASCII data, what you say is all true.

I like your ideas for fixing this.

Ansley



On 8/24/2016 7:20 AM, Ryo Furue wrote:
Hi Ferret users,

I found it sometimes difficult to set a missing value for a variable from a single-precision binary:

file/format=stream/type=r4/var=myvar/grid=... binaryfile.bin
set variable/bad=-9.99e20  myvar
save/file=tmp.nc myvar

The missing value for the original, single-recision variable is -9.99e20. But, ferret converts the variable to double precision.  The second line then assigns a double-precision missing value -9.99e20 to the variable. Because the conversion

  A) single-precision representation of -9.99e20
     --> double-precision

is different from the double-precision representation of -9.99e20, the missing value isn't correctly set.  Indeed, in the netCDF file produced above, the missing-value attribute is -9.99e20 (double precision) whereas the missing value of the converted variable is -9.9900000981736856E+020 (double precision).

I'm attaching a Fortran program that illustrates this problem.

A workaround is

   set variable/bad=-9.9900000981736856E+020  myvar

This BAD value is the result of conversion (A) above.

I can think of two potential solutions to this problem:

1) Implement a BAD qualifier for the FILE command:

   file/type=r4/var="var1,var2,..."/bad="-9.99e20,-1e+34,..." binaryfile.bin

where the BAD values honor the TYPE qualifier.

2) Fix SET VARIABLE/BAD so that the BAD value should honor the original type of the variable.

In passing, I discovered that ncdump doesn't display enough digits. The above missing value is displayed as

   -9.99000009817369e+20

by ndcump but this value is actually different from the correct value -9.9900000981736856E+020 . That is, two different double-precision values are displayed identically.  To find out the result of conversion (A), I used ncdump at first, and was puzzled why it didn't work. Only by running the attached Fortran program did I find the correct value.

Regards,

Ryo

--------
! The following program will print
!
! -9.9900000981736856E+020  -9.9900000000000000E+020 -9.9900000000000000E+020
! d1 == d2
!
! using gfortran 4.8  .
!
program tmp
  implicit NONE
  real(8):: d, d1, d2 ! double precision variables
  character(16):: str = "-9.99e20"

  d  = -9.99e20 ! single-precision constant
  d1 = -9.99d20 ! doulbe-precision constant
  read(str, *) d2 ! double precision read

  write(*,*) d, d1, d2
  if (d1 == d ) write(*,*) "d1 == d"
  if (d2 == d ) write(*,*) "d2 == d"
  if (d1 == d2) write(*,*) "d1 == d2"
end program tmp




[Thread Prev][Thread Next][Index]
Contact Us
Dept of Commerce / NOAA / OAR / PMEL / Ferret

Privacy Policy | Disclaimer | Accessibility Statement