Re: [ferret_users] SAVE precision of var from binary data?

To: ferret_users@xxxxxxxx

Subject: Re: [ferret_users] SAVE precision of var from binary data?

From: "Ansley C. Manke" <ansley.b.manke@xxxxxxxx>

Date: Wed, 5 Jul 2017 10:03:12 -0700

Arc-authentication-results: i=2; mx.google.com; dkim=pass header.i=@noaa-gov.20150623.gappssmtp.com header.b=ohZXYH8z; spf=pass (google.com: domain of ansley.b.manke@xxxxxxxx designates 2607:f8b0:400e:c00::232 as permitted sender) smtp.mailfrom=ansley.b.manke@xxxxxxxx

Arc-authentication-results: i=1; mx.google.com; dkim=pass header.i=@noaa-gov.20150623.gappssmtp.com header.b=ohZXYH8z; spf=pass (google.com: domain of ansley.b.manke@xxxxxxxx designates 2607:f8b0:400e:c00::232 as permitted sender) smtp.mailfrom=ansley.b.manke@xxxxxxxx

Arc-message-signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-archive:list-help:list-post:list-id:mailing-list:precedence :content-language:in-reply-to:mime-version:user-agent:date :message-id:from:references:to:subject:arc-authentication-results :arc-message-signature:dkim-signature:arc-authentication-results; bh=xy4u7G7L8gAKm80QVQXMFRVDqvn3TBUBUh7unTJZooc=; b=zpyu0b+hOm0gNJbjKyfjR7TzFQ4GUFosYeUZsvpWMA2XuHtSfP0M4ExmC8balakL1W xPxL5zY5TRs6EfjRFg4EeuqvLpVuRvNlZSk6kw7mLBeXstw81I0kfp6pV9DSgXKVIrkK u2VlAYkmsuSu90plbvn70pDlAbeVyOIvEKnCWHiQu5nddOZ9kDMjQwI6MZAo4HevR/dc TBgxcwERayhY5uvjya/Knvgpd4oZN1HUS/iY/TPlXJDRIndp8hSkioyAvfqnwkrHfVR5 qtuoOGLHN0GX9NSW3Ya8crCgY+qY6OwzrK2rd4s+R8GxwbgGrFF7Di0pDhpnB2JO23lq QNCg==

Arc-message-signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-language:in-reply-to:mime-version:user-agent:date :message-id:from:references:to:subject:dkim-signature :arc-authentication-results; bh=xy4u7G7L8gAKm80QVQXMFRVDqvn3TBUBUh7unTJZooc=; b=P+upoQUG7jsrVICiZOrhbzdg4Hg9IRDCKoBnm3t4JNl5gt/rGOr4wuGJIDO9WTjzSo 4RTO4jn0PN3pqkqIpT2aQApTlHberN/IKOnPl41LLJrnzAKRyaUC9iS4Qyn6XIeonXnP J6LVm10xHIM3GFHvdmEvAUmdccAHqBimudJ5BPKjUtRIdUnbapsGtnzpt7JsLkOth9dL 1a+P7y8DAkEGS/LtpENCJO1kSUMEnd0Qu2c8JJ4F3hN1DkIUAPGhkLBt48NuC2SgUcro g1mDPw4aZwogaduj4/hsKP3taNJsHbOPnd91PP9SrVpA9168jSzXXK4XcyqYDoCF7vzv RWRQ==

Arc-seal: i=2; a=rsa-sha256; t=1499274198; cv=pass; d=google.com; s=arc-20160816; b=RuYYDdP91PAOiF2LmTUh058z2F7w8IN/sNYR9wHXc+wWvpkFbYHiw9Jgp35eFRDi9v YoMjQM0D9F58v3BShiFw6ZXpc9HD2DKTukE6RTA/mcz4CUaMtaulBs33gxP0Vb+XILa0 7u7qZtrwCoznck9AYEzhAmzuHjCHKnAWONhv+aw9YCpWHRs/5Q0q7Ip5qEkXbjf6EMhN 9ng0A8N0CziTBdURTlUmZ2xBs+0cojsO8OJsLtvbPxg9ougaayPE3ewxQlS6L56YjJxb JenyP0A/I5BkqqMip6x5e+qYpi8WKWstJKTGUZSaqkEUyph6IT64MoCD7iTU5Ofttjxh Liig==

Arc-seal: i=1; a=rsa-sha256; t=1499274194; cv=none; d=google.com; s=arc-20160816; b=rASnIsYUJZX7OM9JkoiVRZ1e9cQ40HeKT06KELsxzIkMCh6HHibLC4fMhQMKQsPfWd ujjyLQJ5PCJT6BqWglBiWfebTYF47uDvPHcONS6Au0FmyOG/YvenA9t7J0bQSMG5YpaA Q1tnjRjzYsiNE8Z7Igl352VPW8/yDTJMQ3i3j71v+Ei6wzX1iagl/zMEEhG32h731G8K 7va23zR+3uQmTRmqOmgZ1w/nXxeQSBVmFxKaTRcvFoNS2706qyEQ++9RMnGz5DcTNm6b AjchJDaA4TG0hq495ISg0ER3aV2G+7wBmmsbLC6U17Cu2MFWnOlp0T65/Rp4U6Een3Mk rWFw==

Dkim-signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=noaa-gov.20150623.gappssmtp.com; s=20150623; h=subject:to:references:from:message-id:date:user-agent:mime-version :in-reply-to:content-language:x-original-sender :x-original-authentication-results:precedence:mailing-list:list-id :list-post:list-help:list-archive; bh=xy4u7G7L8gAKm80QVQXMFRVDqvn3TBUBUh7unTJZooc=; b=gIBaxdZFxWqlLe8RbJF2FjOqewGxUkOG+Tq4v3yW7u+wyhzOP0gmhN9hs697DL5NjH yx8xbSVnAtv8Oc6HNNEnYI4rIKoQ2S9Lfo9MnufIUG9XTnhT6wyrYsUm6P3NBTDzJltc AwrGxM32AEbx5nVIjQnDBM4094joZUDSrbBPxE8Jcw+kuImFLUJHv5xP3KFwLBIoiLFG Ad5bLOBRSn7vTQTWzfbSG5HS3+PgIQszJlY2rWevXrOG7/o9SU1tU1ap+MKJ5hAwL2pX xvuHIQT49evOGmU/mM/XOJKgUCWbzmhzoP0QxvL4Ht5eSPO22omKg+h2chldWnxoGagH YhDw==

In-reply-to: <CAEwQpaJVbAeDozChhF_poCvDrXt3kLvGfedVBDr8N7Psh_jv-g@mail.gmail.com>

List-archive: <https://groups.google.com/a/noaa.gov/group/ferret_users/>

List-help: <https://support.google.com/a/noaa.gov/bin/topic.py?topic=25838>, <mailto:ferret_users+help@noaa.gov>

List-id: <ferret_users.noaa.gov>

List-post: <https://groups.google.com/a/noaa.gov/group/ferret_users/post>, <mailto:ferret_users@noaa.gov>

Mailing-list: list ferret_users@xxxxxxxx; contact ferret_users+owners@xxxxxxxx

References: <CAEwQpaJVbAeDozChhF_poCvDrXt3kLvGfedVBDr8N7Psh_jv-g@mail.gmail.com>

Sender: owner-ferret_users@xxxxxxxx

User-agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:52.0) Gecko/20100101 Thunderbird/52.2.1

Hi Ryo,

The /OUTTYPE option is available for any netCDF write, regardless of the source of the data. In my tests it is working with Ferret v6.93; this has not changed recently.

        NOAA/PMEL TMAP
        FERRET v6.93
        Linux 2.6.32-504.el6.x86_64 64-bit - 11/05/14
         5-Jul-17 09:49

yes? define axis/x=1:10:1 x10
yes? define axis/y=1:5:1 y5
yes? define grid/x=x10/y=y5 g10x5
yes? file/var=myvar/grid=g10x5/format=stream stream_data_link.unf

yes? save/file=vsingle.nc/clobber myvar
LISTing to file vsingle.nc
yes? save/clobber/file=vdouble.nc/outtype=double myvar
LISTing to file vdouble.nc
*** NOTE: Converting data type of missing_value NC_FLOAT to match output type of variable NC_DOUBLE

and the resulting files do have data of those types. I've attached the data file; please test it and let me know what you see.

Note you can say

yes? set list/outtype=double

To apply that setting to all files written in the session. (Often people use SET LIST/OUTTYPE=float or even /OUTTYPE=int, to save disk space when their data does not need double precision output.) The default type for writing netCDF files is the input data type; does your data have more than 6-7 digits? Are the data correct within Ferret when it reads from your binary file?

Ansley

On 7/4/2017 8:37 PM, Ryo Furue wrote:

Hi Ferret users,

It seems that

SAVE/file=some.nc var

saves var in single precision, if var is from a stream, double-precision binary file. In contrast,

LIST/form=STREAM/file=some.bin var

saves it in double precision.

I'm using Ferret 6.93 on Linux.

I guess the OUTTYPE attribute isn't set for a variable from a binary file, or there is some such inconsistency. I hope it's fixed in the future.

Cheers,

Ryo

---------------------------

I have been able to reproduce this problem with

set data levitus_climatology

save/file=tmp.nc/outtype=double temp[k=1,l=1]

cancel data/all

SPAWN ncks -C -b tmp.bin -v TEMP tmp.nc -o tmp2.nc

define axis/x=0:359:1 xax

define axis/y=-89.5:89.5:1 yax

define grid/x=xax/y=yax mygrid

file/form=STREAM/var=temp/type=r8/grid=mygrid tmp.bin

save/file=tmp-new.nc temp

Attachment: stream_example.tar
Description: Unix tar archive