Re: [ferret_users] Compressed ferret output incompatibe with netcdf java apps

["Ansley C. Manke" <ansley.b.manke@xxxxxxxx>]

Hi,
I experimented and found that if the file is written in Ferret using the 
qualifier /SHUFFLE=1 and deflation, then it seems to work correctly, at 
least in Panoply.

In looking at some the netCDF documentation, I find this page that 
discusses the netcdf-java tools.
http://www.unidata.ucar.edu/software/thredds/current/netcdf-java/reference/manPages.html

It says that shuffle=true is the default for nccopy.  So that seems to 
be why running the file through nccopy causes it to work in the 
java-netCDF apps, and this may be what's meant by the "filter type =0" 
setting in the error message. Does this seem like a good solution to you 
Russ?

Ansley

On 1/28/2016 10:52 PM, Russ Fiedler wrote:
> Hi,
>
> This is possibly a problem with netcdf java rather than ferret but 
> anyway...
>
> If we make a variable and save it in compressed netcdf4 format it 
> appears that java based netcdf software han't handle it.
>
> e.g.
>
> yes? let v=i[i=1:20]+j[j=1:20]
> yes? save/file=cmp.nc/ncformat=netcdf4/def v    ! compressed (actually 
> it's larger but just an example)
> yes? save/file=uncmp.nc/ncformat=netcdf4 v      ! uncompressed
>
> Now loading up in Panopoly (and another of our in house apps which is 
> where we spotted the problem).When we try to operate on v
> we get a message about an an unknown filter type being zero for the 
> compressed version
>
> java.lang.RuntimeException: Unknown filter type=0
>     at 
> ucar.nc2.iosp.hdf5.H5tiledLayoutBB$DataChunk.getByteBuffer(H5tiledLayoutBB.java:198)
> etc
>
> The uncompressed version is fine since there are no filters being 
> applied.
>
> Now, here's where it gets weird. If nccopy is used to convert the two 
> files to compressed netcdf4 style the originally compressed file is
> still giving Panopoly grief but the newly compress file is handled 
> just fine!
>
>
> xwing-hf% nccopy -k 3 -d 1 cmp.nc cmp_ncopy.nc          ! Still no good
> xwing-hf% nccopy -k 3 -d 1 uncmp.nc uncmp_ncopy.nc  ! compressed file 
> is fine!!!
>
> Any clues? A known issue?
>
> Occurs for V6.95, V6.96 and earlier by the looks.
>
> Cheers,
> Russ