Hi Martin,
I asked the question on the netcdfgroup email list, and received
these answers:
On 9/6/2012 6:18 PM, Roy Mendelssohn wrote:
Hi Ansley:
I believe the answer is "yes", sort of. Remember, THREDDS unless
you are using the netcdf subset service or another experimental
service John C. is working on, does not deliver netcdf files, it
delivers data through OPeNDAP, which does not have this
compression as part of the description of the protocol. So the
data must be uncompressed and translated to be sent. however,
many OpeNDAP servers, and I believe this is true of TDS, if the
client says it can receive zipped files, will zip the OPeNDAP
stream and then that will be decompressed when it reaches the
client. One question we (ERD) has been experimenting with is the
read penalty with using netcdf4 files in conjunction with OPeNDAP,
and what chunking/compression settings have the best tradeoff
between performance and space
I am probably off on some of this, but I believe it is basically
correct.
-Roy
---
On 9/7/2012 10:08 AM, John Caron wrote:
Correct, thanks Roy.
Anything thats part of the file format is handled on the server.
Standard HTTP deflate (zip) compression is used to send data
across the wire, if client allows. the ncstream protocol is
experimenting with more complicated schemes, but is not ready for
general use.
Figuring out optimal chunking in netcdf4 is now the right game to
play. It depends on the subset request, so you want to optimize
your common requests.
the netcdf group would welcome user experience reports and
timings. posting to this list lets us all mull over the issues.
thanks!
John
There's some general information that might also be of interest in
Unidata's documentation on DAP support:http://www.unidata.ucar.edu/software/netcdf/docs/dap_support.html
We link Ferret with NetCDF libraries that have netcdf-4 enabled.
-Ansley
On 9/6/2012 8:34 AM, Martin Schmidt
wrote:
Good morning,
I have a question concerning the data flow when data are stored on
a THREDDS server.
After opening an opendap connection to a THREDDS server
use "http://......."
i do some operations and write data to a local disk.
let aaa= ....
repeat/l=1:1000 save/append/file=outputfile.nc aaa
I know, data on the THREDDS server are in netcdf-4 format, using
compression. My question is, where
the decompression takes place:
- at the THREDDS server and the uncompressed data are delivered
over the web,
- at the local machine, the compressed data are delivered over the
web and are uncompressed at the local machine?
Greetings,
Martin Schmidt
|