NOTICE: This version of the NSF Unidata web site (archive.unidata.ucar.edu) is no longer being updated.
Current content can be found at unidata.ucar.edu.
To learn about what's going on, see About the Archive Site.
NOTICE: This version of the NSF Unidata web site (archive.unidata.ucar.edu) is no longer being updated.
Current content can be found at unidata.ucar.edu.
To learn about what's going on, see About the Archive Site.
You might check the ChunkSizes attribute with 'ncdump -hs'. The newer netcdf
sets larger default chunks than it used to. I had this issue with 1-d variables
that used an unlimited dimension. Even if the dimension only had a small
number, the default chunk made it much bigger. (Assuming the variable is not
compressed.)
-- Ted
__________________________________________________________
| Edward Mansell <ted.mansell@xxxxxxxx>
| National Severe Storms Laboratory
|--------------------------------------------------------------
| "The contents of this message are mine personally and
| do not reflect any position of the U.S. Government or NOAA."
|--------------------------------------------------------------
On Apr 5, 2016, at 1:44 PM, Val Schmidt <vschmidt@xxxxxxxxxxxx> wrote:
> Hello netcdf folks,
>
> I’m testing some python code for writing sets of timestamps and variable
> length binary blobs to a netcdf file and the resulting file size is
> perplexing to me.
>
> The following segment of python code creates a file with just two variables,
> “timestamp” and “data”, populates the first entry of the timestamp variable
> with a float and the corresponding first entry of the data variable with an
> array of 100 unsigned 8-bit integers. The total amount of data is 108 bytes.
>
> But the resulting file is over 73 MB in size. Does anyone know why this might
> be so large and what I might be doing to cause it?
>
> Thanks,
>
> Val
>
>
> from netCDF4 import Dataset
> import numpy
>
> f = Dataset('scratch/text3.nc','w')
>
> dim = f.createDimension('timestamp_dim',None)
> data_dim = f.createDimension('data_dim',None)
>
> data_t = f.createVLType('u1','variable_data_t’)
>
> timestamp = f.createVariable('timestamp','d','timestamp_dim')
> data = f.createVariable('data',data_t,'data_dim’)
>
> timestamp[0] = time.time()
> data[0] = uint8( numpy.ones(1,100))
>
> f.close()
>
> ------------------------------------------------------
> Val Schmidt
> CCOM/JHC
> University of New Hampshire
> Chase Ocean Engineering Lab
> 24 Colovos Road
> Durham, NH 03824
> e: vschmidt [AT] ccom.unh.edu
> m: 614.286.3726
>
>
> _______________________________________________
> netcdfgroup mailing list
> netcdfgroup@xxxxxxxxxxxxxxxx
> For list information or to unsubscribe, visit:
> http://www.unidata.ucar.edu/mailing_lists/
netcdfgroup archives: