NOTICE: This version of the NSF Unidata web site (archive.unidata.ucar.edu) is no longer being updated.
Current content can be found at unidata.ucar.edu.

To learn about what's going on, see About the Archive Site.

Re: [netcdfgroup] nccopy should use 1 as default-chunksize for unlimited dimension

Hi,

sorry that I have not been clearer, but my main complaint is that

>     NetCDF: Invalid argument
>     Location: file nccopy.c; line 637

isn't a useful feedback to a user trying to chunk some dimensions.


I think it is caused by nccopy using 0 (NC_UNLIMITED) as default for unlimited dimension which doesn't work for chunking. 1 or anything else would be much, much better than 0.

In my world of operational meteorology, file-formats change very slowly - a default of 1 is very similar to what netcdf-3 has been doing with unlimited dimensions and it won't surprise users as a default.


Best regards,

Heiko


On 2014-10-23 16:22, Ed Hartnett wrote:
This gives very poor performance when the number of timesteps in the
file is large.

On Thu, Oct 23, 2014 at 5:25 AM, Heiko Klein <Heiko.Klein@xxxxxx
<mailto:Heiko.Klein@xxxxxx>> wrote:

    Hi,

    when chunking files with unlimited dimension, the unlimited
    dimension must be given explicitly in nccopy, and will usually be
    set to one.

    In a file with time as unlimited dimension, and X and Y as
    dimensions, it is currently required to use

    $ nccopy -k 4 -c "time/1,X/100,Y/100" in.nc <http://in.nc> out.nc
    <http://out.nc>


    When running without time, it does not work:

    $ nccopy -k 4 -c "X/100,Y/100" in.nc <http://in.nc> out.nc
    <http://out.nc>
    NetCDF: Invalid argument
    Location: file nccopy.c; line 637


    Only for unlimited dimensions, one needs to give the dimension
    explicitly, for all other dimensions a useful default (full
    dim-size) is used. I think a useful default for unlimited dimensions
    is 1.


    Heiko



    --
    Dr. Heiko Klein                              Tel. + 47 22 96 32 58
    <tel:%2B%2047%2022%2096%2032%2058>
    Development Section / IT Department          Fax. + 47 22 69 63 55
    <tel:%2B%2047%2022%2069%2063%2055>
    Norwegian Meteorological Institute http://www.met.no
    P.O. Box 43 Blindern  0313 Oslo NORWAY

    _________________________________________________
    netcdfgroup mailing list
    netcdfgroup@xxxxxxxxxxxxxxxx <mailto:netcdfgroup@xxxxxxxxxxxxxxxx>
    For list information or to unsubscribe,  visit:
    http://www.unidata.ucar.edu/__mailing_lists/
    <http://www.unidata.ucar.edu/mailing_lists/>



--
Dr. Heiko Klein                              Tel. + 47 22 96 32 58
Development Section / IT Department          Fax. + 47 22 69 63 55
Norwegian Meteorological Institute           http://www.met.no
P.O. Box 43 Blindern  0313 Oslo NORWAY



  • 2014 messages navigation, sorted by:
    1. Thread
    2. Subject
    3. Author
    4. Date
    5. ↑ Table Of Contents
  • Search the netcdfgroup archives: