Warning on file extension ".cdf" on Windows

• To: "'netcdfgroup@xxxxxxxxxxxxxxxx'" <netcdfgroup@xxxxxxxxxxxxxxxx>
• Subject: Warning on file extension ".cdf" on Windows
• From: Mark Rivers <rivers@xxxxxxxxxxxxxxxxx>
• Date: Mon, 9 Apr 2001 11:15:42 -0500

Folks,

I just ran into a strange and troubling problem using netCDF on Windows (NT
4.0).  I created a large, 430MB netCDF file called "test.cdf".  As soon as I
point Windows Explorer to the directory containing that file I observe
"explorer.exe" consuming 100% of thw CPU and up to 900MB of RAM (on a
workstation with 1GB of RAM).  The workstation is essentially locked up.  It
is not possible to delete or rename the file for many minutes, even from
another workstation, because Windows claims there has been a "sharing
violation".

Here is what I believe is happening:  Windows thinks that files with a
".cdf" extension are "Channel files", which are used to display Web
channels.  As soon as Explorer opens a directory containing a ".cdf" file it
scans that file, even without the file itself being selected.  If the file
is huge it consumes the entire resources of the workstation for many
minutes.

If I create a file called "test.ncd" rather than "test.cdf" things work
fine.  If I rename "test.ncd" to "test.cdf" the problem immediately recurs.

Moral of the story:  don't use the extension ".cdf" for netCDF files on
Windows.

Is this a commonly known problem?  Is there a FAQ for netCDF where this is
documented?

Cheers,

Mark Rivers
CARS
Building 434A
Argonne National Laboratory
Argonne, IL 60439
Phone:  630-252-0422
FAX:    630-252-0443
e-mail: rivers@xxxxxxxxxxxxxxxxx
>From owner-netcdfgroup@xxxxxxxxxxxxxxxx 09 2001 Apr -0700 06:26:10 
Message-ID: <m3vgoecih9.fsf@xxxxxxxxxxxxxxxxxxx>
Date: 09 Apr 2001 06:26:10 -0700
From: Alexey Goldin <Alexey.Goldin@xxxxxxxxxxxx>
In-Reply-To: Russ Rew's message of "Mon, 09 Apr 2001 09:57:42 -0600"
To: Russ Rew <russ@xxxxxxxxxxxxxxxx>
Subject: Re: NC_SHORT alignment, unlimited dimension
Received: (from majordo@localhost)
        by unidata.ucar.edu (UCAR/Unidata) id f39Hrwh02244
        for netcdfgroup-out; Mon, 9 Apr 2001 11:53:58 -0600 (MDT)
Organization: UCAR/Unidata
Keywords: 200104091753.f39HrtL02239
X-Authentication-Warning: allegro.caltech.edu: goldin set sender to 
Alexey.Goldin@xxxxxxxxxxxx using -f
Cc: netcdfgroup@xxxxxxxxxxxxxxxx
References: <200104091557.f39FvhL27446@xxxxxxxxxxxxxxxx>
Lines: 52
User-Agent: Gnus/5.0807 (Gnus v5.8.7) Emacs/20.7
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Sender: owner-netcdfgroup@xxxxxxxxxxxxxxxx
Precedence: bulk
Reply-To: Alexey Goldin <Alexey.Goldin@xxxxxxxxxxxx>


I see. No, unfortunately second dimension is not a good idea, as we
want to make access to data requiring no brains at all. 

What we probably will do is letting it waste space, but then repack
data with fixed time dimension.

Another question --- does anyone know of plotting program similar to
xmgr or grace that can read NetCDF data, but also respects
scale_factor and add_offset attributes?


Thanks!


Russ Rew <russ@xxxxxxxxxxxxxxxx> writes:

> 
> There is a special case that might be of some help.  If there is only
> one record variable, the format drops the restriction that each record
> is 4-byte aligned, so there is no record padding and no wasted space,
> even for shorts or bytes.  For example, this file wastes no space:
> 
>   netcdf t1 {
>   dimensions:
>             time = UNLIMITED ;
>   variables:
>             short array1(time) ;
>   data:
> 
>   array1 = 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
>     1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
>     1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
>     1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
>     1, 1, 1, 1, 1, 1;
>   }
> 
> Brian Eaton's suggestion of adding an extra even dimension such as
> 
>   netcdf t2 {
>   dimensions:
>          d2 = 2 ;
>          time = UNLIMITED ; // (100 currently)
>   variables:
>          short array1(time,d2) ;
>          short array2(time,d2) ;
>  ...
>   }
> 
> is another way to avoid wasting space when storing shorts as record
> variables when you have multiple record variables, but is not
> necessary if there is only one record variable.