[netcdfgroup] Strange and subtle error on cross-compiled Windows x64 netcdf library

• To: NetCDF Group List <netcdfgroup@xxxxxxxxxxxxxxxx>
• Subject: [netcdfgroup] Strange and subtle error on cross-compiled Windows x64 netcdf library
• From: "David W. Pierce" <dpierce@xxxxxxxx>
• Date: Wed, 7 Aug 2013 10:54:03 -0700

Hello all,

I maintain the R interface to netcdf (the ncdf4 package), and have had many
requests that it support 64-bit Windows. A couple months ago I
cross-compiled the netcdf library on Linux with Windows-64 as the target.
It was a pretty straightforward process and seemed to work fine.
(Instructions can be found here:
http://cirrus.ucsd.edu/~pierce/ncdf/how_to_build_on_windows.html). The
fundamental reason for building it this way is because R is not built with
cygwin or visual studio, and so does not include those libraries. I built
with netcdf 4.2.1.1, hdf 1.8.11, and zlib 1.2.8.

However, although everything seems to work fine, a user recently pointed
out a strange problem. I've verified using a simple C program (attached)
that this is a problem with the cross-compiled Win-64 netcdf4 library
itself, rather than anything to do with R. The problem does not occur on
Linux. The symptom is that, at least in some particular cases, using the
netcdf ID from one open file will actually return the data and information
from a different, simultaneously open file.

This doesn't always happen, and I'm not sure on when it does and when it
doesn't. The problem shows up when accessing two similarly-shaped files
that are format 4 and use both compression and chunking. On the other hand,
if one of those version 4 files is tested with a classic version 3 file,
the problem does not occur. I've put links to the relevant files below in
case anyone wants to try it.

Has anyone else seen this kind of error? Any suggestions for a fix?

Regards,

--Dave

-------

Files that trigger the problem can be downloaded from:

http://cirrus.ucsd.edu/~pierce/public/apcp.nc
http://cirrus.ucsd.edu/~pierce/public/dlwrf.nc

These files have different variables, but running the attached C program on
them shows the *same* variable on Windows-64.

A classic, netcdf-3 format file that does not trigger the error, for
comparison:

http://cirrus.ucsd.edu/~pierce/public/movie.nc


-- 
David W. Pierce
Division of Climate, Atmospheric Science, and Physical Oceanography
Scripps Institution of Oceanography, La Jolla, California, USA
(858) 534-8276 (voice)  /  (858) 534-8561 (fax)    dpierce@xxxxxxxx

#include <stdio.h>
#include <stdlib.h>

#include "netcdf.h"

/*======================================================================================================*/
void list_vars( int ncid )
{
        int     ierr, nvars, i;
        char    varname[MAX_NC_NAME];

        ierr = nc_inq_nvars( ncid, &nvars );
        if( ierr != 0 ) {
                fprintf( stderr, "Error trying to get number of vars: %s\n", 
nc_strerror(ierr) );
                exit( -1 );
                }

        printf( "   File has %d vars\n", nvars );

        for( i=0; i<nvars; i++ ) {
                ierr = nc_inq_varname( ncid, i, varname );
                if( ierr != 0 ) {
                        fprintf( stderr, "Error trying to get var name: %s\n", 
nc_strerror(ierr) );
                        exit( -1 );
                        }
                printf( "    %d: %s\n", i, varname );
                }
}

/*======================================================================================================*/
main( int argc, char *argv[] ) 
{
        int     ncid1, ncid2, ierr;

        if( argc != 3 ) {
                fprintf( stderr, "Error, takes 2 args: names of the 2 files to 
test\n" );
                exit(-1);
                }

        printf( "Opening %s...\n", argv[1] );
        ierr = nc_open( argv[1], 0, &ncid1 );
        if( ierr != 0 ) {
                fprintf( stderr, "Error trying to open file %s: %s\n", argv[1], 
nc_strerror(ierr) );
                exit(-1);
                }
        
        printf( "Opening %s...\n", argv[2] );
        ierr = nc_open( argv[2], 0, &ncid2 );
        if( ierr != 0 ) {
                fprintf( stderr, "Error trying to open file %s: %s\n", argv[2], 
nc_strerror(ierr) );
                exit(-1);
                }

        printf( "Vars in file %s:\n", argv[1] );
        list_vars( ncid1 );
        
        printf( "Vars in file %s:\n", argv[2] );
        list_vars( ncid2 );
}