Re: [netcdfgroup] Writing files > 4GB with 16-bit variales

• To: "Natarajan CS" <csnataraj@xxxxxxxxx>, <netcdfgroup@xxxxxxxxxxxxxxxx>
• Subject: Re: [netcdfgroup] Writing files > 4GB with 16-bit variales
• From: "Mark Rivers" <rivers@xxxxxxxxxxxxxxxxx>
• Date: Wed, 4 Nov 2009 13:03:59 -0600

What architecture are you running on? 

 

You have created a variable that is nearly 8GB in size.  Is this on a
64-bit machine? 

 

I am writing short arrays of those sizes OK on 64-bit Linux with IDL.

 

Mark

 

 

________________________________

From: netcdfgroup-bounces@xxxxxxxxxxxxxxxx
[mailto:netcdfgroup-bounces@xxxxxxxxxxxxxxxx] On Behalf Of Natarajan CS
Sent: Wednesday, November 04, 2009 10:09 AM
To: netcdfgroup@xxxxxxxxxxxxxxxx
Subject: [netcdfgroup] Writing files > 4GB with 16-bit variales

 

Hello all,
     I am trying to write a large file by combining multiple files. When
I try to write data (type short) greater than 4GB to a file, I get the
following error  "ncx.c:1810: ncx_put_size_t: Assertion `*ulp <=
4294967295U' failed."
I was wondering if this is an issue with my code or if it is a known and
resolved issue or even if there is a workaround? Appreciate any help!

Cheers.
Code : 
...........
short data[1490][1636][1720];
int NX=1720, NY=1636, NZ=1490;
for (k=1; k<=2; k++) {
                sprintf(name,"test%d.nc",k);
                nc_open(name, NC_NOWRITE, &ncid);

                if (nc_inq_ndims(ncid,&ndims) != NC_NOERR)
                        printf("Another error");
                size_t dim_siz[ndims];
                printf("The file has %d dimensions\n",ndims);
                for (dim =0; dim < ndims; dim++)  {
                        nc_inq_dim(ncid, dim, dim_name, &dim_siz[dim]);
                        printf("Name of dimension %d is %s\n",dim,
dim_name);
                        printf("Size of dimension %d is %d\n",dim,
dim_siz[dim]);
                }
                int varid;
                if (nc_inq_varid(ncid, "segmented", &varid) != NC_NOERR)
                        printf("Error 1");
                if (nc_get_var_short(ncid, varid, &data[745*(k-1)) !=
NC_NOERR)
                        printf("Error 2");
                nc_close(ncid);
        }
        printf("Done reading data\n");
        if(retval=nc_create(FILE_NAME, NC_CLOBBER | NC_64BIT_OFFSET,
&ncid2))
                ERR(retval);
        if ((retval = nc_def_dim(ncid, "x", NX, &x_dimid)))
                ERR(retval);
        if ((retval = nc_def_dim(ncid, "y", NY, &y_dimid)))
                ERR(retval);
        if ((retval = nc_def_dim(ncid, "z", NZ, &z_dimid)))
                ERR(retval);


        dimids[0] = x_dimid;
        dimids[1] = y_dimid;
        dimids[2] = z_dimid;

        if ((retval = nc_def_var(ncid, "segmented", NC_SHORT, 3, dimids,
&varid)))
                ERR(retval);

        if ((retval = nc_enddef(ncid)))
                ERR(retval);

        if ((retval = nc_put_var_short(ncid, varid, &data[0][0][0])))
                ERR(retval);

        if ((retval = nc_close(ncid)))
                ERR(retval);
   printf("*** SUCCESS writing example file simple_xy.nc!\n");


   return 0;
}

• Follow-Ups:
  • [netcdfgroup] Problems with netCDF files in IDL
    • From: Mark Rivers

• References:
  • [netcdfgroup] Writing files > 4GB with 16-bit variales
    • From: Natarajan CS