Re: [netcdfgroup] netCDF C++ Library : writing in loop

• To: 'JongKwan Kim' <kwanu2000@xxxxxxxxx>
• Subject: Re: [netcdfgroup] netCDF C++ Library : writing in loop
• From: Sjur Kolberg <Sjur.A.Kolberg@xxxxxxxxx>
• Date: Wed, 31 Mar 2010 16:25:38 +0200

Hi JongKwan,

float** snow_we is an array of float* pointers. This is the problem, put() 
requires a float pointer.
The &[0][0] provides a float pointer, and the compiler thinks put() will be 
satisfied, but tricking the compiler does not make snow_we a contiguous array 
of floats.

The tcount pointers in snow_we occupy a contiguous range of cells, but these 
cells hold memory addresses, not your data.
Each address points to a (also contiguous) vector of floats, length npix, which 
contains one line of data values.
But these vectors are independently located in memory, so snow_we as a whole is 
not contiguous.

By specifying &[0][0], you give put() the address of the first cell in your 
fist data array, and NetCDF finds happily the first npix values located in 
snow_we[0][0..npix-1]. But then NetCDF just continues, looking for the next 
value in snow_we[0][npix]. This memory location doesn't even necessarily belong 
to your program. The value you want NetCDF to find is snow_we[1][0], but the 
address in snow_we[1] was never provided to NetCDF.

You really need to allocate the array you provide to put() in one single 
operation whether you use C++ new or C malloc. In other words; while NetCDF 
treats the NcVar *data as a multi-dimensional array, the transfer buffer 
snow_we is (required to be) a single-dimensional vector. Hence, you must know 
how this buffer is interleaved, so that the dimensions are not mixed up in the 
transfer.


BTW, since you have declared the NcVar *data as containing only two dimensions 
(xDim and yDim), each of your 20000 iterations will overwrite the previous data 
written to dataFile. Is this intended?

If you really meant to store all the data, you must add a dimension for icall, 
which must be the first dimension
Then, you can allocate a 20000*tcount*npix vector and do one call to 
data->put(snow_we,20000,tcount,npix), after set_cur(0,0,0)
OR you can allocate a tcount*npix vector and do 20000 calls to 
data->put(snow_we,1,tcount,npix), each after set_cur(icall,0,0)
OR you can allocate a npix vector and do 20000*tcount calls to 
data->put(snow_we,1,1,npix), each after set_cur(icall,count,0)
Since you have declared snow_we as an array of float, not an array of float*, 
put() doesn't need the &[0][0] specification.

Unless you are putting/getting the entire NcVar, you need to specify the 
offsets in each dimension. Thus only in the first case, you can get away with 
not using set_cur(), but I recommend you to always use this offset 
specification.

In the last case, you can actually keep your current float** snow_we data 
structure, be sure to pass snow_we[count] as the first parameter to put(). This 
is a float pointer, and snow_we[count] is a contiguous array of floats.

For time considerations, I'd recommend you allocate and de-allocate snow_we 
outside the icall-loop, since the size doesn't change.

Good luck!

Sjur  K :-)


________________________________
From: netcdfgroup-bounces@xxxxxxxxxxxxxxxx 
[mailto:netcdfgroup-bounces@xxxxxxxxxxxxxxxx] On Behalf Of JongKwan Kim
Sent: 31. mars 2010 00:09
To: netcdfgroup
Subject: [netcdfgroup] netCDF C++ Library : writing in loop

Hi all,

I have a trouble in my code below.
When I run this code, I have a segmentation fault in front of red line, even 
though the code is running until 4 times.
That is, when the icall = 0, 1, 2, 3, the code is running well, but icall = 4, 
I have a segmentation fault.

I guess the problem might be creating the netCDF files.

Do you guys have any ideas to solve this problem...
please let me know that.

Thanks a lot.



for (int icall=0; icall<20000; i++)
{// for
int iseed =1, tcount=14608, npix=108, error=2, count=0;
float **snow_we, **soil_uztwc;
snow_we = FloatMatrix2(tcount,npix);
soil_uztwc = FloatMatrix2(tcount,npix);

vector< float > we_val( npix );
vector< float > soil_val( npix );

while ( currentTime <= 14608)
{//while
count++;
error = retrieve_value( allPixels, "we", we_val );
error = retrieve_value( allPixels, "real_soil", soil_val );

for (int j = 0; j < npix; j++) {
   snow_we[count-1][j] = we_val[j];
   soil_uztwc[count-1][j] = soil_val[j];
}
}//while

char snowfname[50] = {0, };
sprintf(snowfname,"snow_scem_iseed%d_iter%d.nc<http://d.nc>",iseed,icall);
NcFile dataFile(snowfname, NcFile::Replace);
if (!dataFile.is_valid()) {
   cout << "Could not open the netcdf snow file ! \n " << endl;
}
NcDim* xDim = dataFile.add_dim("x",tcount);
NcDim* yDim = dataFile.add_dim("y",npix);
NcVar *data = dataFile.add_var("data",ncFloat,xDim,yDim);
data->put(&snow_we[0][0],tcount,npix);

char uztwcfname[50] = {0, };
sprintf(uztwcfname,"uztwc_scem_iseed%d_iter%d.nc<http://d.nc>",iseed,icall);
NcFile dataFile2(uztwcfname, NcFile::Replace);
if (!dataFile2.is_valid()) {
   cout << "Could not open the netcdf uztwc file ! \n " << endl;
}
NcDim* xDim2 = dataFile2.add_dim("x",tcount);
NcDim* yDim2 = dataFile2.add_dim("y",npix);
NcVar *data2 = dataFile2.add_var("data",ncFloat,xDim2,yDim2);
data2->put(&soil_uztwc[0][0],tcount,npix);

FreeFloatMatrix2 (snow_we,tcount);
FreeFloatMatrix2 (soil_uztwc,tcount);

}//for

• References:
  • [netcdfgroup] netCDF C++ Library : writing in loop
    • From: JongKwan Kim