Re: [netcdfgroup] offsets are ignored when creating compound types

• To: Russ Rew <russ@xxxxxxxxxxxxxxxx>
• Subject: Re: [netcdfgroup] offsets are ignored when creating compound types
• From: Felipe Bertrand <felipe.bertrand@xxxxxxxxxxxxxxxx>
• Date: Wed, 18 Nov 2009 18:49:40 +0100

Hu Russ,

I have checked out the beta release (4.1-beta2) and it now works. The
library manages the offsets correctly, as you note in your reply. With this
fixed I believe I will be able to do what I needed to do.

Thanks a lot,

Felipe

On Wed, Nov 18, 2009 at 6:00 PM, Russ Rew <russ@xxxxxxxxxxxxxxxx> wrote:

> Hi Felipe,
>
> > I have found out that NetCDF-4 ignores the offsets that the users
> specifies
> > when creating compound types. Instead, I think it calculates the offsets
> > taking into account the architecture alignment rules.
> >
> > So it is not possible for example to create a compound offset without
> > defining a "struct" in your C program. My problem is that I need to write
> a
> > program that is able to create arbitrary compound objects.
>
> It *is* possible to create a compound offset without defining a struct
> in your C program.  For example, ncgen uses CDL input describing a
> compound type to generate a corresponding netCDF file at run time,
> without a corresponding struct.  Also the new nccopy utility uses only
> the netCDF C API to structurally copy netCDF files, so when it copies a
> netCDF-4 file with a compound type, it must create the necessary type in
> the target file without having a corresponding C struct.
>
> As an example, consider the following CDL file, fb.cdl:
>
>  netcdf fb {
>  types:
>    compound mystruct_t {
>      byte field1;
>      int field2;
>      float field3;
>    }; // mystruct_t
>  variables:
>    mystruct_t cvar;
>  data:
>    cvar = {123, 0, 45.67};
>  }
>
> If you use the ncgen in the current snapshot release (previously named
> ncgen4 in the 4.0.1 release), it creates the corresponding netCDF file,
> as ncdump verifies:
>
>  $ ncgen -k 3 -b fb.cdl
>  $ ncdump fb.nc
>  netcdf fb {
>  types:
>    compound mystruct_t {
>      byte field1 ;
>      int field2 ;
>      float field3 ;
>    }; // mystruct_t
>  variables:
>          mystruct_t cvar ;
>  data:
>
>   cvar = {123, 0, 45.67} ;
>  }
>
> Furthermore, you can use ncgen to generate a corresponding C program,
> compile and run it, and you will still get the same netCDF file.  The
> generated C program does use a C struct for simplicity.
>
>  $ ncgen -k 3 -lc fb.cdl > fb.c
>  $ cc `nc-config --cflags` fb.c -o fb `nc-config --libs`
>  $ ./fb
>  $ ncdump fb.nc
>  [ ... same result as above]
>
> > Another thing that is not possible is to create a compound type that is
> an
> > extract (a subset) of a larger struct. In the following simplified
> example,
> > I have a struct with 3 fields, but I want to write to disk only the 1st
> and
> > the 3rd fields:
> >
> > ////////////////////////////
> >
> > struct mystruct {
> >     char field1;
> >     int field2;
> >     float field3;
> > };
> > struct mystruct myvar;
> > myvar.field1=123;
> > myvar.field3=45.67;
> >
> > nc_def_compound(ncid, sizeof(struct mystruct), "MYSTRUCTEXTRACT",
> &typeid);
> > nc_insert_compound(ncid, typeid, "FIELD1", 0, NC_BYTE)==0);
> > nc_insert_compound(ncid, typeid, "FIELD3", NC_COMPOUND_OFFSET(struct
> > mystruct, field3), NC_FLOAT);
> >
> > nc_def_var(ncid, "CVAR", typeid, 0, NULL, &varid);
> > nc_put_var(ncid, varid, &myvar);
> >
> > ////////////////////////////
> >
> > The file that I create does not contain the right value for "FIELD3", and
> I
> > think the reason is that the library is ignoring the offset that I am
> > supplying.
>
> If you edit the generated C program, fb.c, and comment out the line
> inserting the "field2" member, it still works, and when run shows a
> compound type with only the first and third field and the correct value
> for the field3 member:
>
>  netcdf fb {
>  types:
>    compound mystruct_t {
>      byte field1 ;
>      float field3 ;
>    }; // mystruct_t
>  variables:
>          mystruct_t cvar ;
>  data:
>
>   cvar = {123, 45.67} ;
>  }
>
> It's possible that the different results you are seeing are from bugs in
> an earlier version or indicate a new platform-specific bug we haven't
> seen, but we can't reproduce the problem here.
>
> --Russ
>



-- 
Mission and Data Systems Division
Deimos Space, SLU
http://www.deimos-space.com
Ph. +34 91.806.3450 Ext. 124

• References:
  • [netcdfgroup] offsets are ignored when creating compound types
    • From: Felipe Bertrand
  • Re: [netcdfgroup] offsets are ignored when creating compound types
    • From: Russ Rew