[netcdfgroup] possible bug in 3.6.2 with variables > 4GB

• To: netcdfgroup@xxxxxxxxxxxxxxxx
• Subject: [netcdfgroup] possible bug in 3.6.2 with variables > 4GB
• From: Mario Emmenlauer <mario@xxxxxxxxxxxxx>
• Date: Wed, 12 Sep 2007 16:26:25 +0200

Hi,

I hope this bug hasn't been filed before, I couldn't find it
on the list via a quick check.

This concerns writing large (>4GB) variables in netcdf with
large-file-support. In netcdf-3.6.2/libsrc/var.c around line
400 it reads:

if( varp->xsz <= X_UINT_MAX / product )
  /* if integer multiply will not overflow */
{
  varp->len = product * varp->xsz;
} else {
  /* OK for last var to be "too big", indicated by this special len */
  varp->len = X_UINT_MAX;
}
switch(varp->type) {
  case NC_BYTE :
  case NC_CHAR :
  case NC_SHORT :
    if( varp->len%4 != 0 )
    {
      varp->len += 4 - varp->len%4; /* round up */
      /* *dsp += 4 - *dsp%4; */
    }
  break;
  default:
  /* already aligned */
    break;
}

In the case of NC_BYTE, NC_CHAR and NC_SHORT, varp->len will end
up being X_UINT_MAX+1 instead of X_UINT_MAX. This in turn causes
an assertion when calling ncx_put_size_t later:
ncx.c:1812: ncx_put_size_t: Assertion `*ulp <= 4294967295U' failed.

I could not think about a useful fix despite qualifying the
rounding with the product-overflow (just moved the else-part
further down):

if( varp->xsz <= X_UINT_MAX / product )
  /* if integer multiply will not overflow */
{
  varp->len = product * varp->xsz;
  switch(varp->type) {
    case NC_BYTE :
    case NC_CHAR :
    case NC_SHORT :
      if( varp->len%4 != 0 )
      {
        varp->len += 4 - varp->len%4; /* round up */
        /* *dsp += 4 - *dsp%4; */
      }
    break;
    default:
    /* already aligned */
      break;
  }
} else {
  /* OK for last var to be "too big", indicated by this special len */
  varp->len = X_UINT_MAX;
}

I hope this bug report is useful. If you can send me a better
patch against netcdf-3.6.2 I would highly appreciate it.

Cheers,

  Mario Emmenlauer



-- 
Mario Emmenlauer                             Phone: +49-(0)761-203-8284
Institute for Computer Science               Fax:   +49-(0)761-203-8262
Chair of Pattern Recognition and Image Processing
Albert-Ludwigs-University
Georges-Koehler-Allee 052, room 01-022
79110 Freiburg             mailto:emmenlau * informatik.uni-freiburg.de
Germany          http://lmb.informatik.uni-freiburg.de/people/emmenlau/

• Follow-Ups:
  • [netcdfgroup] DAP-enabled ncdump
    • From: Mark Hess