Re: [netcdfgroup] Bug in nc4file.c

• To: <netcdfgroup@xxxxxxxxxxxxxxxx>
• Subject: Re: [netcdfgroup] Bug in nc4file.c
• From: "Philippe Poilbarbe" <Philippe.Poilbarbe@xxxxxx>
• Date: Thu, 30 Sep 2010 16:19:37 +0200

 Hello,

Here is a patch which corrects the problem.
May be it is not the better way but it works (except for a program which
opens more than 65536 files à the same time: infinite loop in the
function added).

Expecting this will help,
regards,
Philippe.

Le 30/09/2010 14:56, Philippe Poilbarbe a écrit :
>  Hello,
>
> I think I have found a bug in nc4file.c of NetCDF version 4.1.1.
>
> It is the way the external file id is created (in nc_open_file and
> nc_create_file functions).
> The formula to obtain a new external file id (the one seen by programs
> calling the library NetCDF) is:
>     ++last_file_id << ID_SHIFT
> where last_file_id is initialized with 0.
>
> The problem comes when a program open/close many files, the external ids
> are not reused so last_file_id grows each time a nc_open or nc_create is
> called.
>
> When last_file_id reaches 65536, the external file id is 0 (65536
> ...
> As a solution, maybe the external file handle should be reused (as in
> necdf3) or, which may be simpler, the formula "++last_file_id <<
> ID_SHIFT" may be evaluated while the returned number is already in use.
>
> Philippe.
>

--- libsrc4/nc4file.c.orig      2010-09-30 15:09:23.000000000 +0200
+++ libsrc4/nc4file.c   2010-09-30 16:14:17.000000000 +0200
@@ -75,6 +75,27 @@
 
 int nc4_free_global_hdf_string_typeid();
 
+/* Find a valid external file id
+ */
+static int
+find_free_external_file_id (void)
+{
+    int check;
+    int limit; /* to avoid infinite loop */
+
+    /* NOTE: Possibility of infinite loop if 65536 files
+     *       are simultaneously opened */
+    while (1)
+    {
+       check   = (++last_file_id << ID_SHIFT);
+       if (! nc4_find_nc_file(check))
+       {
+           return check;
+       }
+    }
+}
+
+
 /* Set chunk cache size. Only affects files opened/created *after* it
  * is called.  */
 int
@@ -351,7 +372,7 @@
    /* Allocate the storage for this file info struct, and fill it with
       zeros. This add the file metadata to the front of the global
       nc_file list. */
-   if ((res = nc4_file_list_add((++last_file_id << ID_SHIFT))))
+   if ((res = nc4_file_list_add(find_free_external_file_id())))
       return res;
 
    /* Apply default create format. */
@@ -2295,7 +2316,7 @@
 
    /* Allocate the storage for this file info struct, and fill it with
       zeros. */
-   if ((res = nc4_file_list_add(++last_file_id << ID_SHIFT)))
+   if ((res = nc4_file_list_add(find_free_external_file_id())))
       return res;
 
    /* Depending on the type of file, open it. */

• References:
  • [netcdfgroup] Bug in nc4file.c
    • From: Philippe Poilbarbe