Re: Neilley's weather program on linux reading *nix gdbm files

• To: ldm-users@xxxxxxxxxxxxxxxx
• Subject: Re: Neilley's weather program on linux reading *nix gdbm files
• From: "Bryan C. Hahn" <bryanh@xxxxxxxxxxxx>
• Date: Fri, 13 Feb 2004 19:39:10 -0600 (CST)

On Fri, 13 Feb 2004, Neil R. Smith wrote:
>
> Anyone experiencing difficulties with P.Neilley's weather program
> built on linux (say, Red Hat) reading gdbm metar files generated by
> ldm DBFILE action (pqsurf) on another platform/architecture?
>
> Should endian-ness be an issue? Peter's comments in the source code
> file README.linux (ver 4.10) suggests that it shouldn't be an issue.
>

Neil,

I do know that gdbm files are NOT platform independent (at least as of
gdbm version 1.8).  I have personal experience with gdbm files on HPUX 10
(big endian), FreeBSD, and Linux (little endian).  No gdbm file I've ever
created on one platform has been readable by software on another.

One specific problem is the storage of offsets in the file.  The gdbm
library uses system-defined types for the specification of these offsets.

Here's the definition of the header:

typedef struct {
        int   header_magic;  /* 0x13579ace to make sure the header is good. */
        int   block_size;    /* The  optimal i/o blocksize from stat. */
        off_t dir;           /* File address of hash directory table. */
        int   dir_size;      /* Size in bytes of the table.  */
        int   dir_bits;      /* The number of address bits used in the table.*/
        int   bucket_size;   /* Size in bytes of a hash bucket struct. */
        int   bucket_elems;  /* Number of elements in a hash bucket. */
        off_t next_block;    /* The next unallocated block address. */
        avail_block avail;   /* This must be last because of the psuedo
                                array in avail.  This avail grows to fill
                                the entire block. */
}  gdbm_file_header;

The off_t type is system dependent.

On FreeBSD (4.4 and later):

  sizeof ( off_t ) is 8

On Linux (Redhat 7.3):

  sizeof ( off_t ) is 4

So that's an immediate problem.

Endian-ness is an even bigger obstacle.

I'm surprised (and a bit disappointed) that the gdbm developers early on
didn't pay strict attention to developing a format which was
platform-independent.  This problem has caused us to abandon this
otherwise useful data storage mechanism in most cases.  We're using
Berkeley DB in most of the replaced cases.

  - Bryan

> -Neil
> --
> Neil R. Smith, Comp. Sys. Mngr.               neils@xxxxxxxx
> Dept. Atmospheric Sci., Texas A&M Univ.       979/845-6272 FAX:979/862-4466
>

• Follow-Ups:
  • Re: Neilley's weather program on linux reading *nix gdbm files
    • From: Neil Smith