[netcdfgroup] ncgen: include binary data?

• To: netcdfgroup@xxxxxxxxxxxxxxxx
• Subject: [netcdfgroup] ncgen: include binary data?
• From: Ryo Furue <ryofurue@xxxxxxxxx>
• Date: Tue, 6 Oct 2020 00:00:53 +0900

Hi netCDF group,

I was wondering if the developers would consider enhancing ncgen to allow
for binary data?

Currently, ncgen accepts only ASCII data:

    data:
     VAR = 0.5, 1.5, 2.5, 3.5, 4.5, 5.5, 6.5, . . .

I thought it would be nice if the format allowed for an external binary
file in such a way as

data:
  VAR: file("path/to/mydata.bin", stream, big_endian)


[I don't claim this is the best syntax.  This is just for illustration of
the idea.]

I'm trying to find a tool with which it's easy to convert a stream-binary
file into a netCDF format.  Then, it struck me: Why, ncgen must be the best
tool!

So for, Ferret comes close, but it doesn't allow for large binary files and
the Ferret language takes a bit getting used to.

I'm aware of tools that can convert a pair of a GrADS control file and a
stream binary into a netCDF file, but they can handle only single-precision
floats.

I know that you can write a C or Fortran program to do that, but using the
C and Fortran API is too much work (dim-IDs, var-IDs, put_att, enddef,
NF_FLOAT, etc. etc.), compared to the simplicity of the CDL format or the
GrADS control file format.

<netcdfgroup@xxxxxxxxxxxxxxxx>I think this is the very reason very few
scientists around me write programs to generate netCDF files.  Most of them
simply use GrADS with stream binary files because it's so easy to write
GrADS control files.

If you can use the CDL+ncgen combination to generate a netCDF file, I'd be
able to convince them to switch to the netCDF format.

Best regards,
Ryo

• Follow-Ups:
  • Re: [netcdfgroup] ncgen: include binary data?
    • From: Chris Barker