[netcdfgroup] storing sparse matrices data in NetCDF

• To: netcdfgroup@xxxxxxxxxxxxxxxx
• Subject: [netcdfgroup] storing sparse matrices data in NetCDF
• From: Ken Mankoff <mankoff@xxxxxxxxx>
• Date: Mon, 18 Mar 2019 11:58:42 -0700

Hello,

I'm trying to release a data set to a community that likes NetCDF. I'm looking 
for info about if this might be an OK file format given the data structure, and 
if so, how to go about generating the files.

I have a 36-year daily time-series (~14,000 times). Presumably I'll break this 
into yearly files to make working with them easier. At each time, I have 
~20,000 records. Fixed in time is (x,y,lon,lat,elevation), and then a varying 
value. The lon,lat (and associated x,y) are spares points, not a grid.

Is this a data structure that can be reasonably stored in NetCDF, or should I 
stick to a CSV file, one per year, with 365 columns and 20,000+6(header) rows?

Thanks,

  -k.

• Follow-Ups:
  • Re: [netcdfgroup] storing sparse matrices data in NetCDF
    • From: Sourish Basu