NOTICE: This version of the NSF Unidata web site (archive.unidata.ucar.edu) is no longer being updated.
Current content can be found at unidata.ucar.edu.
To learn about what's going on, see About the Archive Site.
NOTICE: This version of the NSF Unidata web site (archive.unidata.ucar.edu) is no longer being updated.
Current content can be found at unidata.ucar.edu.
To learn about what's going on, see About the Archive Site.
We do this for surface observations coming from a variety of sources with a variety of frequencies of update, although no one source at 1000 Hz...
gerry Timothy Hume wrote:
Hi Magnus,I once set up a system to store real-time meteorological data in NetCDF files. One of the difficulties is that you can never be sure of the temporal frequency of the data; some stations report hourly, others three hourly, and others report irregularly. To get around this issue, I used the unlimited dimension for time, and simply "grew" the dimension as new data became available. However, "growing" the unlimited dimension seemed to take a lot of computer time compared to slotting the data into an array where all the dimensions were fixed length. It may be that I wrote my software inefficiently, but I suspect the slowness of "growing" the unlimited dimension is partly due to the structure of a NetCDF file. If it is possible, it may be more efficient to predefine the length of your time dimension (making it large enough to hold all the data you receive), rather than using an unlimited dimension. The downside of this approach is that you are likely to be left with some "empty cells" in your data array.Also, the option of using a text file, as suggested by someone else, may be better. You can trivially append new data to a text file using the shell's ">>" feature.Tim Hume Bureau of Meteorology Research Centre Melbourne Australia -----Original Message----- From: owner-netcdfgroup@xxxxxxxxxxxxxxxx on behalf of Magnus Hagdorn Sent: Fri 2007-01-26 05:44 To: netcdfgroup@xxxxxxxxxxxxxxxx Subject: using netCDF for storing data from realtime source Hello world, we are currently considering to use the netCDF library to store data coming from our instruments. The data represents multiple variables together with a timestamp of the measurement. Some variables are updated more often than others. Sample rates are of the order of 100Hz to 1000Hz. I was thinking of storing the data as a table with 3 columns, timestamp, ID and value together with some appropriate header. Do you think this is a reasonable idea, are there other more suitable standard formats? Regards magnus ============================================================================== To unsubscribe netcdfgroup, visit: http://www.unidata.ucar.edu/mailing-list-delete-form.html ==============================================================================
-- Gerry Creager -- gerry.creager@xxxxxxxx Texas Mesonet -- AATLT, Texas A&M University Cell: 979.229.5301 Office: 979.458.4020 FAX: 979.862.3983 Office: 1700 Research Parkway Ste 160, TAMU, College Station, TX 77843 ============================================================================== To unsubscribe netcdfgroup, visit: http://www.unidata.ucar.edu/mailing-list-delete-form.html ==============================================================================
netcdfgroup archives: