NOTICE: This version of the NSF Unidata web site (archive.unidata.ucar.edu) is no longer being updated.
Current content can be found at unidata.ucar.edu.
To learn about what's going on, see About the Archive Site.
NOTICE: This version of the NSF Unidata web site (archive.unidata.ucar.edu) is no longer being updated.
Current content can be found at unidata.ucar.edu.
To learn about what's going on, see About the Archive Site.
On Mon, May 11, 2015 at 5:13 PM, Michael Powell <mwpowellhtx@xxxxxxxxx> wrote: > On Mon, May 11, 2015 at 3:28 PM, Dave Allured - NOAA Affiliate > <dave.allured@xxxxxxxx> wrote: >> Michael, >> >> On Sun, May 10, 2015 at 2:35 PM, Michael Powell <mwpowellhtx@xxxxxxxxx> >> wrote: >>> >>> It's been a few years since I've looked at netCDF at all, so this is a >>> refresher. Also, this has no doubt been covered a million times, but I >>> am confused about the nature of the data block. >> >> >> You are inquiring specifically about netcdf-3 format (classic and 64-bit >> offset), not netcdf-4 (HDF5), right? > > For now, yessir. Thanks for the follow up. > >> Unless you have some very good reason for attempting to access netcdf-3 >> format directly, please use the netcdf libraries (C, fortran, etc.) to read >> and write. Rough knowledge of the low-level format is helpful for >> understanding performance issues. > > It's good to know. Mostly, morbid curiosity, I enjoy working with file > formats like this, but I did take note that there are C/C++ and other > API readily available. I like to push my architectural skills over > areas like this. Excepting for what I believe is a compiler/platform difference in how readers/writers deal with single and double wide floating point values, I believe I have a purely C++ model/reader/writer functioning properly. Verified on a couple of example NC files for comparison purposes. >>> If I understand the header/data parts correctly (which, I may not), >>> non-record can be 'infinite'? >> >> >> Not infinite, rather about 8 exabytes. But that is probably what you meant. > > Practically infinite, at any rate. > >> This is shown in a good summary table in the older netcdf documentation: >> http://www.unidata.ucar.edu/software/netcdf/docs/netcdf/Large-File-Support.html >> >>> >>> Rather, the non-record, fixed-length >>> parts should occur prior to the record data? There are probably some nuances about record versus non-record metadata, data, and so on, and notwithstanding what I think is compiler/platform read/write differences, I believe I've got it at least presentable. I may post it on github and open it up to feedback. It's presently tested in VS2013 Update 4, and uses current includes, C++11 in places, to facilitate operation. >> Yes. I think the following overview is a little less confusing than in the >> BNF spec. Think of all netcdf-3 files as having five possible parts. All >> parts except the header are optional. The two pads are a minor detail, >> don't worry about them: >> >> * Header >> * Header pad >> * Fixed length variables >> * Data pad >> * Record data I'm not sure what we mean by so-called "header pad". Most fields that call for padding are pretty well self-contained: i.e. for all name/text oriented properties/values. Similar for "data pad"; pretty well, self-contained, self-describing, and/or inherent part of the model (i.e. using std::vectors). >>> How does one tell what is record versus non-record? >> >> >> Many applications don't bother to look. Basic read access works the same >> either way, except for performance issues with large files. >> >> If you care, the manual method is to look for a dimension labeled >> "unlimited" in ncdump -h output. Any variables using this dimension are >> record variables. All others are fixed length variables. >> >> There are inquiry functions which can do this under program control. >> >>> >>> I think this is as a function of the var nelems, dimid, etc, but it's >>> not especially clear from the BNF grammar, or associated verbiage. For >>> example, what is meant by 'interleaved'? As a cross section of the >>> specified variables? >> >> >> Only record variables are interleaved. The sub-arrays ("cross sections") >> for record subscript #0 of all record variables are combined together at the >> start of the variable length section. The combined block for subscript #0 >> is often referred to casually as one "record". The combined block for >> subscript #1 is the next record. And so on. >> >> Suppose you have precip(time, lat, lon) and time(time), and time(time). >> Then the first record is precip(0,*,*) and time(0) together, and so on. >> >>> Also taking into account spiked variable array dimensions? >> >> >> What do you mean by "spiked"? > > Jagged arrays: something like this (turned 90 degrees): > > ++ > ++++ > + > ++++++ > ++++++ > ++ > > And so on. > >>> I'm sure I'm barely scratching the surface here... Insights are welcome. >>> >>> Thank you... >>> >>> Regards, >>> >>> Michael >> >> >> --Dave Thanks again...
netcdfgroup archives: