NOTICE: This version of the NSF Unidata web site (archive.unidata.ucar.edu) is no longer being updated.
Current content can be found at unidata.ucar.edu.
To learn about what's going on, see About the Archive Site.
NOTICE: This version of the NSF Unidata web site (archive.unidata.ucar.edu) is no longer being updated.
Current content can be found at unidata.ucar.edu.
To learn about what's going on, see About the Archive Site.
John Caron wrote:
I think last time we discussed this, I wrote: "The problem is that a location doesnt specify the IOSP needed to handle it. We have to ask all the IOSPs we know about to find out who handles it. The IOSP needs to examine the contents of the dataset to decide if "its mine". Rather than having each IOSP open the file, we pass each the RandomAccessFile, for efficiency. The IOSP design is obviously oriented to using RandomAccessFile.
I figured that if an iosp is defined in the NcML, then there is no need to go door-to-door with an open file, so NetCDF could ignore the "location" and let the IOSP deal with it. However, that would mean different behavior for registered IOSPs and custom IOSPs.
"Now an NcML file can specify the IOSP, so potentially this might be a workaround this problem. The presumed use of NcML is to wrap an existing NetcdfFile. You want to reverse that and let your IOSP's NetcdfFile wrap the NcML, by giving it access to the NcML.
I'm taking my inspiration from things like the GRIB IOSP. My vision of NcML is that it exposes data virtually, in terms of the Common Data Model, enabling the NetCDF API to read it. That is much more powerful than simply enhancing an existing NetCDF file.
My dilemma has been one of metadata. Many of my data sources don't have sufficient information to define a NetcdfFile. I needed a way to provide metadata and NcML seemed like the ideal format. In the interest of not repeating myself, my NcML serves that dual role. Otherwise, I could put the redundant metadata in a different file that I would have to identify in the NcML. Then I'd have maintenance issues.
"I could see trying an experimental version that passes the NcML to the iosp specified in the NcML. I would guess that NcMLReader would not further wrap it, though maybe we can relax that later. Im not sure what issues we'd find, and but we could try it, though i wouldnt commit to supporting it until we have some experience.
Initially, I self-identified the NcML file location in the iospParams. That just felt wrong and meant that I had to edit the file if I wanted to relocate it. I then modified the NetCDF code to give me access to the NcML location from the NcMLReader$NcMLNetcdfFile that is handed to my IOSP's "open". After all, it is a "NcMLNetcdfFile". There were complications with aggregations, so now I provide access to the JDOM "netcdf" Element. It is quite harmless in itself. Worst case, I do recall that JDOM Lists are live so you might be giving the IOSP developer a way to shoot himself in the foot. But if they are modifying those, then they deserve it. :-)
"The question is whether an IOSP is the right level to implement your database access. Alternatively, one could subclass NetcdfFile, which is what other non-file implementations like opendap, thredds, and cdmremote do. The problem is that theres no mechanism to associate the location with a factory that creates those subclasses. I suppose we could create one, though. "So have you considered subclassing NetcdfFile instead of writing an IOSP ?
Again, I'm modeling my use of the IOSP after GRIB and the others. I want a data provider to have a simple way to server data. Defining the data structure in terms of the CDM with NcML seems ideal and very powerful.
Thanks, Doug
netcdf-java archives: