NOTICE: This version of the NSF Unidata web site (archive.unidata.ucar.edu) is no longer being updated.
Current content can be found at unidata.ucar.edu.
To learn about what's going on, see About the Archive Site.
NOTICE: This version of the NSF Unidata web site (archive.unidata.ucar.edu) is no longer being updated.
Current content can be found at unidata.ucar.edu.
To learn about what's going on, see About the Archive Site.
Ethan,
Thanks for the pointers. I'm now creating CDM compliant station observation
files using the CF specification.
I do have a fix for supporting attributes with primitive arrays (char[]) and
strings in compound types. Is there a public SVN server with the NetCDF-java
source? It would be nice to be able to work against the latest source and
provide patches against SVN.
I did try using the CF conventions agains NetCDF-4 w/ complex types and it's
almost there. Being able to do this would-be-nice but not required. I guess
the advantage would be storing all values for a particular station observation
contiguously.
Here's an example of the CDL I tried, with the patch referenced above it loads
in ToolsUI with _CoordinateAxisType attributes correctly assigned. It will
fail when opened as a FeatureType as there are some issues determining
relationships (i.e. finding observations.time dimension across structure
boundary, need to query dimension of observations structure, not time field).
## START
netcdf gsod {
types:
compound observation_type {
int time ;
float tempMean ;
float tempMin ;
float tempMax ;
}; // observation_type
compound observation_att {
string time ;
string tempMean ;
string tempMin ;
string tempMax ;
}; // observation_att
dimensions:
station = 1 ;
observation = UNLIMITED ; // (1 currently)
variables:
int station_id(station) ;
station_id:standard_name = "station_id";
float latitude(station) ;
latitude:units = "degrees_north" ;
float longitude(station) ;
longitude:units = "degrees_east" ;
float elevation(station) ;
elevation:units = "feet" ;
elevation:positive = "up" ;
observation_type observations(station, observation) ;
observations:coordinates = "observations.time longitude latitude
elevation" ;
observation_att observations:units = {"days since 1929-1-1
0:0:0", "degF", "degF", "degF"} ;
// global attributes:
:Conventions = "CF-1.5" ;
:CF\:featureType = "stationTimeSeries" ;
}
## END
Tom Kunicki
Center for Integrated Data Analytics
U.S. Geological Survey
8505 Research Way
Middleton, WI 53562
On Feb 11, 2010, at 5:57 PM, Ethan Davis wrote:
> Hi Tom,
>
> I'm going to leave the netCDF-4 compound type specific questions for
> John Caron who wrote that part of netCDF-Java (he should be back in the
> office on Tuesday).
>
> As for a CDM compliant station file, are you following the CF proposed
> specification for station data:
>
>> https://cf-pcmdi.llnl.gov/trac/wiki/PointObservationConventions
>
> Ah, just noticed your "cdm_datatype" global attribute, so it looks like
> you are following the older (now deprecated) Unidata Obs Dataset Convention:
>
>> http://www.unidata.ucar.edu/software/netcdf-java/formats/UnidataObsConvention.html
>
> The netCDF-Java library should understand station datasets written to
> either convention. However, both conventions are written to the netCDF
> classic data model rather than the enhanced (netCDF-4) data model. So,
> though it may recognize it as a station dataset, I'm not sure (even once
> the bug you are seeing is fixed) whether it will really know how to deal
> with the data as station data.
>
> Have you considered using the netCDF-4 classic model format? It has many
> of the advantages of netCDF-4 (e.g., compression and chunking) without
> breaking backward compatibility with programs written to the netCDF-3
> API (though they do need to be re-linked to the netCDF-4 library). Also,
> at this point many netCDF-4 aware programs don't have general support
> for parts of the enhanced data model (e.g., user defined types).
> Sticking to the classic model avoids this problem as well. It also has
> the advantage of working with conventions that are written to the
> classic model (all current conventions I'm aware of).
>
> There is more information on this in the netCDF FAQ section on the
> various formats, data models, and version:
>
>> http://www.unidata.ucar.edu/software/netcdf/docs/faq.html#formats-and-versions
>
> There is also more detail on the advantages of the netCDF-4 classic
> model format here:
>
>> http://www.unidata.ucar.edu/software/netcdf/papers/nc4_conventions.html#Use_of_the_NetCDF_Classic_Mode
>
> Hope that helps,
>
> Ethan
>
> On 2/8/2010 11:33 AM, Tom Kunicki wrote:
>>
>> Hi,
>>
>> I'm having a problem parsing a NetCDF4 file in NetCDF-java.
>>
>> I've using NetCDF 4.1 (C library with Java JNI bindings) in an attempt to
>> produce a CDM compliant Station file in NetCDF4 format. I want to put each
>> stations observation in a compound field. The file generates and parses
>> using the ncgen and ncdump but I can't open it in NetCDF-Java 4.1. I looks
>> like there's trouble reading char[] or string fields in the compound
>> attribute type. The following stack trace is produced on read:
>>
>> java.lang.NullPointerException
>> at ucar.ma2.ArrayStructure.extractMemberArray(ArrayStructure.java:407)
>> at ucar.nc2.iosp.hdf5.H5header.makeAttributes(H5header.java:587)
>> at ucar.nc2.iosp.hdf5.H5header.makeVariable(H5header.java:1034)
>> at ucar.nc2.iosp.hdf5.H5header.makeNetcdfGroup(H5header.java:388)
>> at ucar.nc2.iosp.hdf5.H5header.read(H5header.java:182)
>> at ucar.nc2.iosp.hdf5.H5iosp.open(H5iosp.java:108)
>> at ucar.nc2.NetcdfFile.<init>(NetcdfFile.java:1325)
>> at ucar.nc2.NetcdfFile.open(NetcdfFile.java:761)
>> at ucar.nc2.NetcdfFile.open(NetcdfFile.java:395)
>> at ucar.nc2.NetcdfFile.open(NetcdfFile.java:365)
>> at ucar.nc2.NetcdfFile.open(NetcdfFile.java:352)
>> at ucar.nc2.NetcdfFile.open(NetcdfFile.java:340)
>> at edu.unidata.ucar.jni.NetCDFTest.main(NetCDFTest.java:147)
>>
>> DIgging through the source it looks like
>> ucar.nc2.iosp.hdf5.H5header.getNCtype(int hdfType, int size) is returning
>> null for a char[] compound field type when trying to map the attributes to
>> the compound fields.
>>
>> With this be supported in the future? Is there another way to assign
>> attributes to compound type fields?
>>
>> Here's the CDL I am trying to use (the observation dim will eventually be
>> UNLIMITED):
>>
>> netcdf gsod {
>> types:
>> compound observation_type {
>> int time ;
>> float tempMean ;
>> float tempMin ;
>> float tempMax ;
>> }; // observation_type
>> compound observation_atts {
>> char time(32) ;
>> char tempMean(32) ;
>> char tempMin(32) ;
>> char tempMax(32) ;
>> }; // observation_atts
>> dimensions:
>> station = 2 ;
>> observation = 3 ;
>> variables:
>> int station_id(station) ;
>> float station_description(station) ;
>> float latitude(station) ;
>> latitude:units = "degrees_north" ;
>> float longitude(station) ;
>> longitude:units = "degrees_east" ;
>> float altitude(station) ;
>> altitude:units = "feet" ;
>> altitude:positive = "up" ;
>> observation_type station_observations(station, observation) ;
>> observation_atts station_observations:units = {{"days since
>> 1930-01-01 00 UTC"}, {"degF"}, {"degF"}, {"degF"}} ;
>>
>> // global attributes:
>> :cdm_datatype = "Stations" ;
>> :stationDimension = "station" ;
>> :observationDimension = "observation" ;
>> :latitude_coordinate = "latitude" ;
>> :longitude_coordinate = "longitude" ;
>> :zaxis_coordinate = "altitude" ;
>> :time_coordinate = "station_observations.time" ;
>> data:
>>
>> station_id = _, _ ;
>>
>> station_description = _, _ ;
>>
>> latitude = _, _ ;
>>
>> longitude = _, _ ;
>>
>> altitude = _, _ ;
>>
>> station_observations =
>> {0, 0, 0, 0}, {0, 0, 0, 0}, {0, 0, 0, 0},
>> {0, 0, 0, 0}, {0, 0, 0, 0}, {0, 0, 0, 0} ;
>> }
>>
>>
>> Thanks,
>>
>> Tom Kunicki
>> Center for Integrated Data Analytics
>> U.S. Geological Survey
>> 8505 Research Way
>> Middleton, WI 53562
>
> _______________________________________________
> netcdf-java mailing list
> netcdf-java@xxxxxxxxxxxxxxxx
> For list information or to unsubscribe, visit:
> http://www.unidata.ucar.edu/mailing_lists/
netcdf-java archives: