Re: [netcdfgroup] attributes not showing up when reading openDAP datasets

• To: Dennis Heimbigner <dmh@xxxxxxxxxxxxxxxx>
• Subject: Re: [netcdfgroup] attributes not showing up when reading openDAP datasets
• From: "Arthur M. Greene" <amg@xxxxxxxxxxxxxxxx>
• Date: Wed, 16 Sep 2009 14:49:21 -0400

Dennis, Jeff,

data:

 pr =
  ; // UNDEFINED

http://esgcet.llnl.gov/dap/ipcc4/20c3m/gfdl_cm2_1/pcmdi.ipcc4.gfdl_cm2_1.20c3m.run1.atm.yr.xml.ascii?pr

Warning::USE_CACHE: not currently supported
Warning::MAX_CACHE_SIZE: not currently supported
Warning::MAX_CACHED_OBJ: not currently supported
Warning::IGNORE_EXPIRES: not currently supported
Warning::CACHE_ROOT: not currently supported
Warning::DEFAULT_EXPIRES: not currently supported
Warning::ALWAYS_VALIDATE: not currently supported
Warning::Lost attribute: cdd
Warning::Lost attribute: time
Warning::Lost attribute: lat
Warning::Lost attribute: lon
Warning::Lost attribute: etr
Warning::Lost attribute: r10
Warning::Lost attribute: bnds
Warning::Lost attribute: r5d
Warning::Lost attribute: height
Warning::Lost attribute: fd
Warning::Lost attribute: hwdi
Warning::Lost attribute: sdii
Warning::Lost attribute: r95t
Warning::Lost attribute: tn90
Warning::Lost attribute: gsl

and in the dump file,

data:

 fd =
  ; // UNDEFINED

----------------------

In Python, where datobj = netCDF4.Dataset(URL),

In [86]: fdobj = datobj.variables['fd']

In [87]: fdobj.shape
Out[87]: (140, 90, 144)

In [88]: fd = fdobj[:]
---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)

/home/amg/work/nhmm/<ipython console> in <module>()

RuntimeError: NetCDF: Variable has no data in DAP request

---------------------

Thanks for the follow-up, hope this is helpful.

Best,

Arthur


Dennis Heimbigner wrote:

Jeff- Please do the following.
1. set the environment variable OCLOGFILE to the empty string
   (e.g. in bourne shell: OCLOGFILE="" ; export OCLOGFILE).
2. rerun the request

http://esgcet.llnl.gov/dap/ipcc4/20c3m/gfdl_cm2_0/pcmdi.ipcc4.gfdl_cm2_0.20c3m.run1.atm.mo.xml

This should give extra information.
I did that request thru a web browser

(http://esgcet.llnl.gov/dap/ipcc4/20c3m/gfdl_cm2_0/pcmdi.ipcc4.gfdl_cm2_0.20c3m.run1.atm.mo.xml.ascii?prw)

and got this message:

Error {
    code = 2;
    message = "Request for 87091200 bytes exceeds limit: 67108864";
}

And I will curious to see if you get that message also.

=Dennis Heimbigner



Jeff Whitaker wrote:

Dennis Heimbigner wrote:

I found and fixed the bug. I did not
implement the libdap algorithm quite
correctly. The fix should be available
in the netcdf snapshot tomorrow.

Thanks for pointing this out.

=Dennis Heimbigner

Dennis: Although the attributes are now appearing correctly, I'm still having trouble accessing the IPCC data. For example,

ncdump -v prw http://esgcet.llnl.gov/dap/ipcc4/20c3m/gfdl_cm2_0/pcmdi.ipcc4.gfdl_cm2_0.20c3m.run1.atm.mo.xml

gives

data:

prw =
 ; // UNDEFINED
}

with the Sept 16 snapshot.  The coordinate variables seem to work fine

data:

lat = -89, -87, -85, -83, -81, -79, -77, -75, -73, -71, -69, -67, -65, -63,

   -61, -59, -57, -55, -53, -51, -49, -47, -45, -43, -41, -39, -37, -35,

-33, -31, -29, -27, -25, -23, -21, -19, -17, -15, -13, -11, -9, -7, -5,

   -3, -1, 1, 3, 5, 7, 9, 11, 13, 15, 17, 19, 21, 23, 25, 27, 29, 31, 33,

35, 37, 39, 41, 43, 45, 47, 49, 51, 53, 55, 57, 59, 61, 63, 65, 67, 69,

   71, 73, 75, 77, 79, 81, 83, 85, 87, 89 ;
}

but all the data grids come back undefined. The same thing happens when trying to access data from the IRI data library

ncdump -v hgt http://iridl.ldeo.columbia.edu/expert/SOURCES/.NOAA/.NCEP/.EMC/.CFS/.MONTHLY_REALTIME/.PRESS/.hgt/dods

hgt =
 ; // UNDEFINED
}


Other dap URLs work fine though, for example

ncdump -v vvelprs http://nomad1.ncep.noaa.gov:9090/dods/reanalyses/reanalysis-2/6hr/pgb/pgb

-Jeff

Jeff Whitaker wrote:

Dennis:  If I open

http://esgcet.llnl.gov/dap/ipcc4/20c3m/gfdl_cm2_1/pcmdi.ipcc4.gfdl_cm2_1.20c3m.run1.atm.mo.xml

with netcdf-4.1-beta2, then get the time variable, it appears there are no attributes associated with it. Browsing the xml file

http://esgcet.llnl.gov/aggregates/ipcc4/20c3m/gfdl_cm2_1/pcmdi.ipcc4.gfdl_cm2_1.20c3m.run1.atm.mo.xml

it does appear these attributes are defined, but somehow the client is not getting them.

Is it a bug in the way there are serving the data, or a bug in the opendap client in netcdf-4.1?

-Jeff

--
*^*~*^*~*^*~*^*~*^*~*^*~*^*~*^*~*^*~*^*~*^*~*^*~*^*~*^*~*^*~*^*~*
Arthur M. Greene, Ph.D.
The International Research Institute for Climate and Society
The Earth Institute, Columbia University, Lamont Campus
Monell Building, 61 Route 9W, Palisades, NY  10964-8000 USA
amg at iri dot columbia dot edu | http://iri.columbia.edu
*^*~*^*~*^*~*^*~*^*~*^*~*^*~*^*~*^*~*^*~*^*~*^*~*^*~*^*~*^*~*^*~*

begin:vcard
fn:Arthur M. Greene
n:Greene;Arthur M.
email;internet:amg@xxxxxxxxxxxxxxxx
tel;work:845-680-4436
tel;fax:845-680-4865
x-mozilla-html:FALSE
version:2.1
end:vcard

• References:
  • [netcdfgroup] attributes not showing up when reading openDAP datasets
    • From: Jeff Whitaker
  • Re: [netcdfgroup] attributes not showing up when reading openDAP datasets
    • From: Dennis Heimbigner
  • Re: [netcdfgroup] attributes not showing up when reading openDAP datasets
    • From: Jeff Whitaker
  • Re: [netcdfgroup] attributes not showing up when reading openDAP datasets
    • From: Dennis Heimbigner