NOTICE: This version of the NSF Unidata web site (archive.unidata.ucar.edu) is no longer being updated.
Current content can be found at unidata.ucar.edu.
To learn about what's going on, see About the Archive Site.
NOTICE: This version of the NSF Unidata web site (archive.unidata.ucar.edu) is no longer being updated.
Current content can be found at unidata.ucar.edu.
To learn about what's going on, see About the Archive Site.
NOTE: The decoders mailing list is no longer active. The list archives are made available for historical reasons.
On Fri, 22 Oct 2004, Dan Swank wrote:
Robb, One final brief question, are the cdl files time independant? (Can we make one and use it as a template for every other narr file)
dan, yes, once made they can be used as long as they are valid for the datasets. gribtocdl is just a utility to assist in making cdls. robb...
Or does a cdl file need to be created for every file we want to convert? -Dan Robb Kambic wrote: >On Thu, 21 Oct 2004, Dan Swank wrote: > > > >>Almost have it. >>The latest version indeed corrects the gribtocdl bug. >>I was able to create the narr-b .nc file going through both steps. >>Next i tried to convert the narr-a files, which have more then 400 grib >>records each. >>gribtocdl wrote this record: >> >> float 03tot_atm(record,y,x) ; >> 03tot_atm:long_name = "Total ozone at entire atmosphere >>considered as a single layer" ; >> 03tot_atm:GRIB_parameter_number = 237 ; >> 03tot_atm:GRIB_level_flag = 200 ; >> 03tot_atm:units = "Kg/m2" ; >> 03tot_atm:_FillValue = -9999.f ; >> 03tot_atm:navigation = "nav" ; >> >>Gribtonc doesn't seem to like the variable name starting with a number >>zero, and throws a syntax error. >> >> > >dan, > >in netcdf no var can start with a number, guess that parameter hasn't been >used before. the problem has been fixed and tested on the file you gave >as a test. sorry but you have to download the latest release: > >decoders-3.0.6.tar.Z > >robb... > >ps the size of netcdf output file grew to ~625mg from ~56 mg input file. >that;s about right > > > > >>This is also strange becuase the NARR doesn' tcontain any ozone fields. >> >>Suggestions? A sample narr-a file to try: >>http://nomads.ncdc.noaa.gov/tmp/narr-a_221_20010101_0000_000.grb >> >>-Dan >> >> >> >>Robb Kambic wrote: >> >> >> >>>On Thu, 21 Oct 2004, Dan Swank wrote: >>> >>> >>> >>> >>> >>>>Robb, >>>> >>>>No luck. I've recompiled the decoders package again and gribtocdl does >>>>the same thing. >>>>We have: >>>>netcdf-3.5.1 (built from source) >>>>udunits-1.11.7 ( Used binary installation since our system had >>>>trouble compiling this from source) >>>>decoders-2.3.8n ( built from source) >>>> >>>> >>>> >>>> >>>> >>>Dan, >>> >>>that's a really old distribution of the decoders package. Your problem >>>sounds like a bug that was fixed back at that time. the latest decoders >>>package is decoders-3.0.5.tar.Z hopefully the newest distribution will fix >>>your problem. >>> >>>robb... >>> >>> >>> >>> >>> >>> >>>>And the output from the compilation can be found here >>>>http://nomads.ncdc.noaa.gov/tmp/decoders-2.3.8n.logs.tar >>>> >>>>OS: >>>>Red Hat Linux release 7.3 (Valhalla) >>>> >>>>We also have a >>>>Red Hat Linux release 8.0 (Psyche) >>>>We can try this if you think it will really make much of a difference. >>>>Avbl but we dont' have root user on either of these so installing >>>>tpackages can be a bit tricky. >>>> >>>>If you need anymore information let me know. >>>> >>>>-Dan >>>> >>>> >>>> >>>>Robb Kambic wrote: >>>> >>>> >>>> >>>> >>>> >>>>>no need to include the others now... >>>>> >>>>> >>>>>On Wed, 20 Oct 2004, Dan Swank wrote: >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>>>Getting a little closer... >>>>>> >>>>>>Set the Env var and now udunits works as expected... however gribtocdl >>>>>>still gets stuck in an infinate loop and don't seem to work right. >>>>>>when i ran it on the narr-b GRIB it got to the following section >>>>>> >>>>>> >>>>>>// Navigation >>>>>>nav_model = "GRIB1" ; >>>>>>grid_type_code = 3 ; >>>>>>grid_type = "Lambert conformal projection" ; >>>>>>grid_name = " " ; >>>>>>grid_center = 7 ; >>>>>>grid_number = 221 ; >>>>>>i_dim = "lon" ; >>>>>>j_dim = "lat" ; >>>>>>Nx = 349 ; >>>>>>Ny = 277 ; >>>>>>La1 = 1.000000 ; >>>>>>Lo1 = -145.500000 ; >>>>>>Lov = -107.000000 ; >>>>>>Dx = 32463.000000 ; >>>>>>Dy = 32463.000000 ; >>>>>>// Pole in proj. plane = North ; >>>>>>centers = 1 ; >>>>>>Latin1 = 50.000000 ; >>>>>>Latin2 = 50.000000 ; >>>>>>Splat = 0.000000 ; >>>>>>SPlon = 0.000000 ; >>>>>>ResCompFlag = 0 ; >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>if you notice the cdl i made didn't have lon and lats. there must be >>>>>something wrong with your gribtocdl build. for a test, i build the >>>>>decoders package on a linux box fedora 9 and gribtocdl created the cdl >>>>>fine. remake the distribution, if you can try it on another box. whats >>>>>your os? >>>>> >>>>>robb... >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>>>lon = -145.50,-146.50,-147.50,-148.50,-149.50,-150.50,-151.50,-152.50, >>>>>> -153.50,-154.50,-155.50,-156.50,-157.50,-158.50,-159.50,-160.50, >>>>>> -161.50,-162.50,-163.50,-164.50,-165.50,-166.50,-167.50,-168.50, >>>>>> >>>>>>Then continues decrementing the longitue values and don't stop. I was >>>>>>forced to CNTL-C it and remove the lon section completely, putting the >>>>>>curly bracket at the end. Know why it might do this? >>>>>>After setting the env var, gribtonc successfully re-created narr.nc from >>>>>>the narr.cdl you supplied. However when I tried my CDL from above i get >>>>>>the following: >>>>>> >>>>>>./narr-b_221_20010101_0000_000.grb >>>>>>ncgen: narr-b2.cdl line 6: dimension length must be positive >>>>>>ncgen: narr-b2.cdl line 7: dimension length must be positive >>>>>>ncgen: narr-b2.cdl line 73: Nx not declared as a variable, fatal error >>>>>> >>>>>>Does gribtocdl require any ENV vars to be set? >>>>>> >>>>>>Appreciate the help. >>>>>>-Dan >>>>>> >>>>>> >>>>>> >>>>>> >>>>>>Robb Kambic wrote: >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>>>On Wed, 20 Oct 2004, Dan Swank wrote: >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>>>I copied the narr.cdl file from the FTP and tried: >>>>>>>> >>>>>>>>gribtonc -v -l ./log -e ./error ./narr.cdl >>>>>>>>narr-b_221_20010101_0000_000.nc < narr-b_221_20010101_0000_000.grb >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>here's the process >>>>>>> >>>>>>># creates cdl file narr.cdl >>>>>>>% gribtocdl -v narr-b_221_20010101_0000_000.grb > narr.cdl >>>>>>> >>>>>>># creates netcdf file narr.nc in verbose mode, logging to screen >>>>>>>% gribtonc -vl - narr.cdl narr.nc < narr-b_221_20010101_0000_000.grb >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>>>Result >>>>>>>>Segmentation fault >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>explained below, set UDUNITS_PATH >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>>>and >>>>>>>>Oct 20 19:31:05 gribtonc[32594]: Starting Up >>>>>>>>in the ./log file. >>>>>>>> >>>>>>>>Any idea whats going on? It is likely gribtonc (or more likely one of >>>>>>>>its dependancies) is not installed correctly on our system (RedHat 7.3) >>>>>>>>Also, what did you use to create this CDL file? The data in cdl seems >>>>>>>>like a translation of the information in the grib PDS, reworked into a >>>>>>>>format that ncgen can understand. >>>>>>>>Any way i can get anymore debug information regarding this? >>>>>>>> >>>>>>>>Only hunch is that it is not happy with the udunits package >>>>>>>> >>>>>>>>-> /usr/local/udunits-1.11.7/udunits-1.11.7/bin/udunits >>>>>>>>udunits(3): Couldn't open units database "/upc/udunits/etc/udunits.dat": >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>ahh, that's the problem. udunits can't find udunits.dat either place >>>>>>>udunits.dat in dir /upc/udunits/etc/ or set environment var >>>>>>> >>>>>>>% setenv UDUNITS_PATH /your/udunits/path/udunits.dat >>>>>>> >>>>>>>replace /your/udunits/path/ with appropriate path >>>>>>> >>>>>>>robb... >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>>>No such file or directory >>>>>>>>Segmentation fault >>>>>>>> >>>>>>>>But, while building the unidata decoders package it only seemed to want >>>>>>>>the .dat .a and .h files within these packages. >>>>>>>>Would this be the source of the problems? >>>>>>>> >>>>>>>>-Dan >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>>Robb Kambic wrote: >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>>>Thanks russ for the clarification. i was assuming you were familar with >>>>>>>>>the decoders process. if i can answer any more questions let me know. i'll >>>>>>>>>try to be more descriptive. >>>>>>>>> >>>>>>>>>robb... >>>>>>>>> >>>>>>>>> >>>>>>>>>On Tue, 19 Oct 2004, Dan Swank wrote: >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>>>Russ >>>>>>>>>> >>>>>>>>>>Actually, i was, at first, trying >>>>>>>>>>GRIB -( gribtocdl )-> CDL -( gribtonc )-> NetCDF >>>>>>>>>>Which i now understand is completely wrong, thanks for the help. >>>>>>>>>>As you have noticed, we are completely unfamiliar with these programs. >>>>>>>>>> >>>>>>>>>>Attempting it the correct way now, i'll let you know how it goes. >>>>>>>>>> >>>>>>>>>>-Dan >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>Russ Rew wrote: >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>>Robb, >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>>i downloaded the file from the last message, created a cdl, and decoded >>>>>>>>>>>>the grb file. there might be something wrong with Dan's decoders build or >>>>>>>>>>>>it could be a platform issue. this was done on a solaris box 5.9 The >>>>>>>>>>>>narr.cdl file is attached and the files narr.cdl, narr.grb, and narr.nc are >>>>>>>>>>>>in the Unidata's ftp dir at >>>>>>>>>>>> >>>>>>>>>>>>ftp unidata.ucar.edu >>>>>>>>>>>> >>>>>>>>>>>>% cd pub/contrib >>>>>>>>>>>>% mget narr* >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>Thanks Robb. The file sizes are: >>>>>>>>>>> >>>>>>>>>>>-rw-rw-r-- 1 rkambic ustaff 3486764 Oct 19 13:32 narr.nc >>>>>>>>>>>-rw-rw-r-- 1 rkambic ustaff 8560 Oct 19 13:32 narr.cdl >>>>>>>>>>>-rw-rw-r-- 1 rkambic ustaff 1398914 Oct 19 13:32 narr.grb >>>>>>>>>>> >>>>>>>>>>>so the netCDF file is about 2.5 times as big as the GRIB file. >>>>>>>>>>> >>>>>>>>>>>I'm guessing the source of the problem may come from using >>>>>>>>>>> >>>>>>>>>>>GRIB -> (via gribtocdl) -> CDL -> (via ncgen) -> netCDF >>>>>>>>>>> >>>>>>>>>>>(Using gribtocdl to generate a very large CDL file and then using ncgen >>>>>>>>>>>to convert that into a netCDF file.) >>>>>>>>>>> >>>>>>>>>>>I think Robb used the following tools instead: >>>>>>>>>>> >>>>>>>>>>>GRIB -> (via gribtocdl) -> CDL >>>>>>>>>>>GRIB and CDL -> (via gribtonc) -> netCDF >>>>>>>>>>> >>>>>>>>>>>(Using gribtocdl to generate a small CDL file describing structure of >>>>>>>>>>>the desired netCDF file and then using gribtonc to convert the GRIB >>>>>>>>>>>data into the netCDF file.) >>>>>>>>>>> >>>>>>>>>>>--Russ >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>-- >>>>>>>>>>Dan Swank <dan.swank@xxxxxxxx> >>>>>>>>>>NOMADS programmer >>>>>>>>>>STG, Incorporated - Government Contractor >>>>>>>>>>151 Patton Avenue, Room 514 >>>>>>>>>>Asheville, NC 28801 >>>>>>>>>>Phone: 828-271-4007 >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>============================================================================== >>>>>>>>>Robb Kambic Unidata Program Center >>>>>>>>>Software Engineer III Univ. Corp for Atmospheric Research >>>>>>>>>rkambic@xxxxxxxxxxxxxxxx WWW: http://www.unidata.ucar.edu/ >>>>>>>>>============================================================================== >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>-- >>>>>>>>Dan Swank <dan.swank@xxxxxxxx> >>>>>>>>NOMADS programmer >>>>>>>>STG, Incorporated - Government Contractor >>>>>>>>151 Patton Avenue, Room 514 >>>>>>>>Asheville, NC 28801 >>>>>>>>Phone: 828-271-4007 >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>============================================================================== >>>>>>>Robb Kambic Unidata Program Center >>>>>>>Software Engineer III Univ. Corp for Atmospheric Research >>>>>>>rkambic@xxxxxxxxxxxxxxxx WWW: http://www.unidata.ucar.edu/ >>>>>>>============================================================================== >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>-- >>>>>>Dan Swank <dan.swank@xxxxxxxx> >>>>>>NOMADS programmer >>>>>>STG, Incorporated - Government Contractor >>>>>>151 Patton Avenue, Room 514 >>>>>>Asheville, NC 28801 >>>>>>Phone: 828-271-4007 >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>============================================================================== >>>>>Robb Kambic Unidata Program Center >>>>>Software Engineer III Univ. Corp for Atmospheric Research >>>>>rkambic@xxxxxxxxxxxxxxxx WWW: http://www.unidata.ucar.edu/ >>>>>============================================================================== >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>-- >>>>Dan Swank <dan.swank@xxxxxxxx> >>>>NOMADS programmer >>>>STG, Incorporated - Government Contractor >>>>151 Patton Avenue, Room 514 >>>>Asheville, NC 28801 >>>>Phone: 828-271-4007 >>>> >>>> >>>> >>>> >>>> >>>> >>>============================================================================== >>>Robb Kambic Unidata Program Center >>>Software Engineer III Univ. Corp for Atmospheric Research >>>rkambic@xxxxxxxxxxxxxxxx WWW: http://www.unidata.ucar.edu/ >>>============================================================================== >>> >>> >>> >>> >>-- >>Dan Swank <dan.swank@xxxxxxxx> >>NOMADS programmer >>STG, Incorporated - Government Contractor >>151 Patton Avenue, Room 514 >>Asheville, NC 28801 >>Phone: 828-271-4007 >> >> >> >> > >============================================================================== >Robb Kambic Unidata Program Center >Software Engineer III Univ. Corp for Atmospheric Research >rkambic@xxxxxxxxxxxxxxxx WWW: http://www.unidata.ucar.edu/ >============================================================================== > > -- Dan Swank <dan.swank@xxxxxxxx> NOMADS programmer STG, Incorporated - Government Contractor 151 Patton Avenue, Room 514 Asheville, NC 28801 Phone: 828-271-4007
============================================================================== Robb Kambic Unidata Program Center Software Engineer III Univ. Corp for Atmospheric Research rkambic@xxxxxxxxxxxxxxxx WWW: http://www.unidata.ucar.edu/ ==============================================================================
decoders archives: