NOTICE: This version of the NSF Unidata web site (archive.unidata.ucar.edu) is no longer being updated.
Current content can be found at unidata.ucar.edu.
To learn about what's going on, see About the Archive Site.
NOTICE: This version of the NSF Unidata web site (archive.unidata.ucar.edu) is no longer being updated.
Current content can be found at unidata.ucar.edu.
To learn about what's going on, see About the Archive Site.
Hi Terry, I did some digging and found a related issue on the GDAL Trac site [1]. It seems like the problem is the "elevation:_ChunkSizes" attribute: in the original dataset it is "1, 946" and in the NCSS version it is "2631, 946". The ticket indicates that GDAL prefers the former. As it turns out, a recent change to THREDDS addresses this issue by attempting to maintain the old chunk sizes when writing new NetCDF-4 data. As a test, I fired up the most recent development version of THREDDS (4.5.1) [2], and ran your original dataset through NCSS. I've attached the result to this email (busselton-final-ncss4.5.1.nc4). Can you run your code on it to see if the issue persists? Cheers, Christian Ward-Garrison [1] https://trac.osgeo.org/gdal/ticket/5082 [2] http://www.unidata.ucar.edu/software/thredds/current/tds/TDS.html On Thu, Jul 24, 2014 at 9:54 PM, <Terry.Rankine@xxxxxxxx> wrote: > Hi Christian > > > > *I think *Thredds has destroyed my PROJCS data which doesn’t allow my > code to rewarp it into a different projection. > > > > busselton-final-meta.nc-thredds-ncss.nc4 -> subset service “request all” > > busselton-final-meta.nc -> original file from HTTP service. > > > > > http://vhirl-dev.csiro.au/thredds/catalog/dem/catalog.html?dataset=testDatasetScan/busselton-final-meta.nc > > > > > > > > ran110@vhirl-dev:~/testing$ gdalwarp > busselton-final-meta.nc-thredds-ncss.nc4 -t_srs "+proj=latlong > +datum=WGS84" -co "COMPRESS=DEFLATE" -r cubic output.wgs84.cubic.nc -wo > OPTIMIZE_SIZE=TRUE -of netCDF -dstnodata -9999 > > Creating output file that is 1152P x 2597L. > > Warning 5: NOTICE: Format set to NC4C because compression is set to > DEFLATE. > > Processing input file busselton-final-meta.nc-thredds-ncss.nc4. > > Using internal nodata values (e.g. -9999) for image > busselton-final-meta.nc-thredds-ncss.nc4. > > ERROR 1: nBlockYSize = 2631, only 1 supported when reading bottom-up > dataset > > ERROR 1: busselton-final-meta.nc-thredds-ncss.nc4, band 1: IReadBlock > failed at X offset 0, Y offset 0 > > ERROR 1: GetBlockRef failed at X block offset 0, Y block offset 0 > > > > ran110@vhirl-dev:~/testing$ gdalwarp busselton-final-meta.nc -t_srs > "+proj=latlong +datum=WGS84" -co "COMPRESS=DEFLATE" -r cubic > output.wgs84.cubic.nc -wo OPTIMIZE_SIZE=TRUE -of netCDF -dstnodata -9999 > > Creating output file that is 1152P x 2597L. > > Warning 5: NOTICE: Format set to NC4C because compression is set to > DEFLATE. > > Processing input file busselton-final-meta.nc. > > Using internal nodata values (e.g. -9999) for image > busselton-final-meta.nc. > > 0...10...20...30...40...50...60...70...80...90...100 - done. > > ran110@vhirl-dev:~/testing$ > > > > > > HEADER INFO > > > > ran110@vhirl-dev:~/testing$ gdalinfo > busselton-final-meta.nc-thredds-ncss.nc4 > > Driver: netCDF/Network Common Data Format > > Files: busselton-final-meta.nc-thredds-ncss.nc4 > > Size is 946, 2631 > > Coordinate System is: > > PROJCS["unnamed", > > GEOGCS["unknown", > > DATUM["unknown", > > SPHEROID["Spheroid",6378137,298.257223563]], > > PRIMEM["Greenwich",0], > > UNIT["degree",0.0174532925199433]], > > PROJECTION["Transverse_Mercator"], > > PARAMETER["latitude_of_origin",0], > > PARAMETER["central_meridian",117], > > PARAMETER["scale_factor",0.9996], > > PARAMETER["false_easting",500], > > PARAMETER["false_northing",10000], > > UNIT["kilometre",1000, > > AUTHORITY["EPSG","9036"]]] > > Origin = (189.659982684810018,6757.539391574399815) > > Pixel Size = (0.250000000000000,-0.250000000000000) > > > > > > VS > > > > ran110@vhirl-dev:~/testing$ gdalinfo busselton-final-meta.nc > > Driver: netCDF/Network Common Data Format > > Files: busselton-final-meta.nc > > Size is 946, 2631 > > Coordinate System is: > > PROJCS["UTM Zone 50, Southern Hemisphere", > > GEOGCS["WGS 84", > > DATUM["WGS_1984", > > SPHEROID["WGS 84",6378137,298.257223563, > > AUTHORITY["EPSG","7030"]], > > TOWGS84[0,0,0,0,0,0,0], > > AUTHORITY["EPSG","6326"]], > > PRIMEM["Greenwich",0, > > AUTHORITY["EPSG","8901"]], > > UNIT["degree",0.0174532925199433, > > AUTHORITY["EPSG","9108"]], > > AUTHORITY["EPSG","4326"]], > > PROJECTION["Transverse_Mercator"], > > PARAMETER["latitude_of_origin",0], > > PARAMETER["central_meridian",117], > > PARAMETER["scale_factor",0.9996], > > PARAMETER["false_easting",500000], > > PARAMETER["false_northing",10000000], > > UNIT["METERS",1]] > > Origin = (189659.982684810005594,6757539.391574399545789) > > Pixel Size = (250.000000000000000,-250.000000000000000) > > > > > > *From:* Christian Ward-Garrison [mailto:cwardgar@xxxxxxxx] > *Sent:* Friday, 25 July 2014 7:49 AM > *To:* Rankine, Terry (Mineral Resources, Kensington) > *Cc:* thredds@xxxxxxxxxxxxxxxx > *Subject:* Re: [thredds] NetCDF Subset service > > > > Hi Terry, > > You might expect that when you request all of the data in a dataset, NCSS > will simply return the original dataset (no subsetting required). That's > not how NCSS works; instead, it will always create a new dataset, and copy > over the requested data (which could potentially be everything) from the > original. > > In this scenario, you might notice that the original and new datasets are > *structurally* different (i.e. different attributes, scale factors, etc), > but they are *semantically* equal (i.e. they contain the same data, though > it might be represented differently). > > As far as I can tell, your 2 example files are semantically equal. Are you > struggling with structural equality issues? > > Cheers, > > Christian > > > > On Mon, Jul 21, 2014 at 12:46 AM, <Terry.Rankine@xxxxxxxx> wrote: > > Hi Guys > > > > Is there any other users currently using the netCDF subset service? I am > having quite a few issues with it and would like to check ‘it’s not just me’ > > > > This: > > http://vhirl-dev.csiro.au/thredds/fileServer/dem/busselton-final.nc > > > > is very very different to this: > > > http://vhirl-dev.csiro.au/thredds/ncss/grid/dem/busselton-final.nc?var=elevation&north=-29.2729&west=113.5915&east=116.2396&south=-35.2422&horizStride=1&accept=netcdf4 > > > > How are others handling THREDDS and CRS changes? > > > > Regards > > Terry > > > _______________________________________________ > thredds mailing list > thredds@xxxxxxxxxxxxxxxx > For list information or to unsubscribe, visit: > http://www.unidata.ucar.edu/mailing_lists/ > > >
Attachment:
busselton-final-ncss4.5.1.nc4
Description: Binary data
thredds archives: