NOTICE: This version of the NSF Unidata web site (archive.unidata.ucar.edu) is no longer being updated.
Current content can be found at unidata.ucar.edu.
To learn about what's going on, see About the Archive Site.
NOTICE: This version of the NSF Unidata web site (archive.unidata.ucar.edu) is no longer being updated.
Current content can be found at unidata.ucar.edu.
To learn about what's going on, see About the Archive Site.
On Thu, Mar 5, 2015 at 6:21 PM, Dumindu Jayasekera < d.jayasekera@xxxxxxxxxxxxxxxxx> wrote: > I need to convert .csv file (see the attached file) which has lat, long, > precip to netCDF? I have over two thousands of these .csv files in 15 > folders. > > Can somebody help to do the conversion? > There are any number of way to do this: NC Operators (NCO) _may_ have a way to process csv files -- I'm not sure but if it doesn't, you'll be well served to use a full featured scripting language, so you can easily control how all this works. If you know MATLAB or R, or ??, I'd use that, but if not, I highly recommend Python -- it's a great general purpose language with very broad support for scientific computing. I'd use the "netCDF4" python library (and numpy of cource ;-) ). Note that your CSV file has NO meta data -- not even variable names. So you'll want o add that, ideally following teh CF standards. I got carried away, so I've enclosed a sample script that reads your csv file and generates a netcdf file -- lots of details to be filled in there. If you don't have Python and all that on your computer -- I recommend the "Anaconda" Python distribution -- it should work out of the box for this. -Chris > I saw the following posting to use in Ferret I new to Ferret. Can somebody > guide me how to use the script in Ferret. Also, I could not found the > script mention in the website. Can I use a similar approach for the > attached .csv file. > > http://permalink.gmane.org/gmane.comp.lib.netcdf/1136 > > Any help is appreciated. > > Thanks, > > Dumindu. > > _______________________________________________ > netcdfgroup mailing list > netcdfgroup@xxxxxxxxxxxxxxxx > For list information or to unsubscribe, visit: > http://www.unidata.ucar.edu/mailing_lists/ > -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@xxxxxxxx
#!/usr/bin/env python
## LOTS of assumptions about how you want to store the data in netcdf!
## I'm assuming that each colum nis a separate array here
## and that it's a time series, but it could be anything
## a Nx# single array, for instance..
## take a look at: http://cfconventions.org/latest.html
## to see how you should store netcdf data
import numpy as np
import netCDF4
## docs for netcdf lib here:
http://netcdf4-python.googlecode.com/svn/trunk/docs/netCDF4-module.html
# load the data
# this load the file into a Nx3 array (three columns)
data = np.loadtxt('TEST_file.csv', delimiter=',')
# create a netcdf Data object
with netCDF4.Dataset('TEST_file.nc', mode="w", format='NETCDF4') as ds:
# some file-level meta-data attributes:
ds.Conventions = "CF-1.6" # if you comply with the convension -- which you
should!
ds.title = 'A succinct description of what is in the dataset.'
ds.institution = 'Specifies where the original data was produced.'
ds.source = 'The method of production of the original data. If it was
model-generated, source should name the model and its version, as specifically
as could be useful. If it is observational, source should characterize it
(e.g., " surface observation " or " radiosonde ").'
ds.history = 'Provides an audit trail for modifications to the original
data. Well-behaved generic netCDF filters will automatically append their name
and the parameters with which they were invoked to the global history attribute
of an input netCDF file. We recommend that each line begin with a timestamp
indicating the date and time of day that the program was executed.'
ds.references = 'Published or web-based references that describe the data
or methods used to produce it.'
ds.comment = 'whatever comment you may want to add'
# defining the dimensions of your arrays:
time = ds.createDimension('time', data.shape[0])
# variables for the columns -- you should use real names
for i in range(data.shape[1]):
var = ds.createVariable('var%i'%i,data.dtype, ('time',))
var[:] = data[:,i]
## adds some attibutes
var.units = 'the_proper_unit_string'
var.long_name = 'a nice long name that describes the data'
var.standard_name = 'a_CF_standard_name'
print ds
netcdfgroup archives: