NOTICE: This version of the NSF Unidata web site (archive.unidata.ucar.edu) is no longer being updated.
Current content can be found at unidata.ucar.edu.

To learn about what's going on, see About the Archive Site.

Re: [netcdfgroup] Need help to convert .csv files to netCDF files

  • To: Chris Barker <chris.barker@xxxxxxxx>
  • Subject: Re: [netcdfgroup] Need help to convert .csv files to netCDF files
  • From: "Signell, Richard" <rsignell@xxxxxxxx>
  • Date: Fri, 6 Mar 2015 17:32:38 -0500
Folks,

If you like python, this seems like a perfect application for xray
(http://xray.readthedocs.org/en/stable/, which adopts a common data
model representation in python.

I found out about xray by reading this blog article:
https://ocefpaf.github.io/python4oceanographers/blog/2014/12/22/xray/

And I adapted it here to create a simple program that  reads CSV data
using Pandas, converts to an xray Dataset, adds metadata, and then
uses an xray method to write to NetCDF:

http://nbviewer.ipython.org/github/rsignell-usgs/notebook/blob/master/csv2netcdf.ipynb

As Chris said, if you use the anaconda python distribution, you can
just do "conda install -c shoyer xray" to install xray.

On Fri, Mar 6, 2015 at 2:29 PM, Chris Barker <chris.barker@xxxxxxxx> wrote:
> On Thu, Mar 5, 2015 at 6:21 PM, Dumindu Jayasekera
> <d.jayasekera@xxxxxxxxxxxxxxxxx> wrote:
>>
>> I need to convert .csv file (see the attached file) which has lat, long,
>> precip to netCDF? I have over two thousands of these .csv files in 15
>> folders.
>>
>> Can somebody help to do the conversion?
>
>
> There are any number of way to do this:
>
> NC Operators (NCO) _may_ have a way to process csv files -- I'm not sure but
> if it doesn't, you'll be well served to use a full featured scripting
> language, so you can easily control how all this works.
>
> If you know MATLAB or R, or ??, I'd use that, but if not, I highly recommend
> Python -- it's a great general purpose language with very broad support for
> scientific computing.
>
> I'd use the "netCDF4" python library (and numpy of cource ;-) ).
>
> Note that your CSV file has NO meta data -- not even variable names. So
> you'll want o add that, ideally following teh CF standards.
>
> I got carried away, so I've enclosed a sample script that reads your csv
> file and generates a netcdf file -- lots of details to be filled in there.
>
> If you don't have Python and all that on your computer -- I recommend the
> "Anaconda" Python distribution -- it should work out of the box for this.
>
> -Chris
>
>
>
>
>
>
>
>
>>
>> I saw the following posting to use in Ferret I new to Ferret. Can somebody
>> guide me how to use the script in Ferret. Also, I could not found the script
>> mention in the website. Can I use a similar approach for the attached .csv
>> file.
>>
>> http://permalink.gmane.org/gmane.comp.lib.netcdf/1136
>>
>> Any help is appreciated.
>>
>> Thanks,
>>
>> Dumindu.
>>
>> _______________________________________________
>> netcdfgroup mailing list
>> netcdfgroup@xxxxxxxxxxxxxxxx
>> For list information or to unsubscribe,  visit:
>> http://www.unidata.ucar.edu/mailing_lists/
>
>
>
>
> --
>
> Christopher Barker, Ph.D.
> Oceanographer
>
> Emergency Response Division
> NOAA/NOS/OR&R            (206) 526-6959   voice
> 7600 Sand Point Way NE   (206) 526-6329   fax
> Seattle, WA  98115       (206) 526-6317   main reception
>
> Chris.Barker@xxxxxxxx
>
> _______________________________________________
> netcdfgroup mailing list
> netcdfgroup@xxxxxxxxxxxxxxxx
> For list information or to unsubscribe,  visit:
> http://www.unidata.ucar.edu/mailing_lists/



-- 
Dr. Richard P. Signell   (508) 457-2229
USGS, 384 Woods Hole Rd.
Woods Hole, MA 02543-1598



  • 2015 messages navigation, sorted by:
    1. Thread
    2. Subject
    3. Author
    4. Date
    5. ↑ Table Of Contents
  • Search the netcdfgroup archives: