NOTICE: This version of the NSF Unidata web site (archive.unidata.ucar.edu) is no longer being updated.
Current content can be found at unidata.ucar.edu.
To learn about what's going on, see About the Archive Site.
NOTICE: This version of the NSF Unidata web site (archive.unidata.ucar.edu) is no longer being updated.
Current content can be found at unidata.ucar.edu.
To learn about what's going on, see About the Archive Site.
Chris,
I created the csv file for a single day and I was able to figure out the
lat, lon dimension declaration as indicated in the following code. But, I
think I am close to obtain the netCDF file.
import numpy as np
import netCDF4
import os
# load the data
path='C:\Users\.spyder2'
os.chdir(path)
# this load the file into a Nx3 array (three columns)
data = np.loadtxt('TRMM_1998_01_0100_newntcl.csv', delimiter=',')
# create a netcdf Data object
with netCDF4.Dataset('TEST_file.nc', mode="w", format='NETCDF4') as ds:
# some file-level meta-data attributes:
ds.Conventions = "CF-1.6"
ds.title = 'Non TC precipitation'
ds.institution = 'USU'
lat = data[:,0] # the first column
lon = data[:,1] # the second column
precip = data[:,2] # the third column
lat = lat.reshape( (161, 321) )
lon = lon.reshape( (161, 321) )
lat = ds.createDimension('latitude', 161)
lon = ds.createDimension('longitude', 321)
I am getting an error when I run following first two lines.
precip = ds.createVariable('precip', np.float32, ('latitude',
'longitude'))
var[:] = data[:,:]
## adds some attributes
var.units = 'mm'
var.long_name = 'Precipitation'
Error:
Traceback (most recent call last):
File "<ipython-input-67-cf31aeae7dd7>", line 1, in <module>
precip = ds.createVariable('precip', np.float32, ('latitude',
'longitude'))
File "netCDF4.pyx", line 1721, in netCDF4.Dataset.createVariable
(netCDF4.c:22247)
File "netCDF4.pyx", line 2313, in netCDF4.Variable.__init__
(netCDF4.c:28523)
RuntimeError: NetCDF: Not a valid ID
I appreciate if you can clarify here a bit or provide me some clues of
what's happening to correct it.
Thanks in advance,
Dumindu.
On Wed, Mar 11, 2015 at 3:19 PM, Chris Barker <chris.barker@xxxxxxxx> wrote:
> Sorry, I really dont have time to do it for you, but a couple hints:
>
> IF you're lucky, you can "reshape" the input arrays in one step:
>
> data = np.loadtxt('TEST_file.csv', delimiter=',')
>
> lat = data[:,0] # the first column -- if that is latitude
>
> lat = lat.reshape( (num_times, num_lats, num_lons) )
>
> That _might put it all in the right order, depending on hoe it's written
> to the file. If that doesn't work, this might:
>
> lat = lat.reshape( (num_times, num_lats, num_lons), order='F')
>
> If that doesn't work, then you may have to loop through all of by hand,
> in the right order:
>
> new_data = np.zeros( (num_times, num_lats, num_lons) )
> i = 0
> for t in range(num_times):
> for lat in range(num_lats):
> for lon in range(num_lons):
> new_data[t, lat, lon] = old_data[i]
> i += 1
>
> You may need to change the order of those loops to match your data.
>
> Also,m I see three columns, not four in your sample file -- is each time
> step in a different file? That wold require you to loop through all the
> files to load each time...
>
> Take a look at python and numpy tutorials online to learn a bit more about
> all this.
>
> -Chris
>
>
>
>
> On Wed, Mar 11, 2015 at 12:03 PM, Dumindu Jayasekera <
> d.jayasekera@xxxxxxxxxxxxxxxxx> wrote:
>
>> Chris,
>>
>> Thanks again. I am unable to create the 3D array since I have limited
>> knowledge in python.
>>
>> I have attached the csv files, python scripts and sample netCDF file.
>>
>> Any help is appreciated.
>>
>>
>>
>>
>>
>> On Wed, Mar 11, 2015 at 8:45 AM, Chris Barker <chris.barker@xxxxxxxx>
>> wrote:
>>
>>> On Tue, Mar 10, 2015 at 5:50 PM, Dumindu Jayasekera <
>>> d.jayasekera@xxxxxxxxxxxxxxxxx> wrote:
>>>
>>>> Thanks again Chris.
>>>>
>>>> I think I dont need to core metadata. I was able to produce the .nc
>>>> file using the code below (as you suggested).
>>>>
>>>>
>>> As Rich suggests, you may be better off using onf the libraries
>>> suggested, Iris, in particular will get all the complex CF metadaat stuff
>>> right for you.
>>>
>>> But you're this close, so...
>>>
>>>
>>>> But, how can I modify to rename the var1 = lat, var2 = lon, var3 =
>>>> precipitation in the code below.?
>>>>
>>>
>>> From the nc file you sent, it looks like you want the precipitiaon to
>>> be a 3-d array: (time X ltitude X longitude), so:
>>>
>>>
>>> # this load the file into a Nx3 array (three columns)
>>>> data = np.loadtxt('TEST_file.csv', delimiter=',')
>>>>
>>>
>>> note -- you dont have a 3-d array here, you have a 2-d array, which is
>>> really three vectors -- you will need to re-shuffle these to get the 3-d
>>> array you want, and the time, lat, and long vectors.... but assumign you've
>>> done that:
>>>
>>>
>>> you'll need three dimensions:
>>> time = ds.createDimension('time', num_times)
>>> lat = ds.createDimension('latitude', num_latitude)
>>> lon = ds.createDimension('longitude', num_longitude)
>>>
>>> Then you want your 3-d precip variable -- I"d call it something more
>>> readable than "r", but maybe you need that...
>>>
>>> r = ds.createVariable('r', np.float32, ('time', 'latitude', 'longitude'))
>>> var[:] = data[:,:,:]
>>> ## adds some attributes
>>> var.units = 'mm'
>>> var.long_name = 'Precipitation'
>>> ... and the others that you need.
>>>
>>> HTH,
>>>
>>> -Chris
>>>
>>> --
>>>
>>> Christopher Barker, Ph.D.
>>> Oceanographer
>>>
>>> Emergency Response Division
>>> NOAA/NOS/OR&R (206) 526-6959 voice
>>> 7600 Sand Point Way NE (206) 526-6329 fax
>>> Seattle, WA 98115 (206) 526-6317 main reception
>>>
>>> Chris.Barker@xxxxxxxx
>>>
>>
>>
>
>
> --
>
> Christopher Barker, Ph.D.
> Oceanographer
>
> Emergency Response Division
> NOAA/NOS/OR&R (206) 526-6959 voice
> 7600 Sand Point Way NE (206) 526-6329 fax
> Seattle, WA 98115 (206) 526-6317 main reception
>
> Chris.Barker@xxxxxxxx
>
netcdfgroup archives: