Re: [python-users] netCDF4/h5Py in the same Python 3 script

• To: Ryan May <rmay@xxxxxxxx>
• Subject: Re: [python-users] netCDF4/h5Py in the same Python 3 script
• From: Aaron Funk <afunk@xxxxxxxx>
• Date: Fri, 24 Jan 2020 15:14:35 -0600

Hi Ryan,

I'm quite new to Python so I may not provide exactly what you asked for
just due to my own ignorance. If you need more traceback info, please let
me know.

What I found is that I can run your example script without error. However,
if I simply change the order of the import statements and place h5py before
netCDF4, I get an error similar to what I was seeing in my own code. Here's
the traceback info printed to the screen:


Traceback (most recent call last):

  File "ucar_test.py", line 12, in <module>

    var[:] = np.arange(200.).reshape(10, 20)

  File "netCDF4/_netCDF4.pyx", line 4950, in
netCDF4._netCDF4.Variable.__setitem__

  File "netCDF4/_netCDF4.pyx", line 5229, in netCDF4._netCDF4.Variable._put

  File "netCDF4/_netCDF4.pyx", line 1887, in
netCDF4._netCDF4._ensure_nc_success

RuntimeError: NetCDF: HDF error


I'll try to be more specific than before about my own environment...

CentOS 7
Pip virtual environment using Python 3.6.4
Installed h5py (2.10.0) and netCDF4 (1.5.3) using pip
I compiled the HDF5 (1.10.6) and netCDF4 (4.7.3) libraries from source and
are installed locally
and pointed to using the LD_LIBRARY_PATH environment var. netCDF4 was
compiled pointing to the HDF5 lib I compiled so I believe everything's
using the same libhdf5.

Thanks,

Aaron



On Fri, Jan 24, 2020 at 2:37 PM Ryan May <rmay@xxxxxxxx> wrote:

> Hi,
>
> I'm not aware of any reason those two modules should interfere--provided
> they're using the same libhdf5. Can you provide the full traceback and text
> of the error? Otherwise we're just shooting in the dark here.
>
> I can say that the following code works without error for me on macOS
> 10.15 and Python 3.8, all from conda-forge:
>
> from netCDF4 import Dataset
> import h5py
> import numpy as np
>
> nc = Dataset('test.nc
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__test.nc&d=DwMFaQ&c=u6LDEWzohnDQ01ySGnxMzg&r=iH6T3D53S142XIlkmrHNIQ&m=FR4YBTjpO9yaHKKoEgwu8zzxAGtf6zGTaoenPC7YirA&s=0s0mS_l3nMrMi1aQ7jAbEyeGVBuIVcXtIWKhrenMe84&e=>',
> 'w')
> nc.createDimension('x', 10)
> nc.createDimension('y', 20)
> var = nc.createVariable('test', np.float64, ('x', 'y'))
> var[:] = np.arange(200.).reshape(10, 20)
> nc.close()
>
> Does this code fail for you?
>
> Ryan
>
> On Wed, Jan 22, 2020 at 10:45 AM Aaron Funk <afunk@xxxxxxxx> wrote:
>
>> Hi all,
>>
>> I'm new to Python and reaching out here for information on an issue I ran
>> into. My issue is that I'm unable to close a netCDF4 object and write a
>> file to disk without error if I import h5py and netcdf4 in the same script.
>> If I remove the "import h5py" line, the script works fine.
>>
>> Is this a known/common issue? Is there a way in which I can use both h5py
>> and netcdf4 modules in the same Python script without generating these
>> types of errors? What's the best workaround for this?
>>
>> I'm not providing an example script here, but some environment details
>> are below. I can provide other info as needed.
>>
>> Environment:
>> CentOS 7
>> Python 3.6
>> pip virtual environment with h5py and netcdf4 and dependencies installed.
>> HDF5 and netCDF-4 C compiled from scratch.
>>
>> Thanks,
>>
>> Aaron Funk
>> _______________________________________________
>> NOTE: All exchanges posted to Unidata maintained email lists are
>> recorded in the Unidata inquiry tracking system and made publicly
>> available through the web.  Users who post to any of the lists we
>> maintain are reminded to remove any personal information that they
>> do not want to be made public.
>>
>>
>> python-users mailing list
>> python-users@xxxxxxxxxxxxxxxx
>> For list information, to unsubscribe, or change your membership options,
>> visit: https://www.unidata.ucar.edu/mailing_lists/
>> <https://urldefense.proofpoint.com/v2/url?u=https-3A__www.unidata.ucar.edu_mailing-5Flists_&d=DwMFaQ&c=u6LDEWzohnDQ01ySGnxMzg&r=iH6T3D53S142XIlkmrHNIQ&m=FR4YBTjpO9yaHKKoEgwu8zzxAGtf6zGTaoenPC7YirA&s=cX6cNiHGF6Bhnq3ogPbdaUiyI7jqUIwI_EUg1FuoLKI&e=>
>>
>
>
> --
> Ryan May, Ph.D.
> Software Engineer
> UCAR/Unidata
> Boulder, CO
>

• Follow-Ups:
  • Re: [python-users] netCDF4/h5Py in the same Python 3 script
    • From: Benjamin Root

• References:
  • [python-users] netCDF4/h5Py in the same Python 3 script
    • From: Aaron Funk
  • Re: [python-users] netCDF4/h5Py in the same Python 3 script
    • From: Ryan May