Re: [netcdfgroup] Reading data from a variable into a numpy.ma.MaskedArray

• To: Constantine Khroulev <c.khroulev@xxxxxxxxx>
• Subject: Re: [netcdfgroup] Reading data from a variable into a numpy.ma.MaskedArray
• From: James Adams <monocongo@xxxxxxxxx>
• Date: Tue, 21 Jan 2014 13:41:11 -0500

Constantine this is a very helpful response, I couldn't have asked for
more.  I'm relatively new to Python and NetCDF so this sort of thing
is very enlightening.  Thanks!

--James

On Tue, Jan 21, 2014 at 11:51 AM, Constantine Khroulev
<c.khroulev@xxxxxxxxx> wrote:
> James,
>
> As far as I know the netCDF4 Python module cannot be told to always
> return a masked array.
>
> Here's a workaround, though:
>
> import numpy as np
>
> def check_record(data):
>     """Check if a record is masked and return True if it is;
>     return False otherwise."""
>
>     try:
>         if data.mask.all():
>             # all entries are masked
>             return True
>         else:
>             # some entries might be masked
>             return False
>     except AttributeError:
>         # the "mask" attribute was not found
>         return False
>
> # and here's some code to test it:
>
> a = np.zeros((10,10))
>
> b = np.ma.array(a)
>
> c = b.copy()
> c.mask = True
>
> print "checking if 'a' is masked:", check_record(a)
>
> print "checking if 'b' is masked:", check_record(b)
>
> print "checking if 'c' is masked:", check_record(c)
>
> In most cases it is better to use exceptions instead of type checks in
> Python code; see http://en.wikipedia.org/wiki/Duck_typing and
> http://en.wikipedia.org/wiki/Python_syntax_and_semantics#Exceptions
>
> I hope this helps.
>
> --
> Constantine
>
>
> On Tue, Jan 21, 2014 at 7:17 AM, James Adams <monocongo@xxxxxxxxx> wrote:
>> When I read data for a lon/lat point from a NetCDF variable I get a
>> MaskedArray if all values are missing and an ndarray if all values are
>> present.  I'd like to instead get it returned in one way or the other
>> in either case, preferably as a MaskedArray so I can check if all
>> values are masked and skip to the next lon/lat point.  Can someone
>> advise as to how I can best go about this?
>>
>> I'm using the netCDF4 Python module from here:
>> https://code.google.com/p/netcdf4-python/
>>
>> My code looks like this:
>>
>>     # get a "chunk" of data from the NetCDF variable
>>     precipChunk = inputPrecipVariable[0:len(inputTimeDimension):1,
>> lonChunkOffset:lonChunkOffset + lonChunkSize:lonChunkSize,
>> latChunkOffset:latChunkOffset + latChunkSize:latChunkSize]
>>
>>     # skip this entire chunk if all values are masked
>>     if (precipChunk.mask.all()):
>>         continue
>>
>> The above works fine if the data (precipChunk) is returned as a
>> MaskedArray, but it bombs out if it's returned as an ndarray.  Maybe I
>> should do some sort of type check on the returned array before
>> checking to see if all values are masked?  Is data only returned as a
>> MaskedArray if *all* values are missing/fill values, or can it happen
>> that a section of data which does contain valid values is also
>> returned as MaskedArray with the valid data values unmasked?
>>
>> Thanks in advance for any suggestions and/or insight.
>>
>> --James
>>
>> _______________________________________________
>> netcdfgroup mailing list
>> netcdfgroup@xxxxxxxxxxxxxxxx
>> For list information or to unsubscribe,  visit: 
>> http://www.unidata.ucar.edu/mailing_lists/

• References:
  • [netcdfgroup] Reading data from a variable into a numpy.ma.MaskedArray
    • From: James Adams
  • Re: [netcdfgroup] Reading data from a variable into a numpy.ma.MaskedArray
    • From: Constantine Khroulev