NOTICE: This version of the NSF Unidata web site (archive.unidata.ucar.edu) is no longer being updated.
Current content can be found at unidata.ucar.edu.
To learn about what's going on, see About the Archive Site.
NOTICE: This version of the NSF Unidata web site (archive.unidata.ucar.edu) is no longer being updated.
Current content can be found at unidata.ucar.edu.
To learn about what's going on, see About the Archive Site.
Hi John,
thank you for your answer. To clarify, I was not trying to realize
long-term persistence of Array objects. I just
needed short term persistence in order to use ehcache to cache arrays
and spill over to disk in case of memory exhaustion.
Ehcache does not seem to have a straightforward way of plugging in a
customized serialization mechanism apart from
using the Java default one, which is why I had to stick to that. It
would probably also be possible to use other serialization
libraries like Kryo or, as you mentioned "protocol buffers" within the
writeExternal/readExternal methods.
I ended up implementing a wrapper object for Array that handles
the serialization and deserialization, maybe not in the most efficient
way, but it currently meets my requirements and looks quite
simple:
public class SerializableArray implements Externalizable {
...
@Override
public void writeExternal(ObjectOutput oo) throws IOException {
if(array!=null) {
oo.writeObject(DataType.getType(array.getElementType()));
oo.writeObject(array.getShape());
oo.writeObject(array.getStorage());
}
}
@Override
public void readExternal(ObjectInput oi) throws IOException,
ClassNotFoundException {
array = Array.factory((DataType)oi.readObject(),
(int[])oi.readObject(), oi.readObject());
}
}
And it passes all tests. So I am currently good to go with that solution.
Thanks again for the pointers!
Nils
Am 13.11.2012 22:35, schrieb John Caron:
> Hi Nils:
>
> Default Java serialization has serious problems. See chapter 11 of
> "Effective Java" for details. Early versions of Netcdf-Java had lots
> of serialization, but I have removed it mostly.
>
> If you want to do serialization, I would create specialized objects
> that take the original object as an argument, and transfer the contents.
>
> Recently Ive been using Google's "protocol buffer" library to do the
> encoding, and I like that approach. Fast, and the serialized objects
> are readable from other languages besides java.
>
> May seem like more work, but in the long run, IMO default java
> serialization is not a good persistence solution except in certain
> limited cases.
>
> John
>
> On 11/13/2012 2:13 AM, Nils Hoffmann wrote:
>> Hi netcdf-java developers,
>> I recently had the requirement of serializing/deserializing a
>> ucar.ma2.Array instance for local client-side caching
>> and was wondering why the current implementation (4.2.32) does not
>> implement java.io.Serializable or java.io.Externalizable?
>> Are there any particular reasons for this, or am I overlooking
>> something?
>>
>> In order to check whether any severe problems would wait down the road,
>> I created a delegate for ucar.ma2.Array and wrote an associated unit
>> test, showing that it is possible to successfully serialize and
>> deserialize arbitrary arrays. I would however prefer to be able to
>> extend ucar.ma2.Array directly, which is currently not possible
>> unless I create the new class within the ucar.ma2 namespace. This is due
>> to three methods in
>> ucar.ma2.Array that are using default visibility (abstract Array
>> createView(Index index), abstract void copyFrom1DJavaArray(IndexIterator
>> iter, Object javaArray),
>> and abstract void copyTo1DJavaArray(IndexIterator iter, Object
>> javaArray)). Their visibility effectively disallows extension
>> of ucar.ma2.Array from outside the ucar.ma2 package.
>>
>> Would it be possible to set the visibility of these methods to public in
>> the netcdf-java api without other negative side effects or would anyone
>> propose a different approach?
>>
>> Maybe someone else has experience with Array serialization and would be
>> interested in sharing or providing me some further pointers?
>>
>> I would greatly appreciate any help on this.
>>
>> Sincerely,
>>
>> Nils
>>
>
> _______________________________________________
> netcdf-java mailing list
> netcdf-java@xxxxxxxxxxxxxxxx
> For list information or to unsubscribe, visit:
> http://www.unidata.ucar.edu/mailing_lists/
--
Nils Hoffmann phone: +49-521-106-4342
Bielefeld University room: U10-144
Faculty of Technology, Genome Informatics
P.O. Box 10 01 31
33501 Bielefeld, Germany
http://www.cebitec.uni-bielefeld.de/~hoffmann
netcdf-java archives: