NOTICE: This version of the NSF Unidata web site (archive.unidata.ucar.edu) is no longer being updated.
Current content can be found at unidata.ucar.edu.
To learn about what's going on, see About the Archive Site.
NOTICE: This version of the NSF Unidata web site (archive.unidata.ucar.edu) is no longer being updated.
Current content can be found at unidata.ucar.edu.
To learn about what's going on, see About the Archive Site.
Hi Thomas: TDS uses standard Java interfaces to the filesystem, so it wouldnt be taking advantage of anything that needed special commands. Both the netcdf library and TDS are thread-safe, so can scale up to large number of simultaneous requests, so it seems likely that a clustered Tomcat environment would work well. Perhaps by distributing data correctly over data nodes, significant improvements might be possible. So much depends on access patterns, so a good way to proceed would be to create a synthentic load (eg script a bunch of requests to the TDS) that mimics what you expect users to need, and measure performance as you modify your system. I dont know enough about GPFS to know what features could be used to go beyond what you get from posix API. Anyone else? John On Thu, Feb 25, 2016 at 2:27 AM, Thomas LOUBRIEU <thomas.loubrieu@xxxxxxxxxx > wrote: > Dear all, > > In our data center, the new high-performance clustered file system we're > going to use is GPFS (General Parallel File System). I am wondering is > java-netcdf library or thredds data server can take benefit of this high > performance file system if the netcdf files are stored on it ? > > Are you aware of work being done or systems working with GPFS or otherwise > on similar high performance systems (HDFS, moosefs, ...). I am definitely > not an expert and any information regarding reading netcdf in java on these > clustered file system (preferably GPFS) would help us very much. > > Thanks, > > Thomas > > _______________________________________________ > thredds mailing list > thredds@xxxxxxxxxxxxxxxx > For list information or to unsubscribe, visit: > http://www.unidata.ucar.edu/mailing_lists/
netcdf-java archives: