Re: [thredds] nco as a web service

• To: Jeff McWhirter <jeff.mcwhirter@xxxxxxxxx>
• Subject: Re: [thredds] nco as a web service
• From: Russ Rew <russ@xxxxxxxxxxxxxxxx>
• Date: Mon, 18 Jun 2012 15:37:26 -0600

Jeff,

> However, we have to keep in mind performance ramifications. It still takes
> a long time to move gigabytes of data across a network. This brings up the
> importance of moving the computation to the data, instead of moving the
> data to the computation. For some data sets and many use cases remote
> access to data works very well so things like brokering are tractable.
> However, for *big* data sets (e.g., climate model output) we need to come
> up with richer mechanisms (like the NCO on local data) to bring computation
> to the data.

See Daniel Wang's SWAMP (the Script Workflow Analysis for
MultiProcessing), built on top of NCO:

  https://code.google.com/p/swamp/

--Russ

• Follow-Ups:
  • Re: [thredds] nco as a web service
    • From: Roy Mendelssohn

• References:
  • Re: [thredds] nco as a web service
    • From: Jeff McWhirter
  • Re: [thredds] nco as a web service
    • From: Ben Domenico
  • Re: [thredds] nco as a web service
    • From: Jeff McWhirter