NOTICE: This version of the NSF Unidata web site (archive.unidata.ucar.edu) is no longer being updated.
Current content can be found at unidata.ucar.edu.
To learn about what's going on, see About the Archive Site.
NOTICE: This version of the NSF Unidata web site (archive.unidata.ucar.edu) is no longer being updated.
Current content can be found at unidata.ucar.edu.
To learn about what's going on, see About the Archive Site.
On Jun 18, 2012, at 4:19 PM, Doug Lindholm wrote: > Hi, > > I've been implementing "filters" for our LaTiS server > (http://lasp.colorado.edu/lisird/tss.html) using the OPeNDAP (DAP2) URL > "function" syntax. For example, > > http://lasp.colorado.edu/lisird/tss/american_sunspot_number.csv?&thin(10)&format_time(yyyy-MM-dd) > > It's not ideal, but it's a start. DAP2 says very little about functions. > Maybe DAP4 could say more. > > I'd also strongly suggest that people think about Functional Programming > paradigms. I suspect that such formalisms will lend themselves better than an > OO biased approach. I've been thinking about just this topic for some time. I am pretty certain that a functional language is the way to go. While the DAP2 specification say little about server functions, Hyrax does come with a few and the parser & evaluator does support composing functions. In addition there are two groups now working on server functions that perform some significant operations: unstructured grid subsetting and satellite data reprojection (satellite corrds to mercator, ...). I'd certainly be interested in talking about this stuff more. James > > Doug > > On 6/18/12 3:56 PM, Roy Mendelssohn wrote: >> But for a useful service, the form and syntax of the URL should be >> independent of the mechanism that does the server side calculations >> (which rules out SWAMP). So for example, both Grads an F-TDS use the >> same format in the URL to say that "this is an expression", but the >> expressions themselves are platform specific. That is not the way to >> get overarching services. >> >> Instead, we need agreement on how in the URL request we signal a >> server-side function, the syntax of that function (independent of the >> engine underneath) and a few simple functions to start (say simple >> data transformations, differencing and averaging on a dimension(s)). >> Then the server back-end can parse the request and use Ferret or >> Grads or NCO or Python or whatever is desired, and like with any good >> service, the back end could change without having any affect on the >> URL or the user. >> >> I know Matthew Arrott at least used to like the approach in Chapter >> 12 of "Python Scripting for Computation Science". But a lot of that >> is the engine underneath. I am more interested in the form of the >> URL. Get some agreement on that, and some real implementation could >> proceed. >> >> -Roy >> >> >> >> >> >> On Jun 18, 2012, at 2:37 PM, Russ Rew wrote: >> >>> Jeff, >>> >>>> However, we have to keep in mind performance ramifications. It >>>> still takes a long time to move gigabytes of data across a >>>> network. This brings up the importance of moving the computation >>>> to the data, instead of moving the data to the computation. For >>>> some data sets and many use cases remote access to data works >>>> very well so things like brokering are tractable. However, for >>>> *big* data sets (e.g., climate model output) we need to come up >>>> with richer mechanisms (like the NCO on local data) to bring >>>> computation to the data. >>> >>> See Daniel Wang's SWAMP (the Script Workflow Analysis for >>> MultiProcessing), built on top of NCO: >>> >>> https://code.google.com/p/swamp/ >>> >>> --Russ >>> >>> _______________________________________________ thredds mailing >>> list thredds@xxxxxxxxxxxxxxxx For list information or to >>> unsubscribe, visit: http://www.unidata.ucar.edu/mailing_lists/ >> >> ********************** "The contents of this message do not reflect >> any position of the U.S. Government or NOAA." ********************** >> Roy Mendelssohn Supervisory Operations Research Analyst NOAA/NMFS >> Environmental Research Division Southwest Fisheries Science Center >> 1352 Lighthouse Avenue Pacific Grove, CA 93950-2097 >> >> e-mail: Roy.Mendelssohn@xxxxxxxx (Note new e-mail address) voice: >> (831)-648-9029 fax: (831)-648-8440 www: http://www.pfeg.noaa.gov/ >> >> "Old age and treachery will overcome youth and skill." "From those >> who have been given much, much will be expected" "the arc of the >> moral universe is long, but it bends toward justice" -MLK Jr. >> >> _______________________________________________ thredds mailing list >> thredds@xxxxxxxxxxxxxxxx For list information or to unsubscribe, >> visit: http://www.unidata.ucar.edu/mailing_lists/ > > _______________________________________________ > thredds mailing list > thredds@xxxxxxxxxxxxxxxx > For list information or to unsubscribe, visit: > http://www.unidata.ucar.edu/mailing_lists/ -- James Gallagher jgallagher at opendap.org 406.723.8663
thredds archives: