Re: Web service for Total Precipitation

• To: Ernest To <ernesto_24_7_365@xxxxxxxxx>
• Subject: Re: Web service for Total Precipitation
• From: John Caron <caron@xxxxxxxxxxxxxxxx>
• Date: Wed, 29 Mar 2006 21:08:48 -0700

Hi Ernest:

We have an experimental web service that will allow you to download the data 
that you want in NetCDF format. You will be able to specify the bounding box in 
lat/lon, so no need to convert to array indices.

You will need to read the file, using the netcdf C library, and convert to your 
format.

It will be ready for you to try in a few days. I can send you a sample of how 
the file will look if you want to get started.


Ernest To wrote:

John,

Thanks for your reply. In essence, the purpose of

the webservice is to allow the lay user to submit data
requests in a standardized manner to the unidata
website and to receive data in a format that is usable
on different platforms (e.g. VB.NET, IDL, etc.)

  The following is the sequence of steps of the
webservice:

1.  accept standardized inputs from the user (i.e. lat
long box, time period, parameter of interest) and
convert them into a URL query for the website
2.  submit the URL query to the website and download
the data (the data can be in any format - ASCII or
binary)
3.  Process the data into standardized text
format,i.e.:
   a. ARCHydro TimeSeries format for one dimensional

arrays, or b. NetCDF ASCII (.CDL file) format for grids or

multi-dimensional arrays.
4.  Incorporate the data text into an XML file and
send it to the user.  For visual basic users, Tim has
developed webservicewrappers that would ingest these
XML files into visual basic variables (e.g. sorted
lists) to make it easy for analysis.

Such webservices have been developed for the NWIS,
Ameriflux, MODIS, DAYMET websites, and we would like
to develop one for UNIDATA that would utilize the same

standardized input format.

For this reason, I am very interested in how current
users are querying and subsetting UNIDATA data,
especially in how OPeNDAP works.  Perhaps (as a wild
guess), to utilize what already has been done, our
webservice could be some sort of wrapper around
OPeNDAP or one of its tools (e.g. Ferret).  Would you
have any suggestions or examples of similar work

that's been done?

We are currently coding in Visual Basic .NET.  Is
there some way to access the Java utility  you
mentioned that converts lat, long coordinates into the
array indices?  Could you tell us what the WMO format
is?

Again, thanks for your reply.

Ernest






--- John Caron <caron@xxxxxxxxxxxxxxxx> wrote:

Hi Ernest:

Ernest To wrote:

John,

My name is Ernest To and I am a student of Dr.
Maidment's.  Tim Whiteaker and I are trying to

write a

webservice that would download total precipitation
data for the continental US at resolution of 20km

from

unidata website for a given time and place.  We

have

so far been experimenting with the following link:

http://motherlode.ucar.edu:8080/thredds/dodsC/model/NCEP/NAM/CONUS_20km/surface/NAM_CONUS_20km_surface_20060324_1800.grib1.html

Can we ask you the following questions?
1.  What is a .grib1 or a .grib2 file?  Is it in
netcdf format?

no, grib1 and grib2 are WMO formats for gridded data

2.  We noticed that in the link above (OPENDAP )

there

are options for getting a binary file or an ASCII
file.  Are those files in netcdf and cdl formats
respectively?

no, they are in opendap specific formats. normally
you would use the opendap client library (C++ or
Java) to access data through opendap, rather than
downloading files.

3.  Our webservice requires the creation of a URL
query based on user inputs (such as coordinates of
bounding rectangle, time period of interest and
parameter of interest) which will then be used to
request data from the website.  Could you provide
information on how to convert geographical

coordinates

(in decimal degrees) and time values (MM/DD/YYYY
hh:mm) into the array indices shown in the website

above? e.g. time[0:1:0] y[0:1:0] x[0:1:0]

this is not trivial, as it depends on the specific
file. there are routines in the netcdf-java 2.2
library that can do it. Are you using java?

what file formats can you handle? what does your web
service do?

 Thank you for your kind attention.  Please let

me

know if anything is unclear.  I can be reached

either

by e-mail: ernesto_24_7_365@xxxxxxxxx or by phone:
512-508-4834.

 Thanks.

Ernest To
PhD student
Center for Research in Water Resources
University of Texas at Austin
512-508-4834




--- John Caron <caron@xxxxxxxxxxxxxxxx> wrote:

Hi David, Ernest:

Have a look at the motherlode catalog:

http://motherlode.ucar.edu:8080/thredds/

we were guessing you will want the NAM 21-km
dataset?

If you want to use WCS protocol, you can get back
geotiff or netcdf-CF. Otherwise you could use
Opendap.

There's a few other wrinkles concerning time
dimension, we can go over.

David Maidment wrote:

Ernest:

Please see the message below from Mohan

Ramamurthy

at Unidata.  Can you

please work with John Caron
to see how we can ingest a field using web

services.  I'd like to stick

to something 2D on the land surface like precipitation, evaporation or soil

moisture.  I believe

those are all in the eta model
output.   Likely what we want to do is to give a

lat-long box, a time

point and a variable, and ask
for a field over the region of the lat-long box

for that variable and

time point, and it might occur
that the field actually covers several forecast

intervals, eg. 3 hour

forecast times for a day.  I'm
not sure in that case if we should treat each

time

slice as one field in

the web services ingest, but
lets look into what is involved and then figure

out what is best.

John -- I'd like to see what can be done with a

WCS service if possible.

Thanks for your help.

David


-----Original Message-----

From: Mohan Ramamurthy [mailto:mohan@xxxxxxxx] Sent: Friday, March 10, 2006 5:40 PM

To: David Maidment
Cc: John Caron; Mohan Ramamurthy
Subject: Web service to Operational Eta model

output

David,

I just spoke to John Caron

(caron@xxxxxxxxxxxxxxxx), our lead developer

on THREDDS, re. our conversation this afternoon

to

create a web service

to access fields from the operational model

output

in the IDD stream.

John will be happy to work the person on your to

make it happen.  It

looks like many of the pieces are already in

place

to enable a web

service for accessing, subsetting, and moving

individual fields, because

the data is served via OPeNDAP.  For example,

here

is the OPeNDAP URL

for today's Eta output:

http://motherlode.ucar.edu:8080/thredds/dodsC/model/NCEP/NAM/Alaska_22km

/NAM_Alaska_22km_20060310_1200.grib1

=== message truncated ==



__________________________________________________
Do You Yahoo!?

Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com