Re: [thredds] Collection configuration help

• To: Ethan Davis <edavis@xxxxxxxxxxxxxxxx>, "thredds@xxxxxxxxxxxxxxxx" <thredds@xxxxxxxxxxxxxxxx>
• Subject: Re: [thredds] Collection configuration help
• From: Kyle Wilcox <KWilcox@xxxxxxxxxxxxxx>
• Date: Tue, 19 Apr 2011 22:00:57 -0400

> The dateFormatMark works on the name of the file, not including the 
> directories

Thanks, that's what I needed to know.  Files will be renamed!
________________________________________
From: thredds-bounces@xxxxxxxxxxxxxxxx [thredds-bounces@xxxxxxxxxxxxxxxx] On 
Behalf Of Ethan Davis [edavis@xxxxxxxxxxxxxxxx]
Sent: Tuesday, April 19, 2011 8:44 PM
To: thredds@xxxxxxxxxxxxxxxx
Cc: 'Jay Titlow'
Subject: Re: [thredds] Collection configuration help

Hi Kyle,

The dateFormatMark works on the name of the file, not including the
directories. You can use the ant-like "**" to indicate all directories
under a given directory. So, if you can rename your files to include the
date, you should be able to do something like

> <collection 
> spec="/data/Weatherflow/ensemble/WRF_ensemble/**/#yyyy.MM.dd.HH#.*\.nc$"
>             ... />

I'm not sure how the dateFormatMark matches up with the regular
expression in ".*\.nc$". You may want to escape the "."s in your
dateFormatMark (or doing so may mess up the dateFormatMark handling).
I'll have to dig into that to be sure.

In case you haven't seen these docs, there's more detail at

> http://www.unidata.ucar.edu/software/netcdf-java/reference/collections/FeatureCollections.html#SimpleDateFormat
> http://www.unidata.ucar.edu/software/netcdf-java/reference/collections/CollectionSpecification.html

Hope that helps,

Ethan

On 4/19/2011 11:19 AM, Kyle Wilcox wrote:
> I've got a bunch of WRF forecasts in NetCDF format I want to collect into an 
> FMRC.
>
> Each run is split up into individual hour files (one time per file).  All 
> hour files for a run are organized by directory, with the directory name 
> containing the information needed to parse out the RunTime for a FMRC.
>
> Sample directory structure:
>
> http://tds.maracoos.org/thredds/catalog/Weatherflow-WRF-Small/Agg-Collection.nc/files/catalog.html
>
>
> Here is how I am specifying the collection:
> ...
> <collection 
> spec="/data/Weatherflow/ensemble/WRF_ensemble/#yyyy.MM.dd.HH#/.*\.nc$" 
> name="WFlow_WRF_Large_Ensemble" recheckAfter="15 min" olderThan="5 min" />
> ...
>
> In catalogInit.log, I get a:
> **Error: DatasetFmrc =Weatherflow-WRF-Large/Agg-Collection.nc directory= 
> </data/Weatherflow/ensemble/WRF_ensemble/#yyyy.MM.dd.HH#> does not exist
>
>
> ]$ ls -1t /data/Weatherflow/ensemble/WRF_ensemble
> 2011.04.19.12
> 2011.04.19.00
> 2011.04.18.12
> 2011.04.18.00
> 2011.04.17.12
> ...
>
> Is it possible to specify the dateFormatMark on a directory name instead of a 
> file?  If not, I can rename files accordingly.
>
> Thanks
>
> ---------
> Kyle Wilcox, Engineer
> Applied Science Associates
> 55 Village Square Drive
> South Kingstown, RI 02879
> p: (401) 789-6224
> e: kwilcox@xxxxxxxxxxxxxx
>
>
> _______________________________________________
> thredds mailing list
> thredds@xxxxxxxxxxxxxxxx
> For list information or to unsubscribe,  visit: 
> http://www.unidata.ucar.edu/mailing_lists/

_______________________________________________
thredds mailing list
thredds@xxxxxxxxxxxxxxxx
For list information or to unsubscribe,  visit: 
http://www.unidata.ucar.edu/mailing_lists/

• References:
  • [thredds] Collection configuration help
    • From: Kyle Wilcox
  • Re: [thredds] Collection configuration help
    • From: Ethan Davis