NOTICE: This version of the NSF Unidata web site (archive.unidata.ucar.edu) is no longer being updated.
Current content can be found at unidata.ucar.edu.
To learn about what's going on, see About the Archive Site.
hope so.
note that the dateFormatMark="MOD13Q1.A#yyyyDDD" is just counting chars, then applying the java SimpleDateFormat, so the date in the filename just has to start at character 9, it doesnt have to match MOD13Q1.A
On 3/26/2013 1:38 PM, Blodgett, David wrote:
Ahhh, so the "/" in the variable name will work to represent groups. I
didn't want to believe that would work, glad it will.
Thanks!
- Dave
On Tue, Mar 26, 2013 at 2:36 PM, John Caron <address@hidden> <mailto:address@hidden <mailto:address@hidden>>><mailto:address@hidden>> wrote:
update stardate 2349873.432874
ok, enhance actually already works, but i ran into another problem
with joinExisting.
So I retreated to joinNew using the kludge of extracting the date
from the filename.
this only works if the filenames are uniform:
<?xml version='1.0' encoding='UTF-8'?>
<netcdf xmlns='http://www.unidata.ucar.edu/namespaces/netcdf/ncml-2.2'>
<aggregation dimName='time' type='joinNew'>
<variableAgg
name='MODIS_Grid_16DAY_250m_500m_VI/Data_Fields/250m_16_days_NDVI' />
<scan location='G:/work/blodgett/' suffix='.hdf'
subdirs='false' dateFormatMark="MOD13Q1.A#yyyyDDD"/>
</aggregation>
</netcdf>
you also have to list the variables you want aggregated explicitly;
i only did one, you can add the others:
<variableAgg
name='MODIS_Grid_16DAY_250m_500m_VI/Data_Fields/250m_16_days_NDVI' />
this form should work with the 4.3.16 release.
let me know if this is adequate for you for now, and any problems.
john
ps: use this as reference:
http://www.unidata.ucar.edu/software/netcdf/ncml/v2.2/AnnotatedSchema4.html#aggregation
On 3/26/2013 1:29 PM, Blodgett, David wrote:
> ... enhance ... enhance ... go back ... enhance ...
>
> Love it!
>
> Thanks John, so no easy path forward on aggregations yet, but the
master
> plan includes it?
>
> I'm happy with that.
>
>
> On Tue, Mar 26, 2013 at 1:59 PM, John Caron
<address@hidden <mailto:address@hidden>
wrote:
>
> Hi David:
>
> I had a look at this again. I changed HdfEosModisConvention
to use a
> 1D instead of a scalar time coordinate.
>
> However, joinExisting isnt working because the dataset needs
to be
> "enhanced" (with HdfEosModisConvention) first, before the time
> coordinate is available. Currently joinExisting aggregation
doesnt
> do that.
>
> I can probably hack that in. Really I want to get the GRID
feature
> collection working, but thats going to be a few months.
Aggregation
> is old and crufty.
>
> Anyway, Ill let you know if i can get "enhanced" working.
>
>
> John
>
>
>
> On 3/20/2013 7:50 AM, Blodgett, David wrote:
>
> Morning John,
>
> I pulled down the latest thredds war and have been
testing a few
> things
> out. Did you get temporal aggregation working on the
structured
> data at
> all? Any hints on how to structure the ncml for that?
>
> Looks like the spatial stuff is working. Fantastic.
>
> Thanks!
>
> - Dave
>
>
>
> On Mon, Mar 4, 2013 at 9:07 AM, Blodgett, David
> <address@hidden <mailto:address@hidden>
<mailto:address@hidden <mailto:address@hidden>>
> <mailto:address@hidden <mailto:address@hidden>
<mailto:address@hidden <mailto:address@hidden>>>> wrote:
>
> OK, we'll keep an eye out for the release, thanks so
much John!
>
> No worries on the aggregation, the tiles are big enough
> that I think
> we can deal without aggregation.
>
>
> On Mon, Mar 4, 2013 at 8:43 AM, John Caron
> <address@hidden <mailto:address@hidden>
<mailto:address@hidden <mailto:address@hidden>>
> <mailto:address@hidden
<mailto:address@hidden>
> <mailto:address@hidden
<mailto:address@hidden>>__>> wrote:
>
> On 3/1/2013 12:03 PM, Blodgett, David wrote:
>
> John,
>
> Really happy you added this stuff in, what's
your
> plan for
> releasing it?
>
>
> going to try to release 4.3.16 in next week or two.
>
>
>
> You think there's any chance that tiled
aggregation
> will work?
>
>
> im reluctant to recommend using whats there
currently.
> But doing
> it right is a challenge and wont be done soon.
>
>
>
>
> - Dave
>
>
> On Tue, Feb 26, 2013 at 8:30 AM,
address@hidden <mailto:address@hidden>
> <mailto:address@hidden <mailto:address@hidden>>
> <mailto:address@hidden
<mailto:address@hidden> <mailto:address@hidden
<mailto:address@hidden>>>
> <address@hidden <mailto:address@hidden>
<mailto:address@hidden <mailto:address@hidden>>
>
> <mailto:address@hidden
<mailto:address@hidden> <mailto:address@hidden
<mailto:address@hidden>>>>
> wrote:
>
> Yes the tiling scheme is consistent for
all the
> MODIS tiles.
>
> Thanks,
>
> Jason Werpy
> Enterprise Architect
> Information Dynamics
> USGS/EROS
> Sioux Falls, SD 57198
> phone: 605-594-2723 <tel:605-594-2723>
> cell: 605-690-3576 <tel:605-690-3576>
> fax: 605-594-2530 <tel:605-594-2530>
> address@hidden <mailto:address@hidden> <mailto:address@hidden
<mailto:address@hidden>>
> <mailto:address@hidden <mailto:address@hidden>
<mailto:address@hidden <mailto:address@hidden>>>
>
>
> NOTICE: This email may contain confidential,
> proprietary,
> or competition
> sensitive bid or proposal procurement
> information. Unauthorized
> disclosure of this information may carry
> criminal penalties as set forth
> in the Procurement Integrity Act, 41
U.S.C. 423, as
> amended. Further,
> the unauthorized disclosure of certain
> commercial information by civil
> servants may result in fines or imprisonment
> under the
> Trade Secrets Act
> (18 U.S.C. 1905). If you have received this
> information in
> error, please
> delete it, including all copies, and
notify the
> sender of
> the error
> immediately.
>
>
> On Feb 26, 2013, at 8:26 AM, "Blodgett,
David"
> <address@hidden
<mailto:address@hidden> <mailto:address@hidden
<mailto:address@hidden>>
> <mailto:address@hidden
<mailto:address@hidden> <mailto:address@hidden
<mailto:address@hidden>>>> wrote:
>
> In this case, the data is 16 day
from about
> 2000 to
> present. So, on the order of 275 time
> steps. There are
> daily products and annual products
though,
> in wich case,
> we are looking 12 to 4000+ time steps.
>
> Jason, is the tile scheme you sent over
> used for all the
> different resolutions? Looking at a
few of
> them it looks
> like that is the case.
>
> - Dave
>
>
>
> On Tue, Feb 26, 2013 at 7:48 AM,
John Caron
> <address@hidden
<mailto:address@hidden>
> <mailto:address@hidden
<mailto:address@hidden>>
> <mailto:address@hidden
<mailto:address@hidden>
> <mailto:address@hidden
<mailto:address@hidden>>__>>
>
> wrote:
>
> Heres an image of this file.
>
> questions about aggregations:
how many
> time steps
> will be in each aggregation ?
how many
> different
> aggregations (tiles?) will you
serve?
>
>
>
> On 2/24/2013 6:55 PM, David
Blodgett wrote:
>
> Sweet, thanks John. There
was data
> in the tile,
> but was hard to find...
BORING, I
> know. Sorry
> about that.
>
> Here's a tile over the great
lakes
> to give you a
> coast to compare locations to.
>
>
http://e4ftl01.cr.usgs.gov/__MOLT/MOD13Q1.005/2000.03.05/__BROWSE.MOD13A1.A2000065.__h11v04.005.2008238024700.1.jpg
>
<http://e4ftl01.cr.usgs.gov/MOLT/MOD13Q1.005/2000.03.05/BROWSE.MOD13A1.A2000065.h11v04.005.2008238024700.1.jpg>
>
http://e4ftl01.cr.usgs.gov/__MOLT/MOD13Q1.005/2000.03.05/__MOD13Q1.A2000065.h11v04.005.__2008238031620.hdf
>
<http://e4ftl01.cr.usgs.gov/MOLT/MOD13Q1.005/2000.03.05/MOD13Q1.A2000065.h11v04.005.2008238031620.hdf>
>
http://e4ftl01.cr.usgs.gov/__MOLT/MOD13Q1.005/2000.03.05/__MOD13Q1.A2000065.h11v04.005.__2008238031620.hdf.xml
>
<http://e4ftl01.cr.usgs.gov/MOLT/MOD13Q1.005/2000.03.05/MOD13Q1.A2000065.h11v04.005.2008238031620.hdf.xml>
>
> - Dave
>
> On Feb 24, 2013, at 6:33 PM,
John
> Caron wrote:
>
> Hi:
>
> Im hacking away at this, but
> the datafiles i
> downloaded _seem_ to have
> nothing but missing
> values. Or im doing
something
> wrong. Can you
> send me a file that you know
> has real data in
> it. thanks
>
> John
>
> On 2/24/2013 9:22 AM, David
> Blodgett wrote:
>
> Hi John,
>
> Actually, I was assuming
> that the
> sinusoidal
projection was
> in there, doh!
> That was based on
> gdal_translate creating
> a geotiff with this
> projection info:
>
> PROJCS["unnamed",
> GEOGCS["Unknown
datum
> based upon the
> custom spheroid",
>
> DATUM["Not_specified_based_on___custom_spheroid",
>
SPHEROID["Custom
>
spheroid",6371007.181,0]],
>
PRIMEM["Greenwich",0],
>
> UNIT["degree",0.__0174532925199433]],
>
PROJECTION["Sinusoidal"],
>
> PARAMETER["longitude_of___center",0],
>
> PARAMETER["false_easting",0],
>
PARAMETER["false_northing",0],
> UNIT["metre",1,
>
AUTHORITY["EPSG","9001"]]]
>
> There's a georeferencing
> section here
>
> (http://www.gdal.org/frmt___hdf4.html
> <http://www.gdal.org/frmt_hdf4.html>) that
> has a bit more info
about
> gdal's
> implementation. Might be
> something
> helpful there? Looks
like
> the ODL text
> you point to is
where its
> coming from.
>
> 4 and 5 are
questions for
> Jason and maybe
> Jordan. Guys?
>
> - Dave
>
>
>
>
>
> On Feb 23, 2013, at 1:22
> PM, John Caron
> wrote:
>
>
> On 2/22/2013
2:07 PM,
> David Blodgett
> wrote:
>
> Welcome back
John.
> Hope you had a
> good trip!
>
> thanks!
>
> I actually just
> spent a bunch of
> time with
this the
> other day and have
> been meaning to
> summarize what
> I'm finding.
>
> The first
issue we
> are running
> into is your #1
> below. In this
> case there
> is an
external xml
> file that
> contains a
lot of
> metadata, including
> information
about
> the projection
>
(sinusoidal), which
> is currently
> unsupported by
> NetCDF-Java?
>
> 1) where do you get
> this xml file?
>
> 2) im looking at
>
>
> MOD13Q1.A2000065.h00v08.005.__2008238080422.hdf
>
> MOD13Q1.A2000065.h00v08.005.__2008238080422.hdf.xml
>
> in your union
> subdirectory. The xml
> file does not
seem to
> have projection
> info in it.
>
> 3) the internal
ODL text
> (StructMetadata) has
>
> ...
>
>
UpperLeftPointMtrs=(-20015109.__354000,1111950.519667)
>
> LowerRightMtrs=(-18903158.__834333,-0.000000)
>
Projection=GCTP_SNSOID
>
> ProjParams=(6371007.181000,0,__0,0,0,0,0,0,0,0,0,0,0)
>
> SphereCode=-1
>
> ...
>
> which may be
enough to
> calculate the
> lat/lon values.
Since
> this is the
> best choice, lets
> investigate it
> before we go
down any
> other paths.
>
> 4) Can anyone track
> down the code in
> the hyrax
handler that
> does this? Or
> does the hyrax
handler
> actually link
> in the hds-eos
libraries ??
>
> 5) how confident are
> you that all the
> files that you
want to
> aggregate have
> the same lat/lon
> geolocation ?
>
>
> John
>
>
>
>
>
>
>
>