NOTICE: This version of the NSF Unidata web site (archive.unidata.ucar.edu) is no longer being updated.
Current content can be found at unidata.ucar.edu.

To learn about what's going on, see About the Archive Site.


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: MD5 checksum, deduplication, etc.



Stonie,

>Date: Thu, 10 Jun 2004 17:54:52 +0000
>From: Stonie Cooper <address@hidden>
>Organization: Planetary Data, Incorporated
>To: Steve Chiswell <address@hidden>,
>To: address@hidden
>Subject: MD5 checksum, deduplication, etc.

The above message contained the following:

> Have a quick question.  We have a config for streaming some data to LDM 
> (6.0.14) that sends the same product twice to the queue - but applies a 
> different feedtype and sequence number.
> 
> Unfortunately, the MD5 dedup catches the duplicate and throws it away.
> 
> Is there a way to turn deduplication off . . .

There's no way to disable the check of the MD5 checksum on the
downstream LDM without modifying the code.

> or use the feedtype and/or 
> sequence number in the MD5 to get a unique product?

Threre's no way for an LDM to change the MD5 checksum without modifying
the code.  Of course, the data-product ingester can do whatever it wants.

> It happens to be GINI 
> imagery . . . so applying a faux sequence number in a faux FOS header is not 
> an option.

Sounds like the ingester isn't under your control.

There might be another way to accomplish the larger task.  What are you
trying to do for which duplicate data-products appears to be a
solution?

> Thanks for any advise.
> -- 
> Stonie R. Cooper
> Planetary Data, Incorporated
> (402) 727-6599

Regards,
Steve Emmerson