NOTICE: This version of the NSF Unidata web site (archive.unidata.ucar.edu) is no longer being updated.
Current content can be found at unidata.ucar.edu.

To learn about what's going on, see About the Archive Site.


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[TIGGE #BXZ-922182]: Re: .missing files & retransmissions



Baudouin,

We've imagined three ways to address your data-verification issue:

1.  Add an option to the LDM to compute the MD5 signature and compare it.

This is the one I originally imagined.  It should be relatively easy to 
implement.  It has a disadvantage of computing the MD5 signature for *all* data 
feeds -- regardless of how important they are.

2.  Modify the grammar of the LDM configuration-file (etc/ldmd.conf) to allow 
for the computation and checking of MD5 signatures on an individual REQUEST and 
ACCEPT basis.

If the LDM is to do the verification, then this is a better solution than #1.  
It would take longer to implement.  It also doesn't address the problem of 
verifying data-products after they've been written to disk.  Which brings us to 

3.  Have the ingester add the MD5 signature to a manifest-file, which is 
transmitted separately, and have a pqact(1)-based mechanism on the receiving 
end use the manifest-file to verify correct reception.

Since you're already using (or going to use) a CONDUIT-like manifest-file 
mechanism to verify reception, this mechanism would seem to be a natural 
extension of that solution.  Besides providing end-to-end verification of 
transmission, it also allows continued confidence in the data-products after 
they've been stored in the archive.

We recommend solution #3 but would like to know what you think.
Regards,
Steve Emmerson

Ticket Details
===================
Ticket ID: BXZ-922182
Department: Support IDD TIGGE
Priority: Normal
Status: On Hold