NOTICE: This version of the NSF Unidata web site (archive.unidata.ucar.edu) is no longer being updated.
Current content can be found at unidata.ucar.edu.
To learn about what's going on, see About the Archive Site.
NOTICE: This version of the NSF Unidata web site (archive.unidata.ucar.edu) is no longer being updated.
Current content can be found at unidata.ucar.edu.
To learn about what's going on, see About the Archive Site.
"Jason J. Levit" wrote: > > Hi all, > > I'm writing an application that will process some real-time data I'm > ingesting through LDM. This application will run around the clock in > the cron, so it needs to be fairly robust. Here's my question - say I > ingest a file via LDM. How do I know that file is complete? Does LDM > place some sort of lock or other information on the file before it's > completed writing? If my application tries to access a file that does > not contain all the necessary data (a file that still has yet to be > completely transmitted, for example), then I need to have my application > not process the file. > > Does anyone else out there have experience with this? I know there's > tricks you can perform, such as checking to see if the file has been > modified in the past 30 seconds before opening, etc., etc., but I was > just wondering if LDM actually places some sort of information with the > file that says "hey, this file isn't done yet". Thanks for any help! > > Jason > > -- > ---------------------------------------------------------------------------- > Jason J. Levit, N9MLA Research Scientist, > jlevit@xxxxxx Center for Analysis and Prediction of Storms > Room 1022 University of Oklahoma > 405/325-3503 http://www.caps.ou.edu/ Hi Jason, I'm not sure if you're talking about when a product is received in the queue, or when a product is being written out to disk from the queue via pqact, although I'm guessing it's the latter. When the LDM receives a product in the queue, that product is not available to pqact or for retransmission to another site until the product is complete. For the purpose of retransmission or actions by pqact, you can consider the product to be an atomic unit. When pqact is writing the product out to file, it is unknown as to when the product is complete. (I am assuming that you're not talking about appending to a file.) This is a frequently asked question. You would have to devise some trick to figure that out. For example, if you know the structure of the product, you could write a program that simply writes out the product, but also identifies when the end of the product is reached and touches some other file to serve as a flag. You would then PIPE the product to your program via pqact. But, to answer your question, the LDM does not have any way to signal that pqact has or has not completely written the product to disk. Hope this helps! Anne -- *************************************************** Anne Wilson UCAR Unidata Program anne@xxxxxxxxxxxxxxxx P.O. Box 3000 Boulder, CO 80307 ---------------------------------------------------- Unidata WWW server http://www.unidata.ucar.edu/ ****************************************************
ldm-users archives: