NOTICE: This version of the NSF Unidata web site (archive.unidata.ucar.edu) is no longer being updated.
Current content can be found at unidata.ucar.edu.
To learn about what's going on, see About the Archive Site.
NOTICE: This version of the NSF Unidata web site (archive.unidata.ucar.edu) is no longer being updated.
Current content can be found at unidata.ucar.edu.
To learn about what's going on, see About the Archive Site.
Hi all, I was wondering if anyone had tried to interface LDM into a database program? I was thinking of embarking on trying to have LDM interface with mysql ( a free database software package) and have it database all of the WMO products. Just wanted to see if it had been done before.. Thanks! Mike Dross mwdross@xxxxxxxxxxxxxxx
I have tried that before. It does work...sort of. The problem is that the volume of data is considerable. It must be stored in BLOBs because of the long variable-length nature of the data. Purging the old data leaves the database filled with many holes which don't get reused efficiently. Therefore you have to frequently perform a reorganize operation on the database to repack it and reclaim the space. (More recent versions of mysql let you do this on the fly, whereas perviously you had to shut down the server to do this) Another thing to consider is that on many platforms, you are limited to a 2GB file, which limits the total size of the database (all tables' data are stored in the same file and using multiple databases as opposed to tables is not convenient in SQL). One hint is that you need to keep the metadata and data in separate tables...i.e. keep the WMO headers, dates, etc in one table and use and index number to the actual data blobs in another table. If you put it all in the same table, searching the metadata becomes seriously degraded in performance. I think this is workable if you stick to a reasonable subset of the available products however, perhaps just the text based products, for example. One note...I tried this a while back and bigger and faster machines are available now and considerable performance improvements have been made to mysql, so it may work better now. On the other hand, the size of the datastreams have increased significantly as well. I plan to revisit the issue sometime soon. -- -------------------------------------------------------- David Wojtowicz, Sr. Research Programmer Department of Atmospheric Sciences Computer Services University of Illinois at Urbana-Champaign email: davidw@xxxxxxxx phone: (217)333-8390 --------------------------------------------------------
ldm-users archives: