NOTICE: This version of the NSF Unidata web site (archive.unidata.ucar.edu) is no longer being updated.
Current content can be found at unidata.ucar.edu.
NOTICE: This version of the NSF Unidata web site (archive.unidata.ucar.edu) is no longer being updated.
Current content can be found at unidata.ucar.edu.
The architecture has been worked-out (multi-threaded, Java-based peers exchanging pieces of data amongst themselves and keeping track of things using a JDBC-based relational database).
The system design roadmap has been worked-out at a high level (i.e., what programs and threads must exist, where they must run, and with whom they must communicate).
The components are being designed and implemented using test-driven development. Currently, the package has 97 Java classes. The components that have been implemented and tested include the following:
Primitive classes (e.g., finite and infinite files, pieces of files, static and dynamic directories, communication connections)
Archive, a class capable of storing over 100 megabytes (460 data-pieces) per second on my workstation with 20 simulated remote peers. The archive comprises two sub-archives:
A data archive for the storing of files and their data-pieces.
A metadata archive for the storing of information about the files and what data-pieces they have. This archive is based on JDBC-compliant Derby and runs as a server so that it can handle multiple instances of the client program.
PieceDecider, a class that makes decisions about what data-pieces should be requested from what peer sites. It can handle over 10,000 such decisions per second on my workstation with 50 simulated remote peers.
NodeTree, a generalization of the individual files that are exchanged by programs such as BitTorrent to a dynamic hierarchy of files and directories. This is a crucial component.
Peer, a multi-threaded Java class for exchanging data-pieces with another version of itself running on another host.
The format of the XML files that are downloaded by the client have to worked-out and appropriate XML decoders must be written.
A class that manages all the peer connections of a client must be written.
A tracker program must be written. Clients will connect to this program in order to learn what peer sites have what desired data.
A data ingestion system must be written. It's possible that the current LDM could be used for this purpose, initially.
A pqact-like backend must be written.