Data management in peer-to-peer systems

The peer-to-peer (P2P) model of distributed computation is characterized by three laws: (i) all peers are born equal; (ii) the set of participant peers is dynamic and may be arbitrarily large; and (iii) peers are distributed over a wide-area network. These three laws pose significant challenges to the development of efficient data-management solutions in the P2P model. The first law requires decentralized design with no reliance on central servers, as well as load balance across peers. The second law requires designing for failures being the norm rather than an exception. The third law requires optimization of data placement and communication for a heterogeneous underlying network where the latency and communication bandwidth between different pairs of peers may be different. We discuss how these challenges may be met in enabling efficient data management solutions over P2P systems. Some of the highlights of our contributions include: (1) Hierarchical overlay-network design optimized for the underlying physical network with improved fault isolation support for efficient data caching. (2) Novel load-balancing algorithms for range-partitioned data that enables efficient range queries. (3) Data-partitioning techniques for multi-dimensional data which, combined with novel routing structures, enable efficient multi-dimensional queries. (4) Indexing schemes for dynamic data that can trade off the cost of maintaining the index against the benefit that the index offers for queries.