Distributed mobile storage systems for non-uniform connectivity

Rapid growth in storage density is continuing to fuel dramatic improvements in cost, form-factor and capacity of storage devices. Compact mobile-storage elements are being embedded in all kinds of electronic devices, including laptops, PDAs, cameras, MP3 players, TV recorders, etc. The trend is toward a world where each user's personal data is distributed among a large number of storage devices, many of which are mobile and distributed across many locations. In this context, data management issues such as efficiently managing coherence, availability, reliability and sharing of data, become challenging. Unfortunately, the network-connectivity options available to help us deal with these challenges are less than perfect. High-bandwidth connectivity is generally available only in small neighborhoods. Although wide-area connectivity is increasingly ubiquitous, its bandwidth will always significantly lag behind that of the short-range options. Therefore, the systems that we design for this environment should be sensitive to this heterogeneity in networking technologies. In particular, the systems need to be judicious in their use of wide-area connectivity, and should rely on storage and short-range connectivity to overcome its limitations. In this thesis, we present two storage systems that deal with distribution and mobility in this environment. The first system, named PersonalRAID , is designed to manage storage on a set of computers distributed across multiple geographic locations by relying only on mobile storage devices and short-range connectivity. The core of this system is a distributed log-structured design, which organizes each storage element in the form of a log. It not only provides an efficient means for propagating updates among different storage elements, but also allows efficient direct I/O accesses to the logs without incurring unnecessary log replay delays. The second system, named Segank, is a generalization of PersonalRAID. By making complementary use of mobile storage, short-range connectivity and wide-area networks, it is able to provide much improved functionality. The Segank design features (1) a topology-aware multicast-like algorithm for locating data, (2) a data coherence mechanism, which enables lazy and independent propagation of data and meta-data in a peer-to-peer fashion, and (3) a distributed snapshot mechanism to support data sharing, which allows users to trade-off freshness of data for access efficiency.