A Permissioned Distributed Ledger for Monitoring Web Content

Online publishing of news and information enables important content to reach a much larger audience than traditional paper publishing, but there is no guarantee of long-term, reliable, and persistent access to the content. Over time, links "decay" because they are not reliably updated when content changes location, or even worse, content is deliberately altered from its original published form or deleted altogether. We present the design of Watchdog, a permissioned distributed ledger that securely and reliably monitors and preserves dynamic web content such as government and news sites. Unlike prior preservation systems, our system design allows nodes to be dispersed across multiple administrative domains, thus eliminating single points of trust and at the same time, is the first to monitor how content changes both over time and across geographical location. Watchdog achieves Byzantine fault-tolerance via a novel interactive consistency algorithm that offers a twist on the traditional definition and use of interactive consistency; the algorithm enables a set of mutually suspicious nodes, with arbitrary-sized, potentially overlapping data collections, to efficiently agree on the exact data collection in possession by each node. We demonstrate how our algorithm enables Watchdog nodes to exchange and agree upon the content each has observed individually, in a fault-tolerant, tamper-proof manner.