Pulse of the City: Spatio-Temporal Twitter Content Analysis

The continuous, online processing of streaming data generated by smart city settings offers opportunities for richer, data-driven understanding of modern urban life. In this paper, we focus on the application of machine learning for highly localized classification of a broad set of urban activities from streaming twitter social media data. We build a platform for continuous ingestion of twitter data, spatio-temporal clustering of the tweets, and online topic and sub-topic extraction through hierarchical classification. We demonstrate results within Manhattan, and show that we can identify urban activities at the spatio-temporal resolution of 3 hours within a single city block. We also demonstrate how our streaming platform can be extended to support multi-modal sensing across dense urban environments, and scale to cover vast geographical areas.