Discovering patterns on financial data streams

With the increasing amount of data in financial market, there are two types of data streams attracting a lot of research and studies, time series index stream and related news stream. In this thesis, we focus on discovering patterns from these data streams and try to answer the following challenging questions, (I) given two co-evolving time series indices, what is the co-movement dependency between them. (II) given a set of evolving time series, could we detect some leaders from them whose rise or fall impacts the behavior of many other time series? (III) could we integrate the news stream information into stock price prediction? (IV) could we integrate the news stream information into stock risk analysis? and (V) could we detect what are those events that trigger time series index movement. For each of the question, we design algorithms and address three technique issues (I) how to detect promising patterns from the noisy financial data; (II) how to update the old patterns when new data arrives in high frequency; (III) how to use the pattern to support the financial applications. We start from investigating the co-movement relationship of multiple time series. We propose techniques to study two aspects of this problem. First, we propose a co-movement model for constructing financial portfolio by analyzing and mining the co-movement patterns among two time series. Second, we presents an efficient streaming algorithm to discover leaders from multiple time series stream. Both of the algorithms are evaluated using real time series indices data and the result proves that co-movement patterns and detected leaders are promising and can support various applications including portfolio management, high frequency trading and risk management. Then, we consider the patterns between news stream and time series indices stream. We first transform the news stream into a set of bursty feature (keywords) time series streams and propose three technique to study their relationship to time series index. First, we explore a Non-homogeneous Hidden Markov Model (NHMM) to predict the stock market process which takes both stock prices and news articles into consideration. Second, we propose a risk analytical model to predict the volatility of price indices by integrating news information. Finally, we devise an algorithm to detect the priming event from text and a time series index. The evaluation on real world dataset suggests the significant correlation exists between news stream and time series stream and our pattern discover algorithm can detect promising patterns from this relationship to support real world applications effectively.