Modeling Of Marketing Processes Using Markov Decision Process Approach

In this paper, an application of Markov Decision Processes (MDP) for modeling selected marketing process is presented. The process is converted into MDP model, where states of the MDP are determined by a configuration of state vector. Elements of the state vector represent most important attributes of the customer in the modeled process. Movement between the states is determined by actions of the customer. In constructed MDP model, individual states with assigned initial reward values then represent consequences of action chosen by the customer resulting in either incresing or reducing the revenue following moving into these states. Value iteration method is then used for computation of the expected final rewards for each state. Based on provided realistic data, customer behavior is analyzed and the best course of action is proposed. Model suitability for future predictions of desired action outcome rate is discussed as well.