使用馬可夫決策過程觀察會員行為並最適化行銷活動 - 以知名咖啡連鎖品牌為例

Abstract

馬可夫決策過程 (Markov Decision Process, MDP) 在關係行銷領域中是顧客生命週期價值 (Customer Lifetime Value, CLV) 最大化問題中的經典模型，許多學者在各式各樣的行銷情境中成功應用此模型協助企業做出更好的行銷決策。本研究使用某國內知名咖啡連鎖品牌 (C公司) 的App會員經營情境作為研究個案，並將研究過程分為兩階段：第一階段使用了馬可夫鏈模型捕捉隨機消費行為，以行為假設為基礎形成模型建構方法並完成會員消費行為的隨機模型後，利用模型預測假設行銷策略不變下的會員價值並觀察分析會員的轉移矩陣、利潤矩陣及長期穩定趨勢。第二階段延伸第一階段的馬可夫鏈，使用馬可夫決策過程建構行銷決策模型，並利用C公司App會員行銷決策情境作為模型設定，完成行銷決策模型後利用蒙特卡羅法模擬估計給定特定行銷策略下的顧客生命週期價值，求解預算限制下的最適行銷決策後，最後根據最適決策做出會員經營策略上的建議。 本研究的兩個重點為： 1. 將App會員消費行為建構成馬可夫鏈形式並做分析觀察。 2. 將App會員行銷實務情境建構成馬可夫決策過程形式並求出最適決策規劃。 研究的成果首先為成功的將C公司的會員行為以生命週期角度建構為馬可夫鏈的形式，透過模型預測顧客在不同折現率下未來一年的生命週期價值，並估計發現C公司目前的行銷策略確實可以在長期得到成效；後續成功的建構出馬可夫決策過程形式的行銷模型，找出了最能準確捕捉顧客動態的分群設定，模擬估計顧客價值並求得C公司未來一年的最適行銷決策。而此最適決策相較C公司過去的行銷策略，在本研究未來一年的模擬中可以顯著提升約2,500萬的顧客金額貢獻。

Traditionally, Markov Decision Process model (MDP) was on top of the models for maximizing Customer Lifetime Value (CLV) in relationship marketing. Many MDP applications have been successfully used in helping corporations with better marketing decisions. In this research, we do a two-phase analysis of a famous coffee shop’s member transactions dataset with Markov models. In phase 1, we introduced a Markov chain model for capturing the member’s dynamics by making some assumptions of its behavior. We further predict the CLVs of the member and observing their transitions and contributions with transition probability matrix and reward matrix. Also, the stationary distribution was used in capturing member’s dynamics and computing the expected retention probability. In phase 2, we build a decision model combined advanced MDP, Monte Carlo simulation, and portfolio optimization of the coffee chain. And also making suggestions of the marketing campaign based on the optimal policy from the model. There are two purposes of this paper: 1. Modeling member’s dynamics with Markov chain and making observations. 2. Use MDP in finding the optimal marketing policy under budget constraints. The results showed that Markov chain and MDPs are useful tools in capturing member dynamics in the context of member apps as illustrated with the coffee chain case study. Furthermore, the optimal policy obtained from MDP outperforms the historical policy according to the model simulations by about 25 million NT dollars.