整合機器學習與啟發式技術建構顧客洞察分析架構之量化研究

Abstract

本研究針對顧客旅程管理、最佳化顧客購買週期三階段規劃、「顧客-通路-產品」理論與變數探討、精準行銷模式建構、顧客終身價值評估、多通路價值最佳化、精準研發模式建構、產品（工業）效用評估及產品規格推薦等實證研究，提出了一個「整合性機器學習與啟發式演算法最佳化技術」的顧客洞察分析系統。該系統以企業真實工作場景和數據為基礎，針對金融業和有終端消費性產品之製造業進行電子商務場域驗證的實證研究。本研究強調的是實務應用，而非方法論的突破，透過應用過往文獻中顧客旅程（Customer Journey）和新產品開發流程（New Product Development）的理論觀點，提出一個整合性分析框架，驗證兩個研究案例，並補足文獻中處理動態數據和最佳化企業資源配置分析方法之缺口。 第一項研究的驗證結果顯示，通過整合分析台灣金融保險行業的多種電子商務顧客數據，將顧客旅程購買週期三階段的動態分析納入現有的顧客價值估計模型，分析各種類型之線上通路的顧客通路互動、點擊行為數據，確實可以最佳化顧客終身價值，同時降低企業的整體轉換成本，並可更深入地了解顧客群體、購買行為、通路接觸點等對電子商務轉換的影響。研究框架涵蓋了各種技術，如集群分析、機器學習、深度學習、整數規劃與二進制差分進化法，目標變數包括點擊、立即購買、轉換收益與顧客終身價值。資料量約為140萬次流量訪問、110萬用戶和56,000筆商務交易的通路互動、點擊、轉換等分析數據。 第二項研究的驗證結果顯示，通過整合自然語言處理技術中的潛在狄利克雷分配主題分析及梯度提升決策樹技術，可以準確地預測亞馬遜消費者評分。並透過與製造公司合作，提出了一個數據驅動的產品服務系統，通過引入品質機能展開程序，根據研發流程自動產生顧客需求與功能重要性排序、顧客需求與產品功能對應之主題特徵矩陣，以及最佳化產品規格推薦，使企業能夠即時識別關鍵研發規格，快速展開新產品開發規劃，藉以支持顧客驅動製造（C2M）商業模式。研究資料量包括從2021年1月到2022年8月的76個產品類別中3,492,632條產品評論觀察記錄，以及來自亞馬遜運營商的銷售數據。研究整合了結構化和半結構化數據，並使用LDA和LightGBM模型進行分析。為了確保模型的準確性，建立了六項評估指標，並在製造應用中進行了概念驗證（POC）。 藉由所提供的多通路顧客價值最佳化解決方案與最佳化產品規格推薦解決方案，分別解決了金控公司數位行銷部門常見的營運管理問題（如顧客價值預測、會員經營管理、行銷通路自動配置、通路資源管理準則）以及製造公司研發部門常見的產品研發問題（如顧客需求分析、產品功能模組設計、新產品規格推薦、產品研發管理準則），對企業在系統化管理顧客購買週期、最佳化顧客旅程管理及產品創新與敏捷開發方面皆有實質性的幫助。機器學習預測和啟發式最佳化技術的整合，提高了動態數據分析和強化企業資源分配的能力。同時，顧客洞察分析系統的分析框架為企業系統提供了數值數據、非結構化分析和自動化分析能力。這為管理科學方法在電子商務研究領域的實務運用提供了參考，該數據分析框架適用於產業中的電子商務部門。

This study focuses on customer journey management, three-stage planning for optimizing the customer purchase cycle, the exploration of the "customer-channel-product" theory and variables, the construction of precision marketing models, customer lifetime value assessment, multi-channel value optimization, precision R&D models, product (industrial) utility evaluation, and product specification recommendations. It proposes a customer insights analysis system that integrates machine learning and heuristic optimization techniques. Based on real-world enterprise scenarios and data, this system is validated through empirical research in the e-commerce domains of the financial industry and manufacturing industries with end-consumer products. This research emphasizes practical applications rather than methodological breakthroughs by applying theoretical perspectives from previous literature on customer journey and new product development processes. It proposes an integrated analytical framework, verifies two research cases, and addresses gaps in the literature concerning dynamic data handling and enterprise resource optimization analysis methods. The validation results of the first study show that by integrating and analyzing various e-commerce customer data from the Taiwanese financial insurance industry, incorporating dynamic analysis of the three-stage customer purchase cycle into existing customer value estimation models, and analyzing customer interactions and click behavior data across various online channels, customer lifetime value can indeed be optimized while reducing overall conversion costs for enterprises. Additionally, it provides deeper insights into customer groups, purchase behavior, and the impact of channel touchpoints on e-commerce conversion. The research framework encompasses various techniques such as cluster analysis, machine learning, deep learning, integer programming, and binary differential evolution, targeting variables including clicks, immediate purchases, conversion revenue, and customer lifetime value. The dataset includes approximately 1.4 million traffic visits, 1.1 million users, and 56,000 business transaction interactions, clicks, and conversion data. The validation results of the second study demonstrate that by integrating Latent Dirichlet Allocation (LDA) topic analysis from natural language processing technology with gradient boosting decision tree techniques, Amazon consumer ratings can be accurately predicted. By collaborating with manufacturing companies, a data-driven product-service system is proposed. This system, through the introduction of the Quality Function Deployment (QFD) process, automatically generates customer needs and function importance rankings, customer needs and product function correspondence topic feature matrices, and optimized product specification recommendations. This enables enterprises to promptly identify key R&D specifications and quickly initiate new product development planning, thereby supporting customer-driven manufacturing (C2M) business models. The research data includes 3,492,632 product review observation records from 76 product categories spanning from January 2021 to August 2022, along with sales data from Amazon operators. The study integrates structured and semi-structured data and employs LDA and LightGBM models for analysis. To ensure model accuracy, six evaluation metrics were established, and a proof of concept (POC) was conducted in manufacturing applications. By providing multi-channel customer value optimization solutions and optimized product specification recommendation solutions, the study addresses common operational management issues in the digital marketing departments of financial holding companies (such as customer value prediction, member management, marketing channel auto-configuration, and channel resource management guidelines) and common product development issues in manufacturing R&D departments (such as customer needs analysis, product function module design, new product specification recommendations, and product R&D management guidelines). This has substantial benefits for enterprises in the systematic management of customer purchase cycles, optimized customer journey management, and product innovation and agile development. The integration of machine learning prediction and heuristic optimization techniques enhances dynamic data analysis and strengthens enterprise resource allocation capabilities. Simultaneously, the analytical framework for customer insights systems provides numerical data, unstructured analysis, and automated analytical capabilities for enterprise systems. This offers a reference for the practical application of management science methods in the field of e-commerce research, and the data analysis framework is applicable to e-commerce departments in various industries.