請用此 Handle URI 來引用此文件：
http://tdr.lib.ntu.edu.tw/jspui/handle/123456789/100216完整後設資料紀錄
| DC 欄位 | 值 | 語言 |
|---|---|---|
| dc.contributor.advisor | 李家岩 | zh_TW |
| dc.contributor.advisor | Chia-Yen Lee | en |
| dc.contributor.author | 李郁恩 | zh_TW |
| dc.contributor.author | Yu-En Li | en |
| dc.date.accessioned | 2025-09-24T16:53:05Z | - |
| dc.date.available | 2025-09-25 | - |
| dc.date.copyright | 2025-09-24 | - |
| dc.date.issued | 2025 | - |
| dc.date.submitted | 2025-08-12 | - |
| dc.identifier.citation | Akiba, T., Sano, S., Yanase, T., Ohta, T., & Koyama, M. (2019). Optuna: A next-generation hyperparameter optimization framework. Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining.
Anderson, M., & Hanish, C. K. (2007). An evaluation of the benefits of integrating run-to-run control with scheduling and dispatching systems. IEEE Transactions on Semiconductor Manufacturing, 20(4), 386–392. Besnard, J., Gleispach, D., Gris, H., Ferreira, A., Roussy, A., Kernaflen, C., & Hayderer, G. (2012). Virtual metrology modeling for cvd film thickness. International Journal of Control Science and Engineering, 2(3), 26–33. Bu, L., Ching, W. L., Ling, H. S., Rhee, M. W., & Fen, Y. P. (2016). 3-D modeling and characterization for die attach process. IEEE Transactions on Components, Packaging and Manufacturing Technology, 6(10), 1567–1575. Butler, S. W., & Stefani, J. A. (1994). Supervisory run-to-run control of polysilicon gate etch using in situ ellipsometry. IEEE Transactions on Semiconductor Manufacturing, 7(2), 193–201. Chan, M. W., Lam, S. K., Yau, W. Y., & Cheung, K. Y. (2013). Bond line thickness control for die attachment. In: Google Patents. Chen, C.-H., Zhao, W.-D., Pang, T., & Lin, Y.-Z. (2020). Virtual metrology of semiconductor PVD process based on combination of tree-based ensemble model. ISA transactions, 103, 192–202. Chin, H. S., Cheong, K. Y., & Ismail, A. B. (2010). A review on die attach materials for SiC-based high-temperature power devices. Metallurgical and Materials Transactions B, 41(4), 824–832. Choi, J., Choi, G. S., & An, S. J. (2019). Reliable low-temperature die attach process using Ag/Sn/Ag sandwich structure for high-temperature semiconductor devices. Scientific reports, 9(1), 555. Fan, S.-K. S., & Chang, Y.-J. (2010). Multiple-input multiple-output double exponentially weighted moving average controller using partial least squares. Journal of Process Control, 20(6), 734–742. Guo, F., Zhou, X., Liu, J., Zhang, Y., Li, D., & Zhou, H. (2019). A reinforcement learning decision model for online process parameters optimization from offline data in injection molding. Applied Soft Computing, 85, 105828. Haarnoja, T., Zhou, A., Abbeel, P., & Levine, S. (2018). Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor. International conference on machine learning. Hasselt, H. (2010). Double Q-learning. Advances in neural information processing systems, 23. He, W., Gao, H., Zhou, C., Yang, C., & Li, Z. (2020). Reinforcement learning control of a flexible two-link manipulator: An experimental investigation. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 51(12), 7326–7336. Huang, D., & Lv, J. (2020). Run-to-run control of batch production process in manufacturing systems based on online measurement. Computers & Industrial Engineering, 141, 106298. Janikow, C. Z., & Michalewicz, Z. (1991). An experimental comparison of binary and floating point representations in genetic algorithms. ICGA. Jianlin, W., Yaoyu, X., Tao, Y., & Liqiang, Z. (2010). Run-to-run optimization for fed-batch fermentation process with swarm energy conservation particle swarm optimization algorithm. Chinese Journal of Chemical Engineering, 18(5), 787–794. Kaelbling, L. P., Littman, M. L., & Moore, A. W. (1996). Reinforcement learning: A survey. Journal of artificial intelligence research, 4, 237–285. Kang, P., Kim, D., Lee, H.-j., Doh, S., & Cho, S. (2011). Virtual metrology for run-to-run control in semiconductor manufacturing. Expert Systems with Applications, 38(3), 2508–2522. Kang, S., An, D., & Rim, J. (2019). Incorporating virtual metrology into failure prediction. IEEE Transactions on Semiconductor Manufacturing, 32(4), 553–558. Khan, A. A., Moyne, J. R., & Tilbury, D. M. (2008). Virtual metrology and feedback control for semiconductor manufacturing processes using recursive partial least squares. Journal of Process Control, 18(10), 961–974. KORABI, T. E., BORODIN, V., Michel, J., & ROUSSY, A. (2021). A hybrid feature selection approach for virtual metrology: Application to CMP process. 2021 32nd Annual SEMI advanced semiconductor manufacturing conference (ASMC). Lee, M. (2006). The squeezing process of complex epoxy patterns in the die-attaching of large IC-chip. 2006 International Conference on Electronic Materials and Packaging. Lenz, B., & Barak, B. (2013). Data mining and support vector regression machine learning in semiconductor manufacturing to improve virtual metrology. 2013 46th Hawaii international conference on system sciences. Li, Y., Du, J., & Jiang, W. (2024). Reinforcement learning for process control with application in semiconductor manufacturing. IISE Transactions, 56(6), 585–599. Li, Z., Wu, D., & Yu, T. (2019). Prediction of material removal rate for chemical mechanical planarization using decision tree-based ensemble learning. Journal of Manufacturing Science and Engineering, 141(3), 031003. Liu, K., Chen, Y., Zhang, T., Tian, S., & Zhang, X. (2018). A survey of run-to-run control for batch processes. ISA transactions, 83, 107–125. Lundberg, S. M., & Lee, S.-I. (2017). A unified approach to interpreting model predictions. Advances in neural information processing systems, 30. Ma, Z., & Pan, T. (2021). A quota-ddpg controller for run-to-run control. 2021 China Automation Congress (CAC). Ma, Z., & Pan, T. (2023). Distributional reinforcement learning for run-to-run control in semiconductor manufacturing processes. Neural Computing and Applications, 35(26), 19337–19350. Maitra, V., Su, Y., & Shi, J. (2024). Virtual metrology in semiconductor manufacturing: Current status and future prospects. Expert Systems with Applications, 123559. Orji, N. G., Badaroglu, M., Barnes, B. M., Beitia, C., Bunday, B. D., Celano, U., Kline, R. J., Neisser, M., Obeng, Y., & Vladar, A. (2018). Metrology for the next generation of semiconductor devices. Nature electronics, 1(10), 532–547. Pandey, V., Wang, E., & Boyles, S. D. (2020). Deep reinforcement learning algorithm for dynamic pricing of express lanes with multiple access locations. Transportation Research Part C: Emerging Technologies, 119, 102715. Raffin, A., Hill, A., Gleave, A., Kanervisto, A., Ernestus, M., & Dormann, N. (2021). Stable-baselines3: Reliable reinforcement learning implementations. Journal of machine learning research, 22(268), 1–8. Sutton, R., & Barto, A. (2018). Reinforcement Learning second edition: An Introduction. In ser. Adaptive Computation and Machine Learning series. MIT Press. Tsang, M., Cheng, D., & Liu, Y. (2017). Detecting statistical interactions from neural network weights. arXiv preprint arXiv:1705.04977. Wang, K., & Han, K. (2013). A batch-based run-to-run process control scheme for semiconductor manufacturing. IIE Transactions, 45(6), 658–669. Wu, S.-R., Chen, C.-F., Luoh, T., Yang, L.-W., Yang, T., & Chen, K.-C. (2016). Virtual metrology for 3D vertical stacking processes in semiconductor manufacturing. 2016 e-Manufacturing and Design Collaboration Symposium (eMDC). Yu, J., & Guo, P. (2020). Run-to-run control of chemical mechanical polishing process based on deep reinforcement learning. IEEE Transactions on Semiconductor Manufacturing, 33(3), 454–465. Zhang, F., Jiang, W., & Wang, H. (2021). Virtual metrology for semiconductor chemical mechanical planarization process using wide & deep learning. Proceedings of the 2021 10th International Conference on Computing and Pattern Recognition. | - |
| dc.identifier.uri | http://tdr.lib.ntu.edu.tw/jspui/handle/123456789/100216 | - |
| dc.description.abstract | 在半導體製程中，製程參數的控制對於提升產品良率與品質具有關鍵影響。本研究提出一套基於深度強化學習的製程參數控制框架，應用於黏晶製程中關鍵品質指標，包括溢膠距離與膠厚的優化控制。該框架整合虛擬量測模型與溫度變化模擬機制，以產生半合成品質指標，進一步設計同時考量品質偏差與參數邊界限制的獎勵函數，引導代理人有效調整製程參數。
實驗結果顯示，所提出的深度強化學習控制器在穩定性與整體表現上均優於傳統的 d-EWMA 方法。在控制策略方面，單一參數調整於多數情境中表現更佳，於個別實驗中可達60.6%的總獎勵提升，並同時降低品質偏差。在預測模型方面，MLP在多數情境中表現優於XGBoost，總獎勵提升達64.4%。此外，固定初始條件有助於強化訓練穩定性，總獎勵較隨機初始化提升達50.1%。雖然整體趨勢在不同產品間大致一致，仍觀察到部分例外，凸顯於強化學習控制策略設計中，需同時考量產品特性與訓練條件的搭配。 | zh_TW |
| dc.description.abstract | In semiconductor manufacturing, the control of process parameters plays a critical role in improving product yield and quality. This study proposes a deep reinforcement learning (DRL)-based control framework, applied to the die bonding process for optimizing key quality indicators, including Epoxy Bleeding Overflow (EBO) distances and Bond Line Thickness (BLT). The framework integrates a virtual metrology model with a temperature fluctuation simulation mechanism to generate semi-synthetic quality indicator values, which supports the design of a reward function that accounts for both quality deviations and process parameter bound constraints. This reward function guides the agent in effectively adjusting process parameters.
Experimental results show that the DRL controller outperforms the traditional d-EWMA method in both stability and quality performance. Regarding control strategies, single-parameter adjustment outperforms multi-parameter adjustment in most cases, yielding up to a 60.6% increase in total reward and a reduction in quality deviations. The MLP outperforms XGBoost in most scenarios, achieving a total reward improvement of up to 64.4%. Fixed initialization further enhances convergence, showing a 50.1% increase in total reward compared to random initialization. While consistent trends are observed across products, certain exceptions underscore the importance of considering product-specific characteristics and training conditions. | en |
| dc.description.provenance | Submitted by admin ntu (admin@lib.ntu.edu.tw) on 2025-09-24T16:53:05Z No. of bitstreams: 0 | en |
| dc.description.provenance | Made available in DSpace on 2025-09-24T16:53:05Z (GMT). No. of bitstreams: 0 | en |
| dc.description.tableofcontents | 謝辭 i
摘要 ii Abstract iii Contents iv List of Figures viii List of Tables x Chapter 1 Introduction 1 1.1 Background 1 1.2 Motivation 2 1.3 Research Objectives 4 1.4 Thesis Architecture 5 Chapter 2 Literature Review 7 2.1 Virtual Metrology in Semiconductor Manufacturing 7 2.1.1 General Applications of Virtual Metrology 7 2.1.2 Virtual Metrology for EBO and BLT Prediction 9 2.2 Run-to-Run Control in Semiconductor Manufacturing 10 2.3 Reinforcement Learning 12 2.4 Summary and Discussion 15 Chapter 3 Virtual Metrology and Simulation 18 3.1 Problem Formulation for EBP module 18 3.2 Data Preprocessing 20 3.2.1 Feature Transformation 21 3.2.2 Feature Selection 21 3.3 Predictive Models 22 3.3.1 Machine Learning Models 22 3.3.2 Multilayer Perceptron 23 3.3.3 Summary 24 3.4 Experimental Setup 24 3.4.1 Data Source 25 3.4.2 Model Evaluation 25 3.4.3 Evaluation Metrics 26 3.5 Results and Discussion 27 3.5.1 Prediction Results 27 3.5.2 Feature Importance Analysis 30 3.5.3 Interaction Analysis 33 3.6 Simulation of Temperature Effect 34 3.6.1 Assumptions and Temperature Modeling 34 3.6.2 Simulation Procedure 36 Chapter 4 Deep Reinforcement Learning 38 4.1 DRL-Based Process Parameter Control Framework 38 4.2 Definition of the MDP Elements 40 4.3 Deep Reinforcement Learning Algorithm 45 4.4 Experimental Setup 49 4.4.1 Experimental Scenarios 49 4.4.2 Environment Configuration 53 4.4.3 Baseline Method 55 4.4.4 Hyperparameters Optimization 56 4.4.4.1 Tuned Settings and Search Ranges 56 4.4.4.2 Sensitivity Analysis of Hyperparameters 58 4.5 Results and Discussion 60 4.5.1 Training Convergence 60 4.5.2 Performance in Testing Environment 64 4.5.3 Policy Visualization and Behavioral Analysis 73 4.5.4 Supplementary Analysis: Effect of Action Clipping 79 4.5.4.1 Clip Rate Analysis in Training Scenarios 79 4.5.4.2 No-Clip Supplementary Experiment in Scenario 7 80 4.5.4.3 Summary 81 Chapter 5 Conclusion and Future Work 82 5.1 Conclusion 82 5.2 Limitation 83 5.3 Future Work 84 Reference 86 Appendix 92 Appendix A Feature Interaction Tables 92 Appendix B Process Settings and Target Quality Values 93 B.1 Bounds and Step Sizes 93 B.2 Target Quality Output Values 93 Appendix C Additional Single-Product Results 94 C.1 Product B 94 C.1.1 Training Convergence 94 C.1.2 Performance in Testing Environment 96 C.2 Product C 98 C.2.1 Training Convergence 98 C.2.2 Performance in Testing Environment 100 Appendix D Analysis of Action Clipping Effects 102 D.1 Average Clip Rate Across Scenarios 102 D.2 Clip Rate Trend in Scenario 7 104 D.3 Supplementary Experiment Design 105 | - |
| dc.language.iso | en | - |
| dc.subject | 黏晶製程 | zh_TW |
| dc.subject | 虛擬量測 | zh_TW |
| dc.subject | 機器學習 | zh_TW |
| dc.subject | 深度強化學習 | zh_TW |
| dc.subject | 製程控制 | zh_TW |
| dc.subject | process control | en |
| dc.subject | die bonding process | en |
| dc.subject | virtual metrology | en |
| dc.subject | machine learning | en |
| dc.subject | deep reinforcement learning | en |
| dc.title | 應用柔性強化學習於黏晶製程參數控制以改善溢膠品質 | zh_TW |
| dc.title | Soft Reinforcement Learning for Die Bonding Process Parameter Control to Improve Epoxy Bleed Out Quality | en |
| dc.type | Thesis | - |
| dc.date.schoolyear | 113-2 | - |
| dc.description.degree | 碩士 | - |
| dc.contributor.oralexamcommittee | 孫紹華;陳以錚;陳建錦 | zh_TW |
| dc.contributor.oralexamcommittee | Shao-Hua Sun;Yi-Cheng Chen;Chien-Chin Chen | en |
| dc.subject.keyword | 黏晶製程,虛擬量測,機器學習,深度強化學習,製程控制, | zh_TW |
| dc.subject.keyword | die bonding process,virtual metrology,machine learning,deep reinforcement learning,process control, | en |
| dc.relation.page | 105 | - |
| dc.identifier.doi | 10.6342/NTU202502896 | - |
| dc.rights.note | 同意授權(全球公開) | - |
| dc.date.accepted | 2025-08-14 | - |
| dc.contributor.author-college | 管理學院 | - |
| dc.contributor.author-dept | 資訊管理學系 | - |
| dc.date.embargo-lift | 2027-08-05 | - |
| 顯示於系所單位： | 資訊管理學系 | |
文件中的檔案：
| 檔案 | 大小 | 格式 | |
|---|---|---|---|
| ntu-113-2.pdf 此日期後於網路公開 2027-08-05 | 4.59 MB | Adobe PDF |
系統中的文件，除了特別指名其著作權條款之外，均受到著作權保護，並且保留所有的權利。