請用此 Handle URI 來引用此文件：
http://tdr.lib.ntu.edu.tw/jspui/handle/123456789/93355完整後設資料紀錄
| DC 欄位 | 值 | 語言 |
|---|---|---|
| dc.contributor.advisor | 孫紹華 | zh_TW |
| dc.contributor.advisor | Shao-Hua Sun | en |
| dc.contributor.author | 洪正維 | zh_TW |
| dc.contributor.author | Cheng-Wei Hung | en |
| dc.date.accessioned | 2024-07-29T16:25:12Z | - |
| dc.date.available | 2024-07-30 | - |
| dc.date.copyright | 2024-07-29 | - |
| dc.date.issued | 2024 | - |
| dc.date.submitted | 2024-07-16 | - |
| dc.identifier.citation | References
[1] Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, et al. Training a helpful and harmless assistant with reinforcement learning from human feedback. arXiv preprint arXiv:2204.05862, 2022. [2] Matej Balog, Alexander Gaunt, Marc Brockschmidt, Sebastian Nowozin, and Daniel Tarlow. Deepcoder: Learning to write programs. In Proceedings of ICLR’17, 2017. [3] Rudy Bunel, Matthew Hausknecht, Jacob Devlin, Rishabh Singh, and Pushmeet Kohli. Leveraging grammar and reinforcement learning for neural program synthesis. In International Conference on Learning Representations, 2018. [4] Xiaocong Chen, Lina Yao, Julian McAuley, Guanglin Zhou, and Xianzhi Wang. Deep reinforcement learning in recommender systems: A survey and new perspectives. Knowledge-based systems, 2023. [5] Xinyun Chen, Maxwell Lin, Nathanael Schärli, and Denny Zhou. Teaching large language models to self-debug. In International Conference on Learning Representations, 2024. [6] Raphaël Dang-Nhu. Plans: Neuro-symbolic program learning from videos. In Advances in Neural Information Processing Systems, 2020. [7] Jacob Devlin, Jonathan Uesato, Surya Bhupatiraju, Rishabh Singh, Abdel-rahman Mohamed, and Pushmeet Kohli. Robustfill: Neural program learning under noisy i/o. In International Conference on Machine Learning, 2017. [8] Xuguang Duan, Qi Wu, Chuang Gan, Yiwei Zhang, Wenbing Huang, Anton van den Hengel, and Wenwu Zhu. Watch, reason and code: Learning to represent videos using program. In Proceedings of the 27th ACM International Conference on Multimedia, 2019. [9] Kevin Ellis, Catherine Wong, Maxwell Nye, Mathias Sablé-Meyer, Lucas Morales, Luke Hewitt, Luc Cary, Armando Solar-Lezama, and Joshua B Tenenbaum. Dreamcoder: Bootstrapping inductive program synthesis with wake-sleep library learning. In Proceedings of the 42nd acm sigplan international conference on programming language design and implementation, 2021. [10] Kavi Gupta, Peter Ebert Christensen, Xinyun Chen, and Dawn Song. Synthesize, execute and debug: Learning to repair for neural program synthesis. In Neural Information Processing Systems, 2020. [11] Vidhi Jain, Maria Attarian, Nikhil J Joshi, Ayzaan Wahid, Danny Driess, Quan Vuong, Pannag R Sanketi, Pierre Sermanet, Stefan Welker, Christine Chan, et al. Vid2robot: End-to-end video-conditioned policy learning with cross-attention transformers. arXiv preprint arXiv:2403.12943, 2024. [12] Hung Le, Yue Wang, Akhilesh Deepak Gotmare, Silvio Savarese, and Steven C. H. Hoi. Coderl: Mastering code generation through pretrained models and deep reinforcement learning. In Neural Information Processing Systems, 2022. [13] Wei-Hsu Lee. Enhancing program synthesis through fine-tuning techniques, contrastive learning, comprehensive training strategies, and real-world evaluation scenarios. Master’s thesis, National Taiwan University, 2024. [14] Guan-Ting Liu, En-Pei Hu, Pu-Jen Cheng, Hung-Yi Lee, and Shao-Hua Sun. Hierarchical programmatic reinforcement learning via learning to compose programs. In International Conference on Machine Learning, 2023. [15] Anthony Manchin, Jamie Sherrah, Qi Wu, and Anton van den Hengel. Program generation from diverse video demonstrations, 2023. [16] Keiron O’Shea and Ryan Nash. An introduction to convolutional neural networks, 2015. [17] Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback. In Neural Information Processing Systems, 2022. [18] Richard E Pattis. Karel the robot: a gentle introduction to the art of programming. John Wiley & Sons, Inc., 1981. [19] Kensen Shi, Joey Hong, Yinlin Deng, Pengcheng Yin, Manzil Zaheer, and Charles Sutton. Exedec: Execution decomposition for compositional generalization in neural program synthesis. In The Twelfth International Conference on Learning Representations, 2024. [20] Richard Shin, Neel Kant, Kavi Gupta, Christopher Bender, Brandon Trabucco, Rishabh Singh, and Dawn Song. Synthetic datasets for neural program synthesis. In International Conference on Learning Representations, 2019. [21] David Silver, Aja Huang, Chris J Maddison, Arthur Guez, Laurent Sifre, George Van Den Driessche, Julian Schrittwieser, Ioannis Antonoglou, Veda Panneershelvam, Marc Lanctot, et al. Mastering the game of go with deep neural networks and tree search. Nature, 2016. [22] Shao-Hua Sun, Hyeonwoo Noh, Sriram Somasundaram, and Joseph Lim. Neural program synthesis from diverse demonstration videos. In Proceedings of the 35th International Conference on Machine Learning, 2018. [23] Dweep Trivedi, Jesse Zhang, Shao-Hua Sun, and Joseph J Lim. Learning to synthesize programs as interpretable and generalizable policies. In Thirty-Fifth Conference on Neural Information Processing Systems, 2021. [24] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Ł ukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in Neural Information Processing Systems, 2017. [25] Letian Wang, Jie Liu, Hao Shao, Wenshuo Wang, Ruobing Chen, Yu Liu, and Steven L Waslander. Efficient reinforcement learning for autonomous driving with parameterized skills and priors. 2023. [26] Yue Wang, Hung Le, Akhilesh Deepak Gotmare, Nghi D.Q. Bui, Junnan Li, and Steven C. H. Hoi. Codet5+: Open code large language models for code understanding and generation. arXiv preprint, 2023. [27] Martin White, Michele Tufano, Matias Martinez, Martin Monperrus, and Denys Poshyvanyk. Sorting and transforming program repair ingredients via deep learning code similarities. In 2019 IEEE 26th International Conference on Software Analysis, Evolution and Reengineering (SANER), 2019. [28] Michihiro Yasunaga and Percy Liang. Graph-based, self-supervised program repair from diagnostic feedback. In International Conference on Machine Learning, 2020. [29] Shafiq Joty Steven C.H. Hoi Yue Wang, Weishi Wang. Codet5: Identifier-aware unified pretrained encoder-decoder models for code understanding and generation. In EMNLP, 2021. [30] Wojciech Zaremba, Tomas Mikolov, Armand Joulin, and Rob Fergus. Learning simple algorithms from examples, 2015. | - |
| dc.identifier.uri | http://tdr.lib.ntu.edu.tw/jspui/handle/123456789/93355 | - |
| dc.description.abstract | 設計和實現能夠模擬機器行為的程式是人工智慧研究的核心任務之一。傳統的自然語言處理（NLP）方法在生成可讀性高的人類文本方面表現優異，然而在程式執行的準確性和一致性方面通常存在顯著差距。相較之下，神經程式合成（NPS）方法能夠通過直接從規範或示例生成可執行的程式，從而更深入地理解和操作機器行為。傳統自然語言處理（NLP）與神經程式合成（NPS）之間的主要差異在於，後者能夠生成可執行的程式碼，這些程式碼可以執行並檢查其行為。這種能力不僅有助於驗證合成的解決方案，還為從執行軌跡中學習開闢了新的途徑。
在本論文中，我們探索神經程式合成（NPS）與程式執行之間的共生關係，突顯程式執行如何成為合成程式的核心要素。本論文包含兩個主要項目：第一個項目探討學習程式執行或合成是否有助於學習另一個方面。通過分析這兩個方面之間的相互作用，我們研究神經網路如何有效地理解並生成可執行程式碼。第二個項目專注於提升從多樣化示範影片中合成程式行為的效率和有效性，同時提出指標驗證其在實際情境中的應用性。透過結構分析、前例分析和實驗驗證，本論文旨在深化對「程式執行對神經程式合成影響」的理解，並探討程式表示和行為之間的複雜性，以促進創造更安全、更強大的人工智慧系統。 | zh_TW |
| dc.description.abstract | Understanding and synthesizing programs that represent machine behavior is a fundamental aspect of artificial intelligence research. Traditional approaches to natural language processing (NLP) have excelled in generating human-readable text but have largely remained detached from program execution. Conversely, Neural Program Synthesis (NPS) offers a paradigm shift by generating executable programs directly from specifications or examples, thereby enabling a deeper understanding and manipulation of machine behavior. The key difference between traditional NLP and NPS lies in the ability of the latter to generate executable code, which can be executed to examine its behavior. This capability not only facilitates the verification of synthesized solutions but also opens avenues for learning from execution traces.
In this thesis, we explore the symbiotic relationship between neural program synthesis (NPS) and program execution, highlighting how program execution serves as a fundamental aspect aiding in the synthesis of programs. This thesis comprises two main projects. The first project investigates whether learning program execution or synthesis aids in learning the other. By analyzing the interplay between these two aspects, we aim to uncover insights into how neural networks can effectively comprehend and generate executable code. The second project focuses on learning to synthesize programs from entire execution traces. We aim to enhance the efficiency and effectiveness of neural program synthesis from diverse demonstration videos and propose realistic metrics to examine whether the method can be applied in more practical scenarios. Through a combination of structural analysis, empirical studies, and experimental validation, this thesis seeks to advance our understanding of the impact of program execution on neural program synthesis. By delving into the intricacies between program representation and behavior, we aim to pave the way for more robust and intelligent AI systems. | en |
| dc.description.provenance | Submitted by admin ntu (admin@lib.ntu.edu.tw) on 2024-07-29T16:25:12Z No. of bitstreams: 0 | en |
| dc.description.provenance | Made available in DSpace on 2024-07-29T16:25:12Z (GMT). No. of bitstreams: 0 | en |
| dc.description.tableofcontents | Acknowledgements i
摘要 iii Abstract v Contents vii List of Figures xi List of Tables xiii Denotation xv Chapter 1 Introduction 1 1.1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1.1.1 Recent Advance in Deep Learning . . . . . . . . . . . . . . . . . . 1 1.1.2 Teach Neural Network to Learn Rules . . . . . . . . . . . . . . . . 2 1.1.3 Programming By Example . . . . . . . . . . . . . . . . . . . . . . 2 1.1.4 Neural Program Induction and Synthesis . . . . . . . . . . . . . . . 2 1.1.5 The Execution Property of Program . . . . . . . . . . . . . . . . . 3 1.2 Research . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 1.2.1 Bridging Neural Program Synthesis and Execution . . . . . . . . . 4 1.2.2 Demo2Program+: Improve Neural Program Synthesis from Diverse Demonstration Videos and Evaluate on Demonstrations of Goal-Oriented Task . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 Chapter 2 Literature Review 9 2.1 Neural Program Synthesis . . . . . . . . . . . . . . . . . . . . . . . 9 2.1.1 Search-Based Program Synthesis . . . . . . . . . . . . . . . . . . . 10 2.1.2 Seq2Seq - Recurrent Neural Networks . . . . . . . . . . . . . . . . 10 2.1.3 Seq2Seq - Transformer . . . . . . . . . . . . . . . . . . . . . . . . 11 2.2 Demo2Program . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 2.3 Programmatic Reinforcement Learning . . . . . . . . . . . . . . . . 14 Chapter 3 Bridging Neural Program Synthesis and Execution 15 3.1 Problem Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 3.1.1 Problem Formulation . . . . . . . . . . . . . . . . . . . . . . . . . 16 3.1.2 Domain Specific Language . . . . . . . . . . . . . . . . . . . . . . 16 3.1.3 Training Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 3.2 Methods and Preliminary Results . . . . . . . . . . . . . . . . . . . 18 3.2.1 Examine the Effects of the Attention Layer in Program Synthesizer and the Effects of the Scheduled Sampling . . . . . . . . . . . . . . 19 3.2.2 Experiment results on Neural Program Synthesis and Unsuccessful results on Neural Program Execution . . . . . . . . . . . . . . . . . 24 3.3 Discussion with Contributions from Team Member . . . . . . . . . . 25 Chapter 4 Demo2Program+ 27 4.1 Problem Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . 27 4.1.1 Karel Environment and Domain Specific Language . . . . . . . . . 27 4.1.2 Problem Formulation . . . . . . . . . . . . . . . . . . . . . . . . . 28 4.2 Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28 4.2.1 Utilizing Large Language Model as Our Encoder-Decoder . . . . . 29 4.2.2 Using Visual Words of Action/Perception as Model＇s Input . . . . 29 4.2.3 Training on Data with Better Generating Heuristic . . . . . . . . . . 30 4.2.4 Employing Action/Perception Prediction and Function/Branch Prediction as Auxiliary Objective Loss Functions . . . . . . . . . . . . 30 4.2.5 Utilizing ExeDec [19] to Break Down Long Horizon Execution Traces 31 4.3 Results and Discussion . . . . . . . . . . . . . . . . . . . . . . . . . 33 4.3.1 Synthesis Results (with method 1 and 2 . . . . . . . . . . . . . . . 33 4.3.2 Comparing the Dataset Generation Method (with method 3) . . . . . 34 4.3.3 Multi Task Objective Loss - function&branch (with method 4) . . . 34 4.3.4 ExeDec (with method 5) . . . . . . . . . . . . . . . . . . . . . . . 35 4.4 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35 References 37 Appendix A — Domain Specific Language 41 A.1 String Transformation Domain . . . . . . . . . . . . . . . . . . . . . 41 A.2 Karel Domain . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42 Appendix B — Program Aliasing 43 | - |
| dc.language.iso | en | - |
| dc.subject | 機器學習 | zh_TW |
| dc.subject | 程式合成 | zh_TW |
| dc.subject | Machine Learning | en |
| dc.subject | Neural Program Synthesis | en |
| dc.title | 探討程式執行對程式合成的影響 | zh_TW |
| dc.title | Investigating the Impact of Program Execution on Neural Program Synthesis | en |
| dc.type | Thesis | - |
| dc.date.schoolyear | 112-2 | - |
| dc.description.degree | 碩士 | - |
| dc.contributor.oralexamcommittee | 李宏毅;謝秉均 | zh_TW |
| dc.contributor.oralexamcommittee | Hung-Yi Lee;Ping-Chun Hsieh | en |
| dc.subject.keyword | 機器學習,程式合成, | zh_TW |
| dc.subject.keyword | Neural Program Synthesis,Machine Learning, | en |
| dc.relation.page | 43 | - |
| dc.identifier.doi | 10.6342/NTU202401358 | - |
| dc.rights.note | 同意授權(全球公開) | - |
| dc.date.accepted | 2024-07-16 | - |
| dc.contributor.author-college | 電機資訊學院 | - |
| dc.contributor.author-dept | 電信工程學研究所 | - |
| 顯示於系所單位： | 電信工程學研究所 | |
文件中的檔案：
| 檔案 | 大小 | 格式 | |
|---|---|---|---|
| ntu-112-2.pdf | 591.71 kB | Adobe PDF | 檢視/開啟 |
系統中的文件，除了特別指名其著作權條款之外，均受到著作權保護，並且保留所有的權利。