請用此 Handle URI 來引用此文件:
http://tdr.lib.ntu.edu.tw/jspui/handle/123456789/6202
完整後設資料紀錄
DC 欄位 | 值 | 語言 |
---|---|---|
dc.contributor.advisor | 劉長遠(Cheng-Yuan Liou) | |
dc.contributor.author | Shin-Rong Huang | en |
dc.contributor.author | 黃信榮 | zh_TW |
dc.date.accessioned | 2021-05-16T16:23:01Z | - |
dc.date.available | 2018-07-26 | |
dc.date.available | 2021-05-16T16:23:01Z | - |
dc.date.copyright | 2013-07-26 | |
dc.date.issued | 2013 | |
dc.date.submitted | 2013-07-16 | |
dc.identifier.citation | [1] G. P. Baerends. The functional organization of behaviour. Animal Behaviour,
24:726–738, 1976. [2] B. Blumberg. Action-selection in hamsterdam: Lessons from ethology. In Proceed-ings of the third international conference on Simulation of adaptive behavior: from animals to animats, volume 3, pages 108–117. MIT Press, 1994. [3] B. Blumberg and T. A. Galyean. Multi-level direction of autonomous creatures for real-time virtual environments. In SIGGRAPH, pages 47–54, 1995. [4] B. M. Blumberg. No bad dogs: Ethological lessons for learning in hamsterdam. In In Proceedings of the Fourth International Conference on Simulation of Adaptive Behavior, pages 295–304. MIT Press, 1996. [5] R. A. Brooks. A robust layered control system for a mobile robot. Robotics and Automation, IEEE Journal of, 2(1):14–23, 1986. [6] C. L. Hull. Principles of behavior, an introduction to behavior theory. D. Appleton-Century Company, Incorporated, New York, London :, 1943. [7] C. Kline and B. Blumberg. The art and science of synthetic character design. In in Proceedings of the AISB 1999 Symposium on AI and Creativity in Entertainment and Visual Art, 1999. [8] C.-Y. Liou, C.-H. Tan, H.-T. Chen, and J.-H. Chen. Agents that have desires and behave adaptively. 1998. [9] A. R. Ludlow. The evolution and simulation of a decision maker. Academic Press, 1980. [10] P. Maes. Modeling adaptive autonomous agents. Artificial Life, 1:135–162, 1994. [11] D. J. McFarland and R. M. Sibly. The behavioural final common path. Philosoph-ical Transactions of the Royal Society of London. Series B, Biological Sciences, 270(907):265–293, 1975. [12] C. W. Reynolds. Flocks, herds and schools: A distributed behavioral model. In SIGGRAPH, pages 25–34, 1987. [13] J. Rosenblatt and D. Payton. A fine-grained alternative to the subsumption archi-tecture for mobile robot control. In Neural Networks, 1989. IJCNN., International Joint Conference on, volume 2, pages 317–323, 1989. [14] R. S. Sutton and A. G. Barto. Reinforcement learning: An introduction. IEEE Transactions on Neural Networks, 9(5):1054–1054, 1998. [15] D. Terzopoulos, X. Tu, and R. Grzeszczuk. Artificial fishes: Autonomous locomo-tion, perception, behavior, and learning in a simulated physical world. Artif. Life, 1(4):327–351, Jan. 1994. [16] N. Tinbergen. The study of instinct. Clarendon Press/Oxford University Press, 1951. [17] T. Tyrrell. The use of hierarchies for action selection. Adapt. Behav., 1(4):387–420, Apr. 1993. | |
dc.identifier.uri | http://tdr.lib.ntu.edu.tw/jspui/handle/123456789/6202 | - |
dc.description.abstract | 我們使用Blumberg的Model來實作出可以自動交易的agent,除此之外,我們加入了learning mechanism來讓agent擁有學習的功能,在learning mechanism 方面,我們implement Q-learning來讓agent可以根據每次的交易結果修正自己的Rule table,最後agent可以完成交易。
在Blumberg的Model裡面,Internal Variable一直扮演很重要的角色,它可以讓agent擁有類似人類的情緒或是慾望,例如如果agent對於時間很渴望的話,換句話說,希望交易可以在最短的時間裡完成,那麼它就會對於價格的堅持較沒有那麼顯著;相反的,如果agent對於價格的渴望較高的話,意思是說,不希望價格降低或是提高太多,那麼就算花較長的時間,價格的讓步還是有限,因此,我們選擇 Internal Variable 來做為學習(learning) 的參數。 我們選擇要實作的Blumberg的Model擁有四個不同的component。Internal Variable(IV) 是用來建構內在狀態,就如我們上段所說,可以根據不同時間下的growth rate and damping rate 和根據正在表現的行為而產生的影響來改變。Releasing Mechanism (RM) 是用來感知外在的刺激,它可以根據每個agent而擁有不同的感知能力。Level of Interest (LI) 是用來建構當時行為所發生的影響。IV、RM、LI這三個component綜合起來決定最後agent選擇哪一個Behavior。在選擇Behavior 的時候,我們會設定一個gain,如果有超過一個以上的Behavior不為零,我們就會利用gain來降低每個Behavior的值,直到出現只有一個Behavior的值不為零為止。 | zh_TW |
dc.description.abstract | In this paper, we present an agent-based electronic marketplace and the desired agents for negotiation. These agents will negotiate with each other in a human-like way. We implement the Blumberg's model and using reinforcement learning mechanism. We use Q-learning to train the internal variables of the agent. The behavior model with Q-Learning, which we propose, can adapt to many environments, such as finance , database, electronic marketplace, and transaction of stock. We will show the experiment of implementation for electronic marketplace. | en |
dc.description.provenance | Made available in DSpace on 2021-05-16T16:23:01Z (GMT). No. of bitstreams: 1 ntu-102-R99922116-1.pdf: 4738842 bytes, checksum: c8a5e5594a2646569f14b401aab0b81d (MD5) Previous issue date: 2013 | en |
dc.description.tableofcontents | 致 謝 i
中文 摘 要 iii Abstract v 1 Introduction 1 1.1 The architecture of the agent . . . . . . . . . . . . . . . . . . . . . . . . 2 1.2 Action selection module used by the desired agent . . . . . . . . . . . . . 2 1.3 The modification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 1.4 Learning Mechanism . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 1.5 Contribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 1.6 Road map . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 2 Mechanisms for Action Selection 7 2.1 Hull [1943] . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 2.2 Tinbergen[1950] . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 2.3 Lorenz[1950] . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 2.4 Baerends[1976] . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 2.5 Ludlow[1980] . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 2.6 Brooks[1986] . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 2.7 Rosenblatt and Payton[1989] . . . . . . . . . . . . . . . . . . . . . . . . 18 2.8 Tyrrell[1993] . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 2.9 Blumberg[1994] . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 3 Behavior System 25 3.1 Behavior module . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 3.2 Releasing Mechanism . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27 3.3 Internal Variable . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28 3.4 Level of interest . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28 3.5 The algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31 4 Reinforcement Learning 33 4.1 Stocastic cellular automata learning mechanism . . . . . . . . . . . . . . 33 4.2 Machine Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34 4.3 Exploration and Exploitation . . . . . . . . . . . . . . . . . . . . . . . . 35 4.4 Element of Reinforcement Learning . . . . . . . . . . . . . . . . . . . . 35 4.5 Value Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38 4.6 Temporal Difference Learning . . . . . . . . . . . . . . . . . . . . . . . 38 4.7 Action Selection Policies . . . . . . . . . . . . . . . . . . . . . . . . . . 40 4.8 On-Policy and Off-Policy Learning . . . . . . . . . . . . . . . . . . . . . 41 4.9 Q-Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41 5 An Electronic Marketplace Implementation 45 5.1 Market Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45 5.2 Q-Learning Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46 5.3 Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47 5.4 Experiment Result . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47 6 Behavior-based Model Discussion and Conclusion 55 6.1 No Bad Dogs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56 6.2 Agents that have desires and behave adaptively . . . . . . . . . . . . . . 57 6.3 Synthetic Character Design . . . . . . . . . . . . . . . . . . . . . . . . . 57 6.3.1 Motivational drives . . . . . . . . . . . . . . . . . . . . . . . . . 58 6.3.2 Emotions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58 6.3.3 Perception . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59 6.3.4 Action selection . . . . . . . . . . . . . . . . . . . . . . . . . . 59 6.4 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59 Bibliography 63 | |
dc.language.iso | en | |
dc.title | 學習行為系統用於自動機代理人 | zh_TW |
dc.title | Auto-agent with Learned Behaviors | en |
dc.type | Thesis | |
dc.date.schoolyear | 101-2 | |
dc.description.degree | 碩士 | |
dc.contributor.oralexamcommittee | 呂育道(Yuh-Dauh Lyuu),黃昭綺(Jau-Chi Huang),鄭為正(Wei-Chen Cheng) | |
dc.subject.keyword | 自動機代理人,機器學習, | zh_TW |
dc.subject.keyword | Behavior system,Reinforcement learning, | en |
dc.relation.page | 64 | |
dc.rights.note | 同意授權(全球公開) | |
dc.date.accepted | 2013-07-16 | |
dc.contributor.author-college | 電機資訊學院 | zh_TW |
dc.contributor.author-dept | 資訊工程學研究所 | zh_TW |
顯示於系所單位: | 資訊工程學系 |
文件中的檔案:
檔案 | 大小 | 格式 | |
---|---|---|---|
ntu-102-1.pdf | 4.63 MB | Adobe PDF | 檢視/開啟 |
系統中的文件,除了特別指名其著作權條款之外,均受到著作權保護,並且保留所有的權利。