政大機構典藏-National Chengchi University Institutional Repository(NCCUR):Item 140.119/142638

政大機構典藏-National Chengchi University Institutional Repository(NCCUR):Item 140.119/142638

English | 正體中文 | 简体中文 | Post-Print筆數 : 27 | Items with full text/Total items : 113325/144300 (79%)
Visitors : 51156125 Online Users : 981

RC Version 6.0 © Powered By DSPACE, MIT. Enhanced by NTU Library IR team.

Scope

please add "double quotation mark" for query phrases to get precise results

please goto advance search for comprehansive author search

Adv. Search

Home ‧ Login ‧ Upload ‧ Help ‧ About ‧ Administer

Goto mobile version

政大典藏 > College of Informatics > Department of Computer Science > Theses > Item 140.119/142638

Please use this identifier to cite or link to this item: https://nccur.lib.nccu.edu.tw/handle/140.119/142638

Title:	透過強化學習設計合作遊戲的夥伴 Designing Game Companions in Cooperative Games Using Reinforcement Learning
Authors:	吳宥衡 Wu, You-Heng
Contributors:	李蔡彥 Li, Tsai-Yen 吳宥衡 Wu, You-Heng
Keywords:	人工智慧強化學習非玩家角色遊戲夥伴 Artificial Intelligent Reinforcement learning Non-player character Game companions
Date:	2022
Issue Date:	2022-12-02 15:19:54 (UTC+8)
Abstract:	電玩遊戲中與玩家互動的非玩家角色（Non-Player Character, NPC）一直是影響玩家遊戲體驗的要素，如何設計一個行為自然又能讓遊戲更加好玩的NPC遊戲夥伴不僅是遊戲業者一直以來努力的方向，也是許多玩家長年來的期待。本研究整理過去設計遊戲夥伴的相關文獻，探討玩家在玩遊戲過程中覺得好玩的原因，以及對遊戲夥伴的期待，發現大多數的玩家期望遊戲夥伴能觀察環境變化，並與玩家相互依賴合作。因此我們於Unity3D遊戲引擎設計一款雙人合作射擊遊戲，有別於過往使用強化學習設計完美通關遊戲的AI，本研究採用ML-Agents套件中近端策略優化（PPO）強化學習演算法的方式，一步一步讓遊戲夥伴學會新的遊戲技術，最後引導遊戲夥伴學會與玩家合作通關遊戲。本研究實驗請20名受試者分別與合作版本與非合作版本的遊戲夥伴一同闖關，透過受試者在實驗後給予的回饋，實驗結果也顯示了大多數玩家認為若遊戲夥伴能在遊戲過程中關注自身的狀態，並且在雙方有難時互相合作，可以更加有助於遊戲的正向體驗。 The non-player character (NPC) that interacts with players in video games has al-ways been an element that affects the players` game experience. How to design an NPC game companion that behaves naturally and makes the games more interesting is not only what the game designers’ striving for, but also the expectation of many players for a long time. This study tries to figure out the reasons why players feel interested in playing games and their expectations of game companions. We have found that most players look forward to game companions to observe changes in the environment and to rely on and cooperate with players. Therefore, we designed a two-player cooperative shooting game in the Unity3D game engine. Differing from using traditional reinforce-ment learning to design game agents in the past, we use proximal policy optimization (PPO) algorithm with ML-Agents toolkit to design our game companions. We try to make game companions learn game skills step by step, and finally learn how to cooper-ate with players to clear the game. We invited twenty participants to participate in our experiment. The participants were asked to play our shooting games in the cooperative version and the non-cooperative version with game companions, respectively. Through the feedback given by the participants, the experimental results show that most players believe that if the game companions can pay attention to players’ state during the games, and cooperate with each other in trouble, it will contribute to more positive playing ex-periences of the game.
Reference:	[1] E. Bouquet, V. Mäkelä, and A. Schmidt, "Exploring the Design of Companions in Video Games," in Academic Mindtrek 2021, 2021, pp. 145-153. [2] K. Emmerich, P. Ring, and M. Masuch, "I`m Glad You Are on My Side: How to Design Compelling Game Companions," in Proceedings of the 2018 Annual Symposium on Computer-Human Interaction in Play, 2018, pp. 141-152. [3] N. Afonso and R. Prada, "Agents that relate: Improving the social believability of non-player characters in role-playing games," in Proceedings of International Conference on Entertainment Computing, 2008: Springer, pp. 34-45. [4] A. Chowanda, M. Flintham, P. Blanchfield, and M. Valstar, "Playing with social and emotional game companions," in Proceedings of International Conference on Intelligent Virtual Agents, 2016: Springer, pp. 85-95. [5] M. Csikszentmihalyi and M. Csikzentmihaly, Flow: The psychology of optimal experience. Harper & Row New York, 1990. [6] J. Chen, "Flow in games (and everything else)," Communications of the ACM, vol. 50, no. 4, pp. 31-34, 2007. [7] M. P. Silva, V. do Nascimento Silva, and L. Chaimowicz, "Dynamic difficulty adjustment through an adaptive AI," in 2015 14th Brazilian symposium on computer games and digital entertainment (SBGames), 2015: IEEE, pp. 173-182. [8] M. P. Silva, V. do Nascimento Silva, and L. Chaimowicz, "Dynamic difficulty adjustment on MOBA games," Entertainment Computing, vol. 18, pp. 103-123, 2017. [9] J. Tremblay and C. Verbrugge, "Adaptive companions in FPS games, "in Proceedings of International Conference on Foundations of Digital Games , vol. 13, pp. 229-236, 2013.. [10] A. Sharifi, R. Zhao, and D. Szafron, "Learning companion behaviors using reinforcement learning in games," in Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment, 2010, vol. 5, no. 1. [11] M. Smith, S. Lee-Urban, and H. Muñoz-Avila, "RETALIATE: Learning winning policies in first-person shooter games," in Proceedings of the AAAI Conference on Artificial Intelligence, 2007, pp. 1801-1806. [12] 王宇軒, "多重代理人之策略競爭遊戲之強化學習方法," 碩士論文, 國立東海大學資訊科學系, 2019. [13] D. Piergigli, L. A. Ripamonti, D. Maggiorini, and D. Gadia, "Deep Reinforcement Learning to train agents in a multiplayer First Person Shooter: some preliminary results," in Proceedings of 2019 IEEE Conference on Games (CoG), 2019: IEEE, pp. 1-8. [14] R. S. Sutton and A. G. Barto, Reinforcement learning: An introduction. MIT press, 2018. [15] J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, "Proximal policy optimization algorithms," arXiv preprint arXiv:1707.06347, 2017. [16] W. IJsselsteijn et al., "Measuring the Experience of Digital Game Enjoyment," in Proceedings of measuring behavior Conference, 2008, Noldus Maastricht, the Netherlands, pp. 88-89. [17] Poels, K., de Kort, Y. A. W., & IJsselsteijn, W. A. (2007). D3.3 : Game Experience Questionnaire: development of a self-report measure to assess the psychological impact of digital games. Technische Universiteit Eindhoven. [18] R. Likert, "A technique for the measurement of attitudes," Archives of psychology, 1932. [19] S. S. Shapiro and M. B. Wilk, "An analysis of variance test for normality (complete samples)," Biometrika, vol. 52, no. 3/4, pp. 591-611, 1965. [20] F. Wilcoxon, "Individual comparisons by ranking methods," Biometrics Bulletin vol. 1, no. 6 (Dec., 1945), pp. 80-83
Description:	碩士國立政治大學資訊科學系 108753123
Source URI:	http://thesis.lib.nccu.edu.tw/record/#G0108753123
Data Type:	thesis
DOI:	10.6814/NCCU202201687
Appears in Collections:	[Department of Computer Science ] Theses

Files in This Item:

File	Description	Size	Format
312301.pdf		2784Kb	Adobe PDF2	141	View/Open

All items in 政大典藏 are protected by copyright, with all rights reserved.

社群 sharing

著作權政策宣告 Copyright Announcement

1.本網站之數位內容為國立政治大學所收錄之機構典藏，無償提供學術研究與公眾教育等公益性使用，惟仍請適度，合理使用本網站之內容，以尊重著作權人之權益。商業上之利用，則請先取得著作權人之授權。
The digital content of this website is part of National Chengchi University Institutional Repository. It provides free access to academic research and public education for non-commercial use. Please utilize it in a proper and reasonable manner and respect the rights of copyright owners. For commercial use, please obtain authorization from the copyright owner in advance.

2.本網站之製作，已盡力防止侵害著作權人之權益，如仍發現本網站之數位內容有侵害著作權人權益情事者，請權利人通知本網站維護人員(nccur@nccu.edu.tw)，維護人員將立即採取移除該數位著作等補救措施。
NCCU Institutional Repository is made to protect the interests of copyright owners. If you believe that any material on the website infringes copyright, please contact our staff(nccur@nccu.edu.tw). We will remove the work from the repository and investigate your claim.

DSpace Software Copyright © 2002-2004 MIT & Hewlett-Packard / Enhanced by NTU Library IR team Copyright © - Feedback