Developing a 5v5 Framework for DotA 2 Bot Competition

(1)

Developing a 5v5 Framework for DotA 2 Bot

Competition

Dennis Nilsson

Malmö University bubblan.dn@gmail.com

Kalle Lindqvist

Malmö University kalle.lindqvist@mykolab.com Faculty of Science

Department of Computer Science

Master Thesis in Computer Science, 15 credits 2020-05-18

Supervisors: José Maria Font Fernandez & Alberto Enrique Alvarez Uribe Examiner: Johan Holmgren

(2)

CONTENTS

I Introduction 3

I-A Motivation . . . 3

I-B Goals . . . 4

I-C Research Questions . . . 4

II Literature Review 4 II-A Data collection . . . 4

II-A1 Game Conferences . . . 4

II-B Systematic search . . . 4

II-B1 Keywords, search string and restrictions . . . 5

II-B2 Databases . . . 5

II-B3 Selection of papers . . . 5

II-C Discussion and results . . . 5

II-C1 Applying artificial intelligence algorithms in MOBA games . . . 5

II-C2 Evolving a Designer-Balanced Neural Network for Ms PacMan . . . 6

II-C3 An Analysis of Artificial Intelligence Techniques in Multiplayer Online Battle Arena Game Environments . . . 6

II-C4 MOBA: a New Arena for Game AI . . . 6

II-C5 MOBA games: A literature review . . . 7

II-C6 On The Development of Intelligent Agents For MOBA . . . 7

II-C7 What Contributes to Success in MOBA games? An Empirical Study Of Defense of the Ancients 2 . . . 7

III Research Method 7 III-A Design Science . . . 7

III-B Experiment . . . 8

III-C Threats to validity . . . 8

IV Frameworks 8 IV-A 1v1 framework . . . 8

IV-B 5v5 framework . . . 10

IV-B1 DotA 2 Addon . . . 10

IV-B2 Web API . . . 10

IV-B3 Client . . . 10

V Experiments 10 V-A Experiment 1 - Hero movement . . . 11

V-B Experiment 2 - Level up hero abilities . . . 11

V-C Experiment 3 - Hero attack . . . 11

V-D Experiment 4 - Buy items . . . 11

V-E Experiment 5 - Sell items . . . 11

V-F Experiment 6 - Use items . . . 11

V-G Experiment 7 - Target Types . . . 11

V-G1 Experiment 7.1 - No Target . . . 11

V-G2 Experiment 7.2 - Toggle . . . 11

V-G3 Experiment 7.3 - Target Point . . . 11

V-G4 Experiment 7.4 - Target Area . . . 11

V-G5 Experiment 7.5 - Target Unit . . . 11

V-G6 Experiment 7.6 - Vector Targeting . . . 12

V-G7 Experiment 7.7 - Combination of Target Point or Target Unit . . . 12

V-G8 Experiment 7.8 - Target Unit with Area of Effect . . . 12

V-H Experiment 8 - Validating usage of all heroes . . . 12

(3)

VI Experiment results 12

VI-A Experiment 1 - Hero movement . . . 12

VI-B Experiment 2 - Level up hero abilities . . . 12

VI-C Experiment 3 - Hero attack . . . 13

VI-D Experiment 4 - Buy items . . . 13

VI-E Experiment 5 - Sell items . . . 13

VI-F Experiment 6 - Use items . . . 13

VI-G Experiment 7 - Target types . . . 13

VI-H Experiment 7.1 - No Target . . . 13

VI-I Experiment 7.2 - Toggle . . . 13

VI-J Experiment 7.3 - Target Point . . . 14

VI-K Experiment 7.4 - Target Area . . . 14

VI-L Experiment 7.5 - Target Unit . . . 14

VI-M Experiment 7.6 - Vector Targeting . . . 14

VI-N Experiment 7.7 - Combination of Target Point and Target Unit . . . 14

VI-O Experiment 7.8 - Combination of Target Unit and Target Area . . . 14

VI-P Experiment 8 - Full length match . . . 15

VI-Q Experiment 9 - Validating usage of all heroes . . . 15

VII Discussion 15 VII-A Future work . . . 15

VIII Conclusion 15

(4)

Abstract—Multiplayer online battle arena (MOBA) games have properties that make them suitable for research in artificial and computational intelligence. MOBA games are generally played team vs team meaning planning and organizing the team is one of the most important aspects. This makes it suitable for research in artificial intelligence agent cooperation.

This paper presents a literature review performed in the area of artificial and computational intelligence regarding MOBA games. The findings are that there is little or no research made in the area. The research found concerning MOBA games is in the context of player behaviour and toxic behaviour in the MOBA game league of legends. What is found is encouragement on developing frameworks supporting 5v5 matches in MOBA games which this paper also presents, a 5v5 framework for bot-matchmaking in DotA 2 making it possible for users to develop bots to be run versus the in-game AI.

Keywords—ai agent framework, DotA 2, moba, agent cooper-ation

I. INTRODUCTION

The topic for this thesis is research within computational intelligence and artificial intelligence. Artificial intelligence has been used to play games for a long time. As mentioned by Julian Togelius in [1], Alan Turing re-invented the Minimax algorithm to play Chess. Furthermore it sheds light on the suitability games provide for artificial intelligence research. This is due to their ability of challenging players and training the cognitive capabilities of players [1]. Different games provide different challenges. Some games challenge one player and some games challenge a team of players. Therefore the research that can be done could differ depending on which game is chosen.

In this thesis a framework has been developed to provide a platform for research of artificial intelligence teamwork. The chosen game for this framework is DotA 2. Conveniently a framework exists for DotA 2 but it can only handle 1v1 matches which takes away the ability to research teamwork of artificial intelligent agents.

DotA 2 is a MOBA game where the ultimate goal is to destroy the opponent’s base. The opponent’s base, called Ancient, is positioned at the opposite end of the map and outputs creeps that pushes the lanes during the game. Creeps are in-game controlled AI units that are either neutral or lane creeps. Neutral creeps are found in the jungle in small camps and provide the possibility for players to farm experience and gold. The lane creeps, spawning every 30 seconds, pushes all lanes automatically towards the enemy’s Ancient. A match traditionally consists of two teams each having their own base to protect and an opponent’s base to attack. Each team consists of five players. Each player controls their own hero i.e. a playable character with multiple abilities and different play styles. Each player is assigned a courier, used to buy and transport items to the player, enabling the player to stay in it’s lane without going back to base to buy items. A hero levels up during the match, increasing it’s strength and abilities. Furthermore the heroes can be equipped with items that are bought with earned gold. The gold is earned by farming, destroying enemy towers and/or heroes. Each team shares three paths leading to the enemy’s Ancient structure. These

paths are referred to as lanes. The three lanes are called bot lane, mid lane, top lane. Figure 1 gives a graphic representation of the DotA 2 map, presenting the meaning of lanes and towers. Mid lane is usually occupied by 1 player from each team. Bot and top lane is usually occupied by 2 players from each team. On top of this there is also a jungle in the map where players can kill neutral creeps for more experience and gold. Players can make their own modifications to the game in the programming language Lua [2]. The Lua sandbox, included in the workshop tools that is downloadable content, provides a comprehensive API to control units and view the game state [3].

Fig. 1. Overview of the DotA 2 map. Radiant team is marked in green and Dire team in red. The circles in green and red represent the teams defending towers [4]

A. Motivation

Ms Pac-Man [5] is a framework for competition in Ms Pac-Man. This is partly used for competition because of the properties of the game. Research within computational and artificial intelligence fields is made by the help of this framework. The complexity of the game is fitting to research due to its challenges in real-time decision-making and hetero-geneous player types. There are other AI-agent frameworks used in competitions around the world such as the TAC SCM framework [6]. The idea behind these competitions is to use them as a testbeds for new techniques and algorithms that can be applied, not just to game AI field, but rather the field of AI research [7]. As an example, in 2017, researchers at Microsoft developed an AI-agent for Ms Pac-Man that scored the highest score ever. Rahul Mehrotra [8] who is a product manager and AI-researcher at Microsoft later stated that the algorithm developed to beat Ms. Pac-Man could be used to help a company’s sales organization make precise predictions about which potential customers to target at a particular time or a particular day [9].

(5)

A framework supporting 1v1 matches currently exists for DotA 2, made by Tobias Mahlmann [10]. This is a good starting point but since it’s only possible to use in 1v1 it lacks the ability of teamwork and therefore also the possibility of testing algorithms that involve multiple agent cooperation and team decision making. Teamwork is an essential part of DotA 2 in which matches usually aren’t played in 1v1. To be able to evaluate computational and artificial intelligence concerning teamwork in DotA 2 a framework supporting 5v5 matches must therefore be developed.

Another important aspect is that the prize pool in DotA 2 [11] is very large and increases every year. Every year the base prize pool is set at $1,600,000. The year 2019, the prize pool went up to $34,330,068 compared to 2018’s prize pool of $25,532,177. The importance of this aspect is that it implies the popularity of the game. There is a project, OpenAI Five, being used for developing AI systems using DotA 2 as a research platform. On April 13th 2019, OpenAI Five defeated the current world champions in a best-of-three match. This was the first time an AI system won against world champions in an esports game. Furthermore it won over 99.4% of encountered human players in a showcase that lasted a couple of days. The name OpenAI Five is somewhat confusing due to the fact that it is unavailable to the public which is easily misinterpreted due to its name. This prevents others from performing research using the OpenAI Five framework which is one of the strong motivations for developing our 5v5 framework [12].

B. Goals

The main goal of this thesis is to upgrade the framework that was developed by Tobias Mahlmann [10] to be able to handle 5v5 matches with bot communication in DotA 2. The framework [13] in its current state uses a client/server implementation that communicates via HTTP calls. The client hosts a web service that the server sends information to in JSON format using HTTP POST calls and get responses/ac-tions using HTTP GET calls. There is an example Client developed in Java by the author of the framework but the client code can be written in any language as long as it implements the functionality required to handle the communication with the server. The subgoals are making sure that all components function as intended. This includes that our artifact allows all heroes to execute all of their functionalities i.e. their individual abilities and basic actions. Furthermore validating that the communication works as intended between the bots within the framework.

C. Research Questions

1) How can the 1v1 DotA 2 AI Framework project be extended to support 5v5?

2) How can the intelligent agents communicate with each other in the extended framework?

3) How could a multiple agent strategy be implemented in the 5v5 framework?

II. LITERATUREREVIEW

A. Data collection

The data has been collected from game conferences and by conducting a systematic literature review.

1) Game Conferences: Competitions held at IEEE COG 2019 [14], one competition that was held is not mentioned here due to its irrelevance.

• 4th Angry Birds Level Generation Competition [15]

• Bot Bowl I [16]

• Fighting Game AI Competition [17]

• First TextWorld Problems: A Reinforcement and

Lan-guage Learning Challenge [18]

• Geometry Friends Game AI Competition [19] • General Video Game AI Competitions [20] • Hanabi Competition [21]

• Hearthstone AI competition [22]

• microRTS AI Competition [23]

• StarCraft AI Competition [24]

• Strategy Card Game AI Competition [25] Competitions held at FDG 2019 [26]

• The Codenames AI Competition [27] • Fifth Aeon Collectible Card Game [28] • GDMC / Chronicle Challenge [29]

Coveted competition papers sought for at TCIAIG 2019 [30].

• Angry Birds Level Generation

• Computer Game Olympiads (including Chess, Amazons, Backgammon, Bridge, Chinese Chess, Dots and Boxes, Draughts, Go, LOA, Shogi, ...)

• DotA 2 Bot • Fighting Game AI • Game Data Mining • General Video Game AI

• Geometry Friends Cooperative Game AI

• microRTS AI

• Ms. Pac-Man VS Ghost Team

• Showdown AI

• StarCraft AI

• Text-based Adventure AI

• Visual Doom AI

As evident by our findings in these game conferences, it lacks a 5v5 DotA 2 bot competition which is due to the fact that a 5v5 framework does not exist for public use. TCIAIG called for DotA 2 bots submissions in 2019 but did not receive any submissions. This is possibly because a 5v5 framework was not available. A 1v1 framework was available at the time [10].

B. Systematic search

In addition to the data collected from the game conferences, a systematic literature review was conducted to find additional research related to our topic.

(6)

TABLE I LIST OF KEYWORDS

Keyword Synonyms

Defence of the ancients 2 DotA 2 Heroes of Newerth HoN Ms Pacman

Minecraft

League of legends LOL Framework

MOBA Intelligent agents

1) Keywords, search string and restrictions: The keywords used to compile the search string are the following and were extracted from the research questions and motivation:

The search string was compiled using the keywords listed in table I and using Boolean operators to make it easy for the reader to understand what was searched for as recommended by Neiva and Silva [31]. The way it’s used in this literature review is the same way an SQL query is executed. The following search string was used:

• ((Defence of the ancients 2 OR DotA 2) OR (Heroes of

Newerth OR HON) OR (Ms Pacman) OR (Minecraft) OR (League of legends OR LOL) OR MOBA) and (Framework OR Intelligence agents)

The search string was adapted to the mechanics used to search each of the different databases. The search string was used to match abstracts, titles and keywords of potential papers.

In addition to the search string the following restrictions were applied when searching the sources for articles:

• Only articles written in English

• Only articles referenced in at least one other confer-ence(s) or journal(s)

The reason for the first restriction is that English is the language this thesis is written in and most works in the field of computer science are published in English. The second restriction is used as a simple quality assurance, if a paper is referenced by external source(s) we can at least be reasonably sure that it is reviewed by one other party.

2) Databases: The following three databases were used in this literature review:

1) Google Scholar [32] 2) ACM [33]

3) IEEE Xplore [34]

We selected ACM and IEEE Xplore since they, to our knowledge, contain the largest amount of published literature in the field of computer science. Additionally, to cover the most ground possible, Google Scholar was used which aggre-gates articles from multiple sources.

3) Selection of papers: The gathered results were subjected to a two stage analysis that ultimately determined if they would be included in this paper. The goal was to select at least 5 articles containing the most relevant information related to our research.

TABLE II

THE INITIAL SEARCH RESULTS

Database Results

ACM 16

IEEE Xplore 8 Google Scholar 50k+

Total 25 + 50k+

In the first stage the titles and abstracts of the articles found were read during the initial search and a decision was made to determine if there was a chance the article could contribute anything to our research. The ones that seemed promising moved on to the second stage.

In the second stage each of the remaining articles were read. If an article contained information that were relevant to our research it was included in our paper.

TABLE III

RESULTS AFTER THE FIRST AND SECOND STAGE OF THE SELECTION OF PAPERS

Source Initial results First stage Second stage

ACM 16 3 1

IEEE Xplore 8 4 2

Google Scholar 50k+ 7 4

Total 24 + 50k+ 14 7

As can be seen in table III the search in Google Scholar yielded a massive amount of articles and since it’s not feasible to read them all, we read the results in sequence until the results were no longer relevant.

TABLE IV

THE FINAL RESULTS AND THE SOURCES THEY WERE FOUND IN

Source Reference ACM [35] IEEE Xplore [36] IEEE Xplore [37] Google Scholar [38] Google Scholar [39] Google Scholar [40] Google Scholar [41]

C. Discussion and results

As evident by the findings in the game conferences and literature review a framework supporting 5v5 in DotA 2 could not be found. Submissions for a competition in a 1v1 match in DotA 2 was sought for by TCIAIG 2019 [30] but none were received. This possibly depends on DotA 2 being a game that relies on teamwork and at the time the framework only supported 1v1 matches. For those who wanted to compete with their AI in 1v1 matches there are possibly more interesting competitions to join such as the Starcraft AI competition.

1) Applying artificial intelligence algorithms in MOBA games: In their paper on AI algorithms in MOBA games [38], authors Niewiadomski et al. highlight the fact that most players in MOBA games prefer to play against bots rather than

(7)

human competition due to the widespread cyberbullying in online MOBA games. This further strengthens our claim that a 5v5 bot framework is needed. Related to our specific topic, Niewiadomski et al. also highlight the need for a framework to have fast response times and provide quick updates on the state of the game so the agent can make the needed sub-second decisions needed in the heat of the battle. Furthermore the paper concludes, by experiments, that the most efficient algorithm in terms of winning the game for map navigation out of Bruce Force, Directional, Split, Monte Carlo and GA, the Split algorithm was the most efficient. The authors describe the algorithm in the following way:

“The Split algorithm starts from computing single steps using all available directions. Then, every obtained point is explored further, but only in two, new directions. An angle formed between the new directions is the same as the angles between the subsequent directions of the first step” [38].

2) Evolving a Designer-Balanced Neural Network for Ms PacMan: In their paper [36] Morsan et al. presents a tool that combines several 3rd party tools made by researchers into a common interface. This tool can be used to generate AI agents of different levels of difficulty for any game. The tool requires a lot of user input and validation to work but the authors claim that it’s a great way to develop AI agents quickly. The authors validate their tool with experiments generating agents for the game Ms Pac-Man.

3) An Analysis of Artificial Intelligence Techniques in Mul-tiplayer Online Battle Arena Game Environments: Waltham et al. tried to incorporate an emotional element in their paper about AI techniques in MOBA games [35]. The authors hypothesis was that by creating an agent that behaves more like a human with emotions instead of performing as efficiently as possible. It would perhaps offer a more interesting experience for the human player(s) who play with or against it. This technique seems promising because more and more players are playing against bots instead of human competition due to toxic behavior in online games against human opponents [38]. An agent that mimics human behavior might offer a more engaging experience for those players.

4) MOBA: a New Arena for Game AI: This paper investi-gates properties that MOBA games can provide to Game AI research. They mention the following phase challenges.

• Pick and Ban Phase, character picks are important

be-cause they can possibly decide the outcome of the match. The challenge of the AI agent is to select a composition of heroes being able to cooperate with each other, be effective on their own and be chosen in regard to the opposite team’s chosen heroes.

• Opening Phase, the first items the hero buys determines its success on it’s lane. The optimal situation for a hero when buying the first items would be to know which hero it would meet on it’s lane. Through experience a human player can guess which hero it would meet on it’s lane. The challenges, for an AI system, in this context is selecting the first items to buy, selecting five heroes and determining which lanes the heroes should occupy.

To realise this they suggest a system that predicts the op-posing team’s strategy i.e. which lanes their heroes would occupy. On top of this they suggest a counter strategy in case the initial guess is not correct, to minimize the risk of losing the match.

• Laning Phase, the challenge here is to determine when to buy new items, when to fight enemy heroes and where to position the hero. Furthermore to be aware of the team heroes status, for instance if they are in a fight and in need of assistance.

• Mid Game, when mid game is reached, the laning phase is traditionally over. The challenges of this phase can differ in many ways because it depends on several factors. However they state that the most important aspect is the cooperation of the team and if they are in sync. Regarding this they present that it is a difficult task due to the complexity of actions that can be made by the players, both opposing and friendly.

• Late Game, last phase of the match. The challenges found in this phase is fast decision making. The complexity of the combats increases due to the capacity of the heroes. In late game, the heroes are stronger and able to fight in more complex ways. Due to the complexity, the challenges of the AI agents would be to make decisions and predict enemy strategies as fast as a professional human player would.

• Side Challenges, the challenges that are inherent in every

match due to the nature of the game. The challenges they point out is the combat outcome of team fights or solo fights, if a set of actions could kill an enemy, enemy team strategy and winner of the phase. These challenges are related to prediction and combat analysis. To deal with these challenges they suggest techniques such as Ant Colony Optimization and Genetic Algorithms. Furthermore they present a model to use to break down the gameplay of MOBA into components so that an AI agent could make decisions on micro and macro levels.

The model they provide is formed by the following compo-nents.

• Item building, to optimize this they suggest that the AI agent can take the enemy team’s status and strategy in account and decide which items to buy.

• Laning, to optimize this phase a system that can simulate micromanagement skills is proposed. This system should be able to teach the AI agent to control the lane, push the lane, execute last hits for farm, taking multiple enemy heroes into account and denying enemy hero farm.

• Team fights, they suggest that the team fight model should

be structured as a one lane map and allow the AI agents to communicate with each other. This model could then be used to simulate scenarios of team fights which could possibly enable the AI agents in a match to perform efficient.

This is valuable to our thesis because we have to develop bots in order to confirm that our 5v5 framework is fully functioning.

(8)

5) MOBA games: A literature review: The authors of this paper performed a literature review and presents a couple of insights valuable to this thesis. The player base and ac-cessibility of MOBA games is massive, meaning that it’s a good environment to perform research in. They state that improvements to the API of MOBA games could help future research or to provide data sets that are relevant to the topic. Furthermore they found that MOBA games, in general, are not investigated which indirectly provides a gap i.e. an opportunity for research. At the time the literature review was written they found that the research being made in the context of MOBA games primarly deals with player behaviour and toxic behaviour in league of legends. They also state that league of legends is the MOBA game that has received most attention in research. This strengthens the claim that a 5v5 framework is needed in the area.

6) On The Development of Intelligent Agents For MOBA: The authors of this paper [37] have identified that using In-fluence Maps is a common approach used for tactical analysis for agents in other game genres but not yet commonly used in MOBA. It’s a method to map out desirable and undesirable locations on a map. The authors try this approach together with what they call a “two-layered agent architecture” and perform experiments to measure it’s effectiveness. The results are however inconclusive and until further research is done on influence maps and “two-layered agent architecture” in a MOBA context we believe this approach should be avoided when developing AI agents.

7) What Contributes to Success in MOBA games? An Empirical Study Of Defense of the Ancients 2: In their paper [41] the author examines what contributes to victory in DotA 2 matches. Among the results presented by the author it is mentioned that kills made by multiple players and the usage of wards contributes heavily on whether the outcome is a win or defeat. This information could possibly be used to optimize the efficiency of a bot developed for our framework.

III. RESEARCHMETHOD

In this section a discussion regarding the chosen research methodology Design science, research approach and data collection is presented.

A. Design Science

The methodology used in this thesis is design science [42]. The reason design science is elected is that it fits this thesis. It is motivated by the process of creating an artifact and evaluating it in iterations which according to March and Smith [43] are the two basic activities of design science.

Design science originates from the natural sciences and was situated in information systems research by Hevner et al. [42] in a framework called Information Systems Research Frame-work. When research is conducted in the field of information science Hevner et al. writes that the process is best approached using conceptional frameworks.

Fig. 2. Overview of the Information Systems Research Framework [42]

The problems identified by the research is defined in the environment. This column contains persons, organizations and the existing or planned technology which in our case is the 1v1 version of the DotA 2 framework developed by Tobias Mahlmann [13] and also our supervisors José Font and Alberto Enrique Alvarez Uribe. The knowledge base contains the foundations upon which we conduct our research, which in our case is the knowledge gathered in section II. The knowledge base also contain our methodologies which is the basis for how we will conduct and validate our research. We discuss our method in length in section III-B. The environment and knowledge base are used to develop theories and artifacts to justify and evaluate them. This is an iterative process that continues until we have completed our research. Each iteration expands our knowledge base and adds to our environment, this is illustrated in figure 2.

Fig. 3. Three cycle view of design science research [44]

Each iteration described above can also be divided into three sub-cycles [44] as shown in figure 3. The rigor cycle provide scientific grounding and additions to the knowledge base. The relevance cycle makes sure the research is relevant in regard to requirements and by doing field tests. The design cycle is where we build and evaluate the artifacts.

(9)

Writing this thesis consisted of doing the following four major iterations:

• Performing a systematic literature review to identify the

gap in existing research regarding a DotA 2 5v5 bot-matchmaking framework. The results of the systematic literature review is presented in section II.

• Selecting an appropriate methodology for conducting our research which is explained in section III-A.

• Developing and testing the artifact. This iteration consists of many smaller iterations where we confirm that the new functionality meets the requirements from the environ-ment, identifying potential problems and bugs and finally adding everything we learned to our knowledge base so that we can develop it further in future iterations.

• Evaluating the functionality of our artifact with the exper-iments defined in section III-B of which the results will be used to answer our research questions and conclude our research.

B. Experiment

The chosen method for this thesis is experiment. The suit-ability of this method for our thesis is that an artifact is devel-oped, a DotA 2 framework supporting 5v5 bot-matchmaking, and to be able to confirm that the artifact work as intended a number of experiments are performed. The quantitative data from the experiments are used to confirm functionality of the artifact and to answer the research questions. The data collected from the experiments are Boolean values. For every action executed by a bot the result is either true or false i.e. that the action has been executed successfully. The actions are defined in the experiments since different experiments evalu-ates different actions available to the bots. The experiments are deemed successful when every action of every hero gives the Boolean result true.

Other quantitative methods, such as case studies or surveys, are not as suitable for our artifact due to a number of reasons. A survey would possibly not contribute in confirming or deny-ing that the artifact is functional in an efficient way. A possible scenario could be that, after the artifact is developed, a survey is performed with participants in a made up competition. This would enable us to question the participants of their experience of the artifact. Which could possibly lead to being overwhelmed with feedback about various functionalities that are not working properly, previously untested due to running no experiments, in a late stage of the artifact development. On the other hand, by running experiments, the answers to our research questions are answered without involving third parties giving us direct feedback. This will allow us to efficiently iterate the development of the artifact to ensure that it functions as intended. Another important motivation for using experiments, as opposed to a case study, is that an experiment is easily reproduced.

C. Threats to validity

Due to the rapid development of DotA 2 leading to frequent updates and the fact that all clients have to be updated to

the latest version to be able to run the game, we can not be certain that our framework will be compatible with future versions of DotA 2. We know for a fact that an update to DotA 2 has broken the 1v1 framework in the past due to renaming of API functions that the framework was using to control the game. We hope we can mitigate this by releasing our framework as open-source under a permissive licence once we have finished developing it. Hopefully a third party will be interested in the functionality of the framework and keep it in sync with eventual future changes to the game API or maybe even improve it, like we did with the 1v1 framework.

Since we only evaluate the functionality of our framework and not it’s usability, we can’t know how easy or difficult it is for other researchers and other interested parties to actually develop agents for it. There is no graphical user interface for our framework, the only way to understand and use it is to read the code of our implementation and the provided example agent. This might not be a problem, since other popular frameworks like Ms Pac-Man [5] and TAC SCM framework [6] rely on the user of the framework to read the code of the example agents to implement their own.

IV. FRAMEWORKS

During a 10-week period the 1v1 framework has been extended to a 5v5 framework for bot-matchmaking in DotA 2. We developed the framework in several iterations. During the first iteration we expanded the framework to support 5v5 as it was the primary goal. During the second iteration we converted the Java web API and client framework to Python to give future bot developers access to machine learning libraries that are available in the Python ecosystem. During our third and final iteration we tested and debugged our framework to make it ready for release. During each iteration we gained further knowledge of what could be applied to the next iteration and discussed what should be done in the next iteration as described in section III-A and described in detail by Hevner et al. [42].

A. 1v1 framework

A 1v1 framework for bot-matchmaking has been developed by J. M. Font and T. Mahlmann and the description of the framework is visualized in figure 5 [10]. DotA 2 sends information about the current game state to the Lua sandbox which is then sent using HTTP post calls, in JSON files, to a web API. The web API reads the information given in JSON files. This information is taken into account by the bot implemented by its user which gives them the ability to choose what actions the bot would execute. The actions are chosen and are converted to JSON and sent back to the Lua sandbox which then forwards it to the game.

An issue with their web API is that it is written in Java which demands a number of libraries to be able to compile. The usability aspect is affected by this since it requires the user to set up the environment in the integrated development environment and installing libraries.

(10)

The original 1v1 framework had support for using items stored in the bot’s inventory. There was no support to actually see how many or which items were in the inventory. This made it near impossible to use items and there also was no functionality to target areas or other units with items, you could only use them on the bot which had them in it’s inventory. All this has been fixed in our adaptation of the framework. The one thing that we could not implement in regards to items is for the bots to use a courier to buy items. The documentation about couriers and their API is completely missing, or the functionality itself is not available within the Lua sandbox. This has prompted us to disable the couriers in our implementation as their existence in the game would provide an unfair advantage to the AI-bots of the enemy team, as the official bots are able to use them, since they are implemented in C++ and have access to the core functionality of the game. In practice this means that for a bot to buy items, it has to be inside the home base or in the vicinity of the shops scattered around the map.

Another issue with the implementation of the Web API is the non-standard use of HTTP verbs. The best practices for developing web API’s is to use HTTP-verbs to describe the functionality of the different routes [45]. A description of the best practices is shown in figure 4.

Fig. 4. Description of the HTTP verbs and the actions they are associated with [46]

The 1v1 framework is using the POST verb for every single route it has which adds unnecessary confusion as it implies that something is created and returned which is not always the case as they use it in the route that updates the world. This is connected with the issue that the 1v1 framework only accounts for 1 bot being able to send its commands in time i.e. each game tick. In the extended framework this is taken into account and for each HTTP POST all bot actions are computed locally and gathered into a single JSON file that is then sent back to the Lua sandbox. If the architecture had been kept from the 1v1 that would resolve in 5 HTTP POST calls for the bots.

Fig. 5. A description of the architecture of the 1v1 framework [10]

During the development of our experiments we also found and corrected numerous issues and bugs in the Lua code. The most prominent were:

• The system that was determining what kind of spell was being cast only had support to cast three different types of abilities, abilities you cast on yourself, abilities you cast on a specific area and abilities you cast on enemies. The functionality to cast toggle abilities was missing completely.

• As mentioned in the section above the ability to see what is in each heroes inventory was missing. During testing we also found that the logic to use testing was just a method that redirected to the use ability logic, which did not work on items.

• When serializing the world into JSON, the code that

gathered information on buildings did not check if they had been destroyed which resulted in nil values in the building array which in Lua signals that the array ends. This meant that once buildings gets destroyed, a lot of buildings that are still standing are missing from the serialized game state.

• Toweraggro, which is when a hero gets attacked by a tower, is information that is vital for the bots survival. The towers deal massive amount of damage and would kill a bot if it does not flee or otherwise mitigate the situation. There are facilities to detect this in the Lua game API but this was never serialized in the game state.

(11)

B. 5v5 framework

Fig. 6. A visualisation of the architecture of our framework

Our implementation of the framework can be divided into four subsystems. In this section we provide a detailed descrip-tion of the implementadescrip-tion of each subsystem. In figure 6 an overview of the framework is presented.

We rewrote most parts of the 1v1 framework as it was not developed with future extension to support more bots. This gave us the opportunity to decouple the subsystems more loosely to allow for future developers to implement their own versions. However, when we submit our framework for AI game developer competitions our intentions is that our full implementation will be used by the competitors. Our addon [47] and framework [48] are hosted on Github and can be visited by following the links in the reference section.

1) DotA 2 Addon: Addons for DotA 2 are written in the programming language Lua and are loaded with the workshop tools which is downloadable content. The game itself is written in C++ which is a statically typed compiled language and Lua which is a dynamically typed scripting language. The Lua API for DotA 2 is extensive and seem to cover most, if not all game functionality [3].

Our addon creates a 5v5 bot match and takes complete control of one team, called the good guys. With each game tick it gathers information about the world (game state) as it is known to the good guys, that means taking fog of war

and what is visible to each individual bot into account. This information is serialized and sent to the web API where it is processed. The final action during each game tick is to gather eventual commands from the web API and translate it into actions to be performed by the in-game bots. All actions are individual to each bot, which means that to make multiple bots move as a single unit you would have to implement that in the Python implementation of the bot. We have implemented the following seven actions, available to each hero in team good guys:

• Buy item

Buy an item from the store available to each team.

• Sell item

Sell an item in the above mentioned store.

• Use item

Use an item from the heroes inventory.

• Move to

Move the hero to specific coordinates on the game map.

• Level up

Level up the hero by increasing the ability points invested in a specific ability.

• Attack

Make the hero use it’s standard attack on an enemy hero or creep.

• Use ability

Use a hero ability. Available targets can be itself, friendly units and enemy units depending on what type of ability is being used.

2) Web API: The web API makes use of Bottle [49], a Python library distributed as a single module. Bottle is a web framework that offers the use of the HTTP calls POST, GET, PUT, DELETE. The routes of the web API is listed in the following bullet list.

• The web API receives JSON data from the Lua sandbox and sends it to the client

• The client decodes the JSON data and provides the information to the bot scripts

• The bot scripts decides which actions to take

• The actions are chosen and JSON data containing the

information is created

• The JSON data is sent back to the Lua sandbox

• The Lua sandbox provides the game with the information 3) Client: The functionality of the client is that it uses one DotA 2 client to connect to the game. When this client connects to the game, the in-game AI bots fill up the server. The bots in our team, called the good guys, are kicked. The in-game AI is either enabled or disabled for both teams. The reason for kicking the bots in our team is to be able to keep the in-game AI enabled for the bots in the opposing team. The connecting DotA 2 client from our framework creates 4 fake clients that connects to the server. These 5 bot clients are controlled by the scripts written by its user.

V. EXPERIMENTS

The following experiments were performed to evaluate the framework.

(12)

A. Experiment 1 - Hero movement

Confirm that heroes can be moved to specific locations

• Start a 5v5 match

• Move heroes

B. Experiment 2 - Level up hero abilities Confirm that heroes can level up abilities

• Start a 5v5 match • Level up heroes abilities

C. Experiment 3 - Hero attack Confirm that heroes can auto-attack

• Push lanes with all bots to validate attack functionality D. Experiment 4 - Buy items

Confirm that bots can buy items

• Buy one specific item with each hero

• Confirm that the bought items are in the inventory of the heroes

E. Experiment 5 - Sell items Confirm that bots can sell items

• Buy one specific item with each hero

• Confirm that the bought items are in the inventory of the heroes

• Sell each of the bought items.

• Confirm that all of the items are sold and no longer in the heroes inventories.

F. Experiment 6 - Use items Confirm that bots can use items

• Let bots buy items that are actively used

• Use items

G. Experiment 7 - Target Types

Heroes in DotA 2 have abilities. These abilities are either active, passive or auto cast. Active abilities are applied when used. Passive attacks are applied permanently or when the requirements for the ability are met. Auto casting abilities are applied when manually cast or when toggled on. There are different targeting types that these experiments validates the functionality of. • No Target • Toggle • Target Point • Target Area • Vector Targeting • Target Unit

Furthermore there are combinations of target types.

• Target Point or Unit

• Target Unit with Target Area

These experiments validates that the bots can utilize their abilities using the corresponding target types. The passive abilities functionality are handled by the game and therefore not included in these experiments.

1) Experiment 7.1 - No Target: This experiment validates functionality of the target type “No Target”. This target type does not target a point or unit and is cast when the command is given i.e. pushing the corresponding button for the ability. In this experiment the heroes Mirana, Slardar, Alchemist, Brewmaster and Clinkz are used.

• Mirana makes use of ability “Starstorm”

• Slardar makes use of ability “Slithereen Crush”

• Brewmaster makes use of ability “Thunder Clap”

• Alchemist makes use of ability “Unstable Concoction”

• Clinkz makes use of ability “Skeleton Walk”

2) Experiment 7.2 - Toggle: This experiment validates functionality of the target type “Toggle”. Target type toggle does not target units or points. They provide a momentarily effect, while toggled on, while draining either mana or health. If an ability is toggled on which drains mana, its toggled off if not sufficient mana remains to keep it enabled. In this experiment the heroes Medusa, Morphling, Witch Doctor, Pudge and Mars are used.

• Medusa makes use of ability “Mana Shield”

• Morphling makes use of abiliy “Attribute Shift (Agility Gain)”

• Witch Doctor makes use of ability “Voodoo Restoration”

• Pudge makes use of ability “Rot”

• Mars makes use of ability “Bulwark”

3) Experiment 7.3 - Target Point: This experiment validates functionality of the target type “Target Point”. Target point type abilities requires the caster to target an area or a point. In this experiment the heroes Techie, Spectre, Morphling, Lina and Mars are used.

• Techie makes use of ability “Techie Suicide”

• Spectre makes use of ability “Spectral Dagger”

• Morphling makes use of ability “Waveform”.

• Lina makes use of ability “Light Strike Array” • Mars makes use of ability “Spear of Mars”

4) Experiment 7.4 - Target Area: This experiment validates functionality of the target type “Target Area”. Target area type requires the caster to target an area. In this experiment the heroes Crystal Maiden, Alchemist, Enigma, Invoker and Brewmaster are used.

• Crystal Maiden makes use of ability “Crystal Nova”

• Alchemist makes use of ability “Acid Spray”

• Enigma makes use of ability “Midnight Pulse”

• Disruptor makes use of ability “Kinetic Field”

• Brewmaster makes use of ability “Cinder Brew” 5) Experiment 7.5 - Target Unit: This experiment validates the functionality of the target type “Target Unit”. Target unit type requires the caster to target a unit. In this experiment the heroes Lina, Vengeful Spirit, Omniknight, Chen and Clinkz are used.

(13)

• Vengeful Spirit makes use of ability “Magic Missile” • Omniknight makes use of ability “Heavenly Grace”

• Chen makes use of ability “Penitence”

• Clinkz makes use of ability “Death Pact”

6) Experiment 7.6 - Vector Targeting: This experiment validates functionality of the target type “Vector Targeting”. Vector targeting abilities requires two locations, a starting location and a direction location. In this experiment the heroes Grimstroke, Pangolier, Dark Seer, Mars and Void Spirit are used.

• Grimstroke makes use of ability “Stroke of Fate”

• Pangolier makes use of ability “Swashbuckle” • Dark Seer makes use of ability “Vacuum”

• Void Spirit makes use of ability “Aether Remnant” • Mars makes use of ability “God’s Rebuke”

7) Experiment 7.7 - Combination of Target Point or Target Unit: This experiment validates the functionality of the com-bination of the target types “Target Point” and “Target Unit”. There are abilities that supports targeting a point and or a unit. That means that the caster can target the unit directly or use a target point to execute the ability. In this experiment the heroes Sand King, Nature Prophet, Lina, Lion and Zeus are used. To test the functionality the abilities are executed using both target types.

• Sand King makes use of ability “Burrowstrike”.

• Nature Prophet makes use of ability “Sprout”

• Lina makes use of ability “Dragon Slave” • Lion makes use of ability “Earth Spike” • Zeus makes use of ability “Lightning Bolt”

8) Experiment 7.8 - Target Unit with Area of Effect: This experiment validates the functionality of the combination of target type “Target Unit” with “Area of Effect”. This target type affects an area surrounding the targeted unit. This target type must be cast on units. In this experiment the heroes Sven and Oracle are used to confirm functionality of target unit with area of effect type. Chen, Lina and Clinkz are added to fill up the team and makes use of their target unit type although these are not area of effect abilities.

• Sven makes use of ability “Storm Hammer”

• Oracle makes use of ability “Fortune’s End”

• Lina makes use of ability “Dragon Slave”

• Chen makes use of ability “Penitence” • Clinkz makes use of ability “Death Pact”

H. Experiment 8 - Validating usage of all heroes

A total of 119 heroes, at the time of writing, exist in DotA 2. A set of 5 heroes can be tested per iteration. This experiment validates that all heroes can be used in the framework.

• Create a list and add all heroes

• Select 5 heroes

• Start match

• Go to step 1 and rerun experiment with next 5 heroes in list

• In the last iteration, only 4 heroes are left in the list so a random hero is added to fill up the team

I. Experiment 9 - Full length match

Confirm that the framework supports full length matches.

• Connect 5 bots to the framework

• Select heroes randomly

• The bots should perform basic actions, such as farming and attacking enemy heroes/towers, during the match

• Complete match from start to end VI. EXPERIMENT RESULTS

To eliminate threats to validity experiments has been con-ducted. The experiments are simple bots written to perform certain actions, as described in each experiment. After the commands to perform the actions are called, the game state is validated to confirm that the expected effects of the com-mands has occurred which means that the functionality being tested works. After an experiment is finished, the results are printed to STDOUT of the terminal in which the experiment was started. The results is a summary of what part of the experiment passed and what failed. If one part of an exper-iment failed then the experexper-iment is considered a failure. All experiments are located in the src/bots folder of our framework and can be tested by anyone who has DotA 2 with our addon running on their computer. The framework and experiments can be found on Github [48]. To run a specific experiment you start our framework with the following command: python framework.py –bot <experiment>, where experiment is the name of the file containing the experiment that should run. A. Experiment 1 - Hero movement

Fig. 7. Screenshot of the result from experiment 1

B. Experiment 2 - Level up hero abilities

(14)

C. Experiment 3 - Hero attack

D. Experiment 4 - Buy items

E. Experiment 5 - Sell items

F. Experiment 6 - Use items

G. Experiment 7 - Target types H. Experiment 7.1 - No Target

Fig. 13. Screenshot of the result from experiment 7.1

I. Experiment 7.2 - Toggle

(15)

J. Experiment 7.3 - Target Point

K. Experiment 7.4 - Target Area

L. Experiment 7.5 - Target Unit

M. Experiment 7.6 - Vector Targeting

N. Experiment 7.7 - Combination of Target Point and Target Unit

O. Experiment 7.8 - Combination of Target Unit and Target Area

(16)

P. Experiment 8 - Full length match

Q. Experiment 9 - Validating usage of all heroes

VII. DISCUSSION

We believe that by having developed a working 5v5 frame-work for DotA 2 we have made a valuable contribution to our field of research, the field of artificial intelligence. As described in our motivation I-A and shown in our literature review II, there was a need for a 5v5 AI-agent framework in the MOBA genre. Our framework fills that void and we prove it’s functionality by performing a series of experiments, described in section V and the results of which are declared in section VI.

We opted to implement our framework in the Python language for it’s simplicity, rich standard library and platform independent nature [50]. We think that our framework would be fitting to use in university courses about AI, to let students implement and test their algorithms in. By using a game, in this case DotA 2, the algorithms implemented in the agents are visualised in real-time in-game. This possibly provides a better understanding of the behaviour of the algorithm the developer has implemented. As a bonus, DotA 2 is a popular game which possibly increases the amount of students that are familiar with the game and therefore understands the core mechanics and finds the assignments appealing. To further increase the potential usefulness of our framework as an educational tool we have created a detailed documentation. It contains a tutorial that is meant to help users setup the framework and implement a basic bot. It also contains a detailed description of the commands available for the developer, both the commands supported by the Lua addon and the API routes needed to be implemented to create an entire new client. An example bot and a number of experiments are provided to give further insight into the possibilities of the framework.

A. Future work

An important part of any DotA 2 match that is missing in our framework is a unit called a courier. In a normal DotA 2 match each player gets assigned a courier that can be used to buy items from the various in-game shops and deliver them to the player. This is a valuable game mechanic that buys each player a lot of time since they won’t have to go back to the shop to buy items. When we developed the 5v5-framework we were not aware of this functionality as it was disabled in the 1v1 framework and neither of us who developed the 5v5-framework had any previous experience playing the game. Once we learned about this functionality we attempted to implement it but were not able to get it in working condition and decided to disable the couriers for both teams. It is not clear from the official documentation [3] how to use the couriers from the Lua sandbox and we discovered this too late in the development process for us to spend any significant amount of time solving it. We even tried to get in contact with the game developer Valve but sadly we never received a response. Implementing the functionality to be able to use couriers in the framework is therefore one of our suggestions of future work that can be done in regards to the 5v5-framework.

Another interesting functionality that could be supported by our framework in the future is the ability to allow both teams to be controlled by custom AI-agents. Currently we only support controlling one of the two teams with our framework and the bots in the opposing team are controlled by Valves in-game AI. This feature is interesting because it would allow for developers to test their agents against each others and arrange tournaments like those being held for the TAC SCM framework [6] and Ms Pac-Man [5]. This feature was not in our scope when we developed our framework but we believe that implementing it would be straight forward and would increase the usability of our framework.

The core of our framework is the Lua addon for DotA 2. It is loosely coupled with a web-API/client written in Python. It is the client that loads the code of the agent that is to control the bots in DotA 2. It would be relatively easy for a programmer to implement a bot client in a different language than Python as we have defined a simple set of API routes and commands that are used for the addon and the bot framework to communicate with each-other. In the future it may be interesting to develop a framework in another language with support for different machine learning libraries than Python. The implementation of our client can be found at Github [48].

VIII. CONCLUSION

The conclusion is that the framework is able to handle 5v5 bot-matchmaking allowing it to be a plausible framework to be used in competitions such as the conference on games [14] and foundations of digital games conference [26]. The final product is a composition of the DotA 2 addon in Lua, a web API and client written in Python. The addon handles the communication between DotA 2 and the web API. The communication consists of data regarding the world and its

(17)

entities serialized in JSON language. The data is forwarded to the client using JSON commands which the client uses to determine a set of actions. The client responds with the actions which is then executed by the addon.

Below we present the answers to our research questions. 1) How can the 1v1 DotA 2 AI Framework project be

extended to support 5v5?

For every game tick DotA 2 streams data to Lua. Lua is then able to forward this data using by sending com-mands to a web API. In the 1v1 framework one client was responsible for handling the data and returning the actions that the user-controlled bot should perform. This could have been extended so that five clients would do the same thing but this could have possibly resulted in loss of actions per game tick. The reasoning is that if the 1v1 architecture was kept, five JSON commands would be sent from Lua every game tick to a chosen web API and the only information that would differ would be the information regarding the bots actions. At any given game tick the world looks identical for every bot involved in the match. Therefore running one client that would be responsible for locally handling all bot actions, by receiving one JSON command per game tick, was more appealing because this would in theory be five times faster. The in-game AI is either turned on or off for both teams in DotA 2 which brought issues when developing the framework. The solution was to kick the bots, except the connecting client, of our own team. The user-controlled bots then connected to the game through the connected client. By doing this, the possibility was enabled to control the bots of our own team while keeping the in-game AI enabled in the opposing team. The setback of this was that the couriers for our own team was lost. To solve this issue attempts were done such as kicking the bots in our team after the couriers had been loaded in the game. Unfortunately this led to the in-game AI buying items for some of the, to be, user-controlled heroes. This unwanted behaviour was disregarded at first and attempts were made to use the couriers for our user-controlled heroes. But it seemed that the connections between heroes and courier were no longer valid. This is possibly because the heroes that they were assigned to by the game were kicked by us. The lack of information regarding couriers in the official documentation [3] made it unfeasible for us to solve this issue during our given time frame. A more detailed explanation of our 5v5 framework is presented in section IV and the improvements we made to the 1v1 framework extending it to a 5v5 framework.

2) How can the intelligent agents communicate with each other in the extended framework?

The architecture of our framework supports communi-cation between the agents in any way possible in the Python programming language. The commands received and forwarded to the Lua addon is handled in a

central-ized Python class. The division of intelligence is up to the implementer of the agents to decide. The power of the Python programming language sets the limitations. The developer of a bot could for example implement a socket server where clients controlling each separate bot could connect to and use the TCP protocol to send messages to each other. Another implementation could use different algorithms for each bot in the team by implementing threads running for them and then use a thread safe global variable for the bot communica-tion. The point is that our architecture allows multiple implementations of the AI agents and the way they communicate.

3) How could a multiple agent strategy be implemented in the 5v5 framework?

We have developed our own example bot as an answer to this question. It is available on Github in the framework repository [48] in the “src/bots/BotExample.py” file. It works roughly in the following way.

• Fetching the names of the heroes to identify them • Assigning a lane to each bot in which they will do

battle

• The bots can switch lanes with each other

• If one of the bots is struggling to maintain its lane, it communicates to its teammates that it needs assistance and the teammates will come to assist

• If the enemy maintains pressure in the lane, the bots switches lane so that the strongest friendly hero stays on the lane

• For each game tick calculating the best possible course of action. For example, if the hero receives damage from a tower it retreats to a safe distance away from it

• If the bot is alone it looks for a group of friendly

creeps to follow and attacks enemies or buildings alongside them

• If the bot has enough gold, it buys one or more

useful items to the hero

There are other courses of action the hero can take and it’s always based on the current game state.

REFERENCES

[1] J. Togelius, “Ai researchers, video games are your friends!” in 2015 7th International Joint Conference on Computational Intelligence (IJCCI), vol. 2, Nov 2015, pp. 5–5, Accessed 2019-12-16.

[2] “The programming language lua,” accessed online 2020-03-19. [Online]. Available: http://www.lua.org/

[3] “Api - valve developer community,” accessed online 2020-01-12. [Online]. Available: https://developer.valvesoftware.com/wiki/Dota_2_ Workshop_Tools/Scripting/API

[4] M. Wu, S. Xiong, and H. Iida, “Fairness mechanism in multiplayer online battle arena games,” in The 2016 3rd International Conference on Systems and Informatics (ICSAI 2016), At Shanghai, China, 11 2016. [5] P. Rohlfshagen, J. Liu, D. Perez-Liebana, and S. M. Lucas, “Pac-man conquers academia: Two decades of research using a classic arcade game,” IEEE Transactions on Games, vol. 10, no. 3, pp. 233–256, Sep. 2018.

(18)

[6] I. Kontogounis, K. Chatzidimitriou, A. Symeonidis, and P. Mitkas, “A robust agent design for dynamic scm environments,” in SETN’06: Proceedings of the 4th Helenic conference on Advances in Artificial Intelligence, vol. 3955, 05 2006, pp. 127–136.

[7] M. Emilio, M. Moises, R. Gustavo, and S. Yago, “Pac-mant: Optimiza-tion based on ant colonies applied to developing an agent for ms. pac-man,” in Proceedings of the 2010 IEEE Conference on Computational Intelligence and Games, Aug 2010, pp. 458–464.

[8] “Rahul mehrotra: Product manager,” Accessed 2019-12-16. [Online]. Available: https://www.microsoft.com/en-us/research/people/ramehrot/ [9] A. Linn, “Divide and conquer: How microsoft researchers

used ai to master ms. pac-man,” May 2018, Accessed 2019-12-16. [Online]. Available: https://blogs.microsoft.com/ai/ divide-conquer-microsoft-researchers-used-ai-master-ms-pac-man/ [10] J. M. Font and T. Mahlmann, “Dota 2 bot competition,” IEEE

Transac-tions on Games, vol. 11, no. 3, pp. 285–289, Sep. 2019.

[11] “Dota 2 Prize Pool Tracker,” Accessed 2019-12-16. [Online]. Available: https://dota2.prizetrac.kr/

[12] OpenAI, “OpenAI Five,” Dec 2019, Accessed 2019-12-16. [Online]. Available: https://openai.com/projects/five/

[13] Lightbringer, “lightbringer/dota2ai,” Accessed 2019-12-16. [Online]. Available: https://github.com/lightbringer/dota2ai

[14] “IEEE COG 2019 Competitions,” Accessed 2019-12-16. [Online]. Available: http://ieee-cog.org/2019/competitions_conference/

[15] “Angry Birds AI Competition,” Accessed 2019-12-16. [Online]. Available: https://aibirds.org/

[16] Njustesen, “Bot Bowl AI Competition,” Sep 2019, Accessed 2019-12-16. [Online]. Available: https://bot-bowl.com/

[17] “Fighting Game AI Competition,” Accessed 2019-12-16. [Online]. Available: http://www.ice.ci.ritsumei.ac.jp/~ftgaic/

[18] “CodaLab Competition,” Accessed 2019-12-16. [Online]. Available: http://aka.ms/textworld-challenge

[19] “GFGAI Competition,” Accessed 2019-12-16. [Online]. Available: https://geometryfriends.gaips.inesc-id.pt/

[20] “General Video Game AI Competitions,” Accessed 2019-12-16. [Online]. Available: http://www.aingames.cn/

[21] “Fireworks Agent Competition,” Accessed 2019-12-16. [Online]. Available: http://hanabi.fosslab.uk/

[22] “Hearthstone AI Competition,” Accessed 2019-12-16. [Online]. Available: http://www.is.ovgu.de/Research/HearthstoneAI.html [23] “Micro RTS AI Competition,” Accessed 2019-12-16. [Online].

Available: https://sites.google.com/site/micrortsaicompetition/home [24] “Starcraft AI Competition,” Accessed 2019-12-16. [Online]. Available:

https://cilab.gist.ac.kr/sc_competition/

[25] “Strategy Card Game AI Competition COG 2019.” [Online]. Available: https://jakubkowalski.tech/Projects/LOCM/COG19/

[26] “FDG 2019 Competitions: San Luis Obispo, California, USA,” Accessed 2019-12-16. [Online]. Available: http://fdg2019.org/competitions.html [27] “The Codenames AI competition,” Accessed 2019-12-16. [Online].

Available: https://sites.google.com/view/the-codenames-ai-competition [28] “Fifth Aeon Collectible Card Game,” Accessed 2019-12-16. [Online].

Available: https://fifthaeon.com/tournament

[29] M. C. Green, “Generative Design in Minecraft,” Accessed 2019-12-16. [Online]. Available: http://gendesignmc.engineering.nyu.edu/

[30] “TCIAIG Special Issues,” Accessed 2019-12-16. [Online]. Available: https://cis.ieee.org/publications/t-games/tciaig-special-issues

[31] F. Weidt and R. Silva, “Systematic literature review in computer science-a prscience-acticscience-al guide,” Relscience-atórios Técnicos do DCC/UFJF, vol. 1, 2016, Accessed 2019-12-16.

[32] “Google Scholar,” Accessed 2019-12-16. [Online]. Available: https: //scholar.google.com/

[33] “Acm Digital Library,” Accessed 2019-12-16. [Online]. Available: https://dl.acm.org/

[34] “IEEE Xplore Digital Library,” Accessed 2019-12-16. [Online]. Available: https://ieeexplore.ieee.org/

[35] M. Waltham and D. Moodley, “An analysis of artificial intelligence techniques in multiplayer online battle arena game environments,” in Proceedings of the Annual Conference of the South African Institute of Computer Scientists and Information Technologists, ser. SAICSIT ’16. New York, NY, USA: ACM, 2016, pp. 45:1–45:7, Accessed 2019-12-16. [Online]. Available: http://doi.acm.org/10.1145/2987491.2987513 [36] M. Morosan and R. Poli, “Evolving a designer-balanced neural network

for ms pacman,” in 2017 9th Computer Science and Electronic Engi-neering (CEEC), Sep. 2017, pp. 100–105.

[37] V. d. N. Silva and L. Chaimowicz, “On the development of intelligent agents for moba games,” in 2015 14th Brazilian Symposium on Com-puter Games and Digital Entertainment (SBGames), Nov 2015, pp. 142– 151.

[38] M. Wi´sniewski and A. Niewiadomski, “Applying artificial intelligence algorithms in moba games,” Studia Informatica : systems and informa-tion technology, vol. Vol. 1-2(20), pp. 53–64, 2016, Accessed 2019-12-16.

[39] V. do Nascimento Silva and L. Chaimowicz, “MOBA: a new arena for game AI,” CoRR, vol. abs/1705.10443, 2017, Accessed 2019-12-16. [Online]. Available: http://arxiv.org/abs/1705.10443

[40] M. Mora-Cantallops and M.-Á. Sicilia, “MOBA games: A literature review,” Entertainment Computing, vol. 26, pp. 128–138, May 2018, Accessed 2019-12-16. [Online]. Available: https://doi.org/10.1016/j. entcom.2018.02.005

[41] B. Xia, H. Wang, and R. Zhou, “What contributes to success in moba games? an empirical study of defense of the ancients 2,” Games and Culture, vol. 14, p. 155541201771059, 05 2017, Accessed 2019-12-16. [42] A. R. Hevner, S. T. March, J. Park, and S. Ram, “Design science in

information systems research,” MIS quarterly, pp. 75–105, 2004. [43] S. T. March and G. F. Smith, “Design and natural science

research on information technology,” Decision Support Systems, vol. 15, no. 4, pp. 251 – 266, 1995. [Online]. Available: http://www.sciencedirect.com/science/article/pii/0167923694000412 [44] A. Hevner, “A three cycle view of design science research,”

Scandina-vian Journal of Information Systems, vol. 19, 01 2007.

[45] M. Biehl, RESTful API design: APIs your consumers will love. API-University Press, 2016.

[46] “What is rest api?” accessed online 2020-06-06. [Online]. Available: https://www.visual-paradigm.com/guide/development/what-is-rest-api/ [47] K. Lindqvist and D. Nilsson, “5v5dota2ai-addon,” May 2020,

accessed online 2020-05-12. [Online]. Available: https://github.com/ ellakk/5v5dota2ai-addon

[48] ——, “5v5dota2ai-framework,” May 2020, accessed online 2020-05-12. [Online]. Available: https://github.com/ellakk/5v5dota2ai-framework [49] “Bottle - python web framework,” accessed online 2020-03-30.

[Online]. Available: http://www.bottlepy.org/docs/dev/

[50] J. V. Guttag, Introduction to Computation and Programming Using Python: With Application to Understanding Data (The MIT Press). The MIT Press, 2016.