by Jesper Hyldahl Fogh
May, 2018
What are they? What do they explain?
How do we work with them?
Examiner: Maria Engberg Supervisor: Simon Niedenthal
TABLE OF CONTENTS
TABLE OF CONTENTS 1 ABSTRACT 3 1 · INTRODUCTION 3 1.1 · Research Question 3 1.2 · Delimitation 3 1.3 · Structure 3 2 · THEORY 5 2.1 · Generic Design 5 Critiques 62.2 · Educational games theory 6
Educational games 7
Games research 7
Play, games, toys, and simulations 7
Mechanics, dynamics and aesthetics 8
Intro to relevant educational theory 9
The cognitive process dimension 9
The knowledge dimension 10
2.3 · What's next? 10
3 · ANALYSIS 10
3.1 · Explorable explanations 11
The categories of analysis 12
General examples 13
Simulating the World In Emojis 14
Introduction to A* 15
Notable examples 15
4D Toys 15
Something Something Soup Something 16
Hooked 17
The Monty Hall Problem 17
Talking with God 17
Fake it to Make It 18
So what are explorable explanations then? 18
What's next? 19
4 · METHODOLOGY 19
4.1 · Sketching and prototyping 19
4.2 · Evaluation 19
5 · DESIGN PROCESS 20
5.1 · The design goals 20
5.2 · Neural networks 20
What are neural networks? 20
Why neural networks make sense as a subject 21
5.3 · The three iterations 21
Iteration 1 · The Visualized Network 21
Evaluating the iteration 22
Iteration 2 · The World's Dumbest Dog 22
Evaluating the iteration 23
Iteration 3 · A Tale of 70.000 Numbers 24
Evaluating the iteration 24
5.4 · A summary of the whole design process 25
6 · REFLECTION & FUTURE WORK 25
6.1 · Reflection 25 Method 25 6.2 · Future work 26 For designers 26 For researchers 27 7 · CONCLUSION 27
ACKNOWLEDGEMENTS 27
REFERENCES 28
APPENDICES 31
ABSTRACT
In this paper, the author examines the concept of explorable explanations. It has emerged as a genre of educational software within the last 7 years, yet descriptions of it are vague at best. The author works with the genre through a generic design approach that consists of an analysis of existing explorables and the design of three iterations of the author's own explorable explanation on the topic of neural networks. 22 examples, of which 9 are
presented in-depth, are analyzed with educational theory and games research theory as tools. It is found that explorable explanations tend to be digital experiences with a high degree of interactivity that attempt to teach facts, concepts and procedures to the user. Furthermore, the author embarks on a design process of creating explorable explanations of their own to understand what can be relevant when designing and evaluating an explorable explanation. The paper is concluded with reflections on the employed method in the project. Future work is also briefly outlined about what impact the analysis and design work can have on the
1 · INTRODUCTION
There is an explorable explanation that teaches how the human voice works (Thapen, 2017). Another one, called Fake it to Make It (Warner, 2017), teaches how and why fake news works. You can even find one that introduces the concept of the fourth dimension through play (ten Bosch, 2017). There are plenty more out there with a variety of other subjects. Yet it is not an easy task to understand if explorable explanations can be considered different from
educational games, data visualization, interactive narratives or similar. There are some definitions out there (Case, 2017; Lambrechts, 2018; Victor, 2011), but none of them are really all that helpful in understanding the field. Goldstein (2015) does an admirable job of trying to nail down definitions for it, but most of his references and conclusions are seemingly pulled from thin air. Within the academic world, my searches have not shown much help either. Granström (2016) works with the genre, but is ultimately more interested in the dissemination of physics than designing and understanding explorable explanations. Other sources (He and Adar, 2017; Kaltman, 2015) merely mention the genre in a passing reference. Before we can compare the genre to others, like those mentioned above, it helps to understand what the genre actually is. This paper seeks to do this by conducting a categorical analysis of 22 explorable explanations, and a design process with three iterations of an explorable
explanation design concept, which attempts to explain the inner workings of artificial neural networks.
1.1 · Research Question
This project focuses on answering the following research question:
How can we characterize explorable explanations, and what design qualities should be considered when designing and evaluating them?
1.2 · Delimitation
This project is not about design guidelines. I am not attempting to find a design process that can be relied upon for future work with explorable explanations. This is also not about evaluating whether explorable explanations are good at teaching their subject matter to players. These are both worthwhile endeavours, but not within the scope of this paper.
1.3 · Structure
First off, I will go through some theory that is necessary for understanding the analysis of explorable explanations as well as the description of my design work that follows it. The analysis has gone through 22 examples of explorable explanations, and presents 9 of these examples in-depth. Following this, the paper introduces my own explorable explanations, which serve to gain a better understanding of what it takes to design explorables. Finally, I will discuss and reflect on the project and the process that I have gone through, before
suggesting possible paths for future work.
This paper is part of a larger story on the research behind it, and there are more aspects to this research than what can be conveyed in the format of a paper. First of all are the
prototypes that were developed as part of the research process. In the hopes that it will give a more vivid image of my research, the latest version of each iteration is available to try online on the following address for at least a year after publication:
The source code for the prototypes will also be available for at least a year on the following Github repository:
https://github.com/jepster-dk/neuralnet-explorable
It is recommended that the prototypes are experienced by the reader at some point during the reading process.
Secondly, the way that this paper is structured does not adequately reflect the research process that lies behind it. Papers are written linearly, but the process behind my research did not occur linearly. Furthermore, there is not enough room to present all of the thoughts, experiences and discussions that have led me to this point. So, in order to accommodate the critique of Zimmerman et al. (2010) on the importance of process documentation in
research-through-design, I have attempted to visualize the trajectory of my work in figure 1.
Figure 1 · My research process, visualized
In general, this visualization shows a process that has featured many explored branching paths, as well as a process that has had to throw away some work in order to trace back to what was important. It is my adaptation of Buxton's idea of design as branching of exploration (Buxton, 2007, p. 388). An example of a branch is the fact that this project started as simply an attempt to create an explorable explanation about neural networks, without regard for
attempting to describe the genre. Another example of a branch would be that I initially wanted to play test my prototypes on an expert in the field of neural networks. Due to time
constraints, this never came to fruition.
Finally, a small note on language. I will sometimes refer to explorable explanations as simply explorables for the sake of brevity. Furthermore, I will use the terms artificial neural network and neural network interchangeably. The words user and player will also be used
interchangeably, since explorables are still poorly defined. They will both be used regardless of whether a referred explorable can be labelled a game or not. With that said, let us move on.
2 · THEORY
Before moving on to the meat of the research process, I wish to establish the theoretical grounding for the project. This includes two main aspects: the employed
research-through-design approach and theory on educational games. When structuring and framing my design work, I am following the notion of generic design thinking as introduced by Wiberg and Stolterman (2014). On the matter of educational games theory, I am relying on
both theory on games and play, as well as more general educational theory.
2.1 · Generic Design
Wiberg and Stolterman (2014) propose their idea of generic design thinking as an answer to the issue of gauging whether a design is novel. Generic design is a way to both conceptualize a new design while relating it to an existing body of designs. The goal is that generic designs can make it easier for researchers to figure out if their own and others' work is novel, and thus eligible as a new contribution to the existing body of knowledge. Wiberg and Stolterman draw the notion of generic design from Warfield (1990), and they describe it as:
"A generic design in HCI can be seen as a design concept [emphasis added] that captures some essential qualities of a large number of particular designs [emphasis added], i.e., it defines a class or design space of interactive systems." (Wiberg and Stolterman, 2014, p. 6)
I have emphasized a few key phrases in this description. The first is that generic designs are design concepts, i.e. they are not just descriptions or ideas, but rather sketches, models, prototypes and the like. Second is that this design concept must capture some essential qualities of a large number of particular designs, which means that the aim of a generic design is not to create a perfect prototype that has been thoroughly user-tested and ready to hit the market. A generic design should be representative of a broad range of designs that reside within the same class and design space. It is important to note that to Wiberg and Stolterman (2014), this applies to both HCI and Interaction Design Research. They make no effort to distinguish between the two fields.
In introducing generic design thinking, Wiberg and Stolterman (2014) identify three existing approaches to working with ideas in design and characterizing designs: proof-of-concept
designs, design guidelines and concept design. They map the three along with generic design on a two-dimensional matrix (see figure 2). The horizontal dimension shows how the approaches work with ideas in the design process. Are the ideas concretely represented in a design (left), or are they abstracted and provide scaffolding (right)? On the vertical dimension, we see whether the approaches deal with particular details and qualities of designs (top) or attempt to build general theories for working with designs (bottom). As such, generic design deals with particular points of reference but on an abstracted layer as scaffolding for design work. Wiberg and Stolterman summarize the main activity of generic design as one of grouping and describing, with the outcome or goal of the method being a characteristic design. Its purpose is to describe design and, if done correctly, will define a design space.
Figure 2 · The matrix of design research (Wiberg and Stolterman, 2014, p. 7)
In this paper, I examine the group of educational software that goes by the name explorable explanations from a generic design perspective. This entails two main research activities: analysis of the qualities of 22 explorables and the design and development of three iterations of an explorable explanation of my own. In generic design terms, this means that I am
grouping and describing 22 explorables, while seeking to articulate a characteristic design. The hope is that this will give other designers and researchers a better understanding of the design space that explorable explanations exist in.
Critiques
Before moving on to educational games theory, I want to address one particular issue with generic design. The approach relies on what Wiberg and Stolterman (2014) call "essential qualities". This implies that there is an essence that can differentiate one design class from others. However, Wiberg and Stolterman do a poor job of giving examples of what this could look like. If you ask Wittgenstein (1953), perhaps this is due to the fact that essential qualities are rarely identifiable. Wittgenstein instead talks about family resemblance. To illustrate his point, he claims that there is no single thing in common between all games, but that they instead share resemblances across the spectrum of their use. This does not mean that descriptions of such concepts are not useful. To Wittgenstein, it still makes sense to say that "these and things similar to it are called games" (Wittgenstein, 1953, §69). He simply points out that in certain cases, it is a futile effort to discover the essence of a concept. In this project, I have had a similar difficulty in trying to nail down the essential qualities of explorable
explanations. Instead, I therefore seek to describe them in terms of their family resemblances. I do this by introducing a handful of examples in tandem with a textual description and a table showing their traits in 8 categories of analysis.
2.2 · Educational games theory
In this part of the paper, I want to introduce a couple of terms and theories that are useful when analysing and describing the examples of explorable explanations. They are similarly useful when it comes to describing and relating my own prototypes in relation to these examples.
I will first introduce the field of educational games, which explorable explanations seems to share features in common with. This serves to contextualize explorable explanations in the larger developments of games. Then I will describe the area of games research that deals with the relationship between games, toys, simulations and play; an area that comes in handy when comparing and juxtaposing various explorables. I will then introduce the MDA
framework as a tool for describing the specific design of explorable explanations, including my own work. Finally, I will introduce an updated version of the classic taxonomy of the cognitive domain (Bloom et al., 1956), commonly referred to as Bloom's taxonomy, as an analytical tool for understanding what type of knowledge and what type of cognitive processes explorable explanations deal with.
Educational games
For various reasons, the field of educational games has seen a kind of fall from grace in the 2000's (Egenfeldt-Nielsen, 2007). The term is on the verge of being substituted with the larger concept of serious games, which includes games for a multitude of subjects, such as health, education, public policy and more (Games for Health Europe Foundation, 2018; Serious Games Interactive, 2018; World Food Programme, 2005). This does not mean that there is not room for educational games in the games industry anymore, but instead that they are now in cahoots with other games with purposes that extend beyond providing
entertainment or artistic expression. This increased scope for educational games makes sense when seen in relation to the growth of explorable explanations as a genre. As we will see in the analysis, explorables deal with a multitude of subject matters, and do not limit themselves to traditional educational environments such as schools and colleges. Instead, they seem to embrace a notion of lifelong learning.
As for how educational games are theoretically grounded, Egenfeldt-Nielsen (2007) states that educational games often draw on the experiential approach to learning. One example is the one formulated by David Kolb (1984). The approach draws connections between concrete experience, reflection, concepts and application, and sees it as a learning cycle. The focus on the concrete experience as part of the cycle lends itself well to educational games, which are aptly suited for providing this experience. The challenge is then to introduce reflection, conceptualization and application as part of the game. Similar challenges and opportunities are seen in explorable explanations.
Games research
Here, I will introduce some theory on the notions of play and games and similar concepts. This will aid me in the analysis of explorable explanations.
Play, games, toys, and simulations
Homo Ludens, first introduced by Huizinga (1938), covers the understanding of humans as inherently playful creatures. In an essay attempting to relate this concept to interaction design, Bill Gaver writes: "Play is not just mindless entertainment, but an essential way of engaging with and learning about our world and ourselves" (Gaver, 2002, p. 1). This notion of play as an engaging activity that enables us to learn about the world is one that resonates with educational games and explorable explanations. I wish to build a bit upon this notion with Salen and Zimmerman (2003). They construct a definition of play that is easier to work with: "Play is free movement within a more rigid structure" (Salen and Zimmerman, 2003, p. 304). This definition of play is accompanied by three categories of play: game play, ludic activity and being playful. Game play constitutes the type of play that occurs when players follow the rules of the game, such as chess. A ludic activity refers to the non-game behaviors that are
still described as playing, such as playing house. Finally, being playful is a more general category that encompasses bringing a playful state of mind into other activities, such as using playful slang.
In relation to play, Salen and Zimmerman go on to define a game as "... a system in which players engage in an artificial conflict, defined by rules, that results in a quantifiable outcome" (Salen and Zimmerman, 2003, chapter 7, p. 11). There are five keywords in this description: system, players, artificial conflict, rules and quantifiable outcome. System refers to a range of elements that interact or interrelate to form a complex whole. Players means that one or more users are actively engaging in playful behavior with the game. Artificial conflict establishes games as being about a contest with others players, the game itself or something third. The outcome of the contest is separate from real life, e.g. games do not lead to actual death, but in-game death. Fourthly, games are defined by rules. These rules are particularly important when distinguishing between the three different categories of play. Finally, games result in a
quantifiable outcome. That is, games have winners and losers, or at least give players a score to show how well they did.
Then there are also toys and how they relate to play and games. One useful description of toys is the one that Sicart gives: "Toys facilitate appropriation: they create an opening in the constitution of a particular situation that justifies the activity of play" (Sicart, 2014, p. 36). To Sicart, play is an appropriative activity, and toys enable this activity. I wish to twist this
description a bit to match the notion of play that I have introduced with Salen and Zimmerman (2003) and Gaver (2002). In this way, toys are facilitators of play, and thus facilitators of a free movement within a rigid structure of engaging with and learning about the world and
ourselves.
Finally, we have simulations. In many ways, they are like games, but according to Egenfeldt-Nielsen (2007) they are missing the key elements of artificial conflict and the player's active engagement. If these are missing, the game seems to become a simulation instead.
I could keep working with these definitions, provide contrasting views, or even go into the discussion of games without goals (Juul, 2007). However, for this project, these definitions are adequate, because they now give me three conceptual boxes in which to sort various
explorable explanations: the game box, the toy box and the simulation box. Furthermore, I can build on top of these boxes with the notion of play.
Mechanics, dynamics and aesthetics
The MDA framework is a common and practical approach to discussing the relationship between a game's system, the game designer and the player (Hunicke, LeBlanc and Zubek, 2004). It works as both a game design and game analysis tool. The framework consists of three parts: Mechanics, Dynamics and Aesthetics. Aesthetics ultimately describes the experience a player has when playing the game. The aesthetics can be described in terms such as expressive, sensational, challenging, and more. Dynamics then refer to aspects of a game that create aesthetics. In a sense, it is where the player meets the game, as in dynamics are what happens when a player engages with the systems of a game. Expressive dynamics, for instance, may include systems that allow the player to leave their mark and create their own things. Finally, mechanics refer to the actions, behaviors and control mechanisms that the player is afforded by the game. They are usually described as verbs. Examples of this include walking, jumping or eating.
in specific terms. Where relevant, the mechanics, dynamics and aesthetics of the analyzed explorables will be described so that they can be compared. It is also used in elements of the design process description.
Intro to relevant educational theory
Since explorable explanations are attempting to teach players something, I have found it useful to introduce some educational theory in order to build an understanding of what types of knowledge and at what level they teach. This entails two aspects of the taxonomy of the cognitive domain: the cognitive process dimension, and the knowledge dimension.
Specifically, these will be invoked in the analysis of explorable explanations as a comparative tool.
Despite being released over 60 years ago, the taxonomy of the cognitive domain (Bloom, Engelhart, Furst, Hill, & Kratwohl, 1956) has remained relevant in educational research with only a few updates. In 2001, Anderson et al. (2001) published an updated version, which makes a few minor changes to the original. The changes were made partly to update the taxonomy to the developments that have happened in society since 1956, and partly to bring the contents of the old version into focus in the new millenium. In this paper, I will rely on the 2001 version.
The cognitive process dimension
The cognitive process dimension is comprised of six different levels of reasoning that a student can achieve. It describes what cognitive processes are involved in a student's
learning. Moving up in the taxonomy, the levels become increasingly abstract and build upon the preceding steps. As such, in order to reach level three, one must first arrive at a basic skill level in level two.
Anderson et al. (2001) changed the naming for the levels of the taxonomy from "Knowledge, comprehension, application, analysis, synthesis and evaluation" to "Remember, understand, apply, analyze, evaluate and create". Notice that the names changed to verbs and evaluation was moved down by one level, while "Create" was added to the mix. Some of the
subcategories were also changed, but I will not go into that much depth here. Figure 3 shows an overview of the levels.
Remember
Understand
Apply Analyze Evaluate Create Retrieve relevant knowledge from long-term memory Construct meaning from instructional messages
Carry out or use a procedure in a given situation Break material into its constituent parts and relate them internally and externally Make judgments based on criteria and standards Put elements together to form a coherent or functional whole; reorganize elements into a new pattern or structure
Figure 3 · Overview of the cognitive process dimension adapted from Anderson et al. (2001, p. 3)
The first level, remember, represents the ability to retrieve relevant knowledge from long-term memory. This can also be thought of as knowing the facts. Secondly, understand, refers to constructing meaning from instructional messages. In practice, this can for instance mean paraphrasing statements, categorizing subjects, contrasting ideas, or constructing
cause-and-effect models. Apply, which is on the third level, refers to carrying out or using a procedure in a given situation, be it familiar or unfamiliar. Examples of this include multiplying two numbers, or using Ohm's law to calculate resistance in a circuit. On the fourth level, we find analyze, which describes the process of breaking material into its constituent parts and determining how these parts relate to one another and to the overall structure. This could, for instance, happen through deconstruction, distinction, selection, or parsing. With evaluate, Anderson et al. (2001) mean making judgments based on criteria and standards. This means to determine and detect the effectiveness or inconsistencies in a product or process in regards to both internal and external criteria. Finally, create, at the top of the taxonomy, is a student's ability to put elements together in a coherent, functional, and novel structure. To demonstrate this, a student could come up with alternative hypotheses based on an observed phenomenon. She could design a procedure for dealing with a new problem. Or perhaps she could produce a new product through the knowledge gained from the learned subject material.
The knowledge dimension
The other dimension of import from the taxonomy is the knowledge dimension. This describes what type of knowledge is involved in the learning process. It includes four levels of
knowledge: factual, conceptual, procedural and metacognitive. The factual level is almost self-explanatory and includes the basic elements a student must know to be acquainted with a discipline. Conceptual knowledge refers to knowledge about the interrelationships between the factual knowledge within a discipline, as well as knowledge of the basic elements that enable the factual to work together in a structure. Thirdly, procedural knowledge includes the ability to perform skills, algorithms, techniques and methods, but goes a bit beyond that, as it
also includes knowledge of the criteria for using these various procedures. Finally, the metacognitive knowledge can be described as knowledge about knowledge. It refers to being aware of one's own level of knowledge, what is expected of one's knowledge level by others and knowledge of different procedures for learning.
2.3 · What's next?
Now that I have established a toolbox for working with and analyzing educational software, I will now move on to the analysis of various explorable explanations.
3 · ANALYSIS
In this section, I will first talk about how explorable explanations are currently defined by various actors in the field, and give a brief history of the genre. This includes a shallow review of related genres. Then I will move on to an analysis of 22 examples of explorable
explanations on the basis of 8 analysed qualities. I will not describe all 22 in depth, but have instead picked two examples that exemplify the most common explorable explanations. Furthermore, in order to encapsulate the variety that exists within the genre, I am also briefly introducing seven other examples that exhibit deviating characteristics in one or more qualities.
3.1 · Explorable explanations
In order to talk about the current state of explorable explanations as a term, I want to do something similar to what Salen and Zimmerman (2004) did when they defined games, albeit in a slightly more limited form. I am going to introduce a couple of existing definitions of explorable explanations and then compare them.
Explorable explanations as a term seems to have its origin in a blog post by Bret Victor (2011) of the same name. It is described like this:
"Explorable Explanations is my umbrella project for ideas that enable and encourage truly active reading. The goal is to change people's relationship with text. People currently think of text as information to be consumed. I want text to be used as an environment to think in." (Victor, 2011)
Others have since taken up the challenge of working with explorable explanations, and they have found a common home on the page Explorable Explanations (Case, 2017). On that page's FAQ, the definition changes to this:
"In short, by "explorable explanation" we mean something that 1) teaches something, and 2) is more interactive than a boring ol' quiz with only one right answer." (Case, 2017)
Third, we have Belgian data journalist Maarten Lambrechts' definition, provided off-the-cuff in an interview with datawrapper.de:
"Explorable explanations (you could also call them dynamic texts or dynamic
documents) are documents users can interact with. They educate people not by just combining text and static graphics, but by integrating interactives. So people can really play with what they’re learning; with what they’re seeing. People can learn something without realizing they are learning something." (Lambrechts, 2018)
Victor's definition does not mention interactivity specifically, but from looking at the
surrounding description in the blog post, it is apparent that interactive elements are part of what he considers explorable explanations. However, what Victor means by text is not necessarily clear. Interestingly, he also talks of active reading, as if reading is a necessity to explorables. This would seem to put it more in the realm of interactive narratives, except within non-fiction.
Then, in Case's definition interaction is included as one of only two criteria, but he is not very specific about what kind of interaction. Whether it is social interaction, human-computer interaction, or something else remains unclear. Case's definition also relies on the unspecific word "something". Looking at the examples on his page, it seems we are dealing with digital artefacts, but this definition does not wish to make this clear. It is not even clear whether we are talking about artefacts at all. In fact, the definition is broad enough that it might even include a birch tree, since I cannot confidently state that birch trees are not things that teach something by being more interactive than a boring ol' quiz.
Figure 4 · Three explorable explanations? (Allen, 2011)
Finally, in Lambrechts' definition, the term explorable explanations is described as
interchangeable with dynamic documents or dynamic texts. Integral to his definition is also the inclusion of interactivity. He goes as far as using the word "play" to describe how the users can engage with the document and its educational content. This playing with the content seems to mirror Victor's notion of explorables as "an environment to think in" (Victor, 2011). However, Lambrechts also seems to feel that explorable explanations are closer to stealth learning games, such as Machineers (Lode et al., 2013), than to overt educational games, when he describes them as teaching without the player knowing that they are being taught.
In seeking to build my own definition, I have chosen to disregard essential qualities, and instead work with family resemblance. In practice, this means that I will not attempt to provide anything like a definition before I have presented the analysis of the examples of explorable explanations. When dealing with family resemblance, it is important to first present the family.
The categories of analysis
In order to analyze the various examples of explorables, I have established 8 categories for comparison. They are subject, time required to play, amount of visuals compared to text,
amount of interaction, highest level of cognitive process, highest level of knowledge, price and platform. All of the 22 explorables were sourced by looking at the Explorable Explanation site (Case, 2017). There are more games on the site than 22, but in order to maintain the scope, I only looked at 22. It should be noted that more examples of explorable explanations exist outside those found on this site. It may seem then that an external categorization is pressed down upon my own analysis. However, Case's site is frequently referenced by others (Lambrechts, 2018; Goldstein, 2015), is open for outside additions through its open-sourced code on Github (explorableexplanations, 2018) and features plenty of examples not authored by Case himself. Therefore it still seems a fitting place to source my examples. Before moving on to the analysis, however, I will briefly explain the basis behind choosing the analysis categories.
Subject refers to what field of knowledge the explorable teaches. It was chosen as a category in order to understand whether explorables are limited to science-based subject matters, which initially seemed to be the case.
Time required to play says something about the scale of explorable explanations, which can help in comparing them to educational games. Some of the explorables are not easily defined in this category, as it is up to the player how long they want to keep playing. Therefore, I have generally looked at how long it minimally takes to experience all of the content the explorable contains.
As we saw in the given definitions of explorable explanations, particularly in Victor's, the required relationship between text and visuals is not necessarily obvious. For this reason, this criteria was also included. Amount of text and amount of visuals was each given a number between 0 and 100 based on how large a percentage of the explorable they constituted. This means that the sum of both numbers would always result in 100. Visuals include graphics, images, videos, animations and similar. Text means words and textual characters. It was a qualitative evaluation, by which I mean that I did not set up strict criteria for how many words, characters, pixels or similar were used in the explorable to find the percentage.
Fourthly, the amount of interaction was similarly qualitatively evaluated on a scale from 0 to 100. In general, it refers to how large a percentage of the explorable is interactive. By interactive, I mean if the content can be affected by a user's input in a way that is not just natively a part of the platform, e.g. highlighting text, or saving images. When seeing the examples, the exact meaning of this will hopefully be more clear.
I also looked at the cognitive dimension and the knowledge dimension of the explorables. I was mostly concerned with what the highest level an explorable reached on each of the dimensions. Through my own qualitative judgment, this was given a number from 1-6 and 1-4, thereby matching the taxonomy introduced by Andersen et al. (2001).
Finally, I looked at the distribution model for the explorable on two accounts: price and
examples included advanced pricing models like free-to-play, subscription-based or similar, this was fairly straightforward. By platform, I mean what delivery method is used to experience the game. In the analysis, only two general methods appeared: directly in the browser, or as downloadable software for either iOS, PC, Mac and Linux.
The analysis does not include a category for game, toy or simulation distinctions. This is due to the fact that certain elements of many explorables can be considered belonging to one distinction, but other elements show more belonging to others. For this reason, these distinctions will be referred to in the expanded descriptions of the analysis, but not in the analysis overviews. A spreadsheet with the analysis of all 22 games can be found in appendix 1.
General examples
First, I want to introduce two examples that constitute what I call general examples of
explorable explanations. They exhibit traits that are shared by many explorable explanations, and do not show major deviation in any of the analysis categories. The two explorables are Simulating the World In Emojis (Case, 2016) and Introduction to A* (Red Blob Games, 2014).
Simulating the World In Emojis
Subject Time Text-visuals ratio Interactivity
Miscellaneous 5-10 50/50 65 Highest cognitive level Highest knowledge level Price Platform 6 3 Free Browser
In a sense, Simulating the World In Emojis (Case, 2016) is one of Nicky Case's most ambitious explorables. Its grand aim is to teach the player how to think in systems. It does this by presenting the user with a series of simulators, initially revolving around the system behind forest fires. Each simulator contains a grid of various sizes with space for an emoji on each individual square. The behavior of the emojis can be tweaked by the user, e.g. so that they appear or disappear more often or turn into other emojis depending on various other rules that are similarly tweakable. The player can also choose to add specific emojis by clicking the simulator, which is then incorporated into the running simulation. The simulators are
interspersed with descriptive and narrative text, and occasionally a simple quiz appears that asks the player what they think the behavior of a complex system will result in. Case has prepared the simulators with specific behaviors and rules to build his point, and has also initially left out some features in order to not overwhelm the player. Finally, the explorable features a "sandbox" that includes all features of the simulator that have been introduced with the added possibility of being able to share one's custom-built simulator.
The even split between textual content, the majority of the content being interactive and its free, browser-based distribution model is what makes Simulating the World (Case, 2016) a general example of an explorable explanation. It is also an excellent example of an
explorable, because it enables teaching the player on a high cognitive level by allowing the user to create their own simulations, and share them with others. In the text, the explorable also acknowledges and talks about how the subject matter is fairly new and that different theories show different results. This can be seen as a type of level three knowledge in that it
deals with criteria for using different procedures. Simulating the World initially introduces the concept of systems thinking by talking about the theme of forest fires, but it manages to build on this by relating to other complex systems, potentially sparking the player's interest in taking their knowledge further outside of the explorable. When it comes to its relation to toys, simulations and games, the explorable can be described, in Salen and Zimmerman terms, as being in-between ludic activity and game. Case sets up some rules that the player is expected to follow, but they are free not to and just play around with the structures set up for them in the simulators. There is no quantifiable outcome outside of the quizzes, and these play such a small part that they are more dressing than salad. In fact, the simulations are seen to be nothing more than dynamics enabled by the mechanics of the explorable. They are barely simulations at all, because they engage the player actively.
Introduction to A*
Subject Time Text-visuals ratio Interactivity
Computing 10-30 35/65 70 Highest cognitive level Highest knowledge level Price Platform 5 3 Free Browser
Introduction to A* (Red Blob Games, 2014) is an interactive primer on the world of path-finding algorithms. It introduces three different algorithms that have certain pros and cons. It also describes some principles that underlie the algorithms, such as early exit, movement costs and frontiers. The explorable explanation features many different maps with different implementations of the various algorithms. The target group is game programmers, so the algorithms are also shown in Python code. Basically, the explorables show how a game system can find paths from a starting point to a goal. So Red Blob Games has made sure that the user can move the goals and starting points at any time to explore how the algorithms work under different circumstances. The player is even able to add and remove walls in some of the maps.
Introduction to A* (Red Blob Games, 2014) is somewhat deviant in regards to the cognitive process dimension, because most analyzed explorables exist on the second level, not the fourth. So what actually makes Introduction to A* a general example of an explorable, like Simulating the World, is that it features many interactive visuals in a relatively short timespan in a free browser-based experience. Introduction to A*, like Simulating the World (Case, 2016), does a good job at introducing a variety of procedures and the criteria for when to use them, while acknowledging its own limits as an explorable of 10-30 minutes.
Notable examples
Hopefully, these two examples give a better understanding of what an explorable explanation looks and feels like in most of the examined cases. However, in the spirit of family
resemblance, I believe it is important to highlight some of the edge cases of explorables. I will introduce seven of these.
4D Toys
Subject Time Text-visuals ratio Interactivity
Math 5-10 35/65 80 Highest cognitive level Highest knowledge level Price Platform 3 1 15€ iPad / Steam
4D Toys (ten Bosch, 2017) is different from most explorables in that it is neither free nor browser-based. 4D Toys is split into two parts: an explorable explanation and a set of toys. The explorable explanation segment takes the player through how the fourth dimension works as a concept. This way, the player will hopefully have a better chance of understanding what is happening and why when playing with the toys. The toys on the other hand do not feature any explanation, nor any quantified outcome, or artificial conflict. They exist within a rigid structure, but by reacting to their environment in unexpected ways, they facilitate play and experimentation.
Pink trombone
Subject Time Text-visuals ratio Interactivity
Biology, linguistics 2-5 5/95 95 Highest cognitive level Highest knowledge level Price Platform 6 1 Free Browser
Pink trombone (Thapen, 2017) is, like 4D Toys (ten Bosch, 2017), a toy-like artefact. It shows the player a model of the human vocal organs. By clicking and moving the mouse around the areas of the model, sounds are generated and change to create various vowels and
consonants, which are played out loud. The explorable barely features any text outside of credits or labels for the various parts of the model. The player is expected to figure out for themselves how to make sense of the model. It stands out as an explorable explanation through this specific lack of explanatory text; a quality which seemed to be inherent to explorables before Pink Trombone was classified as one.
Something Something Soup Something
Subject Time Text-visuals ratio Interactivity
Philosophy 5-10 15/85 85 Highest cognitive level Highest knowledge level Price Platform 2 2 Free Browser
Something Something Soup Something (Gualeni, 2017) presents itself very much like a 3D point-and-click game. The explorable takes the player into the year 2078, where teleportation is being used to transport food, in this case soup, from alien planets to Earth. Due to
translation issues resulting from communication that the player establishes with the alien planets, the "soup" that is teleported does not always look like one would expect. The player is tasked with identifying which teleported objects actually constitute soup so that they can be served to the guests. By doing this, the game hopes to raise questions on the nature of definitions and language. It is described as an interactive thought experiment inspired by Wittgenstein's philosophical work. It differs from most explorables by engaging the player in a 3D world and a narrative. The explorable itself does not explicitly describe its educational intentions, but instead leaves that to the surrounding meta-text. The question of whether the explorable is a game or not is even part of the educational content itself, because as the website says: "Is it even wise or productive to strive for a complete theoretical understanding of concepts like ‘soup’ or ‘game’?" (Gualeni, 2017).
Hooked
Subject Time Text-visuals ratio Interactivity
Psychology, journalism 5-10 20/80 60 Highest cognitive level Highest knowledge level Price Platform 2 2 Free Browser
Hooked (Evershed et al., 2017) is a union between journalism and interactivity. It is perhaps what comes closest to Lambrechts' definition for explorable explanations as dynamic
documents. Its goal is to educate the reader in how the design of slot machines works to build addiction in their players. It does this by introducing a handful of interactive elements, such as a simple button that the user is asked to click as much as they want. It peppers this with video interviews with a former addict and a researcher within the field of gambling. It also includes some non-interactive graphics. All in all, it is a very multimedia experience, and even includes certain game-like elements in that the user is asked to play with simulated slot machines.
The Monty Hall Problem
Subject Time Text-visuals ratio Interactivity
Math 1-2 80/20 15 Highest cognitive level Highest knowledge level Price Platform 2 1 Free Browser
The Monty Hall Problem (Powell, 2014) is by far the shortest explorable that I have examined. It also has a high degree of text compared to most other examples. It does not take long to
read and experience The Monty Hall Problem, and it features very little interaction. The main thing you can do is to run a simulation of the infamous Monty Hall problem, and then later tweak a few variables of that simulation. Despite this, the explorable does a good job of allowing the user to explore and examine the problem by themselves with only these few variables.
Talking with God
Subject Time Text-visuals ratio Interactivity
Philosophy 2-5 100/0 10 Highest cognitive level Highest knowledge level Price Platform 2 2 Free Browser
Talking With God (Stangroom, 2018) is the only examined explorable that contains no visuals. Its interaction is only based on clicking defaultly styled buttons, and the rest is explained in text. The explorable works by putting the player in conversation with God, who asks the player what they think God is capable of and whether God exists. In the end, the player is given an analysis of their results accompanied by an explanation of whether the player's internal logic contains tensions. The inclusion of interactivity works mainly to react to the player's own thoughts on the existence of God. The explorable does not lend itself well to exploration, since the movement through the explorable is very linear, and since the results are only shown once all answers have been given.
Fake it to Make It
Subject Time Text-visuals ratio Interactivity
Media studies 30+ 70/30 70 Highest cognitive level Highest knowledge level Price Platform 5 3 Free Browser
Where The Monty Hall Problem (Powell, 2014) is by far the shortest explorable in the analysis, Fake It to Make It (Warner, 2017) is the longest. The explorable is about how fake news works. It is so long in fact that it includes a saving feature, so that the player can leave the game, and come back later. The saving feature is not only necessary because of the game's length, but also because it asks the player to make their own profile, newspaper and articles through an elaborate interface. Thus it is important that these decisions are saved for later retrieval. Furthermore, Fake It to Make It is definitely a game, and not a toy or simulation. The explorable asks the player early on to set a goal for what they want to buy with their fake news empire funds. This sets up a quantified outcome, an artificial conflict and player engagement.
So what are explorable explanations then?
With this analysis behind us, we get a clearer picture of what an explorable explanation is or can be. In general, they seem to be highly interactive, digital experiences with a mix of visual and textual content, leaning towards the visual side. They teach not only facts, but also concepts and procedures as well as the relationships between said facts, concepts and procedures. Some explorables also teach the criteria for using the taught knowledge. They engage cognitive processes on a medium level, focusing mostly on the player's
understanding of the subject matter. Explorable explanations are most often free and
experienced through a browser. The majority of them are between 5-30 minutes, but they can be shorter or longer. It is all a matter of the scope of the thing to be taught. The subject matter is not limited to neither natural sciences, social science nor humanities. Explorables can exhibit traits of quantified outcome, or artificial conflict, but in most cases will more easily fit in the ludic activity category. They may include a fictional narrative, but will often not.
Simulations can be a tool for engaging with the material, but explorable explanations cannot be classified as such alone.
What's next?
Now that I have described explorable explanations, I will now briefly present my methods for working with my own designs. After that I will take a look at what a design process for an explorable explanation could look like with these designs in focus.
4 · DESIGN METHODOLOGY
My design process has included four main activities: sketching and prototyping, related work research, topic research and evaluation. These activities have not occurred in a chronological manner, but instead I have moved between them seamlessly. To get a better understanding of the process, see figure 5.
Figure 5 · My design process, visualized
4.1 · Prototyping
Prototyping has been the primary method for designing this explorable. I follow the Houde and Hill (1997) notion of the term, where prototypes are tools for exploring the look-and-feel, role and implementation, or all three, of a design concept. I used prototyping primarily as an explorative technique by sketching ideas by hand in words and drawings, but also through software sketching in environments like Processing, P5.js, as well as languages like HTML, CSS and Javascript. I have also prototyped to engage with and understand the subject matter. What I mean by this will be made clearer in the presentation of the first iteration.
4.2 · Evaluation
Evaluation has mainly taken place with two parties: two of my peer interaction design students and my girlfriend. This has taken place as play testing sessions, where the players were first told that they could do nothing wrong, and that the main purpose was to understand how players experience the explorable. They were also asked to speak out loud what they were thinking as they went through it. The play testing was followed up with a brief unstructured interview about their experience. The main purpose of evaluation has been to understand usability issues with my designs, and not to discuss the merits of them as explorables or teaching aids. The sessions were recorded with the consent of the play testers. Evaluation has not played a large role in the project, and is therefore visualized as a smaller circle in figure 5. The consequences of my small amount of evaluation is reflected upon in section 6.
5 · DESIGN PROCESS
In this section, I will introduce the three iterations of my explorable explanation. The explorable attempted to teach the subject of neural networks according to a set of design goals, which will be stated in the next segment. I will describe each iteration in depth and in relation to the previously presented analysis. I will also summarize each iteration with an evaluation of the design. This section is concluded by a brief summary of the design process as a whole.
5.1 · The design goals
To guide the process of designing the explorable explanation, I defined the following design goals:
1. The design concept should be able to reasonably be classified as an explorable explanation
2. The design should be available through the browser
3. The design should make the basic functionality of a neural network understandable for people with no professional experience in programming or mathematics
4. The design should result in an experience that takes a maximum of 10-15 minutes to complete
These goals were not made explicit before the design process initiated, but were instead reframed throughout the process via a research diary, which I kept throughout my research.
5.2 · Neural networks
networks are, and why they matter as a subject. I will attempt a brief, simplified description of that here. This description will only focus on giving a basic understanding of the specific terms that appear in my designs. It will not suffice as a fulfilling description of neural networks.
What are neural networks?
Neural networks are a type of machine learning. Machine learning is a field of computer science that attempts to enable computers to learn tasks without being explicitly programmed to knowing what these tasks are. Neural networks construct outputs by taking one or more inputs, formatted as numbers, and multiplying them by another set of numbers, called weights. Neural networks' main goal is to find the correct weights to multiply by. When it is said that neural networks "learn", what is meant is that they move closer to the correct
weights. This happens by subjecting the network to an algorithm which adds or subtracts from the weights by a slight amount based on how well the neural network performs in a test. After each weight adjustment, the neural network builds an output, before being subjected to the testing and weight adjustment process again. This process is repeated until a satisfactory output is acquired. This will usually involve millions of iterations, which, depending on the complexity of the neural network and the computer performing the calculations, could take hours, days or weeks.
To give an example of how neural networks work with real-life material, let us consider images. Images are made up of thousands of pixels, which, to a neural network, can be formatted as numbers indicating the amount of red, green and blue in each pixel. This would be the input. To make a neural network that can recognize images of handwritten numbers, one could feed in thousands of examples of such images accompanied by labels that indicate what each image shows. After multiplying all the inputs of each image with weights, the neural network would then test whether these weights resulted in the correct output by comparing with the labels. If it did not, it would tweak its weights and try all over again. This process then takes place hundreds, thousands, sometimes millions of times, until the neural network seems to have reached a good track record of identifying images.
So, in essence, a neural network works by trying to guess the correct weights through millions of iterations of trial-and-error based on given examples.
Why neural networks make sense as a subject
Neural networks are notoriously obscure and difficult to understand for not just amateurs, but professionals as well. The main issue is that neural networks function by doing what
computers do well, but humans do poorly, which is iterating on the same task millions of times. They often work with massive datasets, sometimes known as deep learning or deep neural networks, and furthermore do this on powerful computers capable of performing a mind boggling amount of calculations per second. Theoretically, if a mathematician had infinite time to go over the calculations a neural network has done, she would be able to establish how the neural network came to its conclusion. Unfortunately, since no
mathematician does have that time, neural networks have become famous for being black boxes (Card, 2017; Lewis and Monett, 2017; Wolchover, 2017). This is often accompanied by a description of "no one actually knows how they come to their conclusions" (ColdFusion, 2018), which is a bit of an overstatement. However, neural networks are slowly becoming a larger and larger part of our world, and each month, it seems, new neural networks are shown to do incredible things (Elias, 2018; Murgia, 2016; Welch, 2018). The complicated innards of neural networks combined with their increasing influence make it an apt topic for an explorable explanation. Furthermore, for this project, it makes sense for me. It is a topic that has held my interest for a long time, and since I am already a programmer, I might have an easier time
understanding neural networks than I will a subject matter outside of computer science.
5.3 · The three iterations
At this point I want to move on to a description of the three iterations of my explorable explanation. Each of them had different individual goals. They represent three different ways of approaching an explorable explanation, and while some are more high fidelity than others, they will not be subjected to a normative evaluation. Instead, they will be evaluated on their own merits and faults.
Iteration 1 · The Visualized Network
The first iteration had the primary goal of helping me understand how neural networks work and how they can be disseminated. At this point in the process, I was still learning, and prototyping became a tool in that learning process. I have named it The Visualized Network, because it mainly took the shape of the classical representation of a neural network as a bunch of circles with lines connecting them. The lines were missing in my version however. The Visualized Network did not feature any explanatory text. Its main interaction happened via sliders that could be moved with the mouse to affect changes in the weights and inputs in the neural network. The first version attempted to make inputs more easily understandable for the user by manifesting it as red, green and blue values that would increase or decrease in saturation along with the value of the input sliders as set by the player. This was abandoned in order to focus on my learning, before I would eventually move on to a more concerted effort in focusing on the user experience in my design.
Figure 6 · Final version of The Visualized Network
The Visualized Network can be seen as a simple version of Carter and Milkov's Neural
Network Playground (2018), or Crowe's NeuroVis (2018). These represent a common way of illustrating neural networks in an interactive way. The main difference is that, in The Visualized Network, the player takes the role of the neural network, because the weights are not
closer to simulations, and rarely work with data that makes sense to a non-programmer or non-mathematician.
Evaluating the iteration
This iteration was never meant to be the final explorable, because before creation, I had already decided that this was not an appropriate way of teaching neural networks to non-programmers and non-mathematicians due to their separation from easily
understandable data. The only reason for its inception was to aid my understanding, and in collaboration with topic research, as well as talks with a neural network expert and looking at related work, I ended up boosting my understanding of the field.
Iteration 2 · The World's Dumbest Dog
The second iteration revolved around the story of the world's dumbest dog, Neura, not being able to walk straight down a path. The player was then given the task of teaching the dog how to do this, through a series of small tasks.
In the first version of The World's Dumbest Dog, Neura followed a simple set of rules. This was inspired by the movement of the car in Bret Victor's Up and Down the Ladder of Abstraction (2011). It would walk straight ahead, unless it detected that it was either left or right of the path. If it was left, it would make a corrective turn to the right, and vice versa. The first mechanic that was introduced was that the player could adjust the angle of the corrective turn. Through simulated machine learning algorithms, the user would then help Neura find the angle by herself.
Figure 7 · The World's Dumbest Dog
Once I had built the section of the explorable that introduced the self-learning capabilities, I realized that this set of rules was too complicated for a player to easily make sense of. I came to this realization by seeing that the behavior of the car in Victor's Ladder of Abstraction (2011) is chosen exactly because it is complicated and is a good case for why abstracting a problem can help make it more clear. At this point, I remembered that I had watched a video tutorial on neural networks by Daniel Shiffman (2017), which included a way of visualizing neural
networks that was quite similar to the visual I had built for this prototype. So I changed the rules for Neura to instead pick a random direction to go in, and then follow that. The
challenge of the explorable would then be to help her find the right direction. In the end, I did not follow this design to the end, and The World's Dumbest Dog was left incomplete when I moved on to the third iteration.
Evaluating the iteration
When I abandoned this prototype, I did it because I realized that this prototype did not do a good job at relating to what neural networks actually do. Instead it was trying to teach the math behind neural networks visually. This prototype did not even do a very good job at this part, because the math it showed was highly simplified. Still, this prototype represented a step in the direction of a concept that was more like explorable explanations. It included more explanatory text, was more interactive, and the length of the experience was increased. Its major fault was its misguided use of metaphor and mechanic.
Iteration 3 · A Tale of 70.000 Numbers
For the third iteration, I took a look at what neural network tutorials usually do to explain the concept. One of the most common things is to use the MNIST dataset (LeCun et al., 1998), a set of 70.000 handwritten numbers for training and testing a neural network. It seems it is a common approach for tutorials due to multiple factors. First off, this dataset is often used for benchmarking neural network implementations. Secondly, it is free and open-source. Thirdly, the data in it is easily understood by everyone as handwritten numbers. A Tale of 70.000 Numbers was thus built around the story of a woman who wrote down numbers in hand every day of her life until her death. When she died, she wanted her numbers to be distributed among her 10 children according to what number it was. The first child should get the zeroes, the second child the ones and so on. The player is tasked with sorting these 70.000 numbers. At first the explorable tells the player to do it by hand through a drag-and-drop mechanic. However through a chat on the right side, a helper, named J, suggests getting a machine to do it. The player is then taken through the explorable slowly by introducing features to this machine, which is called Augusta 1800, named after Augusta Ada Lovelace. The concept of a weight is introduced as a sort of "magic number" that neural networks use, and in the end, through a series of smaller tasks, Augusta is capable of training herself to find the weights. Behind-the-scenes, A Tale of 70.000 Numbers does not actually run a neural network, but simulates it by faking the behavior of a weight.
The relationship between the amount of text and visuals in A Tale is around the average for the analysed examples of explorable explanations. It is also more interactive than the
previous iterations, and even the text has become interactive by allowing the player to always choose between two options in the chat on the right side. It is considerably longer than the previous iterations, and is the only iteration, which features a complete experience.
Figure 8 · A Tale of 70.000 Numbers
Evaluating the iteration
A Tale of 70.000 Numbers is definitely the iteration that is most representative of explorable explanations that has come out of this design process. It draws on a few features seen in other explorable explanations, but it mixes them in a novel form. A Tale builds a narrative like Something Something Soup Something (Gualeni, 2017), but it does it directly in the browser, and not in an embedded 3D player in the browser. This iteration also introduces a sandbox mode at the end, similar, but not nearly as extensive, as the one seen in Simulating The World In Emoji (Case, 2016). On top of this, the explorable has a quantifiable outcome and artificial conflict, which is not seen in many other explorables. However, there are still elements that are not satisfactory with A Tale of 70.000 Numbers.
First of all, the graphic style of the explorable does not do much to build the narrative or the point of the explorable. It is more or less just the default of the CSS styling library Bootstrap. Secondly, while the explorable is closer to what neural networks are actually capable of than any previous iterations, the explorable relies on a simulation of a neural network. Augusta is not actually capable of identifying numbers. In fact, the algorithm already knows what the correct numbers are, but simply makes fake guesses based on how close the current setting is to a randomly predetermined weight. For more on this, see the Github repository. The choice to keep the neural network a simulation was made in order to make it easier to implement, and control the experience of the explorable. In a future iteration, it might be a better idea to look into Javascript-based neural networks that would actually perform the functions of an image recognition neural network in the explorable explanation. The challenge here would then be to control and disseminate what the weights are doing.
5.4 · A summary of the whole design process
In this section, I have shown how I have gone through a design process of building my own explorable explanation on the topic of neural networks. This has resulted in three iterations of prototypes with quite different approaches. In the next section, I will discuss and reflect on these iterations, the project as a whole and the potential future work in this area.
