Corporatization and Smart Cities: Creating a debate about economic interests in future urban infrastructure

“Technology is the answer. But what is the question?“ Cedric Price (British architect), 1966

Corporatization and Smart Cities

Creating a debate about economic interests in future urban infrastructure.

Malmö University Dennis Bücker - Interaction Design

One-year Master‘s Programme - Spring semester / 2016 / 15 CP Supervisor: Anuradha Venugopal Reddy

Abstract

Smart cities are seen as the answer for specific problems, that we will face in the future: overpopulation, shortage of resources, demographic change, etc. At the same time more people than ever will live in cities. This creates huge difficulties for governmental institutions on the one side, but also profit promising opportunities for companies on the other. This thesis will have a close look at this tension and unpack the vision of the smart cities, as they are created with the help of integrators and service companies. Through the lens of Critical Design, it will result in a fictional artifact to trigger the imagination and a debate about a possible future in which corporate interests changed the relationship between the city and its citizens.

Keywords

Smart Cities, Critical Design, Design Fiction, Urban Interaction Design

Acknowledgments

I am thankful to Anuradha Venugopal Reddy for her invaluable support throughout the project. I would also like to thank Marjo Tikkanen, Sena Çerçi, Carlos Mario Rodriguez Perdomo and Laura Potenti for participating in creating the project. Last, I would like to thank all of my classmates and friends sharing their valuable opinion and feedback during the process.

Table of content

Introduction and Design Landscape ... 4

Part I: Smart Cities ... 6

Concerns of smart cities ... 9

A closer look at the corporatization of smart cities ... 11

Part II: Critical Design–Investigating the preferable Future ... 15

Doing Critical Design ... 16

Related projects ... 21

Part III: Facilitating the debate ... 24

Creating the fiction ... 24

Evaluating the fiction ... 26

Designing the fiction ... 28

Discussion and Conclusion ... 33

Findings ... 33

Evaluation of the process and the results ... 34

Future opportunities ... 35

Conclusion ... 35

Introduction and Design Landscape

Figure 1: Harry Tuttle (Robert de Niro) in the Movie „Brazil“ (Gilliam, 1985), He plays some kind of haunted hacker/guerilla-mechanic in a smart city, who tries to keep the basic needs of the population running

A transformation is taking place in how our urban environments work. A digital landscape overlays our physical world. In the cities of the future, technology isn’t just with us, isolated in different devices, and services as it is today. It surrounds us, supports us, promises to offer ever-richer urban experience by measuring changes of any motion, temperature, air pollution, and more. It uses the context of our environment to empower us in more natural, yet powerful ways. So familiar, so mundane, that we hardly notice. The cities of the future will be smart.

We are aware of the technology in our closer environment, talking about it, questioning, and deciding it. We learn to secure our phones with pass codes, and customize the privacy settings of our social networks to be in control. But companies are already one step ahead: they are evolving the technology we know for a larger scale, globally. A massive collection of cross-referenced data points, involving a massive number of people, and centralized

computational surveillance—that has become intimately linked with the vision of the “smart city”. Smart cities need to know who we are, where we are, what we are, and with whom we are, to keep their promise. Technology in smart cities will tap into a pool of information to understand the world, to react, and predict. This “big data,” as it is increasingly known, will be an immanent force that pervades and sustains our urban world. Are we aware of the powerful ways in which these systems and their use will change our world—our policies, economies and built environment? The movie “Brazil” (Gilliam, 1985) describes for instance a futuristic smart city, in which everything is possible—if you have the right documents. The city services are based on data and technology, controlled by the Ministry of Information, a totalitarian government department functioning like a business. It results into a poorly maintained infrastructure full of malfunctions, where every fix must be applied for (Figure 1).

Today, in our understanding of technology malfunctions are something unexpected; In 2015 the Environment Protection Agency (EPA) found out, that some cars of the German car manufacturer Volkswagen met emissions standards in the laboratory, but not during normal operation (The Guardian, 2015). After more research they detected, that it was not based on a malfunction: the company installed a “defeat device” software that reduces nitrogen oxide emissions while a car is undergoing official tests. Volkswagen did not build a car, that just malfunctions: the company programmed it to cheat intelligently.

In a world of ambient and ubiquitous technology, capable of luring into our cities and taking over control, our environment will be shaped in a way we cannot estimate yet. How can we trust in the invisible technology, when the technology that makes our cities smart is prompted to cheat? Will a self-driving Volkswagen car deliberately slow down to give priority to the higher-priced models? This thesis deals with the question of how a seamless data-collecting technology can determine the culture of a city and its citizens. It will speculate about a future, when companies are helping challenged governments and profit-seeking interest may become as ubiquitous as the technology they are promoting. What plans do the architects of this technology have, and who gets to see the data, make sense of it, and use it?

In the first part I will have a closer look at the concept of the smart city and its concerns. The second part explains Critical Design as a method to look in the future, followed by designing a fiction: based on the research I will present the outcome of a future, in which the balance between the social, community-driven aspect of a smart city and the underlying neoliberal ideology are becoming unequal. Last, I will create an artifact to make the imagined future more accessible.

Research Question

Part I: Smart Cities

Figure 3: Songdo, a newly built city in South Korea. Photo: Gale International LLC

By 2014 the UN announced that 54% of the world’s population lives in urban areas, a proportion that is expected to increase to 66% by 2050(United Nations Population Division, 2015). This leads to a situation in which cities are responsible for 75% of CO2 emissions and energy consumption worldwide(UNEP-DTIE Sustainable Consumption and Production Branch , 2015). Cities must deal with growing inequalities within their own structures, a

diversification of its population and a conversion of their family structures (Damon, 2013). A modern city promotes social cohesion and seeks to specifically include social divides and multiple generations for estimated 6.4 billion people. Urban areas will face massive logistical, ecological, economic and social issues in the near future and all are planned to simply be computed away through early predictions and watchful eyes. In the past 20 years, cities which embraced information and communication technologies (ICT) have been pioneers in embedding digital systems into the nature, structure and enactment of urban infrastructure and been variously labelled as “wired cities” (Dutton, Blumler, & Kraemer, 1987), “cyber cities” (Graham, 2004), “digital cities” (Ishida & Isbister, 2000), “smart cities” (Hollands, 2008) or “sentient cities” (Shepard, 2011). Each of these terms is used in a particular way to conceptualize the relationship between ICT and modern urbanism. Though, they share a common understanding in the effects of ICT on urban canvas, processes and modes of living, and have been become subsumed within the label “smart cities”, a term which has gained traction in business and government, as well as in academia. However, just the embedding of ICT in urban infrastructure is not seen to make a city smart (Hollands, 2008). It is how ICT is used to leverage growth and manage urban development that makes a city smart (Caragliu, Bo, & Nijkamp, 2009). In general, the concept can be divided into two understandings as to what makes a city “smart”. On the one hand, future cities are equipped with a broad range of ICT to manage a city´s assets and meet the resident´s needs. So called “everyware” (Greenfield, 2006), a ubiquitous network of technology, collects data which will be used to centrally monitor, manage

and regulate city flows and processes by public and private organizations (Figure 2) . By this, it is argued that cities transform into more

sustainable, resource-efficient and secure areas (Townsend, 2014). Whereas the first vision of a smart city focuses on ICT and its use in managing and regulating the city from a largely technocratic and technological perspective, the second vision of a smart city is seen to refer more to a place whose economy and governance is being driven by smart people who enact innovation, creativity and entrepreneurship (Kourtit, Nijkamp, & Arribas-Bel, 2012). Here, ICT is seen as being of central importance as the platform for mobilizing and realizing ideas and innovations. In this scenario, networked infrastructures are mere enablers to the enhancement of human capital, education, economic development and governance (Allwinkle & Cruickshank, 2011).

Key in both visions is the city as a massive producer of datasets. These datasets hold the promise of what some see as a truly smart city–one that can be known and managed in real-time and is sentient to some degree–and will be used to reimagine and regulate the urban life. A comprehensive collection and analysis of data means an evidence-based development, new modes of technocratic governance and innovative economics, and empowered citizens through open, transparent information (Batty, Axhausen, Giannotti, Pozdnoukhov, & al., 2012). Previously, data was collected on a non-continues basis and/or rely on samples and are limited in access. So called “big data” now connects through analyzation various datasets and utilizes them in diverse ways. The hype and hope of big data is a transformation in the knowledge and governance of cities through the creation of a data deluge. This seeks to provide much wider-scale, finer-grained, real-time understanding and control of urbanity. Such explosive growth in data is due to a number of different technologies and networked infrastructures and their rapid embedding into everyday life and spaces. In 2008, for the first time, the number of mobile Internet users surpassed the user of home broadband services, like DSL, cable and fiber-optic lines (International Telecommunication Union, 2015). With the fast spreading of mobile devices, the technology became part of our outer world and daily life, but also opened a whole new platform for services and products: the internet of things. The idea of smart cities is based on such internet-based devices, which connect all possible technologies in one central network, accessible and manageable from all over the world. Today, there are already at least two devices connected to the Internet per person. But by 2020 the few billion humans will be outnumbered by 33 billion networked objects prowling through the cyberspace (Cisco, 2016). The development of ubiquitous computing and the embedding of software into all kinds of devices not only enable the accessing and sharing of data, but are also the origin by which much big data are generated (Greenfield, Everyware, 2006). Broadly, big data can be collected in three different ways: directed, automated and volunteered. Directed data is generated by traditional single forms of surveillance and includes mostly a human operator, like collecting fingerprints or checking a passport. Automated data tasks collect meta-information about the task itself, likely how people interact with a machine. In contrast, volunteered data is gifted by the users and includes

interactions on social media platforms etc. (Dodge & Kitchin, Code/space : software and everyday life, 2011). While the first and third way of collecting data can give interesting insights into urban systems and city life, it is automated

Figure 2: A ubiquitous network of technology collects data which will be evaluated centrally and forwarded to further public and private organizations

forms of data generation that catches the attention of those concerned with understanding and managing future cities. Mobile devices, cameras and sensors measure real-time changes of any motion, temperature, air pollution and more. The smart city acts as “constellations of instruments across many scales that are connected through multiple networks which provide continuous data regarding the movements of people and materials’’ (Batty, Axhausen, Giannotti, Pozdnoukhov, & al., 2012, S. 482). As such, the instrumented city offers the promise of an objectively measured, real-time and holistic analysis of urban life and infrastructure. Therefore, advocates are arguing to be able to develop and manage a city free of weak, selective evidence and political ideology.

Figure 3 shows a render of Songdo–a smart city planned from scratch by public and private partners in South Korea. It is notable that Songdo is currently less than half full; only 20% of the commercial office space is occupied, and the streets, cafes and shopping centers still feel largely empty. What keeps new citizens away? The city is undeniable attractive: it has been designed family-friendly with the environment in mind—charging stations for electric cars, or a water-recycling system that prevents clean and drinkable water being used to flush toilets (BBC, 2013). However, whatever improvement we hope to achieve by overlaying our lives with digital mediation, there are a number of concerns, five of which will be examined briefly.

Concerns of smart cities

Data within a smart city is seen as being neutral in political ideology: data is simply data. Sensors and cameras have no politics or agenda and only reflect the truth about the world (Dodge & Kitchin, Code/space : software and everyday life, 2011). Big data in urbanism is inherently a good thing, seeking to make the living space safe,

productive, sustainable etc. However, data is more complicated than that. In its existence it is not independent of the ideas, technologies, people and contexts that conceive, produce, process, manage, analyze and store them. As Gitelman and Jackson, followed by Bowker, put it, “raw data is an oxymoron” and “data is always already ‘cooked’ and never entirely ‘raw’” (2013). What kind of data is generated is the product of choices and constraints, shaped by a system of at ethical considerations, public and political opinion, or technical know-how or resourcing. Then, data is situated, framed and used contextually to try and achieve certain aims and goals. A disciplinary system in law enforcement is influenced by social privilege and social values (Johnson, 2014). There is no doubt that big data systems do produce data which is useful for understanding and managing a city and its assets, but the politics and limitations of such data and the methods to produce and analyze them need to be examined as to the underlying values and agendas.

2. Technocratic governance and city development

The development of information and analytic system into urban managing and regulating promotes a technocratic mode of governance. It presumes into a state of “instrumental rationality” (Mattern, 2013), or what Morozov calls “solutionism” (2013) in which all aspects of a city can be monitored and measured and treated as technical problems which can be addressed through technical solutions. Complex social situations are becoming a decomposition into defined problems that can be solved or optimized through computation (Mattern, 2013). As Hill (2013) puts it: ‘‘[smart city thinking] betrays a technocratic view that the city is something we might understand in detail, if only we had enough data […] and thus master it through the brute force science and engineering.’’ Indeed, Mattern (2013) suggests that big data urbanism suffers from ‘‘datafication, the presumption that all meaningful flows and activity can be sensed and measured.’’ A situation captured by real-time data becomes possibly to be a model, which can then be understood, managed and fixed. Seemingly, it provides city managers with rational and logical decision-making, free of ethical and accountability concerns. However, technocratic forms of governance are failing to take account of wider effects of culture, politics, governance and capital that shape the city life. They do not address the root problems, but rather be more efficient in managing such. Centralized power and data access need to be

complemented with a wide range of policies and practices that are sensitive to the complex ways in which cities are formed.

3. The corporatization of city governance and technological lock-in

Alongside the critique that smart city governance is too technocratic in its nature is a concern that is being captured by the heavy corporate interests in smart city agenda (Greenfield, Against the Smart City, 2013). A number of the world’s largest software services and hardware companies are promoting their vision in broad campaigns:

and energy systems, while at the same time ensuring the safety and security of people and assets“ (Siemens Global, 2016). City governance promises to be a large, long-term potential market for their products. Therefore, companies are seeking to make their solutions a core, indispensable part of how various aspects of life are monitored and regulated. IBM is now offering a “IBM Intelligent Operations Center” which is based on the Rio Operations Centre (COR1_{)–which was before exclusive to the city of Rio–into a single product that can be applied to any other cities.}

Hill (2013) also warns that ‘‘literally hardwiring urban services to a particular device, a particular operating system, is a recipe for disaster, not efficiency […] Put simply, city fabric changes slowly yet technology changes rapidly […] There is a worrying lack of thought about adaptation in this desire to install the consumer tech layer as if it were core building services.”

4. Malfunctioning and hackable cities

Today, many types of service provision to support everyday human activities concerned with food and water, distribution and transport, energy, health, etc. are heavily reliant on computer systems. Traditionally, service access devices are designed and oriented towards users who are engaged in activities that access single isolated services, e.g., to access information, to speak on the phone or to consume media. If we want to access and combine multiple services to support simultaneous activities, we need to use separated devices. In contrast, service offerings tomorrow will not happen on single devices. The use of ubiquitous and pervasive computing in city environments is creating city services and spaces that are dependent on software to function. They will provide more integrated, seamless and ambient service provision (Poslad, 2009). In its core, ubiquitous computing cannot work in isolation (Weiser, 1993). Dodge and Kitchin term this environments “code/spaces” (2011), in which every possible technology must be compatible to its counterpart. This means, when the software for controlling a subway crashes, then all trains do not run. Code/spaces are exposed to viruses, glitches, crashes and security hacks and are in constant need of patches and updates to cope with new contingencies. And as systems become ever more complicated, interconnected and dependent on software, the challenge of producing stable, robust and secure devices and infrastructures increases (Townsend, 2014). The technology has the potential to improve various set of urban problems but what are the implications of taking two highly complex and open systems–cities and information systems–and literally connect them? Or as Townsend (2014) puts it: ‘‘What if the seeds of smart cities’ own destruction are already built into their DNA? What if the smart cities of the future are buggy and brittle? […] The only questions will be how much damage they cause when they crash.’’

5. The panoptic city

Ubiquitous computing could create ethical dilemmas involving user consent. With few exceptions, individuals have traditionally used computer systems consensually. If computer systems become pervasive in all aspects of daily life, this could force people to engage without consent. In fact, individuals might well interact with these systems unwillingly (Townsend, 2014). With the development of various forms of directed, automated and networked

1 _{Centro De Operacoes Prefeitura Do Rio (COR) is a monitoring center of the city Rio de Janeiro, integrating about}

security systems, there have been increasing concerns over the rising level of surveillance in cities, especially acknowledging the data. It is now possible to real-time track individuals and their actions, interactions and transactions during work, consumption, travel, etc. Big data and their centralized control centers–such as the COR in Rio–integrate and bind data streams together into a single point. A society based on the combination of

surveillance (staring at the world) and dataveillance (trawling through and interconnecting datasets) is a world in which all aspects of a citizen’s life are captured and potentially never forgotten (Dodge & Kitchin, "Outlines of a world coming into existence": Pervasive computing and the ethics of forgetting, 2007). The government of China and Chinese largest retailer Alibaba Group is now building up a social scoring system named “Sesame Credit”. By 2020, everyone in China will be enrolled in a mandatory national database that compiles fiscal and government

information that will rank everyone within Chinas borders (BBC, 2015). Such systems create a tension between more effective governance and the right for privacy, confidentiality, and freedom of expression. As more and more aspects of urban life are being captured it is important to keep this tension balanced with individual rights and the benefits of data-driven systems. Not only to preserve the trust in the government, but especially when so much of the data will be processed by corporate systems. Regulated oversight and law enforcement needs to prevent the abuse of data, otherwise a significant resistance and push-back against real-time analytics are very likely by citizens.

A closer look at the corporatization of smart cities

It can be questioned if the smart city will ever exist. Since it is build much as the Internet is build—one website, one application, and one step at the time it is hard to tell when a city reaches the state of being “smart enough”. There is not a one-and-only smart city concept, but components that are supposed to be put together successively, depending on the individual needs. Hollands (2008) thus identifies five main characteristics that define a smart city: a focus on social and human dimension of the city, on the promotion of community agenda aimed at social learning, education and capital, on social and environmental sustainability, an ICT driven urban infrastructure and a business-led urban development with a neoliberal approach to governance. More important, Holland (2008) points out a tension between the social, community-driven aspect of a smart city and the underlying neoliberal ideology that prioritizes market-led technological solutions to city governance and development. At this point of my research I was personally very interested in the economic interest in smart cities and I wanted to understand the balance of this tension.

Major partners are either deeply involved in either building cities from ground up (see Songdo), or partnering with established cities to retrofit their infrastructure with digital technology and data solutions (see Amsterdam). Private partners can have different roles in implementing smart solutions (Frost & Sullivan, 2013):

Integrators: Smart city Integrators are project integrators that bring together various sectors of the industry through pre-packaged platforms, thereby providing a unified, holistic, and end-to-end integration of multiple sectors.

Network Service Providers: Smart city network providers offer collaborative networks, data analytics and enterprise working solutions that connect people, assets, systems and products by leveraging on their network and machine to machine communication capabilities.

Example major partners: Cisco, Huawei, Ericsson, Siemens

Pure-Play Product Vendors: Smart city product vendors provide “hard assets,” such as smart meters and

distribution devices (e.g. automated switches, controllers for capacitor banks and voltage regulators) that operate as the main nodes of connectivity.

Example major partners: Honeywell, Hitachi, Schneider Electric, Siemens

Managed Service Providers: Smart city managed service providers offer round-the-clock monitoring, complete management, compliance monitoring and on-site consulting.

Example major partners: IBM, General Electric, SAIC, Google, Serco, Microsoft

Both parties benefit by this kind of partnership. One major advantage for the public sector to work with the private sector is the increased flexibility of a non-public entity. Private companies do not have the same restrictions and regulations, so they can move more quickly (Kourtit, Nijkamp, & Arribas-Bel, 2012). And for the private sector this is a huge opportunity to be the pioneers in an estimated $1.5 trillion market (Frost & Sullivan, 2013), contrary to the contribution the European Union is willing to invest: since 2011 the project “Smart Cities and Communities“ provides €365 Million for the development of scalable and transferable solutions (European Commission, 2016). If one compares the market opportunities to the GDP of nations in 2016, it will sit above the GDP of Australia, thus making it the 12th largest GDP in the world (The World Bank, 2016). It is perhaps no surprise that big businesses are the strongest advocates for smart city development.

Many cities from around the globe have been keen to adopt the smart city vision and emphasize its more acceptable face supported by the private sector for self-promotional purposes. In addition to assuming there is an automatically positive impact of ICT on the urban form, the smart city label can also be said to assume a rather positive, high-tech future. As such, as Schaffers et al. (2011, S. 437) note, ‘‘smart city solutions are currently more vendor push than city government pull based’’, with companies working to create public-private relationships by putting favorable market conditions in place.

The concern around the involvement of private partners in the creation or conversion of cities is four-fold: First, that it is not guaranteed if these relationships are being long-term, according to Hollands (2008, S. 314): “Public-private partnerships and investment in these areas may in fact backfire, as information technology capital may flow elsewhere depending on what advantages are available to aid further capital accumulation.” Any qualified company or business organization will be allowed to build city infrastructures and services. Such a move actively promotes a neoliberal political economy and the marketization of public services wherein city functions are administered for private profit. Once implemented, the support for a product or a service though depends on its return of investment: Microsoft giving up on their 14-year old operating system Windows XP in April 2014 made total sense for them—

but not for people. It has turned out that 95% of all cash machines in the world are still powered by an now unsupported and exposed operating system (Bloomberg, 2014).

Second, that technocracy, or sensing technology as neutral objects is natural. Our social structure is built upon the assumption that untrustworthy humans living in an honest universe (Goldhill, 2014). That means humans are tempted to not say the truth if they think that they would gain some benefit that outweigh the risks. Ask a person if they always wear their seat belt while driving, and the answer will be at best suspect. But we do not expect objects to do that: if we test the energy consumption of a simple electronic device, we expect an honest response from it. Objects fail, and act unpredictably sometimes, but they are not strategic like humans. The idea of intelligent

computing, controlled by its vendors, is to equip all possible objects with software, and the temptation to teach those objects to lie strategically may be as hard to resist for companies. A company called Renew worked together with the city of London to install around 200 smart recycling bins, which were supposed to show advertisement on screens and the possibility to connect to a public Wi-Fi. But the network of these hotspots did not only show advertisement, it also secretly recorded the unique identifying MAC-address of more than four million smartphones passing by those bins. (Quartz, 2013), without the knowledge of the city or inhabitants. If the smart bin knows, that a specific device is passing by to buy a coffee at the corner every morning around 8:00, it can then flash a targeted ad or a loyalty offer. Even for the people of London, who are already under heavy surveillance through their city-wide CCTV system, this kind of tracking was too much and the bins were shutdown.

Third, that it will provide city-wide systems that are running proprietary software and platforms—a technological lock-in designed and controlled by companies. The danger here is the creation of a corporate path dependency that cannot easily be undone or diverted. Hill (2013) compares the strategy of smart city private partners to American car manufactures in the 1950s. Back then, they influenced urban strategies by promoting a shutdown of public transport and a replacing of it with road building programs to increase traffic. The program then shaped patterns of urban development in the following decades. More recent examples can be found in the smart home movement (as a smart city on a micro-level); the much-hyped Nest thermostat automatically adapts temperature in buildings based on the presence of the residents. It released a software update over the Christmas holidays 2015 which turned the device into a complete malfunctioning gadget that left a number of people without heating, in the middle of winter (The New York Times, 2016), followed by a debate about what the benefits are, of making something as simple and useful as a thermostat smarter, and more important what the point is, of leaving the control of such a crucial device to an external company.

Last, that it leads to ‘one size fits all smart city in a box’ solutions that take little account of the uniqueness of places, peoples and cultures (Townsend, 2014). Generally, infrastructures and buildings generate around the people and communities inhabiting the space. In contrast, a non-modular and technologically deterministic approach in building systems raises the question of appropriation (Chalmers & MacColl, 2003). In their paper on the duality of space and place, Harrison and Dourish (1996) installed the same system at two different places but with an important difference in its design. The one system used complex and invisible-in-how-it-works hardware and was reported as

disappointing, while the other system used low-cost, easily manipulate, visible hardware and was reported as wonderful. Harrison and Dourish propose that the expensive, complex system did not feel to be owned by its users, inhibiting adoption. On the contrary, users could and did play with the less polished system. Chalmers and MacColl

drew from this the inference that systems presented as seamless would be difficult to appropriate, claim and

customize in the ways that seem to make people happiest. Moreover, Bruno Latour (1999) defines this phenomena as a black box: “When a machine runs efficiently, when a matter of fact is settled, one need focus only on its inputs and outputs and not on its internal complexity. Thus, paradoxically, the more science and technology succeed the opaquer and obscure they become.” The input and output of a gramophone is visible. The enormous tube shows how the music can be heard, the spinning record and the needle shows where the music comes from and the turning handle generates the power for the system. In comparison, the iPod leaves the user clueless. Invisible systems lead to an urban development, in which “[…] the people who will be most affected by it, the overwhelming majority of whom are nontechnical, nonspecialist, ordinary citizens of the developed world, barely know that it exists” (Greenfield, Everyware, 2006) and eventually shaped not by culture, but technology–resulting into a space which lacks identity: „Smart cities tend to sense objects not people, infrastructure not culture. While the drive behind monitoring infrastructure is understandable, could it be that it inadvertently generates a less human-centered approach to urban governance? You manage what you measure, after all“ (Hill, 2013).

Cities and their infrastructures are already the most complex structures ever created by mankind. Initially I wanted to understand how balanced the vision of smart cities seems. I found a distinct tendency towards neoliberal approaches which leads me to the conclusion that in the cities of the future companies will have a significant—if not the most significant—influence on how we rethink this infrastructure. With an agenda of growing and expanding in mind not only their products will be pervasive but also their control and interest in a larger profit. Hiding in the complexity they can find numerous opportunities to achieve this goal. When urban planners have to plan for everyone, companies only need to plan for customers. With access to nearly unlimited personal and meta information companies can identify and target each citizen as a possible customer. Based on the findings, the city experience can be manipulated to the point where it becomes a market.

Part II: Critical Design–Investigating the preferable Future

“We watch passively as the world fills up with ubiquitous systems not designed with our interests at heart—at best presenting us with moments of hassle, disruption, and frustration beyond number, and at worst laying the groundwork for the kind of repression the despots of the twentieth century could only dream about.”

Adam Greenfield (Everyware, 2006)

Unpacking the vision of smart cities shows what concerns and consequences the citizens may face in the near future. In the present, the future is determined by government and industry, and although we play a role as consumers and voters, it is a limited one. It is for me very interesting to look at the extend of today’s free market capitalist system in the context of smart cities. To turn this tension into a controversy that addresses the issues of commodification I will explore this topic through the lens of Critical Design. Critical Design is a speculative approach in design research and uses design fiction and controversy design proposals to challenge preconceptions, expectations and to raise

awareness or open a debate about the role technology play in everyday life. It is an attempt to foreground ethics in design practice and technology, to expose hidden agendas and values, and to explore alternative design values. The term Critical Design was first used in the books Hertzian Tales (2005) and later in Design Noir (2001) by Anthony Dunne et al., professor and head of the Design Interactions department at the Royal College of Art in London from 2005 to 2015. In his book “Speculative Everything” (2013) he explains that “there are many people using design as a form of critique who have never heard of the term critical design and who have their own way of describing what they do. Naming it critical design was simply a useful way of making this activity more visible and subject to discussion and debate” (by that Dunne means also design as critique in other disciplines, like architecture, product or fashion design, etc. from the late 1960s and upwards). Being a discipline that originated from fine arts and literature, it started a still ongoing discussion how Critical Design can be situated in HCI (Bardzell & Bardzell, 2013). Similar approaches were already formed before though, see Value-Sensitive Design (Friedmann, 1996) or Reflective Design (Sengers, Boehner, David, & Kaye, 2005).

Critical Design investigates in the future, but not as a destination. The idea is to use speculation about the use of technology in the future as a tool to better understand the present and to discuss the kind of future people may want or not want.

Dunne and Raby refer to futurologist Stuart Candy‘s categorization of the way we separate futures in design research (Figure 4). First, the probable future. Most design methods are affirmative and oriented toward this space. It

describes what is generally expected to happen unless in the extreme event of a crisis (financial, ecological or

political). The second one, the plausible future, is the space of planning and foresight, the space of what could happen. It is mostly used to ensure an organization will be prepared in case the probable future is not going to happen. The third one is the possible future. It includes all possible scenarios, which are difficult to imagine how they are going to happen, but not impossible based on our current understanding of science. Beyond this scope lies the zone of fantasy. The future, in which Critical Design is investigating is another one and is called the “preferable future”, situated in the probable and plausible future (Dunne & Raby, Speculating Everything, 2013).

Figure 4: Stuart Candy‘s categorization of the way we separate futures in design research

The term “preferable future” is intentionally not straightforward, and that’s where the speculation can begin: what does preferable future mean, for whom, and who decides?

Doing Critical Design

Critical Design literature defines Critical Design and offers dozens of examples and inspirations of it, but it says much less about how to do it. The Critical Design process is about thinking through design to engage people and open a debate, which does not immediately lead to useful objects.

Choosing a debate

Among the most basic decisions in a Critical Design project is identifying which aspects of the present world will be speculated on. Though, different to just being science fiction or engineering driven scenarios the design fiction uses narrative elements to envision and explain possible futures for design. Design fiction is a form of what Stuart Reeves calls “envisioning” (2012): using fictional scenarios to work through the benefits, challenges, and implications of a new design idea or technology. Fiction is based on imagination—on the most basic level it is about questioning underlying assumptions in design itself, on the next level it is directed at the technology industry and its market-driven limitations, and beyond that, general social theory, politics, and ideology. The designer can create stories of fictional worlds, utopias/dystopias, enlarged neoliberalism, thought experiments or reduction ad absurdum forms. Also very popular are counterfactuals or what-if scenarios (Dunne & Raby, Speculating Everything, 2013). A design fiction has to imagine a culture of use for a technology or design that has implications for how it is executed and built.

Choosing an approach

All Critical Design tries to offer an alternative vision to how things are: it is a form of constructive design research and not only a form of design practice (Bardzell S. , Bardzell, Forlizzi, Zimmerman, & Antanitis, 2012). Constructive design research details three different approaches: Lab, Field, and Showroom (Koskinen, Zimmerman, Binder, Redstrom, & Wensveen, 2012). In the Lab designers invite other designers for workshops to investigate a question. They then build on the outcomes by making several versions of the same thing, an artifact that investigates a more singular interaction question, and conduct laboratory style experiments to test if the hunches formed in the workshop play out. In the Field examples, design researchers employ lightweight versions of social science methods to understand the current state, and then use methods from design practice to create new things that express a preferred state. For both lab and field, there are sets of methods design researchers can pick and choose from in the design of their research. Showroom, Koskinen et al.’s term for Critical Design, is different. The description in the book provides “tactics” of how methods might be put together to complete a project.

The first one asks the designer to participate in a dialog about the meaning of their work. But not as artists in exhibitions, since exhibitions are often “compilations of many projects collected under an umbrella envisioned by a curator. Typically, the curator places the work into a new framework by juxtaposing things that were not necessarily included in the original research projects. Some research concerns and knowledge might be present in the exhibition, but many are not, and yet others are typically rephrased or substituted. […] This explanatory framework reflects the curator’s interpretation of the research, which may differ significantly from the original goals of the researchers.” Therefore, researchers need to bring the discourse into the real world instead of galleries, to places where the everyday life is happening.

“Critical design might borrow heavily from art's methods and approaches but that is it. We expect art to be shocking and extreme. Critical design needs to be closer to the everyday; that's where its power to disturb lies. A critical design should be demanding, challenging, and if it is going to raise awareness, do so for issues that are not already well known. Safe ideas will not linger in people's minds or challenge prevailing views but if it is too weird, it will be dismissed as art, and if too normal, it will be effortlessly assimilated. If it is labeled as art it is easier to deal with but if it remains design, it is more disturbing; it suggests that the everyday life as we know it could be different, that things could change.”

Dunne & Raby (Speculating Everything, 2013)

Another tactic is to put the prototypes in the real life and to study what happens to them. Through empirical research even very explorative design can turn into research objects. However, “for Showroom researchers, fieldwork is typically not about issues around use but about issues like form.” (Koskinen, Zimmerman, Binder, Redstrom, & Wensveen, 2012) The responses can be collected and compared to the intentions that had been expressed by the designer.

The third tactic is to create the design artifact at a high professional level. This “catches the attention of professional designers, who do not get to label researchers’ designs as art, bad design, or simply not design. If researchers succeed in being taken seriously as designers, they may be able to direct attention to the intention behind the work”

manufactured. Koskinen et al. (2012) refer to the work of the Memphis group, an Italian design and architecture studio that designed postmodern furniture. Through their polarizing, but still well-designed work, they were taken seriously and had eventually a strong impact on the design of furniture, architecture and graphic design (The Guardian, 2001).

Choosing a strategy

Based on the second tactic Vera Khovanskaya et al. applied Critical Design to personal informatics in their project “Everybody Knows What You’re Doing” (2013). The goal was to raise awareness of and reflection on the

infrastructures behind personal informatics. They created a browser extension, which was running in the background and collecting all possible statistics about the internet use while the participants were using the web (Figure 5). For the study they interviewed each of the 23 participants during the course of two weeks, followed by an affinity analysis of the interview data to evaluate the form of the extensions.

Figure 5: The browser extension

More important, they identified three different strategies to execute a Critical Design approach during their process. The first one, “Make it Creepy”, showed the collected data in a frightening way. They wanted to make the constant surveillance and the highly personal aspects visible by pointing out the scale of logging in a commonly perceived anonymity of the user.

“For example, the fact ‘Did you know that we’ve been recording your activity for 5 days? In that time, we’ve seen you online for 200 total hours, and recorded more than 200 sites you’ve visited’ calls attention to the scale and continuous nature of web data logging, as well as the extensive infrastructure that exists for gathering and manipulating users’ data.”

The second one is called “Make it Malfunction” and deliberately display gaps in the gathered data, based on the theory that data is always incomplete. This leads to a misinterpreted or even completely ridiculous picture of the user, which is nevertheless plausible.

“For example, the Chrome API reports how long every tab has been open but not how long it has held focus; if a user has five tabs open for one hour, a naive reading of the data provided by the API makes it appear that the user has been online for five hours and has spent one hour on each site. This aspect of the data led to situations in which the system told the user that s/he

had been online for longer than 24 hours in a single day. […] The conclusion is obviously wrong, but the wrongness is the point.”

The third one, “Make it strange”, shows information in unconventional ways. Normally, data is collected for a specific purpose. Based on the assumption that the purpose shapes the user’s relationship to the data, the extension started to make fun of the personal information.

“To make it strange, we followed a number of approaches. For example, one fact told the user, ‘In the time you've spent on the web, Apollo 11 would have gone to the moon and back 1.5 times.’ or ‘You visited 592 websites this week. That’s .5 times the number of webpages on the whole internet in 1994!’”

Related projects

The United Micro Kingdoms (Figure 6) is divided into four super-shires inhabited by Digitarians, Bioliberals, Anarcho-evolutionists and Communo-nuclearists. Each county is an experimental zone, free to develop its own form of governance, economy and lifestyle. The project imagines the four different cultures and ideology through the field of transportation, represented with engaging models: the digital cars, the giant train, a genetically engineered mutant that is half horse, half ox, a vehicle grown from a portion of yeast and tea. According to an opaque system of prices, you can buy different levels of privacy, size and performance. What might be a paradise of frictionless movement across an open landscape becomes a zone of status envy, marketing and congestion that is not unfamiliar.

Figure 6: Anthony Dunne & Fiona Raby - United Micro Kingdoms (2013)

The engaging fiction is incredible thought-through and legit; the presented models evoke a society by themselves. Even though they created a huge world they concentrated on transportation only, as being a very central part in our lives. This imaginary state is a not wholly unrealistic projection of current technologies–driverless cars are being developed now and a core feature of the smart city, along with the idea that the important thing will be the software that manages them rather than the hardware of the vehicle. The final exhibition is a direct inspiration to this research and contained short films, objects, visualizations and more. The purpose of inventing the kingdoms, however, is not to provide a precise prediction of the future, but to offer physical suggestions of what might happen and to show that the causes and effects of design and technology are political, social and economic.

Hyper-Reality

Hyper-Reality (Figure 7) presents a provocative and kaleidoscopic new vision of the future, where physical and virtual realities have merged, and the city is saturated in media. The six-minute Hyper-Reality video explores Matsuda's concept for a future where augmented reality has been carried to an extreme, with interactive virtual interfaces saturating the urban environment and identities controlled and expressed through digital platforms. It features a central protagonist–42-year-old Juliana Restrepo–who has become disillusioned with her life in Medellin, Colombia. Her vision is filled with games, internet services like Google, and various other functions, alongside adverts that pop-up as she moves around the city.

Figure 7: Keiichi Matsuda - Hyper-Reality (2016)

Starting with a Kickstarter campaign in 2013 Keiichi Matsuda puts a lot of effort into the enormous amount of digital animations overlaying the city. The imagined world shows a society in which physical and virtual realities became intertwined, as a second layer on top of every reality. The digital overlay not only assists; it determines the life. The city experience offers various functions: the "reset" option deletes her digital identity, and accumulate points as she goes about her daily activities, while pop-up avatars act as personal advisors and a help line for the augmented reality system she is plugged into. The visual language is strong. For the viewer the city feels like a product; just a canvas in the background for the digital world to be displayed on. Where companies usually only promote the positive aspects in a bigger picture, the film shows the vision from the perspective of a citizen. At the same time, it stays descriptive, which makes the impact on the viewer even stronger.

Corner Convenience

With film “Corner Convenience” (Figure 8) the Near Future Laboratory want to show how various new technologies might propagate and used that intense normalcy of the corner-shop to do it. It works with humor and stereotypes, is

low budget and was done in 36hrs. Still, each short films within this collection is very clear and understandable even for people who are not deep into design or technology.

Figure 8: Near Future Laboratory - Corner Convenience (2012)

I chose this project especially because the method it is using. The film allows on a very simple, but nevertheless strong way to follow the imagined future. Rather than making an extensive use of visual effects it keeps the displayed scenarios real. The film focusses on details, things that are so tangible as imminent as to be almost ignored. The normalcy of a corner convenience store, the products which were once fantastic, extraordinary, disruptive, are now leftovers. It triggers the imagination in a clever way, by leaving the watcher with the question: If the things we imagine as the future are as normal to be sold in a usual convenience store, then, how does the world around the store must look like? It can be argued, however, that the narratives around the object are misguiding this point. The stories are built around clichés. For example, a truck driver buys glasses (like cheap Google glasses) which show porn. These clichés let the viewer tend to the assumption, that the technology had no impact on the society (or vice versa).

Part III: Facilitating the debate

Creating the fiction

Following the method of Critical Design, a first step was to create the design fiction. Design fiction uses methods of design to create a speculative frame about a potential future by demonstrating that future in a context that a large public audience can understand. More important, the frame provides an opportunity to imagine a technology in a human context. The narrative sets the ground for the debate and expresses the context, engages people, and makes it possible for them to connect and to empathize. I used common human emotions to make the context

understandable–therefore, none of the stories are neutral. Furthermore, I centered each story around a person’s life in a smart city. Based on my research I set a starting point which should reflect the fictional future scenario and be included in the following narratives:

“Facing massive over-population governments and companies created enhanced cities as the solution, with technology as its main driver. In these smart cities companies are looking more and more for power over people, to increase profits and authority. They control and influence every person’s life; people are free in their way of life but only in the options available through the market.”

Previously, I laid out three main concerns about the involvement of companies in future urban development and governance. These three concerns should be represented in three different narratives. Each one hast the potential to trigger its own debate, therefore each story differs in its strategy and positioning. Through these variables it was later possible to evaluate the narrative. The first story, “Dependency” reflects the marketization of public services and is dealing with the power of companies over people. After a powerful corporation’s lobby for a repeal of the ban on slavery, a young woman named Julia is tricked into becoming a slave of a corporation. The story is simple, easy understandable and has a clear, creepy argument; the dystopian aspect is strong. The second story, “Manipulation”, deals with the underlying software which slowly takes the control over the city. Elders are becoming more and more important in urban areas and the city adopts to their presence. The narrative plays with the ambiguity of both, the utopian and the dystopian world. The argument is less plain, and embedded in the world around the main character. The last one, “Separation”, includes the loss of identity in the urban space. It shows the broken relationship between the city and its citizens as a result of the transformation. The story builds on a strange feeling of impenetrability while the main character is clearly lost.

1. Story “Dependency”

After graduating from University of Law in 2024 she is facing her unbearable student debts, but no chance to find a job. In recent years, most of the lawyers were replaced by computers and algorithms; something no one would have expected. But they proofed to be more efficient and cheap so fast, that lawyers nowadays are only needed for rare complicated legal cases.

She is busy working as a waitress to survive, the life in a modern city became very expensive, but there is not really an alternative. One evening she finds this serious-looking web ad, that promises a simple way to get rid of all her pressing debts at once. She is sceptic, but after a meeting with a very enthusiastic salesman she is signing a contract.

Indeed, the contract contains a lot of small prints, but the young woman didn’t really care to read every sentence in the terms and just skimmed through the documents. She trusted the nice salesman with his promising hope and signed a contract. “Congratulations, you are now a slave.” said the salesman. “Ha-ha”, said Julia, and looked confused at the salesman and his trustworthy smile. “Did you not follow the news?”, replied the salesman and pointed to the phrase “agree to an indefinite period of servitude”, on page 546.

2. Story “Control”

By 2050 the number of people with dementia has tripled. Yet, the ongoing surge of retirees has been re-cast as an advantage, and the smart city became an idealized community for elderlies, an urban paradise for the obsolete. The city turned to a “soft-infrastructure”: Street signs are in large print and audio. Transportation is limited to slow speed. Golf is played through the streets on Sundays. Henry, one of the numerous old and retired persons, is waking up in the morning. The sun shines. His room is equipped with responsive photo frames that are supposed to reinforce the identity of his close family via shared ambient presence. Instead it is just showing stock pictures, but Henry doesn’t realize. “Are you feeling well today, Henry?”, an invisible voice is asking him something. “It is time for your pills”, the voice is saying. Henry needs a moment, but then he remembers and grabs the package with dementia pills from the bedside table. “Yes…, thank you Isa”. His personal computer Isa is organizing his life and taking care of his health situation. She has access to all of Henry‘s personal information and connected devices. The computer also decides most of the things for Henry—manipulating him into activities that are efficient for the system. “Get up, Henry, you are going to meet your old friend Gina today”. “Who is Gina?”, is Henry thinking, while he is getting dressed. When he leaves the house, a wagon is waiting for him. The streets are replaced by automated transport and built around the advertisement in big letters. After a short ride the wagon stops and leaves Henry in a street full of Restaurants. Isa guides Henry through the street with instructions he gets with a deaf aid in his hear. He is going inside, meeting a stranger, an elderly too, a good-looking one. Henry likes her. “That’s Gina.”, Isa is saying and but he cannot recognize her. Meanwhile Isa orders something to eat and drink. Henry sits down, and Isa tells him that they know each other from Golf on Sundays. Henry asks Gina: “Do you like Golf?”. They start to have a conversation.

3. Story “Lost”

The pervasive deployment of air pollution masks had become a mandatory feature of the urban experience by the 2010s. At first glance, citizens during the 2040s seem to have much in common—surgical masks turned into audiovisual masks, though, covering for a different kind of pollution. The smart city is experienced as an immersive projection now, with data overlaid onto the physical and constantly overstimulating. Information is flashing, sounds becoming incomprehensible due its overall mixing with the environment. The masks seem a little bulkier than normal glasses, and have apparently integrated filters. This new kind of pollution is cloaked by the glasses, just as all urban noise is cleaned. More people than ever are living in the city; cocooned by the mandatory masks. But they don’t really feel home anymore—it is quite the opposite. Between the overwhelming infrastructure an anonymous person starts wandering the city in a search for unadorned shelter, a space of audiovisual silence, where it can wide open their eyes, ears cocked, experiencing what is left of the unmediated.

Evaluating the fiction

Figure 9: Workshop, second round of reading

After creating the short stories, the next step was to find out if the reader was able to understand each. I let people read the stories with and without explaining the starting point, to see if the argument would become clear without this context. Each reader was interviewed afterwards. Further questions were (with comments):

-

-

-

-

-

-

-

-

-

The feedback was collected with people from different backgrounds and was overall positive. Everybody understood what the stories were about. Some people did not get the context of the smart city. The second and third story was favorited, even though the first one was much clearer in its position. The readers liked the unclarity. Especially story #2 directly resulted in a diverse discussion. Readers told that they needed to go through story #3 multiple times and debated about its meaning.

I changed some misunderstanding parts in the stories and let in a second round a group of people read the stories at the same time (Figure 9). Afterwards I asked them the same questions, but with the intention to open a debate. The city in #1 was pictured dark. People are suffering on the streets, asking for money. The salesman was pictured as a “devil in the suit”. Comments matched that this scenario was very feasible. It was great to see how stories #2 and #3 resulted in a broad discussion, questioning how much technology is “too much”. Story #2 was pictured like Florida, “everybody is wearing pajamas”, with transportation like golf cars and grass instead of pavement. A movie

association was “The Truman Show”. The dementia was initially meant as a metaphor for losing control, or being easily manipulable, but it was hard for the participants to not focus on the disease and computer as caretakers only, though. The debate came always back to that point. It was nonetheless interesting to see how readers, who have a basic understanding of technology, had a more cautious opinion about it; others weren’t less cautious or even happy about company services caring and deciding for elderlies. Story #3 triggered the imagination much broader. The description went from cool citizens, dressed in white and grey and clean surfaces to normal but careless people, wearing weird glasses and isolated from everybody else. The group was also not sure if the object was a provider or— as intended—a blocker of information. After I made that clear, the debate went into the right direction.

Based on the feedback I decided to go with story #3, because the narrative proofed to trigger a debate that I am aiming for. The story includes a narrative, which shows how the relationship to the city as a place for people changed, and a prop, a mandatory mask as part of the urban experience. Nonetheless, I needed to shape the narrative to make the intention clearer for the recipient.

Designing the fiction

Now, the next and last step for me was to decide for an approach. My target participators for the debate (but not limited to) are designers (Interaction Designers in particular). I had to choose a method, or medium that makes this scenario accessible in a way the designers of this field are used to, but at the same time the artifact should make the access of a wider audience possible, too. Based on the recommendation by Sterling (2009) I create a video. Explaining scenarios through a video scenario is a common and powerful method for communicating between Interaction Designers during various stages of the process. It ranges from displaying simple, short sketches up to complex exploration

of a design. Designing a fictional film in Interaction Design evolves from a usual video prototypes, though. Rather than just filming, the designing of style and conventions offers the ability to show systems, structures and processes (Mackay, 2000). At the same time, presenting the issue in the genre of design fiction creates an entry point to the debate for others: non-designers (urban planners, citizens, etc.) are a more than valuable extension. During the process of the scenario building I created different prototypes, which were supposed to help me understanding the challenges and difficulties. I used the prototype model developed by Stephanie Houde and Charles Hill (1997). It helped me to understand what prototypes I exactly needed before starting with the final video and sound recordings with my actors. It defines prototypes in a three dimensional space (Figure 10), named role, look and feel and

implementation. The role helps to understand the context of the prototype. I realized that I did that already: building the scenarios and analyzing their feedback focused mostly on the role of the artifact, and little on the look and feel and implementation (prototype #1). Therefore, I wanted to concentrate prototype #2 around the audiovisual language of the video. Prototype #3 was supposed to help me understanding the limitations of my equipment.

The video is supposed to show three elements: a data-overlaid city, the role of the mask and the narrative of an anonymous person looking for shelter. From the beginning I had to keep in mind my own skills in creating such a video. Due to my lack of experience in 3D art or mapping I decided to work only with footage recorded with a camera and not to try creating complex animations. In that case, the prop needs to be created in a workshop and the sound is very important and key for triggering something that a visual effect would do instead.

Story #3 is told very different than #1 or #2. It is vaguer, not explaining how the city became a place for data

pollution and if people are happy or not. It tells how different a life in such a city is and how the relationship changed from the inhabitants to the city and to each other. The city is apparently not a place for communities anymore, missing the warm, comfortable feeling that comes with it. People live in the city, but not together. The narrative in the video needs a focus on this element: the recordings of citizens are centered around a public space, as a synonym for a place, which is supposed to be filled with people. The persons will stand and sit in the area, just as they do normally. But no interaction will happen between them, showing the isolation the mask creates. The person looking for shelter starts on the square, too. The underlying, hidden technology of the city talks to that person. It sits on the

pavement, looking down, so that the viewer cannot see the person’s face. Then the person will look up, and putting on the mask. The city stops talking and the person stands up. Although the mask is filtering, the person is still not satisfied. It is restless, and feeling very uncomfortable being in the city. The camera moves, triggering a feeling for walking. Eventually the person arrives at a self-storage facility (as a symbol for safety, but also anonymity). It is walking through the space, opening a box, entering and taking off the mask, door is closing.

The technology will be included under and above ground. I will use street lights as synonyms for remote metering data, shots around transportation and buildings for the flow and storage of information. Also, surveillance and other sensor technology will be displayed.

The setting of the video is in the night, using darkness as an amplifier for the discomfort, that the citizens feel. The actors are wearing dark clothing to fit into the setting. The visuals will be underlined by generated and common known data sounds. Furthermore, the city has two kinds of voices: the internal, a computer voice that speaks about status and processes, and the external voice, an adapted human voice, talking to its citizens. The background noise is supposed to add a feeling for the distance between the city and its residents.

The sketching

Based upon that, I created first visual mood and story boards (Figure 11). The boards helped me to understand the ideas that I got as feedback during the scenario building and to combine them with the vision that I had in mind while writing the short story. I started with inspirational pictures, taken from stock websites like shutterstock or videos found mostly on vimeo. Specifically, I searched for photographs and videos that were tagged with keywords like “separation”, “city in the night”, “alone in […]”, “relationships”, etc. With the images for each part at hand, I was able to mix and try out different versions fast and easy.

Figure 11: Example mood and storyboards (created with the free online software celtx)

After deciding for the style and chronology of the pictures (Table 1) I continued to work on the audio. I started with the voices and recorded different people. As planned, I needed two different voices. First I began with the internal computer voice, saying phrases like “checking status”, “looking for entity”, “establishing connection”, “estimating

interest”, etc. The external voice on the other side is saying phrases like “Find your dream home”, “Up to 70% percent on our new collection”, “See our weekly featured products”, “The new way to progress”, etc. The voices are not supposed be completely understandable in the video. It was more important, how they sound. I utilized Adobe Audition to adjust the sound recordings with effects and to create different tones and noise for the background. Furthermore, I searched online platforms like freesound.

I arranged both, the pictures and the sounds using Adobe Premiere for prototype #2. This prototype set the language for the final recording and sound editing and was a crucial help during the preparation. I was ready to choose places and actors for the final recordings.

#1: Slow beginning. showing the technology above ground. Computer voices. Smart Streetlight in a park (as social place) gathering and sending information.

#2: Showing the city with different stills. Still slow. Computer voice and human voice. Maybe look for matching “texts”.

#3: Still of the Smart Buildings checking the status of its residents. first invisible contact of city (computer sound) and citizen (look for lights on/off).

#4: Still of the Smart Traffic. with human voice. Showing blurry images of the traffic (cars, busses, etc), In-vehicle telematics.

#5: First visual moment of the mask by showing “lost“ people (only computer voice): 1. people are waiting.

2. people sitting staring next to the camera.

#6: Person is sitting on the pavement, data overlay at the wall. Personalized sound. Person‘s head is down, you can‘t see the face. Mask lays next to it.

#7: Putting on the mask, laying back. Another shot: closing eyes (close up). Sound stops. Opening eyes again, standing up. Moving right, left, erratically. Mix with tripod and hand free recordings.

#8: Showing the technology underground. Empty parking garage, underground concrete. Human voice. Look for storage-like objects. Mixing the pictures with the person, showing the desperation.Increasing the speed of the images.

#9: Person is going to a self-storage facility. Showing unclear path. All sounds are increasing. Most intense moment. Showing the person walking through the space with a specific location in mind. Entering the storage. Taking off mask. Pause. Sound increasing. Door closes. End.

The creation

I could not bring a full video studio to each spot, so I needed to carefully select my equipment beforehand. I decided to use my own Canon EOS 700D with a fixed focal length lens of 50mm and an aperture of 1.8f plus a tripod. The lens not only helped me to create low light images with little noise, but allows a large depth of field which results into the image effect that I wanted to achieve, too. Furthermore, I used Magic Lantern, to adjust special settings in the camera software. Also a battery powered projector was necessary to make the reflections on the mask possible. The video footage was recorded in three days, mainly in the night. During day one, I visited different parts and storage facilities of Malmö to find matching spots and to plan the recordings. I recorded different shots with different settings and created prototype #3 to understand how I must use the camera in the night. On day two I recorded the stills in the city, on day three the parts with actors. Afterwards I sorted the footage and exchanged the images in prototype #2 with the recordings. At that point I was able to work on the sounds in detail and match them with the final film editing. Eventually I finished the video by stabilizing some shots, working on the final color grading and adding the titles (Figure 12).

Figure 12: The final video, uploaded to vimeo (click the image above or follow this link: https://vimeo.com/167885137) and pictures of the production