Re-entering Hertzian Space
through Affective Wearables
Sonja Rattay
Thesis Project - Interaction Design Master K3 / Malmö University / Sweden
Supervisor: Susan Kozel Examiner: Pelle Ehn
Examination date: 1st June 2015
ALL PICTURES & DRAWINGS IN CHAPTER 9 AND 10 ARE DOCUMENTATION FROM THE PROCESS OF MAKING THE THESIS, EXECUTED BETWEEN 1ST OF APRIL & 25TH OF MAY 2015.
Acknowledgements
During the course of this thesis I was lucky to have the support of a lot of great people, whom I want to thank sincerely. First of all, I want to thank my supervisor Susan Kozel for her valuable input and feedback thoughout the whole project. I also want to thank David Cuartelles and Marcus Johansson for their help in terms of physical prototyping. I want to thank MINC Malmö for the organisation of the wearables hackathon and the allocation of space and material for a long prototyping weekend, as well as the rewarding this project with the first price. I want to thank Nasir Hussein, Billy Lundh and Michael Hoy for joining my team throughout the hackathon, but even more for continuing to work with me on further iterations and for a hopefully succesful collaboration also in the future.
Finally, I want to thank Luisa Fabrizi for her interest in my work and the participation in my testings and Johanna Rochegude for illustrating and sketching out some of my design ideas.
Abstract
This thesis analyzes the domain of wearables and its current developments, and the notion of hertzian space, under the consideration of the emerging areas of the internet of things and ubiquitous compu-ting. It introduces wearables as an opening into hertzian space and presents a possible approach to how a wearable could be designed to function as a touchpoint between human body and hertzian space. It thereby looks into what electronic intimacy can look like and how it can be created and used to bridge an enstrangement between person and device.
This thesis illustrates a whole process of shaping this wearable device, including experimentation, digi-tal sketching, and different testing methods. From the making of this prototype it is concluded that en-hancing electronic characteristics and opening the hertzian space for interaction can create a stronger awareness of our electronic surounding and the invisible fields around us, which thereby also facilitates a better understanding of our position within hertzian space and offers a new personal perspective. This study provides an opening into the field of exploration of the hertzian space through wearables and should be seen as a generative design contribution, which ties aspects in the fields of wearables, hertzian space and interaction design together and also offers openings for further research.
Index
1. Introduction
7
2. Research focus and question
8
3. The evolution of the hertzian space - ubiquitous computing and IoT
9
4. The rise of wearables
11
4.1 Connected wearables 11
4.2 Fashion technology 12
4.3 Extending the context of wearables 13
5. Wearables in Interaction Design
14
5.1 Wearables in Interaction Design 14 5.2 Affective wearables and electronic intimacy 15 5.3 Electronic distortion of body perception 16 5.4 Wearables as opening into hertzian space 17
6. Related work within Interaction Design
19
7. Designing for sensuous discovery and meaning making
22
8. Methods and Design process
23
8.1 EMF as non-physical design material 23
8.2 Material Processes 23 8.3 Digital Sketching 24 8.4 Experiment methods 24 8.4.1 Embodied storming 24 8.4.2 Experience discoveries 24 8.5 Evaluation 25
9. Material exploration
26
9.1 Experiment One - Arduino based EMF Detector 26
9.1.1 Pilot Experiment 27
9.1.2 Wearable Extension 28
9.2 Experiment two - Rotation as output 29 9.3 Experiment Three - Theremin 30
9.3.1 First session
30
9.3.2 Second session
31
9.4 Experiment four - Pulse sensor 33 9.5 Experiment five – Wearables hackathon 33 9.6 Experiment six - creating an audio matrix 35
10. Testing
36
10.1 Testing in a lab environment 36 10.2 Testing in field setting 38
10.2.1 Introduction Interview 38
10.2.2 Probing notebook 38
11. Discussion
40
11.1 Process 40
11.2 Prototype 40
11.3 Research question 40
12. Conclusion & knowledge contribution
42
References
44
Linked products, designers, programs and events
46
Instructions and tutorials for experiments 46
Appendices
47
Appendix 1 - Interview Johanna 47 Appendix 2 - Interview Luisa 48 Appendix 3 - Notebook Luisa 49
1. Introduction
As human beings we assemble our view of the world through what our senses show us of it. However, our perception is rather limited in terms of what our senses are actually able to capture, compared to a lot of other living beings. Our ears are limited to a hearing range of 20 to 20.000 Hz, our eyes are limited to a viewing range of 400 to 790 terahertz. In between, as well as at both ends, lies a spectrum of waves, which still surround us at all times and are used for multiple purposes, like radio transmission and processors, but are not sensable for us.
With the miniaturization of electronics and the rise of ubiquitous computing due to increasingly capable sensors, the amount of devices radiating in this spectrum is expanding while at the same time becoming more sleek and beautiful than ever and hiding their electronic nature.
The space in which waves of this spectrum linger, has been named the hertzian space (Dunne, 1999) and represents an invisible space inhabitat by the radiations and signals of electronic devices surroun-ding us.
Due to the fast evolution of electronic devices, this space underlies an ongoing change in its configu-ration and has since developed into a new environment since its first evaluation by Dunne and Raby (1999 and 2001). The components radiating in this space have become both bigger and smaller, have become more distant and moved closer, as cities become smart and microsensors are integrated into wearables.
This study wants to find out how we would experience our world if these radiations as well as the electronic components which cause them, would not be hidden, but perceivable. It thereby focuses on wearables as an opening to create closeness within hertzian space and the human body under the consideration of what electronic intimacy could be and how it could be created.
2. Research focus and question
The purpose of this study is to uncover our cohabitation with electronic devices in the hertzian space and to create a design concept, which illustrates the entanglement of the human body with the hertzi-an space, through hertzi-an affective wearable. The hertzihertzi-an space is hereby based on the description by An-thony Dunne (1999), but is seen now as evolved into a more dense and almost overpopulated space due to the Internet of Things, an invisible communication space, transmitting messages and information between mobile electronic networked devices. The concept is meant to serve as a research contribution and answer to the research question:
“How can a combination of physiological data and electromagnetic field sensing be designed as a wea-rable to create a sensuous experience in order to uncover electronic intimacy within hertzian space?“ This research question leads to a few other questions, which go along in the exploration and design process: What is electronic intimacy? How can a relation between the wearer and the surrounding electromagnetic climate be created? How do people react to this relation? What kind of insights can be triggered through an artifact which puts electromagnetic fields in relation to our bodies?
As for the scope of this thesis, I intend to analyze both the domain of wearables and the notion of hertzian space. I will rework the notion of hertzian space under consideration of the emerging areas of the Internet of Things (Ashton, 2009) and ubiquitous computing (Weiser & Brown, 1996) and focus on how this has reshaped this area. In the domain of wearables I will focus on current developments and their meaning for the notion of hertzian space. In the design process, I will conduct mainly lab based material exploration and perform material experiments using an adapted version of embodied storming (Schleicher, Jones & Kachur, 2010) to explore an embodied interaction with electromagnetic fields. I will use wearables to create a connection between the exploration of the hertzian space and the personal body perception to create an illustration of the locative entanglement of body and hertzian space.
My intent is neither a utopian nor dystopian stance but to reveal a possible design space in the field of Interaction Design with a reworked take on hertzian space and a contribution to the effect of wearab-les. I hope that my work can become a generative basis for other designers to further explore hertzian space through wearables, as this combination has rarely been used for research, although it offers a very interesting and human centered approach.
3. The evolution of the hertzian space - ubiquitous computing and IoT
“The extra-sensory nature of electromagnetic fields often leads to the EM spectrum being treated as something notional. But while cyberspace is a metaphor that spatialises what happens in computers around the world, hertzian space is real. Cyberspace describes what lies beyond the screen […]. Hertzian Space describes what happens in front of the screen, outside of the object, it is part of the space our bodies inhabit, even though our senses detect only a tiny part of it. Hertzian Space is considered a medium for carrying informa-tion, an invisible alternative to wires and cables. […] Hertzian space is three-dimensional and spatial.”(Dunne and Raby, 2001, p.12)
Hertzian space is a term created by Anthony Dunne (1999) and describes a holistic view on a space consisting of radio space and the cultural and social implications and conditions we share with electro-nic objects. Radio space means the space of radio waves surrounding us, an invisible layer of a physical immateriality inhabited by electro-magnetic fields and radio waves (Dunne, 1999, p.101).
Dunne and Raby (2001) describe radiogenic objects and materials as unwilling interfaces between the abstract electro-magnetic space, and the material culture, which can be used to reveal unexpected points of contact between them due to their functionality and design.
The ideas and concepts, presented in the two books “Hertzian Tales” (Dunne, 1999) and “Design Noir” (Dunne & Raby, 2001), while very concrete and situated, feel rather poetic and focus on located objects. They are dealing with spaces, such as the domestic space or certain areas, which are defined by electro-climates - a specific composition of waves and fields, which identify areas like fingerprints. Since then, the hertzian space has changed due to the development of appliances inhabiting it. The appliances which are nowadays forming the hertzian space we live in, have become bigger, as concepts of smart homes and even connected cities emerge. They have also become smaller, due to the miniatu-rization of electronics. They have become more distant, due to the development of sensors and remote controlling, and have moved closer to our bodies at the same time, as we carry wearables, smart phones and laptops with us wherever we go. They also have become smarter as they start to collect data on their own and form abstract, virtual images of things on the internet and gather together in an exten-sive network, over which they constantly communicate - a data filled network, the Internet of Things (Ashton, 2009).
We have also entered the area of ubiquitous computing which Weiser and Brown (1996) have predic-ted for the time frame of 2005 to 2020, in which instead of us sharing personal computers, a variety of computers will share us:
“The social impact of imbedded computers may be analogous to two other technologies that have become ubiquitous. The first is writing, which is found everywhere from clothes labels to billboards. The second is electricity, which surges invisibly through the walls of every home, office, and car. Writing and electricity become so commonplace, so unremar-kable, that we forget their huge impact on everyday life. So it will be with UC (ubiquitous computing).” (p.5)
Both these visions of digital and electronic devices have evolved and come to life since the year 2000. While Dunne and Raby have focussed on what Weiser and Brown name as the second ubiquitous technology: electricity, imbedded computing as the third ubiquitous technology also feeds of the same hertzian spectrum, just on other frequencies, and thereby shapes the hertzian space into a layer we increasingly use and which changes faster day by day.
Innovative research programs try to bridge distances and to transmit even elusive things like emotions using this space while simultaneously trying to hide as much of it as much as possible from the actual user. New generations of devices (such as wearables and connected appliances) have re-shaped our electronic landscape and also the configuration of radio space.
The idea of the Internet of Things and ubiquitous computing relies on a extensive network of connec-ted appliances, which communicate wirelessly, creating an even denser layer of electronic signals. This additional space of implicit communication is tied into the existing hertzian space and thus reshapes it. This space itself grows with every attempt to create an ubiquitous experience and gains a new inhab-itant with every new device that communicates and radiates.
One of the main features of ubiquitous computing is that much of what is happening is situated in the hertzian space, in order to move activity below the threshold of our attention and awareness (ed. Ekman 2013). Veel (in ed. Ekman, 2013, p.124) points out, that this leads to an oscillation of calm technologies, between the center and the periphery of our attention as Weiser and Brown (1996) envisioned, being impossible, as most of the interaction is happening at a precognitive level (which also reflects the attempts of packaging electronics in a disguising manner).
With this transition of interaction into the hertzian space due to an increase of connected devices, one can assume that the radio space around us has become more dense and is populated by a whole new generation of inhabitants, emitting their own, new type of electronic climate. The space they take is not limited by their materiality but exceeds the borders of their packaging through radio waves and electrical fields surrounding them, extending them to new dimensions and connecting them both to satellites far away and our bodies extremely close, as many wearables use electronic sensors to measure our somatic data and thus create our own electro-climate, which is always surrounding us. The hertzi-an space has thus become more personal hertzi-and more global at the same time.
4. The rise of wearables
In 2014, wearable devices accounted for 65% of the market value for IoT (Internet of Things) de-vices, and is predicted to grow up to 73% in 2018 (The Mars Report, 2014). Aside from industries like the military, wearable technology for personal use has been around in the form of ideas, utopian and dystopian visions, concepts and unique artifacts for over 30 years, with a peak in consumer product releases in 2000 (The Mars Report, 2014).
Steve Mann started creating wearable computer systems in the 1980s with the vision that “smart clothing will significantly enhance our capabilities without requiring any conscious thought or effort.” (Mann, 1996). As his constructions were still bulky and cumbersome, he described the experience of wearing technology as something physically and socially awkward: “People were shocked by the visce-ral combination of human and machine“. (Mann, 1996).
With the ongoing miniaturization of computers and sensors, the design of modern wearables is still trying to tackle this antipathy by covering the electronic nature of the devices behind sleek and elegant surfaces instead of challenging it. Comparing modern commercial wearables aimed at the consumer market the direction is obvious: colorful devices, which hide their electronic-ness behind a fashion character. A part of those devices uses sensors to capture physical input, most prominently the pulse sensor, to allow for novel interaction with the body.
4.1 Connected wearables
Currently the most prominent group of commercial wearables, with the most public awareness, are wristbands and smart watches aimed at self-monitoring and self-quantification. These devices are connected wearables, with the main purpose being data collection and management. Most of them do not operate on their own but are connected to a network, for example with a smart phone or the internet.
The Jawbone1, one of the first fitness wristbands on the market, is positioned as a modern fashion
accessory with added value rather than an electronic gadget (Innovation by Design Conference, 2013). The recently released Apple Watch comes with a variety of different wristbands and is being marketed as the “most personal device Apple has ever created”, referring to an integrated pulse sensor, which personalizes fitness feedback. The Nike+ Fuelband also measures sleep activity in addition to heart rate during workouts.
Img 1 (left): Steve Mann wearing a headpiece with antenna and glasses and an electronic belt Img 2 (right): The Jawbone together with the fitness app, to track physical activity
These examples show that companies are currently trying to market wearables mainly as self-op-timizing gadgets, as a new lens to look at oneself and a new tool to optimize oneself. As stated in the “Wearable Future report” from 2014 by PwC, expectations for wearables are high on both the consumer and producer side. The report attests to a bright future for the industry and the potential to “dramatically reshape the way we live and do business”, presupposing that the industry focuses on a human-centered design approach and turns gathered data into services which actually provide additio-nal value for the user. This group also ties into the area of medical equipment and medical surveillan-ce, both personal and professional.
4.2 Fashion technology
Another group is fashion technology: clothing using sensors to create emotional and interactive pieces. Most of these pieces though, act more as haute couture for entertainment and art purposes (such as the K-Dress by Cute Circuit for Katy Perry or the Spider Dress by Anouk Wipprecht for Intel) or are part of innovation research (such as the Intel innovation program “distance togetherness”) and are rarely available for the common public, with few and very expensive exceptions (like some fashion by Cute Circuit or Elektro Couture).
In the future, wearable technology is seen as the ascent of a new category of product, with conferences exclusively focusing on wearable technology (Wear it, 2014) and big corporations releasing platforms and formats to foster and yield the creativity of the so-called maker movement developing wearable gadgets, based on mini chips and easily modified sensors, such as the Intel initiative “Make it wearab-le” Challenge (2014) and the GE Garages Tour (GE Garages, 2015).
Inside these movements and campaigns, a tech-savvy avant-garde of artists and designers has been creating wearable art and fashion pieces for some years now, using brain activity and bodily functions as touchpoint for interaction and feedback, pushing towards a state, where the human body merges with technology:
“As we move forward, wearable tech is […] promising more and more transformative po-wers in smaller and smaller packages. […] We want technological superpopo-wers, unbound, accompanying us, always ready to provide us with competitive advantages - wherever, whenever.” (Gamper, 2015).
Img 3 (left): The Spider Dress by Anouk Wipprecht for Intel is a futuristic dress with spider leg-like shoulder pieces, which move based on brain activity
Img 4 (right): The Apple Watch uses a heart beat sensor and vibration to imitate the heart beat of another person through the wristband
4.3 Extending the context of wearables
In both groups it becomes apparent that the wearable is foremost connected to the wearer, but barely set in context with the surrounding environment. Also in both cases the electronic characteristic has to be either hidden or packaged in a beautiful shell to avoid a cyborgian appearance as described by Mann (1996). It also shows how strongly the acceptance between digitally enhanced pieces differs - while the electronic fashion is embraced for entertainment purposes, the public acceptance of actual commercial devices, which link the wearer to an all connected network, is much slower. This study draws from both groups and reflects on these aspects, in order to gain insights into how context for a wearable can be created through electromagnetic field awareness.
I want to explore if and how the use of wearables could extend the current focus on efficiency or entertainment. I want to opt for a poetic locative context, which is not particular entertaining or pro-ductive, but rather reflective. I hope to thus create an enhanced awareness and extended insight into hertzian space, as awareness is the foundation for personal interpretation and thus personal choice. A focus hereby lies on how electronic aspects can be integrated in appearance and function and comple-ment each other, rather than being hidden in favor of ease of use and entertaining appearance. The resulting appearance should therefore be seen as a design statement rather than a fashion statement, as it contradicts the current common design assumption and standard of invisibility meaning ease and better understanding, as seen in the named examples of connected wearables.
5. Wearables in Interaction Design
5.1 Wearables in Interaction Design
As described in chapter 4, there are currently two streams in wearable technology - connected weara-bles and fashion technology. The latter can be situated closer to art and entertainment while the first is closer to the commercial market. Wearables in Interaction Design have so far mostly been discussed in terms of use scenarios, meaning seamless, embedded interaction, and clarity of communication and function. Those elements are closer to the area of connected wearables, with wearables being seen mainly as tools and carriers of functions rather than tools for exploration in themselves.
“Pursuing the suggestion that wearables enable a dissemination of immanence by means of intention and attention, the question becomes whether wearables converge with loca-tive media once immanent states radiate outward into shared social spaces. Once the ebb and flow of personal information are mediated by portable, location-aware technologies, like mobile phones, GPS, and Bluetooth, the argument that wearable computing becomes another strand of locative media and open-source digital architectures confronts the argu-ment that it is more accurate to construe wearables as a distinct domain.”
(Kozel, 2007, p. 294)
From this notion of wearables by Kozel I want to take that connected wearables are more than another strand of locative media, as wearables create their very own distinct social space as well as context, sha-ped by the perception of wearables inside our personal space and in correlation to our own bodies. On the other hand, connected wearables today are part of a global and mobile information and service network, connecting them to the idea of the Internet of Things, tying into openings smart phones have created for extended interfaces.
The development of smaller and cheaper sensors enables a broader spectrum of input possibilities - with the current focus in interaction design on developing comfortable, unobtrusive and ubiquitous wearable computing (Swan, 2012).
Instead of explicit interaction between user and interface, wearables equipped with sensors enable implicit communication between body and technology in an ubiquitous manner, through transitioning a lot of what is happening into the hertzian space and thus out of our current window of awareness. The interaction is happening while the user focusses on another main activity, though still being aware of the ongoing, passive interaction with the wearable device (Kranz, Holleis & Schmidt, 2010). Kranz, Holleis and Schmidt (2010) describe the major challenge in this field as ‘embedded interaction’: inter-action “with information that is not disrupting or distracting to their primary tasks and goals”.
For a wearable, which is meant for long-term use, the range of activities it could interfere with is much broader than for stationary devices, which are only in use during a limited time and in particular situa-tions. This has so far led to attempts to create commercial electronic wearables which merge seamlessly with other accessories and do not interfere with other common activities. These attempts lead to what Kranz, Holleis and Schmidt name as the ‘invisibility dilemma’:
“When embedding information and interfaces into objects, a vital design element is to hide this augmentation and leave the original function, look, and feel the same. Howe-ver, this physical disappearance and embedded sensing, actuation and interaction can affect the user’s perception and lead to the invisibility dilemma. Users must still be able to identify digitally enhanced artifacts known and used in everyday life as more potent than meets the eye. In addition, users must recognize this added value to accept and use such artifacts.” (2010, p. 51)
Aside from creating a disguised interface, it also takes away social meaning and expression. Wearables share characteristics with clothing which in itself presents an interface to our environment - what we wear transmits and receives meaning, such as emotions and experiences (Ryan, 2009, p. 312). Hiding an important characteristic of a wearable - in this case digital and electrical enhancement - makes us vulnerable in the way that we become unaware of this aspect and thus of its nature and its drawbacks. Wearables in themselves have not often been discussed in terms of what of them is invisible, but in terms of what is intrusive and could be hidden, and how to create an easy and clear functionality. Sin-ce the point of this thesis is to uncover the imperSin-ceptible, electronic materiality of deviSin-ces in our lives, it is only logical to think about the sensibility of certain characteristics of the wearable.
This thesis does not look into how to make a wearable device seamless and integrated but rather the opposite - how to create a disruptive but still alluring experience based on the invisible electromagnetic climate fabricated by our connected world. It aims to make the user aware of the intimate interaction with technology and its ‘personality’, enhancing the characteristics on both sides of the interaction - the human heart beat and the electronic pulse of the appliances surrounding and communicating around us. A particular focus lies on uncovering this immaterial presence, which extends the material border of appliances through signals and waves and fields, as it is often hidden in order to minimize the visible effect they have on our environment and create an un-intrusive use-experience, which blends in with the appearance of analog appliances, we are used to being surrounded by.
5.2 Affective wearables and electronic intimacy
“The view of affect as referring to a passage from one state to another can be mapped onto mobile, locative media as they encourage or inhibit human exchange. They are fluid, they are portable, they accompany us for hours, days, and seasons, which means they span moods and activities, cycles and rhythms of life. We integrate these little devices into our clothing and our daily gestures include the arm, head, and spine movements associated with using them. We walk and see differently when we use them. Even with something as basic and ubiquitous as a mobile phone, our senses are repatterned, our feeling for space and time folds inward or leaps outward. We carry the other with us in our hearts, in our memories, in our devices.”
(Kozel, 2007, p. 298)
The nature of mobile, locative media as Kozel describes is inherent to wearables as they share space and time with us and thus are used in a variety of contexts which they also shape in return. The device becomes a part of ourselves rather than an entity on its own, enhancing our very own physical abilities and senses, as it exists within our personal space (Mann, 1996). Sharing our personal space with them creates a form of closeness and electronic intimacy.
The Oxford English Dictionary defines intimacy as “the state of having a close personal relationship with somebody”, and “a thing that a person says or does to somebody that they know very well”, a quality mainly used referring to positive, warm interpersonal relationships (Oxford Dictionary). But as technology evolves and becomes more complex in terms of action and reaction, we also build relationships with devices, which hold an intimacy on their own - an electronic intimacy. In terms of electronic intimacy I would distinguish two forms: electronically mediated closeness between people and closeness with the electronic device.
Wearables which strive to bridge distances between people by creating technologically mediated inti-macy try to imitate human touch or human closeness as naturally as possible. Trends like long distance togetherness (Intel, 2014) are met by commercial devices like ‘bond touch’ and ‘tactilu’, both wrist-bands that transmit tactile feedback over long distances in form of pocks and tickles.
Imitating very basic haptic interaction, these and similar devices are examples of how technology enters into an intimate space between two people while at the same time trying to reduce the distance between them. The technology becomes intimate through the connection with another human being and the association of intimate interaction between technical feedback. The mediating device becomes a placeholder, which tries to be as seamless as possible, to vanish in favor of a better imagination of the human being at the other end.
A certain type of intimacy though, can also exist between human and technology itself, as a lot of peo-ple may have experienced with smart phones - containing fragments of our everyday lives, information about people we interact with, our guilty pleasures, our best memories, a phone in itself can be seen as an actor, not only a mediator. As Kozel describes, “wearable devices as they are networked together, between bodies or traversing a single body, bear witness to our constant exchange with alterity as a form of having-the-other-in-ones-skin” (Kozel, 2007, p.283, referring to Diprose 2002, p.115). With the growing distribution of sensor equipped wearables, this formerly rather abstract intimacy is now combined with actual physical closeness.
Technology enters the personal space not only on an emotional level, but also physically, as ‘affective wearables’2 - electronic systems, equipped with sensors and tools, which enable the device to capture physiological and affective patterns of the wearer and are in physical contact with the user in a long-term intimate way (Picard & Healey, 1997). Through somatic data collection combined with behavior patterns, algorithms try to learn to reconstruct emotional states of the user, which allows affective tech-nology to react on a much more intimate and complex level to our emotional state (Picard &Healey, 1997). While affective wearables are designed to be as close to human emotion as possible, connected wearables are more simple in that matter, as interfaces are foremost reduced to simplicity and ease of understanding.
Based on the above discussed points, the fact of electronic intimacy in the field of wearables is import-ant and will therefore be considered for this thesis as it shapes the experience and perception of oneself in the context of hertzian space in a different way than through a basic electromagnetic field detector tool for example. Intimacy in itself has a capturing quality, which creates fragile and therefore more interesting experience of body perception.
5.3 Electronic distortion of body perception
Measuring our heart rate, blood pressure and physical activity, connected wearables capture a new image of bodies, which categorizes them in numbers, bits and diagrams, not only temporary but at all times. And while we feel our bodies, wearables define it in a string of numbers and conditions, enabling us to see our bodies through their sensory, creating a new layer of both understanding and estrangement at the same time. Mapping ourselves through somatic data in a novel form of represen-tation allows us to rediscover our body and find a new awareness for our internal flows if done so in a deliberate manner (such as the Whispers Project by Schiphorst & Kozel, 2002) which addresses both, intimacy between wearables and the actor themself, and between others through the wearables. Putting a device as a mediator between oneself and one’s own physiological data creates a new self-to-self relation. Looking at oneself through another medium can be very insightful and allow a better under-standing and a more interesting exploration of oneself. It can evoke the desire to pay stronger attention to oneself, to aspects that normally happen unconsciously - rhythm, our inner body state like heart beat and sensory inputs (Schiphorst, 2009, p.138).
Not being aware of the foreignness of the mediating device though might also lead to a distortion of one’s own perception of oneself. Rendering the lens through which one looks at oneself as invisible as possible makes it harder to recognize the distorting effect this lens creates.
Current commercial connected wearables (as the in chapter 2 mentioned Jawbone, Apple Watch and Nike Fuelband) create this digitally distorted lens, creating both a stronger connection to what we nor-mally might dismiss on the one hand and an estrangement of our perception on the other.
Intimacy is also marked through vulnerability and the opening of personal space. The PwC ‘Wearable Future Report’ (2014) states, that 82 % of respondents are worried that wearable technology would invade their privacy and 86 % expressed concern that wearables would make them more vulnerable to security breaches. An invasion is only possible in a space that is understood as private, which shows, that the space inhibited by affective wearables is perceived as very private. The articulated and widely spread concerns over the invading potential of wearables in terms of privacy and data security indica-tes that an understanding of the closeness created by affective wearables exists, even though it might be vague and undefined for the majority of consumers.
The described aspects - electronic intimacy, exploration of ones own body and vulnerability - are im-portant to keep in mind when designing a wearable for electronic exploration, as it ties into all of these facts. It is important that the wearable itself keeps its electronic character in a recognizable manner and ties into somatic feedback, to provide a positive, intimate experience while being aware of the tool as an actor in itself.
5.4 Wearables as opening into hertzian space
Just as wearables enter a private space, so do electrical fields and waves without us being aware of it. While home tools have ‘invaded’ (I use the term invaded, because their extended presence is mostly unintended and often gone unnoticed and the perception of electric noise is mostly connoted negati-vely) our domestic space by their leaking electro-magnetic aura, wearables now ‘invade’ our private, intimate space, our bodies, with sensors and fields. The body itself becomes a part of the circuit, both electrical but also behavioral, entering a closer relationship with the gadget. While we shared the hert-zian space and co-existed alongside electric objects before, the invisible layers of both our bodies and the gadget now become entangled. This fact in itself is not novel, since there have always been sensors and wearables for medical purposes, and also power users who are interested in becoming cyborgs, integrating technology in their bodies. The current development addresses a new mindset though: trends towards quantified life and long distance intimacy bring sensor-equipped wearable gadgets into the lives of people who are not particularly aware of electronic nature and also are not (yet) interested in finding out.
Similar to normal home appliances, a lot of effort is made to hide everything electronic and technical that these wearables contain, and to make them sleek and beautiful. Instead of seeing it as the merging of two worlds these gadget are perceived as extensions of ourselves. As described before, this mediation offers a new lens, which does not only uncover, it also alters the way we look at ourselves with more distance. Rather than feeling our heartbeat, we look at numbers and graphs expressing our life functi-onalities in bits and bytes. While on one hand we hide the electronic nature of gadget more and more, we translate on the other hand our bodily functions into their language to understand them better ourselves. While affective wearables are mostly seen as a tool to get to know oneself better, they might also become tools to discover the invisible hertzian space around us.
Few projects uncovering the hertzian space make use of wearables until now. Compared to stationary tools, wearables provide a locative context as well as a more personal connection, as they are rather perceived as a part of oneself than as an external tool (Mann, 1996), and enable a more contextual in-teraction with the hertzian space in terms of position, movement and bodily reaction. They thus pose an interesting medium to illustrate the hidden touch points of the human body and electro-magnetic waves, as the wearable itself is just as entangled into the personal space as are we with the hertzian space. I see wearables as an opening to enable people to recognize oneself in this abstract place, to find
correlations and discover the hertzian space, now shaped and overpopulated by mobile and wearable devices, through exactly these devices.
The current trend of commercial wearables aimed at self-monitoring also provides an opening as it allows for a project contrasting this development and working with an artifact which is not aimed at hiding its “electronic-ness” and fusing seamlessly with the wearer, but rather uncovers the electrosphere around us.
6. Related work within Interaction Design
In terms of working with electromagnetic fields as design material there exist a couple of inspiring projects. In the field of wearables, a lot of projects use somatic input, but mostly for the purpose of surveiling and monitoring body activities, which is not what I want to focus on. Instead of extracting somatic data, I want to interpret it in connection with external data (EMF readings) to create intimacy, and not a stronger distortion. I therefore want to distance this study from how somatic data has been mostly used by wearables (which has also been discussed in the previous chapter about connected wearables and electronic distortion of body perception) and want to focus in this section on work that uses electromagnetic fields in different ways to illustrate how electromagnetic fields can be purposely utilized.
As described beforehand, a lot of effort is put into hiding electricity and electronic in devices from the user. Redström et all (2007) address this fact in their project “Static”, as part of which they design a series of erratic appliances. The project aims to explore how to increase energy awareness by creating objects which directly react on increasing energy consumption and thus pose immediate consequences on the user instead of illustrating abstract and delayed impacts of increasing energy consumption. Redström et all (2007) make the claim to use electricity and energy consumption as a design material in itself and including them in the design and the use experience.
The “erratic radio” of the series of erratic appliances reacts on the amount electronic appliances used in its surrounding. It works with radio waves to detect increasing electric activity in its surrounding, thus using the invisible hertzian layer to “communicate” with the surrounding appliances and trans-lating the detected imbalance into for humans perceivable audio feedback by distorting the received radio waves. The idea is to draw the focus on energy consumption by channeling the measured fre-quencies in the surrounding into audible waves is creative and intuitive. However the interaction with the electromagnetic fields is limited to the radio itself, while the user is made aware of their existence through the radio. The positioning or behavior of the user herself is in no way connected to the shapes of the electro-climate around her, as long as she does not react to the radio.
The approach of using electricity as design material in itself impacts the notion of electromagnetic fields in this study and will be further discussed in the conclusions and reflections.
Img 5 (left): Erratic appliances by Redström & all (2007) Img 6 (right): Humantenna by Microsoft (2012)
Another example of using electromagnetic fields as design material is the “Humantenna” (Cohn & all, 2012) In contrast to the “erratic radio“, the “Humantenna” project uses a device, which turns the human body itself into an antenna, sensing wave feedback using a device strapped to the users back. While using the application, other bigger devices like heating and air conditioning, as well as close-by appliances, are turned off, to level out the unique electro-climate in each home. While this project makes use of the naturally occurring electromagnetic noise in homes as means of sensing the human body, it does not work with the natural variety in composition that occurs within the radio space, caused by the changing appliances that emit electromagnetic fields, since the noise used is based on commonly used appliances like lights and electric lines instead of single devices, as the “erratic radio” does, which acts depending on exactly that. The electric field though is not indicated in itself, but used for “magic” sensing, without actually drawing attention to the nature of the sensing itself. The project thus rather reinforces the immaterial notion of electromagnetic fields, and only highlights the potential of electromagnetic fields as medium rather than interaction with the field itself.
Similar to the “erratic radio”, “electroclimates” is a design proposal by Anthony Dunne and Fiona Raby (2001), addressing the ‘invasion’ of the domestic space by radio waves from foreign emitters outside of the home. Waves from telephone conversations and other wireless communication travel through air and space passing through private and domestic spaces inhabited by other human beings, who then share their space with these waves and fields, mostly without knowing. This instance is illus-trated by a pillow formed object, translating changes in the electromagnetic climate of the space into flickering sound and light output, signaling the passing through of ‘foreign’ signals through the electric climate of the domestic space, which allows the owner to get to know patterns and sounds of their spa-ce also on an electromagnetic level. This proposal illustrates how we perspa-ceive our spaspa-ce in the borders of for humans sensable materiality - vision and sound proof borders block our privacy against intru-ders, but we neglect the space we are unable to perceive and are blind to the interaction happening on these layers. The resulting meaning is poetic and rich in symbolism. I want to use the notion of radio waves in this project to extrapolate on the context not between radio waves and space, but electromag-netic fields and the human body.
In the field of wearables, the project “Fashion Victims” (Agnelli, Buzzini & Drori, 2003) works with phones signals to create clothing and bags, which react to the density of phone signals (and thus to a specific field of the electric climate) in the surrounding. A clothing item reacts to sensed phone signals in the surrounding with a bleeding-like effect - red color leaks into white fabric, creating a strong visual effect, which connects invisible behavior to a physical response. This work illustrates invisible ubiquity of wave signals and the acceptance of and even dependence on this fact in our social behavior. Th-rough the use of clothing, the feedback depends on the whereabout of the user herself, accompanying and tracing every step and also every encounter with other people using phones.
Img 7 (left): Fashion Victims by Agnelli & all (2003) ; Img 8 (right) Electrocliamtes by Dunne and Raby (2001)
The design proposal offered by this research plans to extrapolate on the fact of acceptance of invi-sible ubiquitious electronic signals by connecting the feedback to both somatic input and sensed fields surrounding the wearer. At the same time I want to illustrate the interactions and encounters of signals in a more delicate and reflective manner, as the visualization in “Fashion Victims”comes across as very strong and almost violent, but at the same time quite minimal as it does not indicate strength or change of the signals. I see a potential in mapping also variations and changes in the surrounding electroclima-te in order to represent the hertzian space as the shifting and drifting space it is by nature.
7. Designing for sensuous discovery and meaning making
As the aim of this study is to facilitate and encourage exploration, I want to describe what kind of experience I am aiming for in order to achieve this. The experience created could be seen as playful in a wider sense, but more than that I aim for an aesthetic, poetic and foremost pleasurable experience. The term ‘aesthetics’ derives from the Greek word ‘aesthesis’ and refers to sensory perception and sensori-emotional values and the understanding of this sensuous knowledge. (Stanford Encyclopedia of Philosophy, retrieved 10-05-2015.) Graves applies the meaning of aesthetic to interactions as followed:
“First aesthetic interaction aims for creating involvement, experience, surprise and seren-dipity in interaction when using interactive systems. Second, aesthetic interaction promo-tes bodily experiences as well as complex symbolic representations when interacting with systems.”
(Graves Petersen et al. 2004, p. 274).
As we structure our perception of the world based on connections and relationships, we strive to “de-tect order in chaos or unity in variety” (Hekkert, 2006, p.166). Things are easier to understand when we can clearly identify correlations and systems, and we rely on our sensory systems to spot them. A correlation that does not speak to our sensory system is understandable on a cognitive, abstract level, but very likely to be neglected in our every day life. This is the case with electromagnetic fields - we vaguely know that they exist and use them continuously but are hardly aware of them.
As electronic connections become wireless and electronic character more invisible it is harder for users to easily find relationships and correlations (which I already referred to in chapter 5.1 as the invisibility phenomenon). Mapping the presence of EMF to a sensable output allows us to recognize their materi-ality and to turn them from something abstract into something perceivable, which speaks to our desire to uncover correlation and discover novel insights (Hekkert, 2006, p.167).
I see the poetic part being created through the meaning people draw from this insight, as we can create a novel perspective and onto the hertzian space and draw a wealth of meaning from this reference (Hekkert, 2006, p.165). Based on the described novel sensual access, the insights can be extended to a broader awareness - the awareness, that the input factors which create the output are not random triggers, like buttons on an instrument, which do not have a functionality on their own. Instead each input - EMF readings and pulse - has a complex functionality and presence on their own, which is used to create sense-pleasure. One of these factors is oneself, one’s own alive body, of whose changing and breathing nature we are often only passively aware.
This extended context creates the aesthetic meaning making and thus poetic experience.
The triggers can be changed actively by focussing on speeding up or slowing down the pulse or chan-ging the electromagnetic climate around oneself by switching things off or on or chanchan-ging position, moving closer or further away from a field source. But the triggers in itself also change by nature and are affected by a lot of other circumstances as well. The correlation does not come alive through the user but lives on its own while being open for influence through the user.
Creation and exploration, as well as sensation and discovery as experiential pleasures (Salen & Zim-merman, 2004) are thus happening at the same time.
This allows for an oscillation between states of exploration and play with the player switching back and forth between the explorative goal “what can this object do” and the playful goal “what can I do with this object” (Costello & Edmonds, 2007, p.77).
The experience allows for both surprise and serendipity due to the changing nature of EMF as well as expression and involvement as it is influenceable through position and movement, and a rich symbolic character.
8. Methods and Design process
While this project is not meant to criticize or warn of electro smog or physiological effects of EMFs, it still has characteristics of critical design, as it is meant to act as a comment on the invisible and disgu-ised nature of electronic characteristics in commercial products and on maybe resulting distance to oneself through self quantification.
The concept itself is meant to enable poetic moments created from both the pulse of electronic appli-ances surrounding us and from our own heartbeat by creating an artifact, which borrows from com-mercial affective wearables and creates a private couture piece.
I want to situate this project between a design experiment and critical design, since it is meant to both stimulate discussions and experiment with new extensions of the medium, without being overly critici-zing but more of an aesthetic experience (Dunne & Raby, 2001, p. 58).
8.1 EMF as non-physical design material
In my process I work with somatic data and electromagnetic fields as input material, which both act as influencers in the overall design experience. Therefore the question of electricity as a design material in itself and the notion of electromagnetic fields as an immaterial came into discussion. I see a design material in the field of interaction design as something that can be used to shape and impact an inter-active experience, which is the case in this study, as it facilitates an experience based on the discovery of EMF.
I therefore see both EMF and somatic data as possible design materials, as their nature and the impli-cations they bring are included in the design concept, and not only seen as data provider (as somatic sensing often is) or a problem to solve as described by Redström et all (2007).
The theremin I use for my first experiment can be seen as an illustrating example: It is an instrument which creates music based on interferences caused by the hands of the player in an electromagnetic field. While the original idea is based on electromagnetic fields, today a lot of alternatives exist, which use light and other sensors, to detect the distance between hands and the theremin, without chan-ging the actual user experience. For me, the concept of a theremin itself does not qualify as involving electromagnetic fields as design material. While the “erratic radio” and “electroclimates” directly react to fields in the surrounding, and a change in the input material would clearly affect the experience, the theremin can constitute its electronic component without losing its character. Similarly the “human-tenna” only uses EMF as means to achieve an experience, which could also be achieved through other detection systems.
8.2 Material Processes
My design process is based on lab based material exploration and design experiments as framed by Brandt & Binder (2011) to explore our notion of both ourselves and EMFs around us. The frame of lab is inspired by Koskinen, Binder & Redström (2008), who refer to the lab as a contained space with controlled conditions. Within design research it is related to experimental psychology, as it is based on inserting explanatory variables into the lab environment and observing changing outcomes. The focus hereby is on allowing exemplary insights based on observing perception and reaction to identified variables rather than finalized results (Binder 2007). The process in itself is a hands-on approach to ex-ploration, and I will conduct experiments and prototyping in parallel with reflection and analysis. This leads to an on-going re-framing of the study.
8.3 Digital Sketching
As I work with inperceptable material, working with lo fi prototypes using imagination to mock certain interactions has limited potential. I therefore mainly work with sensing prototypes which require digital and physical sketching to construct actual feedback. As I see electromagnetic fields as well as somatic data as design material in themselves, I call this part of the design process material exploration. I work with the Arduino platform to construct prototypes with different input and output material, such as pulse and EMF sensing on the input side and audio and light as output.
8.4 Experiment methods
For the experiments I adopted two approaches - embodied storming (Schleicher, 2010) and first-person observations in small individual ‘experience discoveries’ (Schiphorst, 2009). The experiments as well as the constructed prototypes focus on how to create an engaging experience for exploration in itself (Hobye, 2014). The purpose is to arrive at a design that invites for experimentation with one’s own presence tied into the hertzian space by generating both vivid and responsive feedback from the user herself and the radio fields around her.
8.4.1 Embodied storming
Embodied storming enables the participant to physically move and react with the whole body: “Embo-died storming posits that we ought to first create the experience of physical performance, not to ideate but to enact experiential awareness.” (Schleicher, 2010, p. 47). In contrast to what is generally seen as “bodystorming”, an embodied prototyping method in context using the whole body to reenact inter-actions, “embodied storming” focuses on physical performance before any situated frame is given. It is seen as a generative mode of participant observation to allow a generative process before even limiting the use space to a certain scenario: “As a new mode of bodystorming, embodied storming helps to cre-ate stories or themes out of the things we observe around us, the things we perceive.” (Schleicher, 2010, p. 48). I chose this method to observe initial reactions and interactions with an electromagnetic field wi-thout a given setting or interpretation. I used this approach to reflect on the embodied behavior around this insensable material of electromagnetic fields, which suddenly becomes very real and physical once a feedback loop is created, even without any guidance or instructions on how to interact with it.
8.4.2 Experience discoveries
As this study is not only concerned with uncovering hertzian space but also building relation between the body and the space it moves through, I also included aspects of self observation in the experiments. Movement expert, Bonnie Bainbridge Cohen (1993) describes embodied experiential practices as mer-ging the conceptual and experiential, shifting between observing and embodying. Cohen states:
“It is a continual dialogue between awareness and action - becoming aware of the relati-onships that exist throughout our body/mind and acting from that awareness. This align-ment creates a state of knowing. There are many ways of working toward this alignalign-ment such as through touch, through movement, visualization, somatization, voice, art, music, meditation, through verbal dialogue, through open awareness, or by any other means.” (1993, p.1)
This state of knowing is what I want to achieve through the designed artifact - knowing about oneself in a space in a way that one has not been able to sense before. The intention behind these individual ‘experience discoveries’ is to observe if the offered interactions are meaningful (Schiphorst, 2009) in terms of reaching this state of knowing.
8.5 Evaluation
I want to evaluate my study based on testing sessions using the developed prototype as well as reflec-tions during the material exploration. The latter includes design decisions which I made based on the outcomes of the explorations and aspects of the design material I see as enhancing the experience, aiming at questions like how the character of the wearable affects the experience. The former is meant to answer questions such as how far I succeeded in creating a relatable experience, that makes the hert-zian space accessible and how do people reflect on the experience afterwards?
I see my study as successful when the notion of electromagnetic fields has changed afterwards and the awareness has shifted from an abstract, use-oriented understanding towards a perception of our coexis-tence in space.
9. Material exploration
The intent of the first exploration phase in the lab is to examine possible options for input and out-put which can be used for the concept artifact. The first series of experiments explores reactions to and experiences with electro-magnetic fields as design material. The second series is centered around physiological input and technology mediated exploration of one’s own body. Aside from the input ma-terials, experiments also include output explorations, which are mainly tested by myself. The material is explored in terms of enjoyability of interaction, responsiveness, movement and expressiveness. The experiments are mainly situated in a lab environment in the Medea studio of Malmö University and in my own apartment. The experiments have been executed in the timeframe of five weeks.
The first series of experiments was focused on finding and interacting with electro-magnetic fields in order to get feedback on how participants perceive and imagine something both very real and abstract at the same time. The experiments addressed this aspect from two sides - in experiment one, partici-pants use an electro-magnetic field detector to find already existing fields, while in experiment two they interact with a specially created field reacting to body presence through audio feedback.
9.1 Experiment One - Arduino based EMF Detector
Participants: First iteration: Three testers plus myself as observer Second Iteration: Me and one other tester
Location: My apartment
Hardware: First Iteration: Arduino Uno, Resistor, LED, Antenna
Second Iteration: Former setup plus a woolen, knitted wristband and three sewn in LEDs
Goal: Gaining insights into electromagnetic climate in a domestic field and observing, how testers react to findings
Instructions: The testers received a short explanation, how the detector works and were invited to search the apartment for electromagnetic fields, with no more specified instructions.
9.1.1 Pilot Experiment
In a first pilot experiment, I built an Arduino-based electro-magnetic field detector (used instructions by Aaron Alai). The device consists of an antenna to pick up EMF, which is connected to a battery powered Arduino board via a resistor. The Arduino runs a sketch, which translates the detected EMF into LED output - the stronger the detected field, the brighter the LED glows. In practice, this trans-lates to the closer the device to an appliance the brighter the LED shines. Adjustments in length for the antenna as well as values in the sketch would make it possible to also detect longer waves and thus lower frequencies.
Three different participants searched the apartment for EMF using the detector taking individual turns. As a silent observer I accompanied them to document where they would find EMF and how they reacted, but tried to stay out of the picture. The device was passed on to the next participant once the active participant was no more interested. Participants enjoyed moving the detector slowly towards and away from appliances, letting the LED oscillate smoothly, in an almost pulsating manner. As it was a hand held device, the participants used wide arm gestures to move the tool closer or further from an appliance, but did not move their own body. The searching for fields had a character of reaching out for them with only the hands, as if they could be touched with the fingertips. The actual embodiment of the sensing was thus limited. After a certain timeframe the interest in finding new EMF decreased, as the presence of EMF was predictable and so was the feedback. After finding three or four stable EMF the experience was seen as too static and the knowledge about the presence of EMF did not seem relevant enough to find more of them, and the detector was passed on.
Another insight was the fact, that touching people with the antenna would lead to a bright LED which caught the participant’s attention. One participant was almost unable to use the device, since the LED would not turn off when held by him. We realized, that his sweater, made from polyester, was very static and caused the LED to react very strongly. This highlighted a paradox of using an electronic de-vice to sense electromagnetic fields created by electronic dede-vices. The detector did not react to its own power supply though and while this paradox (electronic sensing electronic sensing itself) in itself would be worth to be discussed, it did not affect the experiment in itself.
Img 11 & 12: The LED changes brightness according to the strength of the sensed EMF
9.1.2 Wearable Extension
In a second iteration, the Arduino was coupled with a wristband holding a string of three LEDs sewn into it. The wristband emerged kind of naturally as a first shape to be tried for the integration of the sensor in a wearable due to several reasons: As described in chapter 4.1, the wristband is the predomi-nant shape of connected wearables on which I base the underlying design concept.
It is also an easy to produce artifact with very low requirements in terms of fitting. From the first itera-tion of the experiment, the noitera-tion of tapping the field with the hands focused on gestures to find fields, which would be enhanced by a wristband.
The wristband itself was made in a workshop at the makerspace stpln3, and woven from wool which
made it easy to sew on single LEDs. The materiality of the soft woolen wristband and the different LEDs, whose colors matched the colors of the used wool, changed the character of the artifact from a simple tool to a more developed wearable. It was more comfortable since it was worn and not hand held. Although the functionality stayed the same, the reaction to the feedback was different, as the par-ticipant would identify stronger with the location and position in which the fields were detected. The participants rather felt like stumbling into a field than intentionally looking for them.
Despite the detector working well for most of the time, this experiment also illustrated a first problem when working with EMF - the instability of values. While some fields appeared very consistent, others were hardly detectable or readings were not repeatable. Values of the readings constantly changed, and the detector seemed from time to time to be affected by a variety of impacts which are hardly identifiable.
Take-aways:
The light output in itself was not interesting enough to make the search for EMF an exciting experi-ence, as it was too plain and basic. Integrating the detector into a wearable changed the perception of position and location of EMF and tester.
Img 12 & 13: Sketch of wristband and prototype
3 stpln is a open makerspace, available for anyone who wants to create and build things, produce cultural events or experiment with project designs. The space offers tools and workshops. The named workshop was held on 2nd of April 2015 by a volun-teer of stpln as a brief introduction into how to integrate electronics into fabrik and yarns.
9.2 Experiment two - Rotation as output
Participants: Me
Location: My apartment
Hardware: Arduino Uno, Resistor, Servomotor, Antenna
Goal: Testing perception of rotation as output
The second exploration was conducted by myself in my apartment. Based on the insights from the pre-vious experiment, that a single LED as outputs for EMF readings did not appear to be very exciting, I wanted to try a variation and mapped the readings to the rotation of a small 360° servo motor. The readings were mapped in the way that the stronger the signal, the faster the motor would rotate. This mapping turned out to be not as successful, as the readings were to unstable to create a smooth map-ping, and the motor to slow to adapt consistent enough. The motor turned out to vibrate what seemed to be randomly and in no way consistent with the readings.
Take aways:
The used servomotor did not match the required sensibility to interpret the readings in a consistent manner. I therefore did not follow this idea further.
Img 14: Sketches of possible wristband designs using light and rotation as indicators
9.3 Experiment Three - Theremin
Participants: First iteration: Two testers plus myself as observer. Second iteration: Three testers plus myself as observer
Location: Medea Studio at Malmö Högskola
Hardware: Arduino Uno, circuit, speaker, antenna
Goal: Observe interactions with one specially created EMF, which is connected to a
audio feedback loop
Method: Embodied storming
Instructions: The testers received no extensive explanation about how the theremin in itself works, only that the antenna is the origin of an electromagnetic field.
Most of the participants had heard about a theremin before though, and thus a vague understanding of the concept. Aside from the explanation, the instructions were only to explore and play around with no set timeframe or specific intructions on what to try in particular, as I wanted to focus on what the participants would come up with by themselves in order to interact with the field, which goals they would set for themselves and how they would react to an invisible instance reacting to their bodies.
9.3.1 First session
For the third experiment, I built a theremin to create an EMF made for interaction (used instructions by Martin Nawrat, KHM 2008). A theremin is originally an electronic instrument, invented by Leo Theremin in the late twenties, which uses an EMF to detect the distance of hands to an antenna and creates audio feedback from it. The idea of using an EMF to detect a humans position has since been used in different projects and contexts (i.e. “Humantenna” by Microsoft).
The purpose of this experiment was to find out how participants would interact with a single field, of which they know the origin and which allows for a more precise feedback than fields detected from appliances. Also the audio feedback was meant to provide a more interesting feedback than a LED. The audio feedback used the arduino tone library and played different pitched tones depending on the distance between field origin and human body. This way the feedback was not static, but fluent and more lively. The result was a very robotic-like beeping pattern, which enhanced the electronic charac-ter of the whole apparatus.
The theremin was placed on a table in the center of the studio, which is a familiar environment for the participants. All participants knew vaguely how a theremin functions but had never interacted with one. The participants did not receive any instruction before-hand except that the field opened for interactions as soon as the sound set in and they were free to do and act freely and without restrictions, to encourage embodied storming as means of exploring the field with whole body interaction. As an observer, I stood in a distance far away to not interfere with the EMF but close enough to operate (start, end and restart) the system over a laptop, while observing and documenting the interactions. In the first session, the audio feedback was not defined enough to provide a context or make sense of the interaction.
While a certain correlation was detectable, the very unstable values created a lot of noise around the audio pattern. It again became clear, that EMF by character are unstable and flowing, which was reflected in flickering sound and imprecise feedback patterns. This caused the participants to not be able to adjust their movements very well to the field. The whole interaction was perceived as disrupted
patterns, the audio was too irritating and random to create a capturing experience. The participants would stop trying to interact with the fi eld by themselves but articulated to be disappointed, as the invisible materiality of the interactive area did have an intriguing character.
Img 15 & 16: Tester interacting with the theremin
9.3.2 Second session
In the second session, the code was modifi ed to create a more precise and smoothed audio feedback and fi lter out noise caused by jumping reading values. Aside from the participants and myself, a few spectators were also present in the room, so that the interaction felt like a vague performance cha-racter. After the session the participants were asked to fi ll out a questionnaire and to refl ect on their experience.
The three participants of the second session described their experience with the theremin very diff e-rently. None of them seemed to be aff ected by other people being present during their test (maybe also because the bystanders did not seem to pay attention to what was happening). The perception varied from a very stiff interaction (“pressing a button“) to an almost ethereal feeling of fl oating in the fi eld, where even the antenna was perceived as disturbance to the immateriality of the fi eld. All three partici-pants used their whole body to interact, stepping back and forth while trying to fi nd what seemed to be a sweet spot of responsiveness of the fi eld.
Every participant had their own way of movement trying to adjust to the fi eld. One participant jumped and played a lot with positions around the theremin, while another was mostly focussed on recreating sounds and actually playing melodies, trying to fi nd predictive patterns in the fi eld. The third one - the one that perceived the fi eld as stiff - did not change speed or way of movement, but only tried diff erent spots and locations. None of the participants were afraid to move close and even touch the antenna and all participants tried to get as close as possible to the antenna, the origin of the fi eld, as well as as far away as possible without loosing the connection. One participant hugged the antenna, trying to cover as much surface as possible, while another tried to use paper and even another person, to densify the fi eld or stabilize it. In terms of feedback, the sound seemed to be more interesting than the single LED, although the sound was not perceived as pleasant but rather annoying. However, the variety and richness was stronger, and through the aspect of limited unpredictability also more interes-ting.
