IN THE FIELD OF TECHNOLOGY DEGREE PROJECT
MEDIA TECHNOLOGY
AND THE MAIN FIELD OF STUDY
COMPUTER SCIENCE AND ENGINEERING, SECOND CYCLE, 30 CREDITS
STOCKHOLM SWEDEN 2021 ,
Designing for Musical Bodies
An Exploration of the Musician–Instrument Relationship
TOVE GRIMSTAD BANG
KTH ROYAL INSTITUTE OF TECHNOLOGY
SCHOOL OF ELECTRICAL ENGINEERING AND COMPUTER SCIENCE
Ab ac
Dra ing on phenomenolog and the theor of embodied m sic cognition, and the idea that m sic is mo ement and the a e e perience m sic is related to mo ement and to o r bodies, the design and conception of a m sical instr ment is carried o t thro gh a soma design process,
ith the non-d alistic bod at center.
In addition to the se of design fiction as a means to imagine ne design possibilities, practical somaesthetics are a part of the design process thro gh the bodil practices of Dalcro e e rh thmics and instr ment defamiliarisation. These contin o s bodil practices pro ide space for e plorations of m sical, aesthetic sensiti ities and contrib te to the designer's att ning to their o n bod and disco eries of m sical sensiti ities therein.
The e periential q alities of m sic--mo ement related rh thm and repetition are identified, and
reflected in the design of the m sical instr ment. While interacting ith the instr ment does not
necessaril pro ide direct immersion into those e periential q alities, nor an intimate
m sician--instr ment relationship, it might ho e er, open p a fertile breeding gro nd for ne
design directions and make space for ne e periences. The design process itself, leading p to
the instr ment, as a great contrib tion to making room for e ploring interactions ith the bod ,
mo ement and m sic.
Sa a fa g
Med tg ngsp nkt i fenomenologi och teorin om f rkroppsligad m sikkognition, samt id n att m sik r r relse och att h r i pple er m sik r relaterat till r relse och ra kroppar, har designen och skapandet a ett m sikinstr ment tf rts genom en soma-designprocess, med den icke-d alistiska kroppen i centr m.
Ut er an ndandet a designfiktion som ett s tt att f rest lla sig n a designm jligheter, ing r praktisk somaestetik som en del a designprocessen ia dem kroppsliga praktikerna Dalcro e-r tmik och instr ment-fr mmandeg ring. Dessa kontin erliga, kroppsliga praktiker ger tr mme f r tforskning a m sikaliska och estetiska mottagligheter och bidrar till designerns sj l st mning a sin egen kropp, samt ppt ckter a m sikaliska mottagligheter in ti densamma.
De empiriska k aliteterna hos m sik-r relserelaterad r tm och repetition identifieras och
terspeglas d refter i tformningen a m sikinstr mentet. en om interaktion med instr mentet
inte n d ndigt is leder till en direkt pple else a dessa k aliteter, eller en intim
m siker-instr ment-relation, s kan den emellertid l gga en b rdig grogr nd f r n a
designriktningar och skapa tr mme f r n a pple elser. Sj l a designprocessen, som lett fram
till instr mentet, ga stora m jligheter till att skapa plats f r att tforska interaktioner med
kroppen, r relse och m sik.
Designing for Musical Bodies: An Exploration of the Musician–Instrument Relationship
Tove Grimstad Bang
tgbang@kth.se
KTH Royal Institute of Technology Stockholm, Sweden
ABSTRACT
Drawing on phenomenology and the theory of embodied music cognition, and the idea that music is movement and the way we experience music is related to movement and to our bodies, the de- sign and conception of a musical instrument is carried out through a soma design process, with the non-dualistic body at center.
In addition to the use of design �ction as a means to imagine new design possibilities, practical somaesthetics are a part of the design process through the bodily practices of Dalcroze eurhythmics and instrument defamiliarisation. These continuous bodily practices provide space for explorations of musical, aesthetic sensitivities and contribute to the designer’s attuning to their own body and discoveries of musical sensitivities therein.
The experiential qualities of music–movement related rhythm and repetition are identi�ed, and re�ected in the design of the mu- sical instrument. While interacting with the instrument does not necessarily provide direct immersion into those experiential qual- ities, nor an intimate musician–instrument relationship, it might however, open up a fertile breeding ground for new design direc- tions and make space for new experiences. The design process itself, leading up to the instrument, was a great contribution to making room for exploring interactions with the body, movement and music.
CCS CONCEPTS
• Applied computing ! Sound and music computing; • Human- centered computing ! Interaction design process and meth- ods.
KEYWORDS
Soma design, Dalcroze eurhythmics, musical instrument ACM Reference Format:
Tove Grimstad Bang. 2018. Designing for Musical Bodies: An Exploration of the Musician–Instrument Relationship. In Woodstock ’18: ACM Symposium on Neural Gaze Detection, June 03–05, 2018, Woodstock, NY. ACM, New York, NY, USA, 10 pages. https://doi.org/10.1145/1122445.1122456
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for pro�t or commercial advantage and that copies bear this notice and the full citation on the �rst page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior speci�c permission and/or a fee. Request permissions from permissions@acm.org.
Woodstock ’18, June 03–05, 2018, Woodstock, NY
© 2018 Association for Computing Machinery.
ACM ISBN 978-1-4503-XXXX-X/18/06...$15.00 https://doi.org/10.1145/1122445.1122456
1 INTRODUCTION
Movement-based and body-centered designs are ever more present in the �eld of HCI (Human-Computer Interaction) [8, 18, 37]. With the proliferation of technologies such as machine learning and sen- sors like accelerometers, working with movement interaction is becoming more accessible, and new methods for analysing, under- standing and making use of movement interaction is continuing to develop [1, 37]. At the intersection of musical practice and HCI, the communities of NIME (New Interfaces for Musical Expression) and DMIs (Digital Musical Instruments) are following a similar trend, with a wide range of body- and movement-centered designs for human-music interaction. New musical instruments are also designed with the intention of providing a speci�c experience, e.g.
directed towards users with speci�c impairments, or with the in- tention of providing new musical experiences [7, 10, 15, 20, 28, 30].
However, outcomes are often viewed as tech deterministic, and tend to recognise movement as a purely functional component to the in- teraction, rather than an experiential and expressive one [9, 28, 37].
Thus, a design challenge remains to include the experiential body, and making sense of movement, no matter ability and context, as more than mere readings of data streams.
Music can be understood as movement [14]. From a phenomeno- logical perspective, how our bodies move in the world de�nes how we experience the world. Body movement is essential in both mu- sical performance, perception, as well as our experience of music [19, 29]. Through its interface, a musical instrument opens up for a certain kind of movement, and limits others. And the way we move with the instrument, in�uences our experience.
How can we work towards designs that unfold the possibilities for movement and opens up for new musical experiences? And how can we design for musical interactions, moving us in ways which encourages the uncovering and discovering of musical, aesthetic sensitivities, which are already there in the body, only perhaps dormant? Soma design (somaesthetic interaction design), designing with and through the body and paying attention to aesthetic sensi- tivities in all actors, is proposed as one design approach to address the above challenges [17, 28].
This paper presents a project seeking to develop a working proto- type, a tangible musical instrument, through a soma design process.
Based on the idea that through somatic engagement, we make way for broadening of sensory appreciation [18], the goal is for this design process to open up for new design possibilities and for the musical instrument to open up an exploration of an intimate musician–instrument relationship, and the cultivation of musical (aesthetic) sensitivities through the body.
The instrument is envisioned for use in contexts of performance, education and musical and bodily exploration. This paper reports
Woodstock ’18, June 03–05, 2018, Woodstock, NY Tove Grimstad Bang
on the design process behind the musical instrument, and how the experiential qualities informing the instrument design, were uncovered and explored throughout the soma design process, and continuous engagement with musical sensitivities.
2 BACKGROUND
2.1 Embodied Music Cognition
Extending from the phenomenological school of thought, and along the lines of the embodied turn in philosophy spreading into other disciplines over the past decades, Marc Leman introduced the con- cept embodied music cognition to musicology [19, 21]. The ideas of embodied music cognition suggests that the body acts as a mediator for musical meaning-making, and there being a clear link between action and perception:
“It is assumed that human musical action and percep- tion are reciprocal processes that fuel [an interactive]
loop, and that action and prediction are co-determined by constraints of the musical environment, as well as by those of the (corporeal) organism that interacts within it. Music is something that the listener interacts with, using sensorimotor, cognitive, emotional, and energetic abilities that optimize the interaction; it can be seen as an expression of the embodied mind”
- Marc Leman et al. [25], (p. 1) This theoretical framework is today supported by research show- ing that motor areas of the brain are activated from watching some- one else performing an action or hearing the sound of an action performed [19]. Along the same lines, Caramiaux et al. point to how we tend to mimic the sound producing action upon hearing causal sounds, such as champagne glasses clinking or pouring cereal into a bowl. And when hearing sounds where we cannot identify the action, we tend to gesture in a way that ‘draws’ or follows the sound’s acoustic contours [6].
2.2 Musical Instruments, Musical Bodies and Meaning-making
Trained musicians are sometimes heard expressing an idea of be- coming one, or merging with their instrument [31]. In a speculative, conceptual approach introduced by Luc Nijs, based on embodied music cognition, this experience is said to be a necessary condition or aesthetic framework, for a sense of musical meaning-making to take place. He points to musical expression, variations and im- provisation taking place in a state of oneness with the instrument, emerging from a “push-and-pull process between the musician’s in- ternal and external worlds” [31] (p. 50). The internal world is where the musician’s internal image, models and planning of actions on how to technically handle the instrument takes place. Where the musician e.g. actively re�ects upon their grip and posture. The external world is where the musician pays attention to the musical environment and their somatic attunement to the musical perfor- mance or practice. This is the musician’s subjective perception of the experience of playing the instrument, a phenomenological view on the interaction.
For the musician to attend to their musical environment and their living, sentient, purposive body, the instrument needs to become a
natural extension of the musical body such that it disappears from the focus of the musician, and one is pulled towards the external world, away from the technicalities of the internal world. In a state of such an intimate musician–instrument relationship, the boundary between the body and the instrument is erased and the experience of oneness occurs [31]. This process can be understood as an interactive loop of the musician’s subjective experience of playing an instrument.
The push-and-pull between the external and the internal is, as such, in�uenced by the musician–instrument relationship. The instrument interface shapes and determines our relationship with the instrument, which in turn determines our musical experience, opening up for certain movements and aesthetic experiences and limiting others. The form factor, instrument design, interactive interface, size and sonic qualities of musical instruments is essential to how we interact with them, and play a role in how we learn to play them.
2.3 Soma Design and Somaesthetics
With roots in the pragmatist philosophy of somaesthetics, soma design provides a method for designing with and through the body and a theory foregrounding lived experiences through the �rst- person perspective [17, 18]. Somaesthetics is the “critical study and meliorative cultivation of the experience and use of the living body (or soma) as a site of sensory appreciation (aesthesis)” [36]. By joining the two words, soma — the living, sentient, purposive body, and aesthetics — sensory appreciation, somaesthetics is opposing the traditions of mind–body dualism. Unlike most philosophy, so- maesthetics is not only theory, but also include practical exercise as to cultivate, attend to and ameliorate the soma.
Rooted in the interdisciplinary endeavours of somaesthetics, soma design approaches design practice through attending to aes- thetic sensitivities. Soma design methods include estrangement, where one disrupts the habitual and engage with the familiar, and slowing down, in order to access a larger repertoire of experiences [26, 38]. Somatic connoisseurship, collaboration with experts, with a certain experiential sensitivity in a somatic practice, such as a professional musician or a choreographer, is another important part of soma design [17, 18].
Soma design and somaesthetics have on occasions been used in contexts of musicianship [5, 27, 28], such as in the non-tangible interface DogDog by Bigoni and Erkut, where the �rst-person per- spective is applied as a means to bridge musical improvisation and movement–interaction.
In a music–movement interaction related research project, Alexan- der Refsum Jensenius entered the topic of micromotions, move- ments measured in millimeters per second, through slowing down and exploring standstill. Together with a dancer-choreographer, making use of somatic connoisseurship, they engaged in a series of sessions of standstill, taking notes of their subjective experience of the practice. This research project started out with a foregrounding of bodily explorations as a way to gain understanding of micromo- tions of how to further approach sonic interaction design within that scale [20].
Designing for Musical Bodies: An Exploration of the Musician–Instrument Relationship Woodstock ’18, June 03–05, 2018, Woodstock, NY
2.4 Dalcroze Eurhythmics
Dalcroze eurhythmics is a pedagogical approach of teaching music through movement. The approach started with composer and music teacher Émile Jaques-Dalcroze around the beginning of the 1900s, from him noticing how his students at the Geneva Conservatory were lacking musicality despite their solid theoretical knowledge [22, 33]. Dalcroze’ approach to music pedagogy was a practical one based on the idea that music is perceived, performed and under- stood through movement, opposing the tradition of body–mind separation, which was (and still is) the prevailing school of thought in score-based Western music education [19].
The approach typically consists of rhythmics, ear training, solfège, improvisation and development of creative abilities, all while seek- ing to uncover musical knowledge which Dalcroze believed to already be there, present in the body. Today’s teaching of Dalcroze eurhythmics is often accompanied by the what is known as the three mottoes of eurhythmics: “show what you hear, show what you see, show what you imagine" [33].
The Dalcroze approach to music education was in a project by Luc Nijs integrated into a piece of educational technology visual- ising the student’s movements. Nijs argues that through aligning the educational technology with Dalcroze eurhythmics, in terms of theoretical foundation and practice might open up for the re- alisation of the educational goals of the technology. It is further argued that such connections between established music education approaches and technologies are likely to make space for important design considerations, and even new insights into the educational method used without the technology [32].
3 METHOD
Through design �ction and a soma design process including, con- tinuous bodily practice and elements of autoethnography [16], the project seeks to explore how a somaesthetically informed mu- sic–movement interaction might open up for cultivation of musical, aesthetic sensitivities. Soma design methods of estrangement, slow- ing down and use of the �rst-person perspective are used to attend to the body [18]. Throughout the design process, experiential and material qualities (including sound as a material) are extracted and explored to inform the aesthetics of the resulting instrument design.
Engagements with bodily practices took form through participa- tion in lessons of Dalcroze eurhythmics, as a means for myself, the designer, to discover and uncover musical (aesthetic) sensitivities through movement, and through instrument defamiliarisation, with the purpose of slowing and down and defamiliarising with my own instrument (the transverse �ute). The experiences gathered from these practices were documented and kept in a diary format, with my own re�ections.
Furthermore, as a way to open up the design space, and in an attempt to steer clear of being in�uenced by traditional instrument designs, as well as gaining insight into other musician’s subjective experience of the musician–instrument relationship and interaction, a design �ction workshop was arranged with four musicians. The workshop was based on and inspired by the work of Andersen and Wilde, and well as Lepri and McPherson in the context of instrument design, where workshop participants are invited to imagine and create new instruments and new technology through crafting of
non-functional prototypes, guided by a series fast paced, and at times absurd tasks [2, 3, 12, 13, 23].
In the design and conception of a musical instrument, through a soma design process, how might the design of the instrument open up an exploration of an intimate musician–instrument relationship, and the cultivation of musical, aesthetic sensitivities through the body?
4 DESIGN PROCESS
4.1 Design Fiction Workshop
Following a pilot workshop and minor adjustments to the scheduled program, the workshop was organised with four participants with varying musical backgrounds. The participants were of ages 24-32, had between 7 and 15 years of musical practice behind them, across the music genres of electronic, techno, contemporary classical, experimental and improvisation. The instruments they played were synths, computer music, guitar, piano and voice.
Before the workshop, the participants were asked to chose and share a piece of music that they really like, and that they would think other people in their surroundings do not listen to, in order to situate their continued making by their own practice (instru- ment, aesthetic preferences etc.) and avoid discrepancies in how the di�erent participants liked the music, in the case of a choosing a common piece for everyone to listen to together. I listened through all the pieces and assembled a collection of eight, more or less vague
‘musical elements’ present in the music, which were brought to the workshop as laminated cards (see Figure 1). The musical elements were melody, vocal, instrument, rhythm, tempo, emotion, timbre (tone color) and harmony.
Figure 1: Cards with the ‘musical elements’ from the music brought to the design �ction workshop by the participants.
The workshop was organised around the connection between the body, movement and music, and was started with a 15 minute session of Dalcroze eurhythmics inspired exercises, in order to initiate the movement-music connection and make an attempt at providing an experience of estrangement with moving to music in unfamiliar ways. The exercises were guided by me and consisted
Woodstock ’18, June 03–05, 2018, Woodstock, NY Tove Grimstad Bang
of rhythmic practice with recorded music, based around walking, clapping and simple arm gestures.
Then, the participants were asked to lie down and attentively lis- ten to their chosen piece of music with headphones. The cards with the ’musical elements’ where laid out and following the listening, they were asked to chose, among the cards, one musical element from the piece of music they listened to. Next, the participants were asked to think of where in their body does the element resides and how is moves with and in the body. And then, from mundane, everyday materials provided on-site (see Figure 2), they were asked to build a magic machine or an instrument that facilitates, produces or is built around their musical element.
Figure 2: The mundane, everyday materials, of varying size, provided at the workshop for making a non-functional mu- sical instrument.
Making went on for about 20-25 minutes, and when everyone were done, they were asked to present their musical instrument, the musical element they had chosen, the name of the instrument, how it works or how it is played. Their presentations were video recorded. The outcomes were built around the elements emotion, timbre and rhythm (see Figure 3). Some of the instruments mirrored movement and others invited movement. One was wearable, one suspended in the air and two handheld.
A Laban Movement Analysis(LMA)-like systematisation of the movement and touch qualities seen in the workshop outcomes were articulated after re-watching the video recordings [9, 35]. The Music Javelin and Large Heart included a vast set movement and touch qualities, both large and small movements as well as the spatial positioning of the instrument in�uencing the sound. Maxilloscope, picked up on micromotion and included to touch qualities. The Auto Rhythmiciser included only sustained �icks or punches as input with a rich sonic output, producing polyrhythms.
4.2 Material Explorations
In order to gain a deeper understanding of the digital and non digi- tal materials that were to be used in the design, and how various material qualities in�uence the aesthetics of the interaction, ma- terials were explored from an early stage of the project [11, 34].
Conductive materials and textile surfaces were found to provide a rich source of input for movement and touch interaction. Using a Bela board for audio processing, a touch sensor (Trill craft) and PD (Pure Data), I started material exploration using sound as output. A fabric breakout board for the touch sensor was created following an online tutorial [4], for easy, non-permanent conductive connec- tions when testing and exploring materials (see Figure 4). From simple oscillators in PD, with the sensor data controlling frequency and/or amplitude, I explored my way to �nd rich and varying sonic
Figure 3: The instruments created during the design �ction workshop. Assigned name and musical element for each instrument: upper left: The Music Javelin — timbre, upper right: Maxilloscope — timbre, lower left: Large Heart — emo- tion, lower right: The Auto Rhythmiciser — rhythm
Figure 4: Fabric breakout board for the touch sensor.
responses from the touch input, e.g. squeezing or pulling resulting in gradually increasing pitch. Conductive materials in forms of conductive fabric, mesh, thread, yarn and paint were explored in layers with non-conductive materials (wood, plastic, fabric). ‘Sen- sors’ were crafted using various combinations of these materials (see Figure 5).
The nuances of touch input from small and large movements onto the ‘sensor’ materials were identi�ed as the LMA touch qualities of patting, tapping, holding, touching, stroking, gliding, slapping, pressing and kneading. Especially tapping, holding, stroking and kneading translated in a rich variety of sonic feedback from the touch and movement interactions.
Designing for Musical Bodies: An Exploration of the Musician–Instrument Relationship Woodstock ’18, June 03–05, 2018, Woodstock, NY
Figure 5: A variety of ‘sensor’ materials crafted from a mix of conductive and non-conductive materials.
4.3 Bodily Practices
4.3.1 Dalcroze Eurhythmics.I participated in a total of �ve Dal- croze eurhythmics lessons at the Royal College of Music in Stock- holm, with two di�erent undergraduate classes and teachers (course codes FG8011 and FG8012). During the lessons, we would be in- structed to carry out an exercise, and afterwards asked to re�ect upon it in plenary. A focus in these re�ections was placed on our subjective account of the experience. Theory was always introduced after practice.
Variations of walking were often used, and became important sources of estrangement for me. In one exercise, we would be walk- ing freely around the room to the pulse of the music playing, and then start to alternate between walking o� beat and on beat. This was unsettling, and both physically and emotionally destabilising at �rst, and completely disrupted the familiar and mundane act of walking. Walking o� beat was di�cult, it felt unnatural, and from looking at my peers, we resembled chickens with forwards tilting and heavy looking upper bodies. In another similar exercise, we would alternate between walking the pulse and clapping a rhythm, which provided a similarly destabilising experience. Clapping the beat and walking a rhythm turned out to be a real challenge.
Such exercises could include music theory in regards to synco- pation, metre and polyrhythms. And elements of improvisation, collaboration and creativity were also a recurring element in the exercises through e.g. having us chose walking direction of move- ment pattern while adjusting making space for each other. Col- laborative aspects of the practice were often theorised along the lines of playing music with others. Some exercises were organised around switching sensory modalities. As re�ected in the mottoes of eurhythmics, we would take an impression in one modality (e.g.
music) and transform it into another one (e.g. movement).
In one lesson, the teacher (at the piano) would instruct us to step and jump according to series of seemingly random numbers (see Figure 6).
Figure 6: An example of an exercise from a lesson in Dal- croze eurhythmics at the Royal College of Music in Stock- holm. Number of steps on the left and jumps on the right.
This gave a strange sensation of the body moving o� and on beat and the pulse starting to move up into the chest from the legs, destabilising, to the point of almost losing the rhythm, then regaining it when the pulse came down into the legs again. At times, when continuing these exercises on my own, I would get a strong sense of the upper body losing its connection to the legs and the lower body.
After the exercise, the theory was introduced. The steps were crotchets and the jumps were three quavers, with the meter chang- ing every phrase, such that we were going o� and on beat (see Figure 8).
Figure 7: The theory behind an exercise from a lesson in Dal- croze eurhythmics at the Royal College of Music in Stock- holm, showing meter to the left of the notes. Going o� beat on the second phrase, then remaining o� beat on the third, and then back on beat on the fourth and last phrase.
These types of rhythmic exercises could take form through a variety of di�erent movements with the whole body. The notation
Woodstock ’18, June 03–05, 2018, Woodstock, NY Tove Grimstad Bang
in Figure 8 is from an exercise where we would glide or �oat on crotchets or longer notes, and step lightly on quavers. Variations were slowly added, using the whole body improvising with the lightness and weight of the body, and communicating with and at- tending to each other through movement. Here, theoretical parallels were drawn to dynamic musical terms like forte, legato, ritardando, crescendo etc.
Figure 8: The notation from an exercise from a lesson in Dal- croze eurhythmics at the Royal College of Music in Stock- holm.
In some more choreography-like exercises, we were working with repetition and �ne tuning of movement, and getting used to remembering and discovering movement with the body. In one lesson, with a focus on working with the weight of the body, we were �nding new ways to turn from one side of the body to the other.
Exercises working with rhythm and displacing the pulse around the body became strong reference experiences for me, that I con- tinued revisiting throughout the design process.
4.3.2 Instrument Defamiliarisation.I started out revisiting a classic music education exercise involving repetition, again with the pur- pose of disrupting the habitual. When learning to play a new piece of music on an acoustic instrument, one of the tips you typically receive is to slow down, play it slowly until you get a hang of it, and then, when it has become readily available in your body, you speed up again.
I decided to work this exercise with a piece I enjoy a lot, but �nd challenging to play, Debussy’s Prelude à l’après-midi d’un faune.
Before playing and repeating the �rst four phrases of the score, I performed the same repetitive exercise of the E-major scale, as in the score. In slowing down, I was attending to my grip, and the movement of my lips, head, back and shoulders, and how for example, a change in my head posture alone completely changes both the sound of the �ute and how long or deeply I breathe. When speeding up the scale repetitions, I once noticed how my hands and �ngers moved in staccato motions, and that I needed to get some heat into my hands. The last time I had this sensation when
playing, must have been when as a kid I would play outside in a marching band during the cold spring.
I was repeating this exercise, going from scale to score, for about a week before I started approaching it from di�erent angles, playing only parts of the scale, or parts of the score, or playing the scale with rhythmic variations and changing articulation. As I during the same period was developing digital, sound material and Pd patches, I started recording and playing back loops and layering the �ute. I was focusing on how I could work with movement in new ways, such as slapping the �ute for percussion-like sounds, using the voice together with playing, or rolling the tongue while playing, and how much I could layer until there was no more space.
Throughout my defamiliarisation with the �ute, I was much inspired by listening to and also playing with music recordings.
From a contemporary jazz rendition of Stravinsky’s The Rite of Spring, I was playing around, repeating the main theme from the beginning phrases. From improvising small rhythmic and metric variations I would easily loose myself and forget what I was doing, and then suddenly be pulled back when stumbling on a grip or note, or losing the rhythm. I continued with these types of repetitions, both with and without looping myself, as these were exercises that would easily push me into improvisation.
5 MUSICAL INSTRUMENT
From bodily practice, the various music-movement related experi- ences of rhythm and repetition stood out as especially in�uential on my musical sensitivities during this work. And with several reference exercises as described in Section 4.3, these experiential qualities were revisited and continuously explored throughout the design process.
Movement and touch qualities, as identi�ed and explored through the design �ction workshop and material explorations, were now together with the experiential qualities used to inform the design and conception of the musical instrument (see early drafts in Figure 9).
Figure 9: Early design drafts, showing a variety of di�er- ent approaches to embedding syncopation and large move- ments into the interaction.
Designing for Musical Bodies: An Exploration of the Musician–Instrument Relationship Woodstock ’18, June 03–05, 2018, Woodstock, NY
A variation of touch-sensing interfaces were made from machine knitted cotton and conductive thread (see Figure 10) providing nu- anced sonic output from stroking, dabbing, holding and squeezing.
Figure 10: Crafted touch-sensing materials used in the mu- sical instrument.
The structure of the instrument suspended from the roof and is large in size. It is primarily made up wooden rings, of 90 cm diameter on the largest.
Inspired by Dalcroze exercises, syncopation and varying musi- cal meter is knitted into the fabric of the instrument (see Figure 12). Varying distance between the conductive areas, imitating note sequences of changing musical meter, gives the sonic feedback of syncopation when sliding over them at an even pace.
The conductive areas of the instrument are connected to the touch sensor breakout board (as seen in Section 4) and the sound is processed through a Bela board running a 4-channel looper and sound synthesis in Pd (�nd code here1). The sound of the instrument is increasing in pitch across its height, and the sound source is at present in form of a speaker located on the �oor, under the instrument. The looper is controlled from pedal-like boxes the �oor, also through the touch sensor breakout board.
6 DISCUSSION
We experience the world through our bodies. The way in which my instrument invites me to interact with it, through its design, shapes the way I am in this world, and how I experience the world.
Designs carry the load of shaping our experiences, and as a designer you have the opportunity (and perhaps the responsibility) to create designs that open up for new experiences. Thus, in the design and conception of a musical instrument, through a soma design process, how might the design of the instrument open up an exploration of an intimate musician–instrument relationship, and the cultivation of musical, aesthetic sensitivities through the body?
1https://github.com/togrba/designing_for_musical_bodies
Figure 11: Final design of the tangible musical instrument.
Figure 12: Conductive areas imitating syncopation as in- spired by Dalcroze exercises.
Through bodily and material explorations, a set of movement qualities as well as the experiential qualities of music–movement related rhythm and repetition are identi�ed, and later re�ected in the design of a musical instrument, through patterns of syncopation, an interface inviting large and small movement, and a 4 channel looper.
Woodstock ’18, June 03–05, 2018, Woodstock, NY Tove Grimstad Bang
After having completed the construction of the musical instru- ment, and having interacted with it, I am now of the opinion that the experiential qualities as felt in the non-technology oriented reference exercises, from Dalcroze eurhythmics and instrument defamiliarisation, are leaving a bigger aesthetic impression, than from my interaction with the instrument.
The experience and discovery of moving the musical pulse around the body, created a sensation of a strong push into the unfamiliar, with the pulse in the upper body, and a strong pull back to the habitual (and in a way more comfortable), with the pulse back in the legs and the lower body. After becoming more familiar with this musical experience, I felt much more supple in my ways of understanding music. I could listen to and experience ‘old’ music in new ways. On a pure personal level, I have, and am, going through a widening of musical sensibilities, tied to the disruptive and unfamil- iar experiences I have had with music and movement throughout the design process.
The Dalcroze eurhythmics lessons were especially formative in the development of my musical, aesthetic sensitivities, and it provided a space with what felt like a constant �ow of somatic knowledge being transferred from teacher to students. Now, how can this type of knowledge transfer be integrated into the interac- tion with and the design of a musical instrument? Perhaps with continued exploration with the instrument, with bodily practices integrated, along the lines of Nijs’ work with technology and Dal- croze eurhythmics [32], new design possibilities could arise and provide new openings for cultivation of aesthetic sensibilities.
As of now, interacting with the design does not necessarily pro- vide any sense of intimacy. However, I am eager to continue the work and would myself expect some sense of intimacy while inter- acting with it in combination with e.g. working with repetitions into improvisation on the �ute. The entire design process has taken form through a personal (and interpersonal) endeavour in search of musical sensitivities, and one might thus wonder what value the uncovering and cultivation of our aesthetic sensibilities has if it remains personal, or even private? I am of the opinion that there is real value in cultivating aesthetic sensibilities, even if the endeavour remains private, personal or interpersonal, as it prioritises values beyond pure intrinsic, hedonistic ones. And with human-music interactions often setting out with the sole purpose of discovering and uncovering musical sensitivities on a personal level, a soma design process seems a god �t for human-music related design work. The same applies to music education contexts, as these tend to be centered around interpersonal relationships, and aesthetic and bodily knowledge transfer therein.
In regards to the representation of the experiential qualities in the musical instrument, it now remains purely symbolic. As with the ‘patterns’ of syncopation embedded into the fabric, I do not think it necessarily invites to interacting with it in a way where you would actually hear the syncope, except if you were instructed to or showed how to do so. As for the looper, I believe that to have a stronger potential of providing a sense of repetition. However, a repetition from using the looper is the only interactive aspect of the instrument where the sound is detached from the action or movement. This is typical of electroacoustic instruments and many DMIs and NIMEs, and sort of represents one of the changing ways of how we interact with music. Much of the music theory, including
that in Section 2, is based primarily on acoustic instruments. How we interact with all types of instruments might demand for revised or new theoretical backdrop [21].
There might be a presumption that instruments in the form of non-tangible, immersive spaces are better enablers of ‘thinking with the body’, which might help the musician with placing their focus apart from the technicalities of handling the instrument. I wanted to explore the physical musician–instrument relationship, and would argue that, depending on design choices, tangible and even large instruments can just as much invite for thinking with the body.
As for the design �ction workshop, there was a clear di�erence in the outputs between those trained in a music conservatory setting and those who were not. While there were only four participants, the two with music conservatory training made instruments with very few or limited inputs, perhaps suggesting more attention paid to subtleties in the trained musicians [24]. All the workshop out- puts were far removed from acoustic instruments, and the musical elements and placing them around the body seemed to work well as a way to prompt the making, and might have played an important role in opening up for new ideas and thoughts.
In retrospect, it might have been of a larger contribution to my design process, if I had organised the workshop later on, after having reached the experiential qualities. Then I could have ask- ing the workshop participants to pick a piece of music based on rhythm or repetition instead, and organised the crafting around elements within those, rather than the eight ‘musical elements’ that I provided.
Regarding the Dalcroze eurythmics inspired exercise arranged at the workshop, I doubt that I managed to provide similar expe- riences of estrangement as those I was having in lessons with the educated Dalcroze teacher. Having a person more knowledgeable and experienced in Dalcroze’ methodology than me, instructing that sequence of the workshop would have been a great improve- ment. I do however think that the exercises were of some worth.
Further reorganizing the workshop to integrate the attentive listen- ing together with Dalcroze eurhythmics exercises would also have been a valuable way of expanding on the movement practice.
6.1 Future Work
Expanding on the sound design of the instrument would be a great improvement, perhaps even through vocal sketching sessions, in- cluding textural and material change across the instrument re�ected in e.g. a timbral change across registers. Also adding situated sound sources, with the sound coming from the conductive areas touched, rather than from one speaker placed on the �oor, would make a great technical improvement.
As already brie�y discussed, continuing the study to further in- corporate bodily practice in interaction with the musical instrument would be an interesting departure for new potential discoveries of musical sensitivities. Additionally, further developing a push-back and exploring cause and e�ect relations in the instrument would be an interesting way forward. It could somehow be a way of address- ing the lessons learnt from the bodily practices and bringing them into the design, and create a sort of dialogue between the musician and the instrument.
Designing for Musical Bodies: An Exploration of the Musician–Instrument Relationship Woodstock ’18, June 03–05, 2018, Woodstock, NY
7 CONCLUSION
This paper provides a practical example of the use of a soma design process in a designing musical instruments. Through patterns of syncopation, an interface inviting both large and small movement with the body, and a 4-channel looper, a musical instrument is designed and conceived inspired by and informed by a soma design process. The experiential qualities rhythm and repetition were identi�ed through the bodily practices of Dalcroze eurhythmics and instrument defamiliarisation.
Interaction with the instrument does not necessarily provide any direct immersion into those experiential qualities, nor an inti- mate experience. But the design process did however provide solid insights into musical, aesthetic sensitivities which might open up new doors to new design work in the future. Dalcroze eurhythmics provided a solid ground for practical somaesthetics of music.
REFERENCES
[1] Sarah Fdili Alaoui, Baptiste Caramiaux, Marcos Serrano, and Frédéric Bevilacqua.
2012. Movement Qualities as Interaction Modality. In Proceedings of the Designing Interactive Systems Conference (Newcastle Upon Tyne, United Kingdom) (DIS ’12).
Association for Computing Machinery, New York, NY, USA, 761–769. https:
//doi.org/10.1145/2317956.2318071
[2] Kristina Andersen. 2017. Making magic machines. Ph.D. Dissertation. KTH Royal Institute of Technology.
[3] Kristina Andersen and Ron Wakkary. 2019. The Magic Machine Workshops:
Making Personal Design Knowledge. In Proceedings of the 2019 CHI Confer- ence on Human Factors in Computing Systems (Glasgow, Scotland Uk) (CHI
’19). Association for Computing Machinery, New York, NY, USA, 1–13. https:
//doi.org/10.1145/3290605.3300342
[4] Becky Stewart Bela Blog. 2020. Making E-Textile Interfaces with Trill Craft. https:
//blog.bela.io/2020/09/29/e-textile-interfaces-trill-craft/
[5] Francesco Bigoni and Cumhur Erkut. 2020. DogDog: Soma-Based Interface Design for an Improvising Musician. In Proceedings of the 7th International Conference on Movement and Computing (Jersey City/Virtual, NJ, USA) (MOCO ’20). Association for Computing Machinery, New York, NY, USA, Article 23, 4 pages. https:
//doi.org/10.1145/3401956.3404242
[6] B. Caramiaux, F. Bevilacqua, T. Bianco, N. Schnell, O. Houix, and P. Susini. 2014.
The Role of Sound Source Perception in Gestural Sound Description. ACM Trans.
Appl. Percept. 11, 1, Article 1 (April 2014), 19 pages. https://doi.org/10.1145/
2536811
[7] Doga Cavdir and Ge Wang. 2020. Felt Sound: A Shared Musical Experience for the Deaf and Hard of Hearing. In Proceedings of the International Conference on New Interfaces for Musical Expression, Romain Michon and Franziska Schroeder (Eds.). Birmingham City University, Birmingham, UK, 176–181. https://www.
nime.org/proceedings/2020/nime2020_paper34.pdf
[8] Sara Eriksson, Åsa Unander-Scharin, Vincent Trichon, Carl Unander-Scharin, Hedvig Kjellström, and Kristina Höök. 2019. Dancing With Drones: Crafting Novel Artistic Expressions Through Intercorporeality. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland Uk) (CHI ’19). Association for Computing Machinery, New York, NY, USA, 1–12.
https://doi.org/10.1145/3290605.3300847
[9] Sarah Fdili Alaoui, Jules Françoise, Thecla Schiphorst, Karen Studd, and Frederic Bevilacqua. 2017. Seeing, Sensing and Recognizing Laban Movement Qualities.
In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (Denver, Colorado, USA) (CHI ’17). Association for Computing Machinery, New York, NY, USA, 4009–4020. https://doi.org/10.1145/3025453.3025530 [10] Emma Frid, Hans Lindetorp, Kjetil Falkenberg Hansen, Ludvig Elblaus, and
Roberto Bresin. 2019. Sound Forest: Evaluation of an Accessible Multisensory Music Installation. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland Uk) (CHI ’19). Association for Computing Machinery, New York, NY, USA, 1–12. https://doi.org/10.1145/3290605.3300907 [11] Elisa Giaccardi and Elvin Karana. 2015. Foundations of Materials Experience: An Approach for HCI. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (Seoul, Republic of Korea) (CHI ’15). Association for Computing Machinery, New York, NY, USA, 2447–2456. https://doi.org/10.
1145/2702123.2702337
[12] Antonella Nonnis Paul Stapleton Kristina Andersen Tom Mudd John Bowers Pete Bennett Sam Topley Giacomo Lepri, Andrew McPherson. 2020. 10.000 MUSICAL INSTRUMENTS FOR A SEMI-CONNECTED WORLD. http://instrumentslab.org/
research/10000-instruments.html
[13] Antonella Nonnis Paul Stapleton Kristina Andersen Tom Mudd John Bowers Pete Bennett Sam Topley Giacomo Lepri, Andrew McPherson. 2020. Play Make Believe: Exploring Design Fiction and Absurd Making for Critical NIME Design.
http://instrumentslab.org/data/giacomo/NIME2020Workshop.pdf
[14] Rolf Inge Godøy and Marc Leman. 2010. Musical gestures: Sound, movement, and meaning. Routledge.
[15] Ethan Hein and Sumanth Srinivasan. 2019. The Groove Pizza. Springer Interna- tional Publishing, Cham, 71–94. https://doi.org/10.1007/978-3-319-92069-6_5 [16] Kristina Höök. 2010. Transferring Qualities from Horseback Riding to Design. In
Proceedings of the 6th Nordic Conference on Human-Computer Interaction: Extend- ing Boundaries (Reykjavik, Iceland) (NordiCHI ’10). Association for Computing Ma- chinery, New York, NY, USA, 226–235. https://doi.org/10.1145/1868914.1868943 [17] Kristina Höök. 2018. Designing with the Body: Somaesthetic Interaction Design.
MIT Press, Cambridge.
[18] Kristina Höök, Baptiste Caramiaux, Cumhur Erkut, Jodi Forlizzi, Nassrin Ha- jinejad, Michael Haller, Caroline Hummels, Katherine Isbister, Martin Jonsson, George Khut, Lian Loke, Danielle Lottridge, Patrizia Marti, Edward Melcer, Flo- rian Müller, Marianne Petersen, Thecla Schiphorst, Elena Segura, Anna Ståhl, Dag Svanæs, Jakob Tholander, and Helena Tobiasson. 2018. Embracing First-Person Perspectives in Soma-Based Design. Informatics (Basel) 5, 1 (2018), 8.
[19] Alexander Refsum Jensenius. 2007. Action-sound: Developing methods and tools to study music-related body movement. Ph.D. Dissertation.
[20] Alexander Refsum Jensenius. 2017. Sonic Microinteraction in “the Air”. In The Routledge Companion to Embodied Music Interaction (1 ed.). Routledge, 429–437.
[21] Alexander Refsum Jensenius. Forthcoming. Sound Actions: Musicking in an electronic world.
[22] Marja-Leena Juntunen and Heidi Westerlund. 2001. Digging Dalcroze, or, Dissolv- ing the Mind-Body Dualism: Philosophical and practical remarks on the musical body in action. Music Education Research 3, 2 (2001), 203–214. https://doi.org/10.
1080/14613800120089250 arXiv:https://doi.org/10.1080/14613800120089250 [23] Giacomo Lepri and Andrew McPherson. 2019. Making Up Instruments: Design
Fiction for Value Discovery in Communities of Musical Practice. In Proceedings of the 2019 on Designing Interactive Systems Conference (San Diego, CA, USA) (DIS ’19). Association for Computing Machinery, New York, NY, USA, 113–126.
https://doi.org/10.1145/3322276.3322353
[24] Giacomo Lepri and Andrew P. McPherson. 2019. Fictional instruments, real values:
discovering musical backgrounds with non-functional prototypes. In Proceedings of the International Conference on New Interfaces for Musical Expression, Marcelo Queiroz and Anna Xambó Sedó (Eds.). UFRGS, Porto Alegre, Brazil, 122–127.
https://doi.org/10.5281/zenodo.3672890
[25] Micheline Lesa�re, Pieter-Jan Maes, and Marc Leman. 2017. The Routledge companion to embodied music interaction. Routledge, Chapter Introduction, 1.
[26] Lian Loke and Toni Robertson. 2013. Moving and Making Strange: An Embodied Approach to Movement-Based Interaction Design. ACM Trans. Comput.-Hum.
Interact. 20, 1, Article 7 (April 2013), 25 pages. https://doi.org/10.1145/2442106.
2442113
[27] Mary Mainsbridge. 2019. Soma-based Non-Physical Instrument Design in Elec- tronic Music Performance. Leonardo (2019), 1–8.
[28] Juan P Martinez Avila, Vasiliki Tsaknaki, Pavel Karpashevich, Charles Windlin, Niklas Valenti, Kristina Höök, Andrew McPherson, and Steve Benford. 2020.
Soma Design for NIME. In Proceedings of the International Conference on New Interfaces for Musical Expression, Romain Michon and Franziska Schroeder (Eds.).
Birmingham City University, Birmingham, UK, 489–494. https://www.nime.org/
proceedings/2020/nime2020_paper93.pdf
[29] Fred Everett Maus. 2010. Somaesthetics of Music. Action, criticism, and theory for music education 9, 1 (2010), 9–25.
[30] Andrew McPherson, Fabio Morreale, and Jacob Harrison. 2019. Musical Instru- ments for Novices: Comparing NIME, HCI and Crowdfunding Approaches. Springer International Publishing, Cham, 179–212. https://doi.org/10.1007/978-3-319- 92069-6_12
[31] Luc Nijs. 2017. The merging of musician and musical instrument: Incorporation, presence, and levels of embodiment. In The Routledge companion to embodied music interaction. Routledge, 49–57.
[32] Luc Nijs. 2018. Dalcroze meets technology: integrating music, movement and visuals with the Music Paint Machine. Music Education Research 20, 2 (2018), 163–183.
[33] Eva Nivbrant Wedin. 2015. Playing Music With The Whole Body - eurhythmics and motor development. Gehrmans Musikförlag AB. 13–15 pages.
[34] Charlotte Nordmoen, Jack Armitage, Fabio Morreale, Rebecca Stewart, and An- drew McPherson. 2019. Making Sense of Sensors: Discovery Through Craft Practice With an Open-Ended Sensor Material. In Proceedings of the 2019 on Designing Interactive Systems Conference (San Diego, CA, USA) (DIS ’19). As- sociation for Computing Machinery, New York, NY, USA, 135–146. https:
//doi.org/10.1145/3322276.3322368
[35] Thecla Schiphorst. 2009. Soft(n): Toward a Somaesthetics of Touch. In CHI
’09 Extended Abstracts on Human Factors in Computing Systems (Boston, MA, USA) (CHI EA ’09). Association for Computing Machinery, New York, NY, USA, 2427–2438. https://doi.org/10.1145/1520340.1520345
Woodstock ’18, June 03–05, 2018, Woodstock, NY Tove Grimstad Bang
[36] Richard Shusterman. [n.d.]. The Encyclopedia of Human-Computer Interaction (2 ed.). Chapter Somaesthetics.
[37] Diego Silang Maranan, Sarah Fdili Alaoui, Thecla Schiphorst, Philippe Pasquier, Pattarawut Subyen, and Lyn Bartram. 2014. Designing for Movement: Evaluating Computational Models Using LMA E�ort Qualities. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Toronto, Ontario, Canada) (CHI ’14). Association for Computing Machinery, New York, NY, USA, 991–1000.
https://doi.org/10.1145/2556288.2557251
[38] Danielle Wilde, Anna Vallgårda, and Oscar Tomico. 2017. Embodied Design Ideation Methods: Analysing the Power of Estrangement. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (Denver, Colorado, USA) (CHI ’17). Association for Computing Machinery, New York, NY, USA, 5158–5170. https://doi.org/10.1145/3025453.3025873
TRITA-EECS-EX-2021:63
