Processing of task-irrelevant sounds while performing a visual task: A study of auditory steady-state evoked potentials

Department of Psychology Thesis 15 HE credits Psychology

Psychology III (30 credits) Autumn term 2019 Supervisor: Stefan Wiens

Processing of task-irrelevant

sounds while performing a

visual task of varying load

A study of auditory steady-state evoked potentials

Jenny Arctaedius

1

PROCESSING OF TASK-IRRELEVANT SOUNDS WHILE

PERFORMING A VISUAL TASK OF VARYING LOAD: A

STUDY OF AUDITORY STEADY-STATE EVOKED

POTENTIALS

Jenny Arctaedius

Abstract

Perceptual capacity and selection in attention have for long been an interest in cognitive science, with early theories of early selection to late selection. Hearing is an important subject to investigate when it comes to attention and early auditory processing can be investigated by using auditory steady state responses (ASSRs). Studies on ASSRs to irrelevant sounds have investigated the 40 Hz ASSR and concluded no effect of load. The aim of this study was to investigate the effects of visual load on frequencies other than 40 Hz ASSR to investigate if this yielded the same results. As such, this study investigated the 20 Hz and 80 Hz amplitude modulations using EEG. The visual stimuli were a rapid stream of letters and varied between no load, low load, and high load. The auditory stimuli were an amplitude-modulated tone with a carrier frequency of 500 Hz, with three varying modulation frequencies: 20 Hz, 40 Hz, and 80 Hz. Load level and amplitude modulation alternated over 18 blocks. Participants subjective workload was measured in terms of effort and mental demand. The statistical analysis was comprised of t tests and Bayes Factor. Results provided support for the null hypothesis for the 20 Hz frequency but were inconclusive for the 80 Hz frequency. Further research is needed to give a conclusion for effect of load for the 80 Hz frequency. As there were no convincing results for the higher frequencies, this study cannot give any conclusions on the temporal activation of ASSRs.

Keywords: Attention, Auditory Processing, Perceptual load, ASSRs, Filtering, EEG

Introduction

Have you ever wondered how you sometimes can get so caught up in something that you seem to not be aware of what is going on around you? Imagine you are sitting in school doing an exam. You are so concentrated on answering the questions on the exam that you don't even realize that half the class has already left the room when you finally look up. How can this be?

2

When you actually are listening, you know that the door slams shut loudly. Does the brain turn off the processing of sound when busy with other tasks or does the brain in fact process the sound but turn off its distracting effects? This is a subject of interest in research on perception and attention.

Earlier theories of perception and attention

One question that has interested researchers in cognitive science for a long time is that of perceptual capacity and selection in attention. Researchers have investigated if we have a limited perceptual capacity, and if there is a selection of what stimulus the brain is processing. In the selective filter model of attention, Broadbent (1958) proposes the theory of early

selection and argues that the perceptual capacity is limited. Attention, hence, operates at an early level and before stimulus identification occurs. Processing takes place in stages, and in the initial stage, all stimuli that are important to the fundamental needs of the individual are processed. After this first stage, stimuli are filtered and only the relevant stimuli are processed and stored in long-term memory. As the perceptual capacity is limited, only the goal-oriented stimuli and the high-intensity stimulus are selected to be processed. Further, Broadbent argues that parallel processing can only take place if two simultaneous messages contain little

information.

Treisman (1960) found that when participants heard two messages in a selective listening task, the participants almost remain unaware of the rejected message. What Treisman did find, that is not completely in line with Broadbent's selective filter model (1958), is that the filter could occasionally let through words from the rejected channel of the messages. The explanation to this could be that the threshold for words of a certain class can be raised or lowered in relation to other words, according to the “dictionary decoding system”. The selective filter is acting on physical cues such as intensity, time, and differences in frequency, instead of characteristics of the meaning of words as in Broadbent's theory. Messages are analyzed by the nervous system and the selection of messages takes place during this analysis (Treisman, 1964a). At first, the general physical features are analyzed, then the identification of words and their meanings. This means that the filtering does not block irrelevant stimulus, as Broadbent meant, instead the filtering attenuates the irrelevant stimulus (Treisman, 1960; Treisman, 1964b). Words that are of high importance or that are relevant to the participant would then still be noticed by the participant (Treisman, 1960; Treisman 1964b; Treisman & Geffen, 1967).

A different view is that of the late selection theory. In Deutsch and Deutsch’s (1963) review of several studies of the behavioral findings on attention, they came to the realization that there must be an additional and more complex system of filtering. Broadbent’s filter theory does not work with complex situations with more discriminations of stimuli, as such, there must be some other system at work. In their theory, they propose that all sensory messages that impinge on the organism are analyzed perceptually, but only the most important messages will be remembered and pursued in actions. As all messages are analyzed at the highest level, the selection of what message to act on, what will be stored in memory and what messages that will be filtered out are at a late state. Deutsch and Deutsch also explained that attention and selection in perception has to do with arousal. When an individual is asleep, they will only respond to the most important messages. When aroused, the individual will attend to all messages, but if a more important message is received it will take up more attentional resources.

3 Newer theories of perception and attention

A more contemporary theoretical contribution is that of Lavie and Tsal (1994). They suggested that the current view is a compromise between early selection theory and late selection theory. In reviewing several studies, they concluded that perceptual load is the important part in selective attention. In high perceptual load (e.g. when reading a book on a complicated topic), there is no more perceptual capacity left to process the stimulus and this enables early selection. On the other hand, in low perceptual load, there is perceptual capacity left and this leads to late selection. Even when given instructions to ignore distractors,

participants are not able to do this while performing a low load task (Lavie, 2005; Rees, Firth & Lavie, 1997). Selective attention is thereof only enabled in tasks of high load, not in low load tasks. Lavie and Viding (2004) give further evidence for a hybrid of the theories of early selection and late selection in their five experiments. They also concluded that distractors only can be ignored when the perceptual load is high, as the perceptual capacity then is exhausted. According to a more adaptive model of filtering in auditory attention, the filtering is dynamic and depends on the present attentional demand (Giard, Fort, Mouchetant-Rostaing & Pernier, 2000). Thus, in more attention demanding tasks, the filtering will take place at the most peripheral structures and will be more active and efficient. In tasks that do not demand a lot of attention, more resources are available to process stimuli that are not task related. Thus, attentional researchers have gone back to the idea of early selection in attention and evidence supports a new and improved early filter model of attention. In studying the processing of sounds, Marsh and Campbell (2016) suggested a new early filter theory. As in Broadbent's early filter model (1958), there is a limitation of how much information the human mind can process, and selection of information will occur at an early stage of processing. What the new early filter model is presenting, and what is different from that of Broadbent’s, is that the individual’s prior knowledge and their working memory can affect what will be attended to. Early filtering of auditory stimuli takes place in corticopetal-corticofugal loops. The early filtering in these loops will be more selective when the to-be stimulus is predictable.

Predictable and unattended noise will thus be damped, and the selected predictable stimulus will be enhanced.

Hearing and attention

Hearing is an important sense because even though the individual is tired or under the influence of drugs, intense auditory stimuli can still influence the individual into action (Henneman, 2009). As such, hearing is a warning system that alerts us of danger and incoming threats. This could be because hearing inherently demands more attention and auditory stimuli thus may capture attention even though the individual is attending to something else. Audition is not limited in space, as for vision, and the individual can hear sounds coming from all possible directions.

To navigate in the environment, the individual must be able to rapidly choose the sensory information that is of importance to their current goals and to shift their attention when new, threatening, or possibly rewarding stimuli are detected (Corbetta, Patel & Shulman, 2008). In later research, it has been shown that two different networks interact to help the individual to reorient the incoming information. The dorsal frontoparietal network is goal-oriented (top-down) and keeps track of signals that is in line with current goals and the preexisting

information. The ventral frontoparietal network is stimulus-oriented (bottom-up) and responds to targets and objects that are behaviorally relevant. The interaction of these networks

determines to what and where the individual's attention is directed. The individual must have an adaptive behavior and answer to stimuli that are outside of their current attention if this

4

new information could change their situation. But. not all stimuli give important new information and to have this auditory warning system can both be a blessing and a curse. When it comes to everyday life, it is important for the individual to mute some of the distracting auditory inputs to be able to be productive in their lives (Szychowska & Wiens, 2019). To study hearing, attention and distracting input is important because this can teach us how individuals responds to and reacts to different environments. For example, how a noisy classroom or open-plan office space affects the individual. If the individual is not able to ignore the distracting inputs, the individual might not be able to fully concentrate on the tasks they are supposed to.

Auditory steady-state responses

The balance between the goal-oriented and the stimulus-oriented attention has been studied by investigating how much the goal-oriented attention to a visual task filters the processing of task-irrelevant auditory stimuli (Szychowska & Wiens, 2019). Early auditory processing can be detected and studied by using auditory steady-state responses (ASSRs) (see Figure 1a). The ASSR is an auditory evoked potential (AEP) and can be recorded from the brain in response to an auditory stimulus (Korczak, Smart, Delgado, Strobel & Bradford, 2012). ASSRs are evoked by fast and repetitive sounds in the ear that form a periodic response (Bohórquez & Özdamar, 2008; Mahajan, Davis & Kim, 2014). The sounds are usually amplitude modulated tones, clicks, tone bursts (Korczak et al, 2012; Galamboš, Makeig, & Talmachoff, 1981), or beeping sounds (Mahajan, Davis & Kim, 2014). The ASSRs are beneficial to use because the amplitude modulation of the tones/auditory stimuli will be represented in the neural responses. When the modulated frequencies are attended to, the neural activity is influenced so that the neural responses match the attended event in time. As such, the temporal event in which this takes place can be studied. This enables the measuring of multiple moderated frequencies at the same time.

(Korczak et al., 2012)

Figure 1. A: The figure shows the timeline for responses to an auditory stimulus. After a stimulus onset, the first response is the auditory brainstem response (ABR), with a latency of 1.5-5.5 ms (Korcak et al., 2012). The middle latency response (MLR) takes place at 12-60 ms, and then the auditory late response (ALR) takes place at 60-250 ms. After the ALR there is the auditory P300 response. (Van Dun, 2008) B: The 40 Hz amplitude modulation tone increases in amplitude every 25 ms and responses follow (Korcak et al., 2012). From the second presented tone, the ABR wave V overlap with the MLR component Pa (and with the following amplitude increases, the ABR wave V overlaps with Pb and Pc and so forth).

According to the superposition hypothesis, there is a special relationship between the 40-Hz ASSR and auditory brainstem response and the mid latency response. Because the ABRs and

5

the MLRs to consecutive amplitude increases overlap in time, and this overlap is the strongest and the most synchronized in response to the tones with amplitude modulation of 40-Hz, the 40 Hz ASSRs are the strongest measured ASSR (Bohórquez & Özdamar, 2008). Specifically, at every amplitude increase, there is a brain response (see Figure 1b). At first the auditory brainstem response at the first 12 ms and then the mid latency response at around 12 to 100 ms. From the second amplitude peak, the first MLR overlaps nicely with the second ABR, and this overlap continues with every amplitude increase. This can be displayed by a series of soundwaves where there is a peak in the waves every 25 ms at the 40 Hz frequency

(Galamboš et al., 1981). Although tones with the amplitude modulation of 40 Hz evoke the largest ASSR, other frequencies can also evoke ASSRs (e.g. the 80-Hz).

Studies on ASSRs and perceptual load

Parks, Hilimire and Corballis (2011) investigated distractor filtering with both visual and auditory distractors. The task was a rapid serial visual representation (RSVP) of crosses: one cross appeared every 400 ms on the screen. Two conditions of perceptual load were used. The presence of visual, auditory, or no distractors varied over trials. The auditory distractors were 500 Hz tones at an amplitude modulation between 0-70 dB SPL at a rate of 40 Hz. The results showed that the effects of perceptual load were modality specific. The processing of visual distractors was modulated by varying visual perceptual load (e.g. when both task and distractor were of the same modality (visual), the signals of the visual distractors were reduced in high perceptual load). However, the processing of auditory distractors was not modulated by a high level of visual perceptual load and the ASSRs did not decrease in

strength from low to high load. This suggests that the processing of 40 Hz auditory distractors are not affected by visual perceptual load.

Mahajan et al. (2014) investigated how selective sustained attention modulates ASSRs as a function of different modulation frequencies presented contralateral/ipsilateral to the ears. They used white noise as their experimental stimuli. The white noise was presented for 30 seconds at around 70-75 dB SPL. Four different amplitude modulations were used: 16 Hz, 23.5 Hz, 32.5 Hz, and 40 Hz. At each trial, two different amplitudes were presented, one in each ear. The target stimuli consisted of a change of amplitude modulations for two seconds, two to four times every 30 second trial. The participants were instructed to look at a screen and attend to the cues ‘RIGHT’ or ‘LEFT’ that were presented. Results showed that selective attention influenced the ASSRs to the modulated frequencies of 16 Hz and 23.5 Hz (e.g. enhanced the ASSR for contralateral activation and suppressed the ASSR for ipsilateral activation). The results showed no effect of attention on ASSRs for the modulated frequencies of 32.5 Hz and 40 Hz.

In their study, Szychowska and Wiens (2019) recorded ASSRs on irrelevant sounds as the participants performed a visual task (modelled after that of Parks et al. 2011 above) with low and high visual load. A target cross was shown in the middle of the screen on every 500 ms trial. In the low load condition, participants were instructed to press the spacebar at every red cross (upright or inverted). In the high load condition, participants were instructed to press the spacebar at every upright yellow and inverted green cross. The auditory stimulus was a 500 Hz tone with an amplitude modulation of 40.96 Hz. Tones were presented at 60 dB SL. The results showed no effect of visual load on ASSRs (BF01 = 4.5). This indicated that the processing of the auditory stimulus was not affected by either high or low visual load. The studies reviewed above concluded that the 40 Hz ASSR is not decreased by load or selective attention. What these studies have in common, and what most studies on the subject

6

investigates, is the 40 Hz amplitude. What is lacking in research on the subject is thus the investigation of other amplitudes and their effect on load.

Studies on ASSARs and temporal activation

A few studies have investigated the temporal activation of ASSRs in the brain. In general, the findings indicate that low frequency ranges have more cortical responses and that high

frequency ranges have more sub-cortical responses.

In an MRI study, Giraud, Lorenzi, Ashburner, Wable, Johnsrude, Frackowiak and

Kleinschmidt (2000) investigated the temporal envelope of the processing of sounds in the brain. The auditory stimuli were white noise, amplitude modulated at frequencies ranging between 4-256 Hz. The findings gives evidence that the auditory system is comprised as a hierarchical filter bank and that every level of processing responds to a specific amplitude modulated frequency. The lowest frequencies (4–16 Hz) showed the strongest cortical

responses (e.g. from auditory cortex) to the amplitude modulations. The strongest responses to the highest modulated frequencies (128-256 Hz) were found in the lower brainstem.

By using EEG, Herdman, Lins, Roon, Stapells, Scherg, and Picton (2002) investigated the intracerebral sources of ASSRs. The auditory stimuli were a 1000 Hz tone at 70 dB SPL, presented to the participant’s left or right ear. They used the amplitude modulations 12 Hz, 39 Hz, and 88 Hz. The results showed that the responses to the 88 Hz mostly occurred in the brainstem, e.g. the auditory pathways of the brainstem. The responses to the 39 Hz were mostly activated in cortical sources, e.g. auditory cortex and thalamus. As the responses were smaller to the 12 Hz it was harder to get a clear answer, but the results suggest a combination of brainstem and cortical responses.

Luke, De Vos, and Wouters (2017) analyzed the sources of ASSRs in hearing using EEG. For the acoustic stimuli, they used amplitude modulated speech-weighted noise that was presented at 70 dB SPL to the participants left, right or both ears simultaneously. As the goal was to investigate ASSRs to the entire auditory pathway, they used five amplitude modulated rates; 4, 10, 20, 40 and 80 Hz. The 80 Hz amplitude modulation activated sources in the brainstem and the 40 Hz modulation showed thalamic activation. The results confirmed the findings from earlier studies, that higher amplitude modulations activate the brainstem and lower amplitude modulations activate the auditory cortex and thalamus.

As higher modulations activate the brainstem, they seem to take place at an earlier stage of the auditory pathway, and lower modulations in later stages of processing. As such, the ASSRs to an amplitude modulation of 80 Hz should be smaller than that of 20 Hz.

The aim of this study was to investigate the effects of visual load on frequencies other than 40 Hz ASSR to see if this yielded the same results. By investigating different frequencies, the results can teach us if higher and lower frequencies are processed in different stages and areas in the brain. To this end, the difference between the no load and high load conditions was computed at the amplitude modulation of 20 Hz and 80 Hz. If visual load decreases processing of irrelevant tones, the ASSR should be smaller during high load than no load. Accordingly, the difference of no load minus high load should be positive. For completeness, the study also recorded a low load condition (to confirm that high load was more difficult than low load) and a 40 Hz amplitude modulation. However, these results are only of secondary importance.

7

Method

Participants

The final sample consisted of 13 right-handed participants (5 men, 8 women) with an age span of 19 to 36 years (mean = 26.7). The participants were recruited via online billboards and via social media. All the participants had normal or adjusted to normal vision, normal hearing, no psychological disorders and no history of neurological disease. Before the

experiment, the participant provided written consent. The participants were compensated with a 100 SEK gift voucher for their participation.

The threshold of hearing was tested before the experiment to ensure normal hearing among the participants and to adjust the auditory stimuli presentation so that all the participants heard the auditory stimulus at the same level. This was tested by using pure tone audiometry at 500 Hz (this one was relevant to the study), 750 Hz and 1000 Hz. With over-ear headphones, low beeping sounds, starting at 20 dB HL (hearing level), were first presented to the right ear and then to the left ear. The participant was instructed to push a button every time they heard these sounds. If they missed to press the button on one of the beeps, the sounds got louder. For thresholds at 20 dB HL or below, hearing was considered normal. If the participant’s threshold was over 20 dB HL they were excluded from the study.

Material and Apparatus

The visual task was presented on a computer screen in a quiet room. A chin rest on the desk was used to make sure all participants had the same distance to the screen. The height of the desk could be adjusted so that all participants could rest their chin comfortably and at the same time see the screen well, regardless of their length.

The visual stimuli were uppercase and lowercase letters in five different colors: yellow, green, red, blue, and violet. The letters that were used were H, M, X, and W for the first 8

participants. For the last five participants, the letters H, M, R, and K was used. In the

experiment, the participant would see a stream of the letters. At every 500-ms trial, a letter in one of the five colors was shown for 100 ms in the middle of the screen (see figure 2). The size of the letters was 3.2° x 3.2°.

Figure 2. Modified after Szychowska and Wiens (2019). The stimuli during the visual task were identical in the no, low, and high load except for which stimulus was the designated target stimulus. This avoided any potential confounds from physical differences between the tasks. In the no load condition, there were no targets. In the

8

low load condition, the target letters were in one specific color. In the high load condition, there were two letter and color combinations as the designated stimulus.

The auditory stimuli were an amplitude-modulated tone with a carrier frequency of 500 Hz, with three varying modulation frequencies: 20 Hz, 40 Hz, and 80 Hz. Tones were presented at 60 dB SL (sensation level) to make sure all participants had the same experience of the

sounds. The participants were presented with the auditory stimuli by in-ear tube earphones that the experimenter inserted in the opening of the ear-canal by carefully lifting the

participant’s ear. By using foam on these in-ear tube earphones, background sounds could be closed out.

EEG was used to measure auditory steady-state responses (ASSRs) to the auditory stimuli. EEG is a direct and non-invasive method for measuring human brain activity (Woodman, 2010). EEG is a useful method for studying cognitive processing and especially for measuring attention and perception at a millisecond level. To do this, researchers compute event-related potentials (ERPs) from the EEG to see when electrical activity in the brain occurs, and thus to see how the brain processes a stimulus. In this study, the EEG data were recorded from six electrodes on the scalp at standard 10/20 positions (Fpz, Fz, FCz, Cz, P9, and P10) using a 64-electrode EEG cap and pin 64-electrodes. Two additional 64-electrodes were used on the scalp for reference: the CMS as the internal reference electrode and the DRL as the ground electrode. Two electrodes were used on the face: one on the cheek to measure eye blinks and eye

movements (if too extreme, the participant’s data would be excluded from the study), and one electrode on the nose for reference. These two were recorded with flat electrodes attached with adhesive disks.

The NASA Task Load Index (TLX) (2019) was used to measure the participants' subjective and self-reported experience of the demands of the tasks (e.g. self-reported task load). This study used two of the original questionaries’ single item subscales: the mental demand of the

task and the effort put in to complete the task. The two single item questions were answered

using the Borg centiMax scale (CR100) (Borg & Borg, 2002). The participants had the Borg CR100 scale and explanations (Borg & Borg, 2001) on the meaning of certain numbers on the scale on paper to look at for reference during the experiment.

Procedure

Before the experiment began, participants were informed of what would happen and what tasks they were supposed to perform. When starting the script on the screen, participants were showed trial sessions of the tasks so they would know how to do it. The trial sessions were identical to the experimental sessions. In the trial, participants were showed the questions of mental demand and effort and were explained how to answer them using the Borg CR100. When the trial run was over the experimenters left the room and closed the door. The participant started the experimental sessions by pressing the space key. The script for the experiment was divided into 18 blocks of about two minutes each and included three different visual tasks that alternated over blocks. In the no load condition, the participant was asked to passively observe a stream of letters on the screen. In the low load condition, the participant was asked to press the space key at a specific color on the letters on the screen. In total there were eight different targets in the low load condition. The task could, for example, be to press the space key every time a yellow letter was shown on the screen. In the high load condition, the participant was asked to press the space key on two different letters, each with a different color combination. In total there were four different targets in the high load condition. The task could, for example, be to press the space key at all blue letter H and all green letter M. In

9

both the low load and the high load condition, the participant was instructed to press the space key as fast as possible when seeing the target letters, both for uppercase and lowercase letters. On every low load and high load block, there were 247 trials. In 20% of these trials, the letters on the screen were targets (e.g. 48 targets and 199 non-targets). There were nine combinations of load (no, low and high) and modulated frequency (20 Hz, 40 Hz, and 80 Hz) and all were presented twice. The targets in specific load condition were the same across trials for a given subject (e.g. if the first low load trial for a subject had target letter in the color yellow, all low load trials had yellow target letters for that subject). Before each block, instructions of the task were shown on the screen. On all blocks, the auditory stimulus was administered in one of the three different amplitude modulated frequencies and the participant was instructed to ignore the sounds and focus on the visual tasks. After each block, the participant was asked to rate their subjective task load in terms of mental demand and effort. Instructions on how to fill in the two task load questions using the Borg CR100 was always shown on the screen before filling in the questionnaire. In total, the questionnaire was filled in 18 times.

EEG and ASSR analysis

Each two minute block was divided into averaged epochs at around 1.56 seconds each (77 epochs per block). By using Fast Fourier Transform (FFT), mean waveforms were then translated to the frequency spectrum. FFT is an algorithm that can summarize data (Bullard, 2019). What the algorithm does is transforming signals from its original domain to its representation in the frequency domain. With this transformation, the original complicated structure is converted mathematically to a more comprehensive frequency structure. The frequency spectrum shows the amplitudes of many frequencies, so we can see that there is a large amplitude peak at the target frequencies of 20 Hz, 40 Hz, and 80 Hz (see figure 3 for example). As other, neighboring, frequencies are not of interest to use, they have small amplitudes and we consider them as noise. Amplitude SmN (signal minus noise) was then computed from the data. For the low and high load conditions, ERPs were computed for targets and non-targets. ERPs were computed across all trials for the no load condition.

Figure 3. The average audio amplitude spectrum and mean ERP for 40 Hz.

Statistical analysis

The analyses were mainly paired sample t-tests to test the difference between no load and high load or low load and high load conditions. Bayesian paired t-tests were also used. The Bayesian t-test compares the likelihood of two theories to estimate witch one is the more

10

plausible explanation (Dienes, 2011). This is used because to only test if a theory is true or false (significance testing) might ignore other possible explanations. The Bayesian t-test can also tell us how robust nonsignificant results are. This was computed in the statistical software program JASP (2018) with a default one-tailed Cauchy prior (0.707) to represent the idea that the difference of no load (or low load) minus high load should be positive for ASSR (i.e., because ASSR should be larger during no/low load than high load).

Results

The main question was whether load would affect ASSRs. Figure 4 shows mean amplitudes (signal minus noise) in ASSR for the three frequencies (20, 40, and 80 Hz) and the difference (no load minus high load). Table 1 shows the results of t tests for the mean difference in ASSR (no load minus high load), separately for the three frequencies. Results suggest no statistically significant effects of load on ASSR because all 95% confidence intervals (CIs) are rather narrow and overlap zero. In support of the notion that load did not affect ASSR, the Bayes Factor (BF01) provided evidence for the null hypothesis for the 20 Hz frequency (BF01 = 5.73); however, the evidence was inconclusive for 40 Hz (BF01 = 1.86) and 80 Hz (BF01 = 1.33) (see Appendix for supplementary analyses and figures).

Figure 4. Mean amplitude (signal minus noise) and 95% confidence intervals (CI) for all three frequencies. No load minus high load (NvsH) is presented for all frequencies. Note that the 95% CIs of the load effect (NvsH) overlap zero and are relatively small. This suggests that load did not affect ASSR.

Table 1. Results of t tests for differences (no minus high load) in ASSR amplitude (signal minus noise), separated by frequency.

95% Cl for Mean Difference _______________________

Mean SE

t df p Difference Difference Lower Upper

AmpSmNno20Hz – AmpSmNhi20Hz -0.769 12 0.457 -0.046 0.060 -0.176 0.084

AmpSmNno40Hz – AmpSmNhi40Hz 0.761 12 0.461 0.032 0.042 -0.060 0.123

AmpSmNno80Hz – AmpSmNhi80Hz 1.075 12 0.304 0.020 0.018 -0.020 0.059

During the experiment, participants rated their subjective experience of the different tasks in terms of effort and mental demand. Figure 5 shows mean ratings and confidence intervals on the two single item questions of mental demand and effort respectively, separated by

11

differences (no minus high load) in the two single item questions of mental demand and effort respectively, separately for the three frequencies. For each frequency, no load is rated at least 20 points less than high load (with a lower limit of 11.20 points in the 95% CIs) for both mental demand and effort. These results support that high load was rated as more mentally demanding and effortful than no load.

Figure 5. Mean ratings and confidence intervals (Cl) for mental demand and effort, separated by frequency and load condition.

Table 2. Results of t tests for differences (no minus high load) in mental demand and effort, separated by frequency.

95% Cl for Mean Difference _______________________

Mean SE

t df p Difference Difference Lower Upper

Mentalno20Hz– Mentalhi20Hz -4.894 12 < .001 -21.731 4.441 -31.406 -12.055 Mentalno20Hz – Mentalhi20Hz -5.043 12 < .001 -20.385 4.042 -29.192 -11.577 Mentalno20Hz – Mentalhi20Hz -4.065 12 0.002 -24.385 5.999 -37.455 -11.315 Effortno20Hz – Efforthi20Hz -4.214 12 0.001 -23.192 5.504 -35.184 -11.200 Effortno40Hz – Efforthi40Hz -5.678 12 < .001 -24.692 4.349 -34.168 -15.217 Effortno80Hz – Efforthi80Hz -4.004 12 0.002 -26.615 6.648 -41.100 -12.131

As only the low and high load conditions had a task that included targets, the mean differences of d’ and reaction time is calculated between the low and high load. Figure 6 shows mean differences (low load minus high load) for d’ and reaction time (RT), and Table 3 shows the results of t tests of mean differences (low minus high load) for RT (reaction time, in ms) and D prime (d´), separately for the three frequencies. The results show that reaction time was significantly higher in high load than in low load for all frequencies, as the 95% Cl is wide apart and far from zero. In high load, reaction time was at least 106,77 ms slower than in low load (with a lower limit of 80.53 ms slower, in the 95% CIs). The results also show that there was a strong difference in the false alarms in hit rate for low and high load, as the main difference between low and high load and the 95% Cls are significantly greater than zero.

12

Figure 6. The mean differences (low minus high) and confidence intervals (Cl) for d’ and reaction time (RT), separated by frequency.

Table 3. Results of t tests for differences (low minus high load) in d´ and reaction time (RT), separated by frequency.

95% Cl for Mean Difference _______________________

Mean SE

t df p Difference Difference Lower Upper

dprlo20Hz– dprhi20Hz 9.798 12 < .001 2.095 0.214 1.629 2.560 dprlo40Hz– dprhi40Hz 10.924 12 < .001 1.997 0.183 1.598 2.395 dprlo80Hz– dprhi80Hz 10.747 12 < .001 2.211 0.206 1.763 2.660 meanRTmslo20Hz – meanRTmshi20Hz -8.863 12 < .001 -106.774 12.048 -133.023 -80.525 meanRTmslo20Hz – meanRTmshi20Hz -12.461 12 < .001 -110.082 8.834 -129.331 -90.834 meanRTmslo20Hz – meanRTmshi20Hz -9.674 12 < .001 -111.987 11.576 -137.208 -86.766 Discussion

The results suggest that there is no effect of load on ASSR. Specifically, the 95% CIs are rather narrow and overlap zero, and, the Bayes Factor (BF01) showed that there is support for the null hypothesis for the 20 Hz frequency (BF01 = 5.73), but for the 40 Hz and 80 Hz it is inconclusive. As for the self-reported task load, the results show that high load was rated more demanding and effortful than no load, with the 95% Cls far apart and the mean differences in effort and mental demand ranging from -20.39 to -26.62. This result is also supported by the mean differences and 95% Cls for RT and d’, as the Cls are wide apart and not close to zero. E.g. reaction time and the false alarms in hit rates were higher in high load than in low load.

ASSR

As for the ASSRs, there seems to be no effect of load when looking at the t test results. For both 20 Hz, 40 Hz and the 80 Hz frequency, there is no statistically significant difference between no and high load. This is further confirmed by the 95% Cl being narrow and

overlapping zero. The main difference between no and high load for 20 Hz are -0.05 (95% Cl = [-0.18, 0.08]), for 40 Hz 0.03 (95% Cl = [-0.06, 0.12]), and for 80 Hz 0.02 (95% Cl = [0.02, 0.06]). To investigate this further the Bayes Factor (BF01) was computed. The BF01

confirmed the result of the null hypothesis of no effect of load on ASSR for the 20 Hz

frequency (BF01 = 5.73). Because a BF01 between 3 to 10 means that the null hypothesis is 3 to 10 times more likely than the alternative hypothesis, this is the typical range to talk about moderate evidence for the null. Because BF01 = 5.73, it provides moderate evidence for the null hypothesis (e.g. the null hypothesis is almost six times more likely than the alternative

13

hypothesis) and the null hypothesis is concluded as true for the 20 Hz frequency. The BF01 is small and thus inconclusive for the 40 Hz (BF01 = 1.86) and 80 Hz (BF01 = 1.33) frequency. As the limit is 3 these results are inconclusive for the effect of load. To be able to say if the 40 Hz and 80 Hz frequency ASSRs are affected or unaffected by load, more data must be

collected. What can be said is that the signal for the 40 Hz frequency is clear and not affected by noise. The signal for the 80 Hz frequency, however, was not clear and strong to begin with. As such, the result of no effect may not be surprising for the 80 Hz.

Self-reported task load

The self-reported task load confirmed that this study manipulated load. The ratings show a clear difference in the difficulty of the task for the no and high load with mean ratings ranging from about 12 to 40 (e.g. week till somewhat strong) on the CR100 scale. Table 2 and Figure 5 showcase that the demands of the task are rated higher in the high load condition when having the 80 Hz frequency (95% Cl = [-37.46, -11.32]) as the auditory stimulus than the lower frequencies (95% Cl = [-31.41, -12.06] for 20 Hz and 95% Cl = [-29.19, -11.58] for 40 Hz). The negative values on the 95% Cl are explained by high load being rated as more difficult than no load, and when calculating the mean difference, we take no minus high load, e.g. the values are negative. The same pattern was found for the effort put in to accomplish the tasks for the high load condition: the 80 Hz frequency was rated as stronger (e.g. more effort put in) (95% Cl = [41.10, 12.13]) than the lower frequencies (95% Cl = [35.18, -11.20] for 20 Hz and 95% Cl = [-34.17, -15.22] for 40 Hz). No load (on all three frequencies) was rated as weak/light (e.g. not as demanding and little effort was needed to put in to do the tasks) and the high load (on all three frequencies) was rated as more or less as somewhat

strong (e.g. fairly demanding and effortful). The mean differences between no and high load

on both mental demand and effort make it clear and confirm that the high load condition was, in fact, more difficult than the no load condition with the mean differences ranging from -20.39 to -26.62.

D prime and Reaction time

D prime (d’) measures shows how well the participant can separate signals from no signals (e.g. if they press hit even when the target letter was not presented). As the no load condition did not include any targets, the d’ and RT is calculated between low and high load. The result shows that there is a significant difference between low and high load for d’, as the difference is significantly different from zero. The 95% confidence interval for 20 Hz; Cl = [1.63, 2.56], for 40 Hz; Cl = [1.60, 2.40], and for 80 Hz; Cl = [1.76, 2.66]. For all three frequencies, the false alarms in hit rate is smaller in low load than in high load. The reaction time further confirms that the high load condition is, in fact, more trying. The mean differences between the low and high load condition is statistically significant for all three frequencies. The main difference (low minus high load) for the 20 Hz is -106.77 ms (95% Cl = [-133.02, -80.53]), for the 40 Hz -110.08 ms (95% Cl = [-129.33, -90.83]), and for the 80 Hz -111.99 ms (95 % Cl = [-137.21, -86.77]). E.g. it takes about 107 ms longer to react to a target letter in the high load condition than in the low load condition.

Theories of perception and attention

As this study did not find an effect of load on processing of irrelevant auditory stimuli, the results cannot confirm the theories such as Broadbent’s (1958) early selection and Treisman’s

14

(1960) moderate selection view. Both these theories state that there is a selection of what to attend to at an earlier stage, but the results from this study indicate that all stimuli (visual task and auditory stimuli) are processed. It might, then, be more likely that Deutsch and Deutsch’s (1963) late selection view is applicable, as this theory states that all stimuli are analyzed at the highest level and that the filtering is at a late stage of processing.

If instead discussing the findings in relation to later theories, the theories state that when filtering takes place depends on the perceptual load and the current attentional demands (Rees et al., 1997; Lavie & Tsal, 1994; Giard et al., 2000; Lavie, 2005; Lavie & Viding, 2004). As such, the results from this study should indicate that there is an effect of load (e.g. irrelevant auditory stimuli can be ignored in high perceptual load). The results indicate no effect of load and cannot confirm these theories. The result of no effect can be because the high load is not high enough, but this was investigated and both the self-rated task load, d’ and reaction time make it clear that the high load task was, in fact, more demanding than the no or low load task.

Studies on ASSRs

The result of no effect of load (according to the t tests) on the 40 Hz ASSR confirms the results of no effect of load in the reviewed studies on ASSRs (Parks et al., 2011; Mahajan et al., 2014; Szychowska & Wiens, 2019). The auditory stimuli and load manipulations in Parks et al. (2011), and Szychowska and Wiens (2019) is similar to that of this study, which makes the result more probable. Mahajan et al. (2011), on the other hand, investigated lower

frequencies and found results that 16 Hz and 23.5 Hz influenced the ASSRs (e.g. enhanced the ASSR for contralateral activation and suppressed the ASSR for ipsilateral activation). The results could not confirm this result with the 20 Hz frequency as none of the investigated frequencies enhanced ASSRs. As such, the results of no effect of load on the 20 Hz and 80 Hz might be an indication that there is no effect of load, neither on the 20 Hz, 40 Hz, or 80 Hz frequency. What this can mean in reality is that, irrespective of the difficulty of a task, an individual may still not be able to ignore distracting sound (e.g. when working or studying in noisy environments, it can be hard to concentrate).

As mentioned, the study could not make any conclusions on effect of load on the 40 Hz and 80 Hz ASSR, according to the Bayes Factor. With more data there might have been more clear answers for the 40 Hz, but not for the 80 Hz as the signal was not clear for this frequency. However, the study did find support for no effect of load on the 20 Hz ASSR (BF01 = 5.73). The 40 Hz ASSR already has compelling evidence of no effect of load in previous studies and might not be in need for further investigation. As such, the 80 Hz ASSR is more interesting to further investigate in another study.

According to studies investigating the temporal activity of ASSRs, higher frequencies activate the brainstem and seem to take place at an earlier stage of processing (Giraud et al., 2000; Herdman et al., 2002; Luke et al., 2017). As such, the results on ASSRs should show that the ASSRs to the frequency of 80 Hz is smaller than the ASSRs the 20 Hz. This study could not confirm the results from these studies, as the signal for the 80 Hz frequency was not strong enough to give convincing results or draw conclusions from, and the 20 Hz and 80 Hz could thus not be compared.

15 Methodological discussion

With another research design, the problem with inconclusive results for the 40 Hz might have been resolved. By using a more extensive repeated measurement design, the results could have been more reliable as this approach has more control for variability and it is easier to detect an effect size even when having few participants. At the same time, performing repeated measurements is more time consuming as the participant must participate twice and there is a risk of practice effect (e.g. when doing a test twice, the participant might benefit from the first test occasion and perform better the second time). However, the current design has repeated measurements as all participants did every load and amplitude combination (in total nine combinations) twice. Hence, the current design has controlled for variability that is due to individual differences. Another explanation to the inconclusive results could be that, because the experiment was long (about 45 minutes or longer, depending on how long each participant rested between blocks), participants might have been more focused and

concentrated on the tasks in the beginning of the experiment.

As the result for mental demand and effort are almost identical, the is a question if there really is a difference between these two subscales of the NASA TLX. As they show the same

results, it can be that they do not measure two distinct things: rather the same thing. When rating the subjective task load, the participants only answered one question on mental demand and one question on effort. This might not be enough to separate the two phenomena. In turn, this could affect the reliability of the self-rated task load results. To yield more reliable results in future studies, an option is to not have mental demand and effort as single items. Instead, mental demand and effort could be more efficiently separated by using multiple items on each subscale. An alternative is to not separate them into two subscales and claim that they are two separate things, but instead incorporate them into the same dimension of self-reported task load.

Conclusions

In conclusion, the results on ASSRs indicates that there is no effect of load on the 20 Hz frequency. Further research is needed to draw conclusions for effect of load on the 40 Hz and 80 Hz frequencies. However, several studies indicate that there is no effect of load on the 40 Hz frequency. The results of no effect should not be a result of load manipulations not being strong enough, as the self-rated task load, d prime and reaction time gives convincing

evidence that there is a strong difference between no and high load (or low and high load). As there were no convincing results for the higher frequencies, this study cannot give any

16

References

Bohórquez, J., & Özdamar, O. (2008). Generation of the 40-Hz auditory steady-state response (ASSR) explained using convolution. Clinical Neurophysiology, 119(11), 2598-2607. doi: 10.1016/j.clinph.2008.08.002

Borg, G., & Borg, E. (2001). A new generation of scaling methods: Level-anchored ratio scaling. Psychologica, 28(1), 15-45.

Borg, E., & Borg, G. (2002). A comparison of AME and CR100 for scaling perceived exertion. Acta Psychologica, 109(2), 157-175. doi: 10.1016/S0001-6918(01)00055-5 Broadbent, D. E. (1958). Perception and communication. London: Pergamon

Bullard, E. (2019). Fast Fourier transform (FFT). Salem Press Encyclopedia of Science. Retrieved 2020-01-21 from

http://search.ebscohost.com.ezp.sub.su.se/login.aspx?direct=true&db=ers&AN=12177299 3&site=eds-live&scope=site

Corbetta, M., Patel, G., & Shulman, G. L. (2008). The Reorienting System of the Human Brain: From Environment to Theory of Mind. Neuron, 58(3), 306-324. doi:

10.1016/j.neuron.2008.04.017

Deutsch, D., Deutsch, A. J. (1963). Attention: Some theoretical considerations. Psychological

Review, 70(1), 80-90. doi: 10.1037/h0039515

Dienes, Z. (2011). Bayesian Versus Orthodox Statistics: Which Side Are You On?.

Perspectives on Psychological Science 6(3), 1-17. doi: 10.1177/1745691611406920

Galamboš, R., Makeig, S., & Talmachoff, P. (1981). A 40-Hz auditory potential recorded from the human scalp. Proceedings of the National Academy of Sciences of the United

States of America, 78(4), 2643-7. doi: 10.1073/pnas.78.4.2643

Giard, M-H., Fort, A., Mouchetant-Rostaing, Y., & Pernier, J. (2000). Neurophysiological mechanisms of auditory selective attention in humans. Frontiers in bioscience, 5, D84-94. doi: 10.2741/Giard

Giraud, A-L., Lorenzi, C., Ashburner, J., Wable, J., Johnsrude, I., Frackowiak, R., &

Kleinschmidt, A. (2000). Representation of the temporal envelope of sounds in the human brain. J Neurophysiol. 84(3), 1588-1598. doi: 10.1152/jn.2000.84.3.1588

Henneman, H, R. (1952). Vision and audition as sensory channels for communication.

Quarterly Journal of Speech, 38(2), 161-166. doi: 10.1080/00335635209381758

Herdman, A.T., Lins, O.G., Roon, P.V., Stapells, D.R., Scherg, M., & Picton, T.W. (2002). Intracerebral Sources of Human Auditory Steady-State Responses. Brain Topography,

15(2), 69-86. doi: 10.1023/A:1021470822922

Korczak, P., Smart, J., Delgado, E. R., Strobel, T., & Bradford, C. (2012). Auditory Steady-State Responses. Journal of the American Academy of Audiology 23(3), 146-70 doi: 10.3766/jaaa.23.3.3

Lavie, N. (2005). Distracted and confused?: Selective attention under load. Trends in

Cognitive Sciences 9(2), 75-82. doi: 10.1016/j.tics.2004.12.004

Lavie, N., Hirst, A., De Fockert, W, J., & Viding, E. (2004) Load Theory of Selective Attention and Cognitive Control. Journal of Experimental Psychology: General, 133(3), 339–354. doi: 10.1037/0096-3445.133.3.339

17

Lavie, N., Tsal, Y. (1994). Perceptual load as a major determinant of the locus of selection in visual attention. Attention Perception & Psychophysics. 56(2), 183-197. doi:

10.3758/BF03213897

Luke, R., De Vos, A., & Wouters, J. (2017). Source analysis of auditory steady-state responses in acoustic and electric hearing. NeuroImage, 147, 568-576. doi: 10.1016/j.neuroimage.2016.11.023

Mahajan, Y., Davis, C., & Kim, J. (2014). Attentional Modulation of Auditory Steady-State Responses. PLoS ONE 9(10): e110902. doi:10.1371/journal.pone.0110902

Marsh, J. E., & Campbell, T. (2016). Processing complex sounds passing through the rostral brainstem: The new early filter model. Frontiers in Neuroscience, 10(136). doi:

10.3389/fnins.2016.00136

National Aeronautics and Space Administration. (2019). NASA (TLX) Task Load Index. Retrieved 2019-10-10 from https://humansystems.arc.nasa.gov/groups/TLX/

Parks, N., Hilimire, M., & Corballis, P. (2011). Steady-state Signatures of Visual Perceptual Load, Multimodal Distractor Filtering, and Neural Competition. Journal of cognitive

neuroscience, 23(5), 1113-1124. doi: 10.1162/jocn.2010.21460

Rees, G., Frith, D, C., & Lavie, N. (1997). Modulating Irrelevant Motion Perception by Varying Attentional Load in an Unrelated Task. Science, 278(5343), 1616-1619. doi: 10.1126/science.278.5343.1616

Szychowska, M., Wiens, S. (2019). Visual load does not decrease the auditory steady-state response (ASSR) to 40-Hz amplitude-modulated tones. (Draft Manuscript)

Treisman, A. M. (1960). Contextual Cues in Selective Listening. Quarterly Journal of

Experimental Psychology, 12(4), 242–248. doi: 10.1080/17470216008416732

Treisman, A. (1964a). Verbal Cues, Language, and Meaning in Selective Attention. The

American Journal of Psychology, 77(2), 206-219. doi:10.2307/1420127

Treisman, A. (1964b). Monitoring and storage of irrelevant messages in selective attention.

Journal of Verbal Learning and Verbal Behavior, 3(6), 449-459. doi:

10.1016/S0022-5371(64)80015-3

Treisman, A., & Geffen, G. (1967). Selective Attention: Perception or Response? Quarterly

Journal of Experimental Psychology, 19(1), 1–17. doi: 10.1080/14640746708400062

Van Dun, Bram. (2008). Improving Auditory Steady-State Response Detection using

Multichannel EEG Signal Processing (Doctoral thesis, Katholieke Universiteit, Leuven).

Retrieved 2019-12-08 from

https://www.researchgate.net/publication/28360757_Improving_Auditory_Steady-State_Response_Detection_using_Multichannel_EEG_Signal_Processing

Woodman, G. F. (2010). A brief introduction to the use of event-related potentials in studies of perception and attention. Attention, perception & psychophysics, 72(8), 2031-2046. doi:10.3758/APP.72.8.2031

18