1: The Language of Noise and the Surreal: A Lynchian Perspective on Sonic Aberrations in HCI

Storytelling via recording historical events, ideological and theological concepts, entertainment, and social events for stronger bonding between groups has been a cornerstone of society since the dawn of time. This was originally strongly rooted in a ritualistic purpose not of the artist themselves but as a reflection and recollection of their surroundings [36]. One of the driving vehicles for storytelling is to posit ideas or situations in a way that essentially transports the viewer into a situation or dilemma that is not directly related to their own lived experiences. To promote a sense of immersion and facilitate the imaginative leap which places the reader or listener in an unfamiliar situation, authors and storytellers will often leverage narrative tools to shift the perspective of a story through the eyes of the characters to convey "the multifaceted nature of a character’s experience" [11]. While storytelling may often be associated with the written or spoken word, telling stories can be done in many forms: orally, written, painted, carved, played, sung, or acted. Effective delivery frequently combines multiple modalities into new artistic mediums, such as movies, video games, or installation experiences. It is through this multi-modal approach that the mood and feeling of an art piece can become a fundamentally abstracted or surreal experience.

A seemingly relaxing scene of nature and its innocent inhabitants can suddenly become deeply unsettling if the audio is presented in a way that diverges from the expectations of the spectator. This was where the initial idea of this work began, in the form of an aptly named short film David Lynchenborough. In this short film which premiered at the CalArts Digital Arts Expo in 2020, we presented relaxing scenes of nature alongside various animals and insects. These images were combined with dissonant symphonic drones, multiple layers of harsh noise, and a fragmented narration by Kaylee SooHyun Lee. The result, which was purposely intended to be flippant and provocative in a playful manner (idiotic title, egregiously over-the-top audio compression, persistent tape de-synchronization on the dialogue track), it also spurned a keen interest in the juxtaposition of light/dark, peaceful/unsettling, clean/harsh. The interest in the purposeful combination of disparate elements rather quickly brought to mind the beauty and mastery of filmmaker David Lynch. Throughout much of his work, he has deftly explored these extremes as not only an aesthetic identifier but as a means for venturing into the creation of worlds that seem at once dreamlike and all too real. This liminal realm of dreams and the surreal is at the core of the work we propose.

1.1 Music and Sound in Movies

Music and sound design in movies have long been a strong creative tool in effectively conveying emotion and narrative. These musical elements in visual media can stimulate both positive and negative emotions as well, taking many different forms and approaches [7]. One such approach is through the use of thematic musical cues, which are typically nondiegetic (i.e. heard by the viewers but not the characters) yet impactful in their storytelling ability by showing the inner thoughts or emotional state of the characters on screen [12]. Even though these musical cues are not specifically occurring on the screen, their impact is nevertheless crucial to the overall trajectory and meaning of the scenes. Themes from television and films can not only become recognizable for their connection to the works themselves but can also convey emotions and intentions when dissociated from the visual medium [20]. The most prominent aspects of Bernard Herrmann’s score for the film Psycho feature what has become known as the "Hitchcock Chord", a dissonant sonority of minor and major thirds. This sonority is not only effective in conveying the tension of the film’s narrative but operates as a means for eliciting that sense of dread and fear in the audience through its atypical resolutional tendencies [26].

Utilizing a combination of visual, sonic, and expressive means to create a succinct whole experience can afford not only an effective artistic exploration but also present a stronger and more enriching result for the viewer [8]. In some instances, a more psychoacoustic approach is used for affective sound in visual media, which "is concerned with the relationships between the physical characteristics of sounds and their perceptual attributes" [25]. One such use case is known as the "Shepard’s Tone". This auditory illusion involves a tone, typically a pure sine wave, that moves upwards in pitch and then is combined with a duplicate tone beginning at the same starting point, causing the illusion of an ever-rising pitch [29]. This auditory illusion has been used in contemporary musical and visual works for similar effect,such as Hans Zimmer’s score for Dunkirk and Björk’s song "Cosmogony" from her album Biophilia.

Related to the Shepard’s Tone is the use of high-frequency sounds. These can convey a stronger sense of closeness between the viewer and the subject, creating "a sense of encirclement and presence, and bringing psychological pressure to the audience" [23]. These high frequency tone applications can effectively convey an array of emotions, such as tension, unease, or anticipation.

1.2 The Potential of Sonic Aberrations

Our proposed design approach is called Sonic Aberrations. The core philosophy behind Sonic Aberrations is guided by the works of David Lynch and his use of sound as a meaningful and effective way of conveying emotions and meaning. Often times in his work, sound is the vehicle through which hidden meanings and abstract settings come together. Much in the way that Lynch has utilized his unique approach to sound art in his films, we imagine settings where a similar strategy could be used for both artistic and utilitarian purposes within HCI. Using sound as a way of creating a discomforting feeling can be used as a way to gently nudge somebody towards an intended or desired task or be used as a way to alert individuals of potential dangers or threats. The use of psychoacoustic elements, much like the ones found in Lynch works and other related artists, can be an effective and subtle way to add some surrealistic characteristics to a flexible range of applications. The design approach that we are proposing is not a fundamentally reconstructed one. It is simply a new lens through which to approach applications that could benefit from this more subtle method, or even lead to new experiences for both the users and the designers themselves. In this fashion, we built on the HCI tradition of uncomfortable interactions [1], discomfort as a design tool [14], and creepiness of technology [34]. Specifically, we focus on how sound manipulation and the purposeful creation of Sonic Aberrations can foster a sense of abnormality in subtle ways, to alert, provoke thoughts, or entertain individuals as they interact with technology in everyday life.

Contribution Statement

With our proposed design methodology, we will contribute a new approach to sound design and sonic affect which can flexibly be utilized within and around the HCI community. A brief overview of some related works as they apply to the sonic qualities of Lynch’s work will show some context for how his aesthetic has permeated throughout the visual arts and beyond. This will be expanded upon through a discussion on some of the biggest sonic hallmarks of David Lynch’s filmography and potential HCI applications of this approach. Finally, we discuss the ethical aspects of Sonic Aberrations, particularly concerning the risk of manipulation, and propose potential safeguarding strategies. Our contributions are the following:

1) A multi-modal experience that demonstrates a Lynch-inspired design language in relation to the content of the publication;

2) An introduction and analysis of music and sound on human emotion;

3) An analysis of Sonic Aberrations and the discussion of its use in HCI, followed by example use cases for HCI.