Attention à la marche !

Mind the Gap!

Thinking Electronic Literature in a Digital Culture
Penser la littérature électronique en culture numérique

sous la direction de / edited by
Bertrand Gervais
& Sophie Marcotte

ouvrage créé par / [e]book publication created by
Emmanuelle Lescouët & Gwénaël Graindorge

Les Presses de l’Écureuil

Équipe de direction

Bertrand Gervais, UQAM, directeur / Chair

Caitlin Fisher, Université York, co-directrice / Co-Chair

Joanne Lalonde, UQAM, direction du comité artistique / Artistic Committee Chair

Sophie Marcotte, Université Concordia, direction du comité scientifique / Scientific Committee Chair

Marcello Vitali-Rosati, Université de Montréal, direction du comité francophone / French Committee Chair

Dene Grigar, WSUV, présidente de ELO / ELO President

Ariane Savoie, UQAM, NT2, chargée de projet / Project Manager

Gina Cortopassi, UQAM, NT2, coordination générale / Event Coordinator

Élaine Després, UQAM, centre Figura, coordination générale / Event Coordinator

Michaël Trahan, Figura-NT2-Concordia, coordination Concordia / Concordia Coordinator

Lisa Tronca, UQAM, NT2, Exposition Uchronia | What If? Exhibition

Jean-Michel Berthiaume, UQAM, NT2, assistant à la coordination / Coordination Assistant

Robin Varenas, UQAM, NT2, équipe médiatique / Media Team

Sylvain Aubé, UQAM, NT2, équipe médiatique / Media Team

Sophie Julien, direction artistique / Artistic Director

Hugo Marchand, coordination, direction artistique / Artistic Director Coordinator

Kevin Nicolas, direction technique / Technical Director

Marc-Antoine Goyette, assistant / Assistant

Vanessa Duval, graphisme / Graphic Designer

Sarah Grenier-Millette, Benoit Bordeleau, Charles Marsolais-Ricard, UQAM, NT2

Équipe éditoriale

Bertrand Gervais et Sophie Marcotte, directeurs de la publication / Editors

Gina Cortopassi, chargée de projet / Project Manager

Emmanuelle Lescouët, éditrice / Publisher

Gwénaël Graindorge, éditeur / Publisher

Alexandra Martin, réviseure et traductrice / Copy editor and Translator

Anastasia Llewellyn, traductrice / Translator

Technique

Emmanuelle Lescouët, édition sémantique et mise en page sous Stylo / Semantic editing and layout in Stylo

Emmanuelle Lescouët & Gwénaël Graindorge, créations des publications numérique et papier / Creators of digital and paper publications

Polices de caratères utilisées / fonts used : Garamond MD Std, Georgia, Roboto, Sofia, Open Dyslexic

© Les Presses de l’Écureuil – ALN / NT2, 2020
Dépôt légal : novembre 2020
ISSN : en cours – ISBN : 979-10-384-0005-4

Cet ouvrage a été réalisé avec le concours des organismes suivant :

Tables des matières  /  Table of contents

Introduction

Bertrand Gervais & Sophie Marcotte

PARTIE 1.
Des sons, des bruits et des voix / Sounds, Noises and Voices

Listening to Electronic Literature:
Sounds at the heart of works by Jeremy Hight, Stuart Moulthrop, and Mohamed Habibi

John F. Barber

Speech to Text:
Between the Real and the Unreal

Simon Biggs

Cling-Clang-Cornelius:
Digital Sound Poetry as Embodied Posthumanism

Christophe Collard

Demagogic speech and rhetorical emptiness

Verónica Paula Gómez

Transmedia: an improvisualization

Kedrick James • Ernesto Peña • Yuya Takeda and the Digital Literacy Centre

PARTIE 2.
Le langage comme interface / Language as Interface

The language that machines read

John Cayley

What Natural Language Generation Means for Authorship and Why We Should Care

Leah Henrickson

Spark Gap:
Digital Language, Interstitial User

Sarah Ciston

Balpe & Chatonsky :
Esthétique générative, surproduction et mémoire à l’ère numérique

Paulo Silva Pereira

PARTIE 3.
Plateformes et narrations / Platforms and Narratives

RPG Maker as an E-Literature Platform

Roman Kalinovski

for the sleepers in that quiet earth.:
An Artbook Generated by a Deep Learning Agent

Sofian Audry

Making participatory interactive experiences (PIE):
Closing the gap between story and experience through Electronic Literature and Digital Games

Jon Saklofske

Love letters to strangers

Cristiane Costa

Haptic onto-epistem-ologies in Tendre Claw’s Pry

Abraham Avnisan • Amaranth Borsuk

Digital Deep-Sea Diving:
Navigating the narrative depths of E-lit and VR

Melinda M. White

Micronarrative, Virtual Reality, and Medium Specificity:
Circa 1948 as VR installation and Mobile App

Jim Bizzocchi • Reese Muntean

PARTIE 4.
Esthétiques numériques / Digital Aesthetics

TOPO – TRAJECTOIRES.
20 ans de présence en arts et littérature numériques

Michel Lefebvre • Eva Quintas

The Book of Disquiet Digital Archive as a role-playing experiment

Manuel Portela • Cecília Magalhães

Sortir de l’écran : Lucette, Gare de Clichy

Françoise Chambefort

J’écris double mais je me soigne

Boris du Boullay

Midiazine Research Performance ELO 2018 / Eastern Bloc, Montreal, August 15

Chris Funkhouser

Trump’s Tweets, or the True and Authentic Electronic Literature

Alan Sondheim

PARTIE 5.
Lire et écrire en culture numérique / Reading and Writing in a Digital Culture

Exploring Literary Gaps in S. by Doug Dorst and J. J. Abrams

Gabriella Colombo Machado

From Codex to Code:
Computational Poetics and the Emergence of the Self-Reading Text

Andrew Klobucar

Expand Readings:
a visual / motor Activity Signifiant in Textual Reality

Rolando Rodríguez • Jessica Rodríguez • Alejandro Brianza

Swarm Reading:
“Deformance and Interpretation” at Scale

Kyle Booten

Secret sharing,
a pedagogical tool for e-lit practices

Augusto Vinicius Marquet

eLITE-CM Project:
Developing enriching interactivity in children’s digitized literature and e-lit

María Goicoechea de Jorge

Is there a gap in the classroom?

Ana Maria Machado • Ana Albuquerque e Aguilar • Alice Atsuko Matsuda

PARTIE 6.
Perspectives diachroniques sur la littérature électronique / Reading E-Lit Across Time

La littérature électronique de papier :
réflexions sur la troisième génération de littérature électronique

Anaïs Guilet

“Completing the circle” ?
The curious counter-canonical case of the Eastgate Quarterly Review of Hypertext (1994-1995)

Astrid Ensslin

Raconter l’internet : matérialité et effets

Sylvain David

« Tout était vrai et donc rien n’était vrai ; tout était faux et donc rien n’était faux. »
La (ré)invention des grands récits à l’ère du numérique

Sophie Marcotte

Avoiding digital insanity:
The attachment to constructivism in Brazilian literature

Andréa Catrópa • Gilbertto Prado

Littérature numérique :
10 marches à franchir ?

Serge Bouchardon

Donner aux dyslexiques accès à la lecture en adaptant la typographie : l’enjeu est crucial et on l’a compris, c’est par ici !

Introduction
Attention à la marche !
Penser la littérature électronique en culture numérique

Bertrand Gervais

Attention à la marche !
(jump to english version)

Comme un train entré en gare, la littérature électronique s’est arrêtée en 2018, le temps d’une escale, dans ce vaste immeuble tout neuf que représente la culture numérique. Le train avait déjà un certain âge, tandis que la gare était récente. Il apparaissait évident que la rencontre entre les deux n’était pas parfaite. À tout moment, on risquait de trébucher, surtout si on ne faisait pas attention où l’on mettait les pieds. Il y avait un écart, une faible distance, entre le marchepied et le quai. Faible, de sorte que de loin, on ne le remarquait guère, mais assez marqué pour provoquer des faux pas.

Le congrès de l’Electronic Literature Organization, Attention à la marche !, tenu à Montréal à la fin de l’été 2018, entendait questionner de façon explicite la place de la littérature électronique dans la culture numérique. Quelle est la nature de la relation entre les deux ? Comment une forme déjà vieille de cinquante ans parvient-elle à rester pertinente dans un environnement culturel fortement marqué par le numérique ? Comment une pratique d’avant-garde développée en culture du livre réussit-elle à s’ajuster aux principes d’une culture de l’écran ?

Les expressions « littérature électronique » et « culture numérique » viennent d’être employées comme s’il s’agissait de pratiques ou de champs homogènes aisés à définir. Il en va évidemment tout autrement. En effet, la littérature électronique n’a pas connu un développement homogène, elle n’a pas suivi une progression linéaire ; et on en serait même rendu, comme le dit Leonardo Flores (2019), à une troisième génération. Les deux premières générations, définies par N. Katherine Hayles, permettaient de distinguer, d’une part, une littérature pré-web, fondée sur des formes complexes de textualité, axées notamment sur les fonctionnalités de l’hypertextualité, mais respectant globalement les conventions de la culture de l’imprimé et du livre ; et d’autre part, à partir de 1995, un ensemble de pratiques basées sur le web et intégrant le multimédia et l’interactivité (Hayles 2008). La troisième génération est celle de la littérature numérique en tant que telle, une littérature qui se sert de l’ensemble des dispositifs et des plateformes de diffusion rendus disponibles par les développements informatiques récents, notamment les développements de l’infrastructure technologique qu’est Internet. Cette dernière génération englobe les deux premières qui continuent à avoir leurs adeptes et épigones.

De la même façon, il n’y a pas une seule culture numérique, mais un ensemble de pratiques culturelles, artistiques et textuelles, diverses et non conventionnelles, pour ne pas dire divergentes, en marge bien souvent des institutions (quoique celles-ci cherchent de plus en plus à les intégrer), qui se servent de ces dispositifs et plateformes de diffusion ou de l’infrastructure d’Internet. Comprendre les manifestations et la logique sous-jacente des cultures numériques exige des formes de recherche et de recherche-création innovantes, ainsi que des approches à la croisée des disciplines, telles que l’herméneutique, l’esthétique ou l’ethnographie numériques (Miller 2011), l’archéologie des médias, les humanités numériques, etc.

Le principal objectif du congrès ELO 2018, duquel découle la publication que ce texte introduit, était d’ailleurs de chercher à multiplier les perspectives critiques sur les formes de communication et d’imbrication, mais aussi de résistance entre littérature électronique et culture numérique, en réunissant un très grand nombre de chercheurs et d’artistes préoccupés par leur développement. La rencontre a ainsi servi à favoriser le renouvellement des perspectives de recherche et de recherche-création en littérature électronique, à mieux définir les zones de recoupement entre cette forme littéraire et les humanités numériques, à prendre en compte les technologies mobiles, de plus en plus présentes au quotidien, dans leur impact sur les habitudes d’écriture et de lecture et à investiguer la dimension politique du numérique, dans ses liens à l’art, à la littérature et à la culture. Parmi les autres thèmes abordés, on a pu noter un intérêt accru pour les principes de réalité augmentée et de réalité virtuelle, pour les formes et enjeux de la traduction (en termes de codes, de langues ou de médias), pour les spécificités langagières et culturelles de la littérature électronique, pour les liens entre le numérique et la culture urbaine, ainsi que pour les pratiques éditoriales destinées aux jeunes (les lecteurs de demain).

*
*    *

Il est difficile de résumer une rencontre où une plus de cent cinquante communications et conférences ont été prononcées, un congrès accompagné d’une importante exposition de 60 œuvres numériques, ainsi que d’un festival de performances échelonné sur 3 soirées. Il est tout aussi difficile de structurer les 35 textes qui ont été proposés pour publication et qui sont réunis ici. Le congrès ayant été bilingue, les textes publiés sont ainsi en français et en anglais. Nous n’avons pas choisi de les regrouper en fonction de la langue de rédaction, mais plutôt en regard des enjeux abordés.

L’ouvrage est composé de six sections. La première, intitulée « Des sons, des bruits et des voix », explore la dimension acoustique de notre relation à la littérature électronique. On ne fait pas que lire lorsque confrontés à de telles œuvres, on regarde des vidéos, on explore une architecture virtuelle et, dimension trop souvent oubliée, on écoute toute sortes de bruits et de sons. La deuxième section, « Le langage comme interface », suit de près cette première exploration, en portant l’attention cette fois sur la programmation et le code, dans leur capacité à structurer une expérience et à l’orienter de manière précise. Les pratiques numériques reposent sur du langage qui renouvelle notre rapport aux œuvres.

La troisième section, « Plateformes et narrations », examine diverses plateformes, allant des jeux vidéo aux dispositifs de Réalité Virtuelle, utilisées, voire détournées, afin de générer des œuvres inattendues, qui renouvellent notre compréhension des arts numériques. Sous le titre très large des « Esthétiques numériques », la quatrième section s’arrête sur quelques œuvres récentes, présentées bien souvent par les artistes qui les ont produites. Cette section permet de donner un aperçu des projets qui ont été exhibés dans le cadre du congrès et de son exposition.

La cinquième section « Lire et écrire en culture numérique » s’interroge sur nos stratégies de production et de réception des œuvres de littérature électronique, y compris dans les salles de classe. Cette interrogation permet de revenir de façon importante sur les enjeux pédagogiques de notre passage à la culture numérique, dimension qui devient de plus en plus importante.

Finalement, la sixième et dernière section s’arrête sur des « Perspectives diachroniques sur la littérature électronique ». Cette section est l’occasion d’approfondir notre connaissance des développements de la littérature numérique, depuis les 50 dernières années. L’hypothèse des trois générations de cette littérature, discutée d’entrée de jeu, y trouve un écho explicite.

Ce regard diachronique vient en quelque sorte boucler la boucle. Les sections se sont enchainées en fonction d’une spirale centrifuge, commençant au plus proche du texte et de son expérience, du point de vue acoustique ou du code, et s’ouvrant peu à peu aux plateformes et dispositifs exploités, aux principes esthétiques qui ont servi à la production et à la réception des œuvres, aux stratégies de lecture et d’écriture, abordées de façon analytique et, ultimement, aux transformations historiquement datées que cette pratique littéraire et artistique a connues.

Références

Flores, Leonardo. 2019. «Third Generation Electronic Literature». Electronic Book Review. Accéder au lien.

Hayles, Katherine N. 2008. Electronic Literature. New Horizons for the Literary. Indiana : Notre Dame University Press.

Miller, Vincent. 2011. Understanding Digital Culture. London : Sage.

We consider accessibility as crucial. Give dyslexic readers access by adapting the typography: click here!

Introduction
Mind the Gap!
Thinking electronic Literature in a Digital Culture

Bertrand Gervais

Mind the Gap!
(aller à la version française)

Like a train entering the station, electronic literature stopped in 2018 in the vast new building that is digital culture. The train was already a certain age, while the station was brand new. It was obvious that the meeting between the two was not perfect. At any time, there was a risk of stumbling, especially if you weren’t careful where you set foot. There was a gap, a small distance, between the step and the platform. It was small, so that from a distance you could hardly notice it, but it was marked enough to cause you to stumble.

The Electronic Literature Organization’s conference Mind the Gap!, held in Montreal in late summer 2018, was intended to explicitly question the place of electronic literature in digital culture. What is the nature of the relationship between the two? How does a form that was already fifty years old manage to remain relevant in a cultural environment strongly marked by digital technology? How does an avant-garde practice developed in the context of book culture succeed in adjusting to the principles of a culture heavily engaged in screens, networks and mobile devices.

We’ve just used the terms “electronic literature” and “digital culture” as if they were practices or homogeneous fields that are easy to define. This is obviously false. Indeed, electronic literature has not developed homogeneously, it has not followed a linear progression; and we are even in the presence, as Leonardo Flores (2019) says, of a third generation. The first two generations, as defined by N. Katherine Hayles, made it possible to distinguish, on the one hand, a pre-web literature based on complex forms of textuality, focusing in particular on the functionalities of hypertextuality but generally respecting the conventions of print and book culture; and on the other hand, from 1995 onwards, a set of web-based practices integrating multimedia and interactivity (Hayles 2008). The third generation is that of digital literature as such, a form that makes use of all the devices and platforms made available by recent computer developments, particularly developments in the technological infrastructure that is the Internet. This last generation encompasses the first two which continue to have their followers and epigones.

Similarly, there is not a single digital culture, but a set of diverse and unconventional–not to say divergent–cultural, artistic and textual practices, often at the margins of institutions (although these are increasingly seeking to integrate them), which use these devices and platforms or the internet infrastructure. Understanding the manifestations and underlying logic of digital cultures requires innovative forms of research and research-creation, as well as cross-disciplinary approaches, such as hermeneutics, digital aesthetics or ethnography, media archaeology, digital humanities, etc. (Miller 2011).

The main objective of the ELO 2018 conference, the origin this publication, was moreover to multiply critical perspectives on forms of communication and interaction, but also of resistance between electronic literature and digital culture, by bringing together a large number of researchers and artists concerned about their development. The meeting thus served to encourage research and research-creation perspectives in electronic literature, to better define the intersections between this literary form and digital humanities, to take into account mobile technologies which are increasingly present in everyday life, their impact on writing and reading habits, and to investigate the political dimension of digital technology, in its links to art, literature and culture. Other themes addressed during the four days of the conference included an increased interest in the principles of augmented and virtual reality, the forms and challenges of translation (in terms of codes, languages or media), the linguistic and cultural specificities of electronic literature, the links between digital and urban culture, and editorial practices aimed at young people, that is, the readers of tomorrow.

*
*    *

It is difficult to summarize a meeting where some one hundred and fifty papers and conferences were delivered, a congress accompanied by a major exhibition of 60 digital works, as well as a festival of performances spread over three evenings. It is just as difficult to structure the 35 texts that have been proposed for publication and that are gathered here. The conference was bilingual, thus the texts published are in French and English. We have not chosen to group them according to the language of writing, but rather according to the themes addressed.

The book is divided into six sections. The first, entitled “Sounds, Noises and Voices,” explores the acoustic dimension of our relationship to electronic literature. We don’t just read when confronted with such works, we watch videos, explore a virtual architecture and, a dimension too often forgotten, listen to all sorts of noises and sounds. The second section, “Language as Interface,” follows closely on from this first exploration, this time focusing on programming and code, in their capacity to structure an experience and orient it in a precise manner. Digital practices are based on language that renews our relationship to works.

The third section, “Platforms and Narratives,” examines various platforms, from video games to Virtual Reality devices, which are used, or even hijacked, to generate unexpected works that renew our understanding of digital art. Under the broad title of “Digital Aesthetics,” the fourth section focuses on some recent works, often presented by the artists who produced them. This section provides an overview of the projects that were exhibited at the conference and its exhibition.

The fifth section, “Reading and Writing in Digital Culture,” looks at the strategies used for producing and consuming electronic literature, including in the classroom. This questioning allows us to return in a significant way to the pedagogical stakes of our transition to digital culture, a dimension that is becoming increasingly important.

Finally, the sixth and last section focuses on “Reading E-Lit across Time.” This section is an opportunity to deepen our knowledge of developments in digital literature over the last 50 years. The hypothesis of three generations of e-Lit, mentioned earlier, is explicitly echoed here.

This diachronic view closes the loop, as it were. The sections have followed one another in a centrifugal spiral, starting as close as possible to the text and its experience, from the acoustic or coding point of view, and gradually opening up to the platforms and devices exploited, the aesthetic principles that have served for the production and reception of the works, the reading and writing strategies, approached analytically and, ultimately, the historically dated transformations that this literary and artistic practice has undergone.

Did we succeed in closing the gap? Mind you… We barely managed to look at the space itself and figure its form! But in doing so, we continued to explore a literary form that still evolves and questions our ability to represent our world.

References

Flores, Leonardo. 2019. «Third Generation Electronic Literature». Electronic Book Review. Link.

Hayles, Katherine N. 2008. Electronic Literature. New Horizons for the Literary. Indiana : Notre Dame University Press.

Miller, Vincent. 2011. Understanding Digital Culture. London : Sage.

1

Des sons, des bruits et des voix
Sounds, Noises and Voices

John F. Barber
Listening to Electronic Literature: Sounds at the heart of works by Jeremy Hight, Stuart Moulthrop, and Mohamed Habibi

Simon Biggs
Speech to Text: Between the Real and the Unreal

Christophe Collard
Cling-Clang-Cornelius: Digital Sound Poetry as Embodied Posthumanism

Verónica Paula Gómez
Demagogic speech and rhetorical emptiness

Kedrick James • Ernesto Peña • Yuya Takeda and the Digital Literacy Centre
Transmedia: an improvisualization

We consider accessibility as crucial. Give dyslexic readers access by adapting the typography: click here!

Listening to Electronic Literature:
Sounds at the heart of works by Jeremy Hight, Stuart Moulthrop, and Mohamed Habibi

John F. Barber

Abstract

This essay argues that sound(s) in electronic literature, experienced through listening, prompt(s) readers to re-think their relational experiences with literary worlds, real or imagined, with others in these spaces and places, and with themselves. This centrality of sound(s) is examined through discussions of works of electronic literature by Jeremy Hight et al., Stuart Moulthrop, and Mohamed Habibi. Each has something to say about listening to electronic literature.

Résumé

Cet article démontre que le(s) son(s) dont on fait l’expérience dans la littérature électronique poussent les lecteurs à repenser leurs expériences avec le monde littéraire, réel ou imaginé, avec les autres qui sont présents dans ces espaces et ces lieux, ainsi qu’avec eux-mêmes. La centralité des sons est examinée à travers l’analyse des œuvres de littérature électronique de Jeremy Hight et al., de Stuart Moulthrop et de Mohamed Habibi. Chacune de ces oeuvres nous dit quelque chose sur le son et l’écoute de la littérature électronique.

Introduction

Electronic literature, as a field of research and creative practice evolving from electronic and digital computing technologies, displays its artifacts primarily on screens of various devices. And, given its iterative and experimental evolution, one might suggest what one sees when considering electronic literature depends upon the perspective from which one looks. This essay suggests another way of considering electronic literature: by listening.

Printed text–literature or poetry, print or pixels–has long afforded readers opportunities to create immersive virtual worlds using their memories and imaginations. Sound(s) make(s) these worlds believable. Listening is the portal. To explore these ideas, this essay first defines sound as the experience of acoustic energy that conveys information. Listening promotes the understanding of this information. Next, the centrality of sound, especially the speaker’s voice, and its forward inclusion through print and pixels is outlined. To examine these ideas, pioneering works of electronic literature by Jeremy Hight et al., Stuart Moulthrop, and three contemporary video poems by Mohamed Habibi are considered. As will be suggested, these works of electronic literature each promote our engagement through their effective use of sound. This essay concludes by considering the significance of this alliance.

Conceptual Framework

The term “sound,” as used here, signifies vibrations of acoustic energy that travel through air, or some other medium. These vibrations are experienced (heard) when they reach one’s ears and register in one’s brain as sound(s).

According to Walter Murch, hearing is one of the first human senses to develop, and for approximately 4.5 months before birth one is bathed in a rich soup of sounds heard within the womb. After birth, a baby reacts immediately to sound, before its vision fully develops (Murch 2005).

Different from hearing, listening is the active processing of auditory stimuli, a carefully and purposefully conducted activity, involving conscious effort and attention. French critic and composer Michel Chion, in his discussion of aural relationships to sound in cinema, suggests one listens to obtain information about a sound source, to appreciate the sound itself, and to learn what is communicated by the sound (Chion 1994). Nicole Brittingham Furlonge calls listening “a physiological form of reception,” a practice that is at once “interpretive, situated, and reflective.” Listening to sound, as analytic “storyhearers” says Furlonge, prompts engagement that is participatory, interactive, and experiential (Furlonge 2018, 60, 83).

With Murch’s positioning of awareness of sound(s) as primary, Chion’s suggestion of listening to learn, and Furlonge’s notion of engagement as storyhearers, Bruce R. Smith posits that knowing the world through active listening is fundamentally different from knowing the world through passive viewing (Smith 2003). Michael Bull and Les Back echo this idea when they say sound is primary for knowing and being in the world. One hears a sound and it provides immediate clues about one’s location, and what to expect. If the sound is acousmatic, originating from an unseen source, one can use experience, memory, or imagination to discern its source and meaning (Bull and Back 2003).

At the risk of oversimplifying the work of Walter Ong and others regarding orality (both primary and secondary) and literacy, one might consider speech as expressing and sharing abstract ideas, histories, memories, and cultures, as spoken narratives for understanding external and internal worlds (Ong 1982). With the advent of writing, sounds of speech were graphically transcribed. Thus, writing provides scripts for reproducing the sounds of the spoken voice, whether internally, as part of silent reading, or externally, as the result of reading aloud. In either case, the dynamics of voice are transmitted to the reader, despite its remediation.

Although speech and writing are fundamentally different–speech being based upon hearing and writing upon seeing–at the heart of each is the sound(s) of the speaker’s voice. Listening to these sounds directly engages our memories and imaginations, helps us rethink our relational experiences with others, with ourselves, and the spaces and places we inhabit. Listening to sound(s) provides a way of knowing and being in a world, whether real or imagined.

Discussion

To examine these ideas, we can consider two pioneering works of electronic literature–34 North 118 West by Jeremy Hight et al. (2003), and Under Language by Stuart Moulthrop (2007) (2020)–as well as three contemporary video poems by Mohamed Habibi–Just Words, Matchbox, and Mug. These examples are chosen for their strong emphasis on sound.

34 North 118 West

Jeremy Hight, with Jeff Knowlton and Naomi Spellman, created and released 34 North 118 West in 2002-2003. This pioneering work of locative electronic literature combined multiple media and Global Positioning System (GPS) technology to create an interactive series of narratives about a once thriving railroad depot situated at 34 North latitude, 118 West longitude in downtown Los Angeles, California, during the first half of the twentieth century (Hight, Knowlton, and Spellman 2003).

Participants walked through the four-block area formerly occupied by the railroad station and other businesses with a laptop computer, a GPS device, and headphones. GPS tracked and overlaid their position on a map displayed on the computer screen. Easily identifiable locations were also displayed. Approaching these “hot spots,” participants triggered recorded narratives and soundscapes created from historical, ethnographic, and architectural information about the area.

Other sound effects–squeaking wooden cart wheels and musicians entertaining on busy street corners, for example–were triggered by hidden GPS locations, each waiting to be discovered by wandering participants. These sounds were purposely designed to connect physical locations with events, activities, narratives, and lives of a past dismissed by urban change. Signs, displays, and other physical artifacts or details at each location augmented the narratives, providing metaphors and information for interaction(s) with the characters and history of the area defined by the geographic coordinates 34 North 118 West.

Hight, Knowlton, and Spellman investigated the half-square mile area around the former railroad freight station for more than a year, digging through the histories of the buildings to learn about people who worked and lived there. They crafted narratives from the hidden or lost information they recovered.

For example, at the site of a former tire factory, a worker describes how bits of rubber rained down on Los Angeles after the factory caught fire. A waitress at the train station restaurant talks about the harried passengers she serves. A railroad worker recounts cleaning the tracks after people committed suicide by stepping in front of trains. A cook, the station clock inspector, and others provide additional narratives.

As participants moved throughout the area, triggering multiple sound-based narratives, they uncovered the hidden history of this once thriving part of downtown Los Angeles, and developed a sense of the work’s larger scope and concept. These narratives of forgotten or faded histories, lost buildings, tensions still present of past persons, all buried in memory, could, Hight contends, return with sufficient ability to sustain listeners simultaneously in two separate realities at the same location, one present, the other past (Hight 2013).

Hight contends this approach helps organize forgotten historical and cultural information into meaningful narratives about a place, a time, and people. Hight calls these narratives “sonic archaeology in the urban landscape,” or “narrative archaeology” (Hight 2013).

Under Language

Pioneering electronic literature author Stuart Moulthrop released Under Language in 2007. Moulthrop calls this work a “textual instrument,” an artifact akin to literature but structured like a game (Moulthrop 2020). Few instructions are provided for interaction, however, leaving the reader and / or player to learn the rules for the work. Using a game-like screen display interface that responds to mouse clicks, users select ten lines of text for a poem. These ten lines of text are displayed on screen, along with a closing graphic, chosen by Moulthrop’s textual instrument to reflect the quality of the final poem. Repeat as many times as desired for different poems.1

Moulthrop uses the term “under language” in two ways. First, the title, Under Language, speaks to the underlying computer code that drives the work. Under language, the concept, speaks to both the necessity to notice how writing intersects code, and the consequences of a collision (collusion?) when poetry meets code.2

So, under language, the concept, underlies and infuses Under Language, the work, which is, fundamentally, a generative textual work, meant to be experienced visually, on a screen. But the brilliance of this work is Moulthrop’s sonification of the underlying five layers of computer code. The first is a series of computer-voiced renditions of ActionScripts programmed by Moulthrop that operate the work. The second layer is a series of ambient recorded collages of tunings across radio broadcasts. The third level consists of pseudo-code, again voiced by text-to-speech technology. Comments and summaries, ostensibly voiced by the ten-line poem at the heart of this work, constitute the fourth level. The fifth level is an audio collage, where the poem’s ten lines are each vocalized, as well as comments seemingly from the poem’s self awareness of its creation.

These vocalized narratives of the under language for Moulthrop’s work are not specifically ordered, but rather assembled from user choices of lines of text for a generated poem. Still, the result provides unprecedented access to the interactive affordances beyond the program’s screen-based visual displays.

Moulthrop’s point is that under language (the underlying code) is the language of computer programming, and is inseparable from the work titled Under Language. With Under Language, the work, Moulthrop argues that to experience electronic literature one must appreciate the underlying code, which is vocalized as part of the work.

One might extend Moulthrop’s point by arguing that Under Language is an example of sound-based electronic literature. Hearing the under language vocalized, one understands the presence of hidden narratives concurrently creating and commenting upon one’s experience of the work’s visualization, and, indeed, writing the larger context for its experience. Rather than visuals augmented by sound, Under Language is a work of electronic literature where sound is augmented by visuals, where sound is at the heart of the literary experience.3

Mohamed Habibi

Contemporary examples of Arabic electronic literature by Mohamed Habibi extend the idea of sound-based electronic literature. Habibi, a Saudi scholar and poet working in the genre of video poetry, has for the past decade shared his work through his YouTube channel (Habibi 2018). This essay examines three of Habibi’s video poems: Just Words, Matchbox, and Mug.4

Just Words

Just Words (2:11) begins with a flashlight beam illuminating what appears to be a blank wall. All else is black, and silent. The focus changes and the beam illuminates Arabic writing on a piece of paper. Sound begins–insects and rhythmic clicking. Text appears superimposed on the screen, below the image of the paper, reading,

We dream

Almost unseen, a hand turns the paper to expose another, again with Arabic writing,

dream

The sound continues. One now has the sense of discovering this text in a darkened room, at night. Turning the paper–might one say “pages” now?–reveals another, again with writing. The superimposed translation reads,

in the morning

The rhythmic clicking becomes more rapid, intense, almost insistent. Another page flip…

We carry our dreams

page flip…

to dry it out.

Another page flip…

it won’t fly away;

Two page flips…

as we’ll clasp them.

At 1:35 the scene changes to ten white “pages,” each inscribed with Arabic writing, secured on a line by clothespins–three groups of two pages, one of three pages, and one of one page–near a window. Daylight illuminates the room, and the pages.

One has the sense that these pages represent the dream portrayed in the first scene of Habibi’s video poem, and that these parts of the dream are drying in the morning light. At first this tableau is silent, but the sound of these pages rustling fades in as they are seen moving in an air current from the nearby window.

A dissolve at 1:58 returns us to the first scene: a piece of paper illuminated by a flashlight beam, the rest of the room in darkness.

the clasps

The night insects and rhythmic clicking sounds begin. The page is turned …

are… Just words

Fade to black, and end (Habibi 2018).

Matchbox

Matchbox (1:52) begins with a black screen. Sounds are quickly heard–footsteps running, dogs barking, some vocalizations that hint at physical effort. Memories of similar sounds suggest this is perhaps someone running.

At 0:30 the running stops, the runner seems to take a breath, and one hears a series of sliding, scratching sounds. Silence as Arabic writing appears on the screen, from top right across, then back to the right margin before beginning a new line. The screen fills with writing.

The soundtrack begins anew–breathing, running, dogs, the same sliding, scratching sound. More Arabic writing appears on the screen, right to left, appearing to be written as one watches.

More scratching, sliding sounds until a match lights in the darkness, briefly illuminating a person’s face.

Four seconds of silence and blackness passes until the soundtrack–running feet, breathing, dogs barking–returns. English writing appears on the screen, left to right, as if being typed in real time.

The child
–who runs out of the door
like a pullet
who gets up to glance at his injured
elbow
who walks by anyone to reassure
himself
is you.

They send you out
to the neighborhood
along the dark street,
where the walls are painted with ghosts
to get a matchbox.

A match is lit, again briefly illuminating the face of a young person. End (Habibi 2018).

Mug

Like Matchbox and Just Words, Mug (3:59) begins with a black screen. The rhythmic clicking, heard in Just Words, begins the soundtrack and is quickly followed by sounds of birds, geese, and a crowing rooster. Night sounds follow immediately–insect sounds and human vocalizations in the background.

At 0:46 a small circle of white light appears in the lower right quadrant of the otherwise black screen. It begins to gyrate, seemingly synchronically with the background insect sounds.

At 1:20 the insect sounds stop. Silence until 1:26 when the sound of liquid being poured onto some surface is heard. Suddenly, the white light is interrupted by small, circular waves. Immediately it is clear: the liquid heard is being poured into a container. The white light is a reflection in that container.

Silence as the white light, now tiny and surrounded by a ring, like an orbiting moon, moves on the surface of what is now understood as liquid.

At 2:00 the sounds of ducks and geese return. Arabic writing appears on an entirely black screen. Sounds of ducks and geese and rhythmic clicking continue. A rooster crows. Sounds, seemingly of something walking, slowly, carefully, on gravel. Slow zoom into text overlaid on the screen.

At 2:25 a second screen of Arabic writing appears. The sound changes to nighttime insects. Slow zoom into another, different text.

The white light, now understood as a reflection in liquid, reappears. Night insect sounds continue as a third screen of Arabic text overlays the visual. The reflected light dances behind and in the center of the text filling the screen.

The text disappears. More liquid is poured into the container. The reflected light is disturbed. The sound fades out as a final screen of Arabic text appears, as if being written onto the screen.

Translation of the text to English speaks to images and sounds.

It is not surprising that the ear depends upon imagination
surrounded by sounds.
A rooster crows for the time of prayer.
Frogs croaking foretell the smell of rain.
A dog passes nearby, pauses, continues on his way.
Your eyes, staring up for a long time,
suddenly look down to the mug into which you are pouring water.
You were in the mug where the water reflected the moon.

The reflected light of the moon dances in the mug at the bottom of the screen. Fade to black. End (Habibi 2018).

Significance

These works of electronic literature by Hight et al., Moulthrop, and Habibi each promote engagement through their effective use of sound, vocal, environmental, or mechanical.

For example, the sounds of 34 North 118 West, according to Hight, positioned one in both past and present at specific locations comprising the work.

The original 34 North 118 West is no longer available as an in situ experience. But, the sound files are available, and several were given to the author by Hight, along with permission to remix them so as to portray a sense of the original, larger work. For all practical purposes, these sounds–a train crossing, a street band, a street salesperson, narratives of individuals living in the area–are all that remain of the work.

In a 13:00 recombination of these sounds, the author utilized an Aristotelian, linear narrative structure of beginning, middle, and end, bookended by the sound of a passing train. Other approaches would be equally valid. The result from any approach is for the sound-based narratives of 34 North 118 West to provide effective access to this work–no longer available–a facsimile for the intent and content of the original work (Hight, Knowlton and Spellman 2018).5

Under Language is still available in its original context thanks to a website maintained by the author, Stuart Moulthrop. Like Hight, Moulthrop was generous in providing the author with sound files from this work, as well as permission to create experimental remixes.

Using sound files provided by Moulthrop, the author created a 14:20 audio narrative by arranging individual sound files following their numbering from Moulthrop’s original content database to create a serendipitous narrative. Other methodologies could easily be used. As with the original work, there are five layers (as described previously) to this re-conceptualized narrative. The reader and/or player is responsible for making sense of the artifact (Moulthrop 2020).6

Missing in this effort is the direct interaction the user / reader / player / participant has with choosing the lines of the poem. But, remaining is the direct experience with the under language, the vocalization(s) of the computer programming underlying, supporting, creating the original work.

The three works by Habibi, while each an example of video poetry, incorporate strong, and one might argue essential, sound elements. In Just Words, one sees a hand turning individual pieces of paper on which are inscribed Arabic words and / or phrases. But it is the sound of these pages turning that sparks one’s imagination, encouraging one to ask for more perspective on this literary experience. What are these pages? How do they relate to one another? Who is turning the pages? What is the reader and / or viewer to think of these words and their display?

In another scene one sees these individual pages hanging from a line, like fish or dried fruits, evidence of their collection and display outside their native element. But again, it is the sound of these pages, this time rustling in the air moving through a nearby window, that makes clear what one sees is a context, a place, a space in which these words might be experienced. Taken out of context, do these words still convey the same meaning(s) as when bound with others into the form of a book? Again, what is the reader and / or viewer to think of these words and their display?

In Matchbox, one sees nothing but blackness. Only at the end of this video poem, when a match is struck and thus illuminates the face of a young person, does one understand what is seen is the darkness of night. Only with the appearance of the text superimposed at the end of the video does one have full context for the poem. Again, it is sound, this time the striking of a match, that provides a sense of presence in a particular place and brings the visual elements together, providing an interface with which one can engage the essential nature of the poem.

At the heart of Mug, is, one might argue, an acoustic listening experience, where one does not know the source of the sound(s) heard. One might also argue that the substance of the visual experience is unknown, too. Beyond the text, it is the sound of liquid being poured into a container of more liquid that provides the crucial key for understanding this video poem and its examination–interrogation?–of what is, ultimately, a reflection. At this point, as noted in the text of the poem, one is present in the mug. That space, created by sound, defines one’s interaction with the poem.

What does all this mean? Why is it important? A first response might be to note that sound provides a fundamental sensory input and communication channel for human culture. Sound is the phoneme for speech (verbalization of abstract thought). Sound can be a central component of narrative (the recounting of a sequence of events and their meaning) and storytelling (the addition of setting, plot, characters, logical unfolding of events, a climax).

Allan Moore describes how motifs that remind of and return lyrical references provide embedded meaning in popular music (Moore 2012). This same connection is demonstrated in the examples of electronic literature and poetry discussed in this essay. Each uses sound to connect and surround readers with multiple, concurrent aspects of their storytelling. If readers are fully engaged with the imaginative situation–whether musical or literary–the result can be quite powerful, according to Tim Crook, who says sound effectively prompts life from little details seen in the listeners’ mind’s eye (Crook 1999).

Alan Hall describes how this phenomenon works. Listening carefully and purposefully to sound(s) offers, according to Hall, “a portal through which a deeper, often inarticulate, consciousness can be glimpsed… The intention is to find deeper and wider resonances within–and without–the listener” (Hall 2010, 99, 104).

Michael Bull and Les Back say sound provides a place in which embodied social and cultural traces can be carried, often without the awareness of their bearers. Therefore, it is good to actively and deeply listen to the sounds of the world in which one lives. In this way, according to Bull and Back, one opens new approaches to thinking about and appreciating the social experience, memory, time, and place–the auditory culture–of sound (Bull and Back 2003).

The works by Hight et al., Moulthrop, and Habibi considered in this essay encourage consideration of how different aesthetic conceptualizations and material practices of voice and other sound(s) inform literary expression. For example, sound and poetry might be considered an exchange between language and code, and thus at the center of our understanding of language arts. The desired outcome is to expand understanding of literature and textuality as vehicles for exchanges in and across media, languages, and cultures.

Finally, this focus foregrounds an approach to literature characterized by what Edmund Carpenter calls the verbal, musical, and poetic traces and fragments (figures) of oral culture (Carpenter 1970). The works by Hight et al., Moulthrop, and Habibi discussed in this essay show how sound provides a way of knowing and being in a literary world, real or imagined. In this regard, sound is the basis for literature (written works considered to possess lasting artistic merit) and the various practices and cultures associated with its production and consumption (reading, writing, and listening). Listening, as real and concrete participatory practice, evokes aural experiences across a wide range of theory and practice through which listeners can derive information about the world they inhabit. This suggests a way forward for electronic literature, through careful listening.

References

Patterned after the first video game, Pong, Moulthrop’s Sc4nda1 in New Media (2012) converges philosophical meditation with a retro video game. The sounds provide context(s) in this new space. The author maintains an archival webpage for a remix he created using original sound files supplied by Mouthop (Barber 2020b).

Barber, John. 2020a. “Radio Salience.” 2020. Link.

Barber, John. 2020b. “Sc4nda1 in New Media. radioELO.” 2020. Link.

Barber, John. 2020c. “Under Language.” 2020. Link.

Bull, Michael, and Les Back, ed. 2003. The Auditory Culture Reader. Oxford : Berg.

Carpenter, Edmund. 1970. They Became What They Beheld. New York : Outerbridge & Dienstfrey.

Chion, Michel. 1994. Audio-Vision Sound on Screen. Translated and edited by Claudia Gorbman. New York : Columbia University Press.

Crook, Tim. 1999. Radio Drama Theory and Practice. Abingdon, UK : Routledge.

Furlonge, Nicole Brittingham. 2018. Race Sounds: The Art of Listening in African American Literature. Iowa City : University of Iowa Press.

Habibi, Mohamed. 2018. “habibi1366.” YouTube. 2018. Link.

Hall, Alan. 2010. “Cigarettes and Dance Steps.” In Reality Radio: Telling True Stories in Sound, ed. by John Biewen and Alexa Dilworth, 96‑107. Chapel Hill : University of North Carolina Press.

Hight, Jeremy. 2013. “Narrative Archaeology.” In NeMe. March 25. Link. Also available here.

Hight, Jeremy, Jeff Knowlton, and Naomi Spellman. 2003. “34 North 118 West.” In radioELO. Link.

Moore, Alan. 2012. Song Means: Analyzing and Interpreting Recorded Popular Song. Farnham, NY : Ashgate.

Moulthrop, Stuart. 2020. Under Language. 2020. Link.

Murch, Walter. 2005. “Womb Tone.” Transom Review 5 (1). Link.

Ong, Walter J. 1982. Orality and Literacy: The Technologizing of the Word. York : Methuen.

Smith, Bruce R. 2003. “Tuning into London c. 1600.” In The Auditory Culture Reader, ed. by Michael Bull and Les Back, 127‑35. Oxford : Berg.

Under Language shared the 3rd International Digital Literature Award Ciutat de Vinarós Prize for Digital Narrative with Isaias Herrero Florensa’s Universo Molecula. In his artist statement, Moulthrop describes the origin of the term “textual instrument.”
[Textual instruments is] a term I borrowed from John Cayley many years ago to describe things that might look like literature, but also like structures for play, though not necessarily what we would call games. In fact, this one lies pretty close to game space, having rules, a scoring system (albeit invisible), and even a simple agon in which you compete against the perversity of the puzzle-maker, and constraints of the clock (Moulthrop 2020). [back]

Moulthrop maintains an archival webpage for Under Language, where he provides information and listening opportunities. In the “About” section of this website he describes the inspiration for the term under language.
The phrase “under-language” was invented by the comics artist Alan Moore in an interview he gave in the early 1980s. He used it to describe the essence of comics art, which is neither verbal nor visual, but something that underlies and infuses both modes. The term gets at the essence behind Moore’s great genius for irony and verbal-visual puns. It also provides a convenient reminder that everything, these days, tends to mean more than it seems (Moulthrop 2020).
The author maintains an archival webpage for a remix he created using original sound files supplied by Moulthrop (Barber 2020c). [back]

Similar to Under Language, Moulthrop’s Radio Salience (2007) is an interactive image-text-sound instrument with a game-like interface that explores indeterminacy, accident, and resonance, taking as its muse the breathless voice of the airwaves and radio. The author maintains an archival webpage for a remix he created using original sound files supplied by Moulthrop (Barber 2020a). [back]

The author met Saudi poet and scholar Mohammed Habibi and was introduced to his video poetry at the Arabic Electronic Literature: New Horizons and Global Perspectives conference. Perhaps the first focusing on Arabic electronic literature, this conference was convened at Rochester Institute of Technology, in Dubai, United Arab Emirates, Feb. 25-27, 2018. Organization website / Conference program [back]

The author maintains an archival webpage providing information and listening opportunities. [back]

The author maintains an archival webpage providing information and listening opportunities. [back]

Biographies

Barber, John F.
Back  /  Retour

John F. Barber convenes with The Creative Media & Digital Culture Program at Washington State University Vancouver. His scholarship, teaching, and creative endeavors arise from intersections of art, humanities, and technology and manifest as literary media art where he feels that practice-based research discovers and puts into action new knowledge. His radio and sound art are broadcast and exhibited internationally. His publications appear in Digital Humanities Quarterly, Digital Studies, ebr, Hyperrhiz: New Media Cultures, Leonardo, MATLIT (Materialities of Literature), Scholarly Research and Communication, and elsewhere. Barber curates The Brautigan Library, a collection of unpublished manuscripts, and was featured on This American Life. See his website, for more information and access to his work.

John F. Barber est professeur dans le programme des Médias créatifs et de la Culture numérique à l’Université de l’État de Washington à Vancouver. Ses travaux d’érudition, d’enseignement et de création sont issus des intersections de l’art, les humanités et la technologie, et se manifestent sous la forme d’un art littéraire médiatique où il estime que la recherche basée sur la pratique découvre et met en pratique de nouvelles connaissances. Ses œuvres radiophoniques et sonores sont diffusées et exposées dans le monde entier. Ses publications paraissent dans Digital Humanities Quarterly, Digital Studies, ebr, Hyperrhiz: New Media Cultures, Leonardo, MATLIT (Materialities of Literature), Scholarly Research and Communication et ailleurs. Barber est le conservateur du Brautigan Library, une collection de manuscrits non publiés, et a été présenté sur This American Life. Voir son site web, pour plus d’informations et pour accéder à son travail.

Textes optimisés
pour
dyslexiques

Texts optimized
for dyslexic readers

Donner aux dyslexiques accès à la lecture en adaptant la typographie : l’enjeu est crucial et on l’a compris, retourner en arrière.

Introduction
Attention à la marche !
Penser la littérature électronique en culture numérique

Bertrand Gervais

Attention à la marche !
(jump to English version)

Comme un train entré en gare, la littérature électronique s’est arrêtée en 2018, le temps d’une escale, dans ce vaste immeuble tout neuf que représente la culture numérique. Le train avait déjà un certain âge, tandis que la gare était récente. Il apparaissait évident que la rencontre entre les deux n’était pas parfaite. À tout moment, on risquait de trébucher, surtout si on ne faisait pas attention où l’on mettait les pieds. Il y avait un écart, une faible distance, entre le marchepied et le quai. Faible, de sorte que de loin, on ne le remarquait guère, mais assez marqué pour provoquer des faux pas.

Le congrès de l’Electronic Literature Organization, Attention à la marche !, tenu à Montréal à la fin de l’été 2018, entendait questionner de façon explicite la place de la littérature électronique dans la culture numérique. Quelle est la nature de la relation entre les deux ? Comment une forme déjà vieille de cinquante ans parvient-elle à rester pertinente dans un environnement culturel fortement marqué par le numérique ? Comment une pratique d’avant-garde développée en culture du livre réussit-elle à s’ajuster aux principes d’une culture de l’écran ?

Les expressions « littérature électronique » et « culture numérique » viennent d’être employées comme s’il s’agissait de pratiques ou de champs homogènes aisés à définir. Il en va évidemment tout autrement. En effet, la littérature électronique n’a pas connu un développement homogène, elle n’a pas suivi une progression linéaire ; et on en serait même rendu, comme le dit Leonardo Flores (2019), à une troisième génération. Les deux premières générations, définies par N. Katherine Hayles, permettaient de distinguer, d’une part, une littérature pré-web, fondée sur des formes complexes de textualité, axées notamment sur les fonctionnalités de l’hypertextualité, mais respectant globalement les conventions de la culture de l’imprimé et du livre ; et d’autre part, à partir de 1995, un ensemble de pratiques basées sur le web et intégrant le multimédia et l’interactivité (Hayles 2008). La troisième génération est celle de la littérature numérique en tant que telle, une littérature qui se sert de l’ensemble des dispositifs et des plateformes de diffusion rendus disponibles par les développements informatiques récents, notamment les développements de l’infrastructure technologique qu’est Internet. Cette dernière génération englobe les deux premières qui continuent à avoir leurs adeptes et épigones.

De la même façon, il n’y a pas une seule culture numérique, mais un ensemble de pratiques culturelles, artistiques et textuelles, diverses et non conventionnelles, pour ne pas dire divergentes, en marge bien souvent des institutions (quoique celles-ci cherchent de plus en plus à les intégrer), qui se servent de ces dispositifs et plateformes de diffusion ou de l’infrastructure d’Internet. Comprendre les manifestations et la logique sous-jacente des cultures numériques exige des formes de recherche et de recherche-création innovantes, ainsi que des approches à la croisée des disciplines, telles que l’herméneutique, l’esthétique ou l’ethnographie numériques (Miller 2011), l’archéologie des médias, les humanités numériques, etc.

Le principal objectif du congrès ELO 2018, duquel découle la publication que ce texte introduit, était d’ailleurs de chercher à multiplier les perspectives critiques sur les formes de communication et d’imbrication, mais aussi de résistance entre littérature électronique et culture numérique, en réunissant un très grand nombre de chercheurs et d’artistes préoccupés par leur développement. La rencontre a ainsi servi à favoriser le renouvellement des perspectives de recherche et de recherche-création en littérature électronique, à mieux définir les zones de recoupement entre cette forme littéraire et les humanités numériques, à prendre en compte les technologies mobiles, de plus en plus présentes au quotidien, dans leur impact sur les habitudes d’écriture et de lecture et à investiguer la dimension politique du numérique, dans ses liens à l’art, à la littérature et à la culture. Parmi les autres thèmes abordés, on a pu noter un intérêt accru pour les principes de réalité augmentée et de réalité virtuelle, pour les formes et enjeux de la traduction (en termes de codes, de langues ou de médias), pour les spécificités langagières et culturelles de la littérature électronique, pour les liens entre le numérique et la culture urbaine, ainsi que pour les pratiques éditoriales destinées aux jeunes (les lecteurs de demain).

*
*    *

Il est difficile de résumer une rencontre où une plus de cent cinquante communications et conférences ont été prononcées, un congrès accompagné d’une importante exposition de 60 œuvres numériques, ainsi que d’un festival de performances échelonné sur 3 soirées. Il est tout aussi difficile de structurer les 35 textes qui ont été proposés pour publication et qui sont réunis ici. Le congrès ayant été bilingue, les textes publiés sont ainsi en français et en anglais. Nous n’avons pas choisi de les regrouper en fonction de la langue de rédaction, mais plutôt en regard des enjeux abordés.

L’ouvrage est composé de six sections. La première, intitulée « Des sons, des bruits et des voix », explore la dimension acoustique de notre relation à la littérature électronique. On ne fait pas que lire lorsque confrontés à de telles œuvres, on regarde des vidéos, on explore une architecture virtuelle et, dimension trop souvent oubliée, on écoute toute sortes de bruits et de sons. La deuxième section, « Le langage comme interface », suit de près cette première exploration, en portant l’attention cette fois sur la programmation et le code, dans leur capacité à structurer une expérience et à l’orienter de manière précise. Les pratiques numériques reposent sur du langage qui renouvelle notre rapport aux œuvres.

La troisième section, « Plateformes et narrations », examine diverses plateformes, allant des jeux vidéo aux dispositifs de Réalité Virtuelle, utilisées, voire détournées, afin de générer des œuvres inattendues, qui renouvellent notre compréhension des arts numériques. Sous le titre très large des « Esthétiques numériques », la quatrième section s’arrête sur quelques œuvres récentes, présentées bien souvent par les artistes qui les ont produites. Cette section permet de donner un aperçu des projets qui ont été exhibés dans le cadre du congrès et de son exposition.

La cinquième section « Lire et écrire en culture numérique » s’interroge sur nos stratégies de production et de réception des œuvres de littérature électronique, y compris dans les salles de classe. Cette interrogation permet de revenir de façon importante sur les enjeux pédagogiques de notre passage à la culture numérique, dimension qui devient de plus en plus importante.

Finalement, la sixième et dernière section s’arrête sur des « Perspectives diachroniques sur la littérature électronique ». Cette section est l’occasion d’approfondir notre connaissance des développements de la littérature numérique, depuis les 50 dernières années. L’hypothèse des trois générations de cette littérature, discutée d’entrée de jeu, y trouve un écho explicite.

Ce regard diachronique vient en quelque sorte boucler la boucle. Les sections se sont enchainées en fonction d’une spirale centrifuge, commençant au plus proche du texte et de son expérience, du point de vue acoustique ou du code, et s’ouvrant peu à peu aux plateformes et dispositifs exploités, aux principes esthétiques qui ont servi à la production et à la réception des œuvres, aux stratégies de lecture et d’écriture, abordées de façon analytique et, ultimement, aux transformations historiquement datées que cette pratique littéraire et artistique a connues.

Références

Flores, Leonardo. 2019. «Third Generation Electronic Literature». Electronic Book Review. Accéder au lien.

Hayles, Katherine N. 2008. Electronic Literature. New Horizons for the Literary. Indiana : Notre Dame University Press.

Miller, Vincent. 2011. Understanding Digital Culture. London : Sage.

We consider accessibility as crucial. Give dyslexic readers access by adapting the typography: click here to go back!

Introduction
Mind the Gap!
Thinking electronic Literature in a Digital Culture

Bertrand Gervais

Mind the Gap!
(aller à la version française)

Like a train entering the station, electronic literature stopped in 2018 in the vast new building that is digital culture. The train was already a certain age, while the station was brand new. It was obvious that the meeting between the two was not perfect. At any time, there was a risk of stumbling, especially if you weren’t careful where you set foot. There was a gap, a small distance, between the step and the platform. It was small, so that from a distance you could hardly notice it, but it was marked enough to cause you to stumble.

The Electronic Literature Organization’s conference Mind the Gap!, held in Montreal in late summer 2018, was intended to explicitly question the place of electronic literature in digital culture. What is the nature of the relationship between the two? How does a form that was already fifty years old manage to remain relevant in a cultural environment strongly marked by digital technology? How does an avant-garde practice developed in the context of book culture succeed in adjusting to the principles of a culture heavily engaged in screens, networks and mobile devices.

We’ve just used the terms “electronic literature” and “digital culture” as if they were practices or homogeneous fields that are easy to define. This is obviously false. Indeed, electronic literature has not developed homogeneously, it has not followed a linear progression; and we are even in the presence, as Leonardo Flores (2019) says, of a third generation. The first two generations, as defined by N. Katherine Hayles, made it possible to distinguish, on the one hand, a pre-web literature based on complex forms of textuality, focusing in particular on the functionalities of hypertextuality but generally respecting the conventions of print and book culture; and on the other hand, from 1995 onwards, a set of web-based practices integrating multimedia and interactivity (Hayles 2008). The third generation is that of digital literature as such, a form that makes use of all the devices and platforms made available by recent computer developments, particularly developments in the technological infrastructure that is the Internet. This last generation encompasses the first two which continue to have their followers and epigones.

Similarly, there is not a single digital culture, but a set of diverse and unconventional–not to say divergent–cultural, artistic and textual practices, often at the margins of institutions (although these are increasingly seeking to integrate them), which use these devices and platforms or the internet infrastructure. Understanding the manifestations and underlying logic of digital cultures requires innovative forms of research and research-creation, as well as cross-disciplinary approaches, such as hermeneutics, digital aesthetics or ethnography, media archaeology, digital humanities, etc. (Miller 2011).

The main objective of the ELO 2018 conference, the origin this publication, was moreover to multiply critical perspectives on forms of communication and interaction, but also of resistance between electronic literature and digital culture, by bringing together a large number of researchers and artists concerned about their development. The meeting thus served to encourage research and research-creation perspectives in electronic literature, to better define the intersections between this literary form and digital humanities, to take into account mobile technologies which are increasingly present in everyday life, their impact on writing and reading habits, and to investigate the political dimension of digital technology, in its links to art, literature and culture. Other themes addressed during the four days of the conference included an increased interest in the principles of augmented and virtual reality, the forms and challenges of translation (in terms of codes, languages or media), the linguistic and cultural specificities of electronic literature, the links between digital and urban culture, and editorial practices aimed at young people, that is, the readers of tomorrow.

*
*    *

It is difficult to summarize a meeting where some one hundred and fifty papers and conferences were delivered, a congress accompanied by a major exhibition of 60 digital works, as well as a festival of performances spread over three evenings. It is just as difficult to structure the 35 texts that have been proposed for publication and that are gathered here. The conference was bilingual, thus the texts published are in French and English. We have not chosen to group them according to the language of writing, but rather according to the themes addressed.

The book is divided into six sections. The first, entitled “Sounds, Noises and Voices,” explores the acoustic dimension of our relationship to electronic literature. We don’t just read when confronted with such works, we watch videos, explore a virtual architecture and, a dimension too often forgotten, listen to all sorts of noises and sounds. The second section, “Language as Interface,” follows closely on from this first exploration, this time focusing on programming and code, in their capacity to structure an experience and orient it in a precise manner. Digital practices are based on language that renews our relationship to works.

The third section, “Platforms and Narratives,” examines various platforms, from video games to Virtual Reality devices, which are used, or even hijacked, to generate unexpected works that renew our understanding of digital art. Under the broad title of “Digital Aesthetics,” the fourth section focuses on some recent works, often presented by the artists who produced them. This section provides an overview of the projects that were exhibited at the conference and its exhibition.

The fifth section, “Reading and Writing in Digital Culture,” looks at the strategies used for producing and consuming electronic literature, including in the classroom. This questioning allows us to return in a significant way to the pedagogical stakes of our transition to digital culture, a dimension that is becoming increasingly important.

Finally, the sixth and last section focuses on “Reading E-Lit Across Time.” This section is an opportunity to deepen our knowledge of developments in digital literature over the last 50 years. The hypothesis of three generations of e-lit, mentioned earlier, is explicitly echoed here.

This diachronic view closes the loop, as it were. The sections have followed one another in a centrifugal spiral, starting as close as possible to the text and its experience, from the acoustic or coding point of view, and gradually opening up to the platforms and devices exploited, the aesthetic principles that have served for the production and reception of the works, the reading and writing strategies, approached analytically and, ultimately, the historically dated transformations that this literary and artistic practice has undergone.

Did we succeed in closing the gap? Mind you… We barely managed to look at the space itself and figure its form! But in doing so, we continued to explore a literary form that still evolves and questions our ability to represent our world.

References

Flores, Leonardo. 2019. «Third Generation Electronic Literature». Electronic Book Review. Link.

Hayles, Katherine N. 2008. Electronic Literature. New Horizons for the Literary. Indiana : Notre Dame University Press.

Miller, Vincent. 2011. Understanding Digital Culture. London : Sage.

1

Des sons, des bruits et des voix
Sounds, Noises and Voices

John F. Barber
Listening to Electronic Literature: Sounds at the heart of works by Jeremy Hight, Stuart Moulthrop, and Mohamed Habibi

Simon Biggs
Speech to Text: Between the Real and the Unreal

Christophe Collard
Cling-Clang-Cornelius: Digital Sound Poetry as Embodied Posthumanism

Verónica Paula Gómez
Demagogic speech and rhetorical emptiness

Kedrick James • Ernesto Peña • Yuya Takeda and the Digital Literacy Centre
Transmedia: an improvisualization

We consider accessibility as crucial. Give dyslexic readers access by adapting the typography: click here to go back!

Listening to Electronic Literature:
Sounds at the heart of works by Jeremy Hight, Stuart Moulthrop, and Mohamed Habibi

John F. Barber

Abstract

This essay argues that sound(s) in electronic literature, experienced through listening, prompt(s) readers to re-think their relational experiences with literary worlds, real or imagined, with others in these spaces and places, and with themselves. This centrality of sound(s) is examined through discussions of works of electronic literature by Jeremy Hight et al., Stuart Moulthrop, and Mohamed Habibi. Each has something to say about listening to electronic literature.

Résumé

Cet article démontre que le(s) son(s) dont on fait l’expérience dans la littérature électronique poussent les lecteurs à repenser leurs expériences avec le monde littéraire, réel ou imaginé, avec les autres qui sont présents dans ces espaces et ces lieux, ainsi qu’avec eux-mêmes. La centralité des sons est examinée à travers l’analyse des œuvres de littérature électronique de Jeremy Hight et al., de Stuart Moulthrop et de Mohamed Habibi. Chacune de ces oeuvres nous dit quelque chose sur le son et l’écoute de la littérature électronique.

Introduction

Electronic literature, as a field of research and creative practice evolving from electronic and digital computing technologies, displays its artifacts primarily on screens of various devices. And, given its iterative and experimental evolution, one might suggest what one sees when considering electronic literature depends upon the perspective from which one looks. This essay suggests another way of considering electronic literature: by listening.

Printed text–literature or poetry, print or pixels–has long afforded readers opportunities to create immersive virtual worlds using their memories and imaginations. Sound(s) make(s) these worlds believable. Listening is the portal. To explore these ideas, this essay first defines sound as the experience of acoustic energy that conveys information. Listening promotes the understanding of this information. Next, the centrality of sound, especially the speaker’s voice, and its forward inclusion through print and pixels is outlined. To examine these ideas, pioneering works of electronic literature by Jeremy Hight et al., Stuart Moulthrop, and three contemporary video poems by Mohamed Habibi are considered. As will be suggested, these works of electronic literature each promote our engagement through their effective use of sound. This essay concludes by considering the significance of this alliance.

Conceptual Framework

The term “sound,” as used here, signifies vibrations of acoustic energy that travel through air, or some other medium. These vibrations are experienced (heard) when they reach one’s ears and register in one’s brain as sound(s).

According to Walter Murch, hearing is one of the first human senses to develop, and for approximately 4.5 months before birth one is bathed in a rich soup of sounds heard within the womb. After birth, a baby reacts immediately to sound, before its vision fully develops (Murch 2005).

Different from hearing, listening is the active processing of auditory stimuli, a carefully and purposefully conducted activity, involving conscious effort and attention. French critic and composer Michel Chion, in his discussion of aural relationships to sound in cinema, suggests one listens to obtain information about a sound source, to appreciate the sound itself, and to learn what is communicated by the sound (Chion 1994). Nicole Brittingham Furlonge calls listening “a physiological form of reception,” a practice that is at once “interpretive, situated, and reflective.” Listening to sound, as analytic “storyhearers” says Furlonge, prompts engagement that is participatory, interactive, and experiential (Furlonge 2018, 60, 83).

With Murch’s positioning of awareness of sound(s) as primary, Chion’s suggestion of listening to learn, and Furlonge’s notion of engagement as storyhearers, Bruce R. Smith posits that knowing the world through active listening is fundamentally different from knowing the world through passive viewing (Smith 2003). Michael Bull and Les Back echo this idea when they say sound is primary for knowing and being in the world. One hears a sound and it provides immediate clues about one’s location, and what to expect. If the sound is acousmatic, originating from an unseen source, one can use experience, memory, or imagination to discern its source and meaning (Bull and Back 2003).

At the risk of oversimplifying the work of Walter Ong and others regarding orality (both primary and secondary) and literacy, one might consider speech as expressing and sharing abstract ideas, histories, memories, and cultures, as spoken narratives for understanding external and internal worlds (Ong 1982). With the advent of writing, sounds of speech were graphically transcribed. Thus, writing provides scripts for reproducing the sounds of the spoken voice, whether internally, as part of silent reading, or externally, as the result of reading aloud. In either case, the dynamics of voice are transmitted to the reader, despite its remediation.

Although speech and writing are fundamentally different–speech being based upon hearing and writing upon seeing–at the heart of each is the sound(s) of the speaker’s voice. Listening to these sounds directly engages our memories and imaginations, helps us rethink our relational experiences with others, with ourselves, and the spaces and places we inhabit. Listening to sound(s) provides a way of knowing and being in a world, whether real or imagined.

Discussion

To examine these ideas, we can consider two pioneering works of electronic literature–34 North 118 West by Jeremy Hight et al. (2003), and Under Language by Stuart Moulthrop (2007) (2020)–as well as three contemporary video poems by Mohamed Habibi–Just Words, Matchbox, and Mug. These examples are chosen for their strong emphasis on sound.

34 North 118 West

Jeremy Hight, with Jeff Knowlton and Naomi Spellman, created and released 34 North 118 West in 2002-2003. This pioneering work of locative electronic literature combined multiple media and Global Positioning System (GPS) technology to create an interactive series of narratives about a once thriving railroad depot situated at 34 North latitude, 118 West longitude in downtown Los Angeles, California, during the first half of the twentieth century (Hight, Knowlton, and Spellman 2003).

Participants walked through the four-block area formerly occupied by the railroad station and other businesses with a laptop computer, a GPS device, and headphones. GPS tracked and overlaid their position on a map displayed on the computer screen. Easily identifiable locations were also displayed. Approaching these “hot spots,” participants triggered recorded narratives and soundscapes created from historical, ethnographic, and architectural information about the area.

Other sound effects–squeaking wooden cart wheels and musicians entertaining on busy street corners, for example–were triggered by hidden GPS locations, each waiting to be discovered by wandering participants. These sounds were purposely designed to connect physical locations with events, activities, narratives, and lives of a past dismissed by urban change. Signs, displays, and other physical artifacts or details at each location augmented the narratives, providing metaphors and information for interaction(s) with the characters and history of the area defined by the geographic coordinates 34 North 118 West.

Hight, Knowlton, and Spellman investigated the half-square mile area around the former railroad freight station for more than a year, digging through the histories of the buildings to learn about people who worked and lived there. They crafted narratives from the hidden or lost information they recovered.

For example, at the site of a former tire factory, a worker describes how bits of rubber rained down on Los Angeles after the factory caught fire. A waitress at the train station restaurant talks about the harried passengers she serves. A railroad worker recounts cleaning the tracks after people committed suicide by stepping in front of trains. A cook, the station clock inspector, and others provide additional narratives.

As participants moved throughout the area, triggering multiple sound-based narratives, they uncovered the hidden history of this once thriving part of downtown Los Angeles, and developed a sense of the work’s larger scope and concept. These narratives of forgotten or faded histories, lost buildings, tensions still present of past persons, all buried in memory, could, Hight contends, return with sufficient ability to sustain listeners simultaneously in two separate realities at the same location, one present, the other past (Hight 2013).

Hight contends this approach helps organize forgotten historical and cultural information into meaningful narratives about a place, a time, and people. Hight calls these narratives “sonic archaeology in the urban landscape,” or “narrative archaeology” (Hight 2013).

Under Language

Pioneering electronic literature author Stuart Moulthrop released Under Language in 2007. Moulthrop calls this work a “textual instrument,” an artifact akin to literature but structured like a game (Moulthrop 2020). Few instructions are provided for interaction, however, leaving the reader and / or player to learn the rules for the work. Using a game-like screen display interface that responds to mouse clicks, users select ten lines of text for a poem. These ten lines of text are displayed on screen, along with a closing graphic, chosen by Moulthrop’s textual instrument to reflect the quality of the final poem. Repeat as many times as desired for different poems.1

Moulthrop uses the term “under language” in two ways. First, the title, Under Language, speaks to the underlying computer code that drives the work. Under language, the concept, speaks to both the necessity to notice how writing intersects code, and the consequences of a collision (collusion?) when poetry meets code.2

So, under language, the concept, underlies and infuses Under Language, the work, which is, fundamentally, a generative textual work, meant to be experienced visually, on a screen. But the brilliance of this work is Moulthrop’s sonification of the underlying five layers of computer code. The first is a series of computer-voiced renditions of ActionScripts programmed by Moulthrop that operate the work. The second layer is a series of ambient recorded collages of tunings across radio broadcasts. The third level consists of pseudo-code, again voiced by text-to-speech technology. Comments and summaries, ostensibly voiced by the ten-line poem at the heart of this work, constitute the fourth level. The fifth level is an audio collage, where the poem’s ten lines are each vocalized, as well as comments seemingly from the poem’s self awareness of its creation.

These vocalized narratives of the under language for Moulthrop’s work are not specifically ordered, but rather assembled from user choices of lines of text for a generated poem. Still, the result provides unprecedented access to the interactive affordances beyond the program’s screen-based visual displays.

Moulthrop’s point is that under language (the underlying code) is the language of computer programming, and is inseparable from the work titled Under Language. With Under Language, the work, Moulthrop argues that to experience electronic literature one must appreciate the underlying code, which is vocalized as part of the work.

One might extend Moulthrop’s point by arguing that Under Language is an example of sound-based electronic literature. Hearing the under language vocalized, one understands the presence of hidden narratives concurrently creating and commenting upon one’s experience of the work’s visualization, and, indeed, writing the larger context for its experience. Rather than visuals augmented by sound, Under Language is a work of electronic literature where sound is augmented by visuals, where sound is at the heart of the literary experience.3

Mohamed Habibi

Contemporary examples of Arabic electronic literature by Mohamed Habibi extend the idea of sound-based electronic literature. Habibi, a Saudi scholar and poet working in the genre of video poetry, has for the past decade shared his work through his YouTube channel (Habibi 2018). This essay examines three of Habibi’s video poems: Just Words, Matchbox, and Mug.4

Just Words

Just Words (2:11) begins with a flashlight beam illuminating what appears to be a blank wall. All else is black, and silent. The focus changes and the beam illuminates Arabic writing on a piece of paper. Sound begins–insects and rhythmic clicking. Text appears superimposed on the screen, below the image of the paper, reading,

We dream

Almost unseen, a hand turns the paper to expose another, again with Arabic writing,

dream

The sound continues. One now has the sense of discovering this text in a darkened room, at night. Turning the paper–might one say “pages” now?–reveals another, again with writing. The superimposed translation reads,

in the morning

The rhythmic clicking becomes more rapid, intense, almost insistent. Another page flip…

We carry our dreams

page flip…

to dry it out.

Another page flip…

it won’t fly away;

Two page flips…

as we’ll clasp them.

At 1:35 the scene changes to ten white “pages,” each inscribed with Arabic writing, secured on a line by clothespins–three groups of two pages, one of three pages, and one of one page–near a window. Daylight illuminates the room, and the pages.

One has the sense that these pages represent the dream portrayed in the first scene of Habibi’s video poem, and that these parts of the dream are drying in the morning light. At first this tableau is silent, but the sound of these pages rustling fades in as they are seen moving in an air current from the nearby window.

A dissolve at 1:58 returns us to the first scene: a piece of paper illuminated by a flashlight beam, the rest of the room in darkness.

the clasps

The night insects and rhythmic clicking sounds begin. The page is turned …

are… Just words

Fade to black, and end (Habibi 2018).

Matchbox

Matchbox (1:52) begins with a black screen. Sounds are quickly heard–footsteps running, dogs barking, some vocalizations that hint at physical effort. Memories of similar sounds suggest this is perhaps someone running.

At 0:30 the running stops, the runner seems to take a breath, and one hears a series of sliding, scratching sounds. Silence as Arabic writing appears on the screen, from top right across, then back to the right margin before beginning a new line. The screen fills with writing.

The soundtrack begins anew–breathing, running, dogs, the same sliding, scratching sound. More Arabic writing appears on the screen, right to left, appearing to be written as one watches.

More scratching, sliding sounds until a match lights in the darkness, briefly illuminating a person’s face.

Four seconds of silence and blackness passes until the soundtrack–running feet, breathing, dogs barking–returns. English writing appears on the screen, left to right, as if being typed in real time.

The child
–who runs out of the door
like a pullet
who gets up to glance at his injured
elbow
who walks by anyone to reassure
himself
is you.

They send you out
to the neighborhood
along the dark street,
where the walls are painted with ghosts
to get a matchbox.

A match is lit, again briefly illuminating the face of a young person. End (Habibi 2018).

Mug

Like Matchbox and Just Words, Mug (3:59) begins with a black screen. The rhythmic clicking, heard in Just Words, begins the soundtrack and is quickly followed by sounds of birds, geese, and a crowing rooster. Night sounds follow immediately–insect sounds and human vocalizations in the background.

At 0:46 a small circle of white light appears in the lower right quadrant of the otherwise black screen. It begins to gyrate, seemingly synchronically with the background insect sounds.

At 1:20 the insect sounds stop. Silence until 1:26 when the sound of liquid being poured onto some surface is heard. Suddenly, the white light is interrupted by small, circular waves. Immediately it is clear: the liquid heard is being poured into a container. The white light is a reflection in that container.

Silence as the white light, now tiny and surrounded by a ring, like an orbiting moon, moves on the surface of what is now understood as liquid.

At 2:00 the sounds of ducks and geese return. Arabic writing appears on an entirely black screen. Sounds of ducks and geese and rhythmic clicking continue. A rooster crows. Sounds, seemingly of something walking, slowly, carefully, on gravel. Slow zoom into text overlaid on the screen.

At 2:25 a second screen of Arabic writing appears. The sound changes to nighttime insects. Slow zoom into another, different text.

The white light, now understood as a reflection in liquid, reappears. Night insect sounds continue as a third screen of Arabic text overlays the visual. The reflected light dances behind and in the center of the text filling the screen.

The text disappears. More liquid is poured into the container. The reflected light is disturbed. The sound fades out as a final screen of Arabic text appears, as if being written onto the screen.

Translation of the text to English speaks to images and sounds.

It is not surprising that the ear depends upon imagination
surrounded by sounds.
A rooster crows for the time of prayer.
Frogs croaking foretell the smell of rain.
A dog passes nearby, pauses, continues on his way.
Your eyes, staring up for a long time,
suddenly look down to the mug into which you are pouring water.
You were in the mug where the water reflected the moon.

The reflected light of the moon dances in the mug at the bottom of the screen. Fade to black. End (Habibi 2018).

Significance

These works of electronic literature by Hight et al., Moulthrop, and Habibi each promote engagement through their effective use of sound, vocal, environmental, or mechanical.

For example, the sounds of 34 North 118 West, according to Hight, positioned one in both past and present at specific locations comprising the work.

The original 34 North 118 West is no longer available as an in situ experience. But, the sound files are available, and several were given to the author by Hight, along with permission to remix them so as to portray a sense of the original, larger work. For all practical purposes, these sounds–a train crossing, a street band, a street salesperson, narratives of individuals living in the area–are all that remain of the work.

In a 13:00 recombination of these sounds, the author utilized an Aristotelian, linear narrative structure of beginning, middle, and end, bookended by the sound of a passing train. Other approaches would be equally valid. The result from any approach is for the sound-based narratives of 34 North 118 West to provide effective access to this work–no longer available–a facsimile for the intent and content of the original work (Hight, Knowlton and Spellman 2018).5

Under Language is still available in its original context thanks to a website maintained by the author, Stuart Moulthrop. Like Hight, Moulthrop was generous in providing the author with sound files from this work, as well as permission to create experimental remixes.

Using sound files provided by Moulthrop, the author created a 14:20 audio narrative by arranging individual sound files following their numbering from Moulthrop’s original content database to create a serendipitous narrative. Other methodologies could easily be used. As with the original work, there are five layers (as described previously) to this re-conceptualized narrative. The reader and/or player is responsible for making sense of the artifact (Moulthrop 2020).6

Missing in this effort is the direct interaction the user / reader / player / participant has with choosing the lines of the poem. But, remaining is the direct experience with the under language, the vocalization(s) of the computer programming underlying, supporting, creating the original work.

The three works by Habibi, while each an example of video poetry, incorporate strong, and one might argue essential, sound elements. In Just Words, one sees a hand turning individual pieces of paper on which are inscribed Arabic words and / or phrases. But it is the sound of these pages turning that sparks one’s imagination, encouraging one to ask for more perspective on this literary experience. What are these pages? How do they relate to one another? Who is turning the pages? What is the reader and / or viewer to think of these words and their display?

In another scene one sees these individual pages hanging from a line, like fish or dried fruits, evidence of their collection and display outside their native element. But again, it is the sound of these pages, this time rustling in the air moving through a nearby window, that makes clear what one sees is a context, a place, a space in which these words might be experienced. Taken out of context, do these words still convey the same meaning(s) as when bound with others into the form of a book? Again, what is the reader and / or viewer to think of these words and their display?

In Matchbox, one sees nothing but blackness. Only at the end of this video poem, when a match is struck and thus illuminates the face of a young person, does one understand what is seen is the darkness of night. Only with the appearance of the text superimposed at the end of the video does one have full context for the poem. Again, it is sound, this time the striking of a match, that provides a sense of presence in a particular place and brings the visual elements together, providing an interface with which one can engage the essential nature of the poem.

At the heart of Mug, is, one might argue, an acoustic listening experience, where one does not know the source of the sound(s) heard. One might also argue that the substance of the visual experience is unknown, too. Beyond the text, it is the sound of liquid being poured into a container of more liquid that provides the crucial key for understanding this video poem and its examination–interrogation?–of what is, ultimately, a reflection. At this point, as noted in the text of the poem, one is present in the mug. That space, created by sound, defines one’s interaction with the poem.

What does all this mean? Why is it important? A first response might be to note that sound provides a fundamental sensory input and communication channel for human culture. Sound is the phoneme for speech (verbalization of abstract thought). Sound can be a central component of narrative (the recounting of a sequence of events and their meaning) and storytelling (the addition of setting, plot, characters, logical unfolding of events, a climax).

Allan Moore describes how motifs that remind of and return lyrical references provide embedded meaning in popular music (Moore 2012). This same connection is demonstrated in the examples of electronic literature and poetry discussed in this essay. Each uses sound to connect and surround readers with multiple, concurrent aspects of their storytelling. If readers are fully engaged with the imaginative situation–whether musical or literary–the result can be quite powerful, according to Tim Crook, who says sound effectively prompts life from little details seen in the listeners’ mind’s eye (Crook 1999).

Alan Hall describes how this phenomenon works. Listening carefully and purposefully to sound(s) offers, according to Hall, “a portal through which a deeper, often inarticulate, consciousness can be glimpsed… The intention is to find deeper and wider resonances within–and without–the listener” (Hall 2010, 99, 104).

Michael Bull and Les Back say sound provides a place in which embodied social and cultural traces can be carried, often without the awareness of their bearers. Therefore, it is good to actively and deeply listen to the sounds of the world in which one lives. In this way, according to Bull and Back, one opens new approaches to thinking about and appreciating the social experience, memory, time, and place–the auditory culture–of sound (Bull and Back 2003).

The works by Hight et al., Moulthrop, and Habibi considered in this essay encourage consideration of how different aesthetic conceptualizations and material practices of voice and other sound(s) inform literary expression. For example, sound and poetry might be considered an exchange between language and code, and thus at the center of our understanding of language arts. The desired outcome is to expand understanding of literature and textuality as vehicles for exchanges in and across media, languages, and cultures.

Finally, this focus foregrounds an approach to literature characterized by what Edmund Carpenter calls the verbal, musical, and poetic traces and fragments (figures) of oral culture (Carpenter 1970). The works by Hight et al., Moulthrop, and Habibi discussed in this essay show how sound provides a way of knowing and being in a literary world, real or imagined. In this regard, sound is the basis for literature (written works considered to possess lasting artistic merit) and the various practices and cultures associated with its production and consumption (reading, writing, and listening). Listening, as real and concrete participatory practice, evokes aural experiences across a wide range of theory and practice through which listeners can derive information about the world they inhabit. This suggests a way forward for electronic literature, through careful listening.

References

Patterned after the first video game, Pong, Moulthrop’s Sc4nda1 in New Media (2012) converges philosophical meditation with a retro video game. The sounds provide context(s) in this new space. The author maintains an archival webpage for a remix he created using original sound files supplied by Mouthrop (Barber 2020b).

Barber, John. 2020a. “Radio Salience.” 2020. Link.

Barber, John. 2020b. “Sc4nda1 in New Media. radioELO.” 2020. Link.

Barber, John. 2020c. “Under Language.” 2020. Link.

Bull, Michael, and Les Back, ed. 2003. The Auditory Culture Reader. Oxford : Berg.

Carpenter, Edmund. 1970. They Became What They Beheld. New York : Outerbridge & Dienstfrey.

Chion, Michel. 1994. Audio-Vision Sound on Screen. Translated and edited by Claudia Gorbman. New York : Columbia University Press.

Crook, Tim. 1999. Radio Drama Theory and Practice. Abingdon, UK : Routledge.

Furlonge, Nicole Brittingham. 2018. Race Sounds: The Art of Listening in African American Literature. Iowa City : University of Iowa Press.

Habibi, Mohamed. 2018. “habibi1366.” YouTube. 2018. Link.

Hall, Alan. 2010. “Cigarettes and Dance Steps.” In Reality Radio: Telling True Stories in Sound, ed. by John Biewen and Alexa Dilworth, 96‑107. Chapel Hill : University of North Carolina Press.

Hight, Jeremy. 2013. “Narrative Archaeology.” In NeMe. March 25. Link. Also available here.

Hight, Jeremy, Jeff Knowlton, and Naomi Spellman. 2003. “34 North 118 West.” In radioELO. Link.

Moore, Alan. 2012. Song Means: Analyzing and Interpreting Recorded Popular Song. Farnham, NY : Ashgate.

Moulthrop, Stuart. 2020. Under Language. 2020. Link.

Murch, Walter. 2005. “Womb Tone.” Transom Review 5 (1). Link.

Ong, Walter J. 1982. Orality and Literacy: The Technologizing of the Word. York : Methuen.

Smith, Bruce R. 2003. “Tuning into London c. 1600.” In The Auditory Culture Reader, ed. by Michael Bull and Les Back, 127‑35. Oxford : Berg.

Under Language shared the 3rd International Digital Literature Award Ciutat de Vinarós Prize for Digital Narrative with Isaias Herrero Florensa’s Universo Molecula. In his artist statement, Moulthrop describes the origin of the term “textual instrument.”
[Textual instruments is] a term I borrowed from John Cayley many years ago to describe things that might look like literature, but also like structures for play, though not necessarily what we would call games. In fact, this one lies pretty close to game space, having rules, a scoring system (albeit invisible), and even a simple agon in which you compete against the perversity of the puzzle-maker, and constraints of the clock (Moulthrop 2020). [back]

Moulthrop maintains an archival webpage for Under Language, where he provides information and listening opportunities. In the “About” section of this website he describes the inspiration for the term under language.
The phrase “under-language” was invented by the comics artist Alan Moore in an interview he gave in the early 1980s. He used it to describe the essence of comics art, which is neither verbal nor visual, but something that underlies and infuses both modes. The term gets at the essence behind Moore’s great genius for irony and verbal-visual puns. It also provides a convenient reminder that everything, these days, tends to mean more than it seems (Moulthrop 2020).
The author maintains an archival webpage for a remix he created using original sound files supplied by Moulthrop (Barber 2020c). [back]

Similar to Under Language, Moulthrop’s Radio Salience (2007) is an interactive image-text-sound instrument with a game-like interface that explores indeterminacy, accident, and resonance, taking as its muse the breathless voice of the airwaves and radio. The author maintains an archival webpage for a remix he created using original sound files supplied by Moulthrop (Barber 2020a). [back]

The author met Saudi poet and scholar Mohammed Habibi and was introduced to his video poetry at the Arabic Electronic Literature: New Horizons and Global Perspectives conference. Perhaps the first focusing on Arabic electronic literature, this conference was convened at Rochester Institute of Technology, in Dubai, United Arab Emirates, Feb. 25-27, 2018. Organization website / Conference program [back]

The author maintains an archival webpage providing information and listening opportunities. [back]

The author maintains an archival webpage providing information and listening opportunities. [back]

Biographies

Barber, John F.
Back / Retour

John F. Barber convenes with The Creative Media & Digital Culture Program at Washington State University Vancouver. His scholarship, teaching, and creative endeavors arise from intersections of art, humanities, and technology and manifest as literary media art where he feels that practice-based research discovers and puts into action new knowledge. His radio and sound art are broadcast and exhibited internationally. His publications appear in Digital Humanities Quarterly, Digital Studies, ebr, Hyperrhiz: New Media Cultures, Leonardo, MATLIT (Materialities of Literature), Scholarly Research and Communication, and elsewhere. Barber curates The Brautigan Library, a collection of unpublished manuscripts, and was featured on This American Life. See his website, for more information and access to his work.

John F. Barber est professeur dans le programme des Médias créatifs et de la Culture numérique à l’Université de l’État de Washington à Vancouver. Ses travaux d’érudition, d’enseignement et de création sont issus des intersections de l’art, les humanités et la technologie, et se manifestent sous la forme d’un art littéraire médiatique où il estime que la recherche basée sur la pratique découvre et met en pratique de nouvelles connaissances. Ses œuvres radiophoniques et sonores sont diffusées et exposées dans le monde entier. Ses publications paraissent dans Digital Humanities Quarterly, Digital Studies, ebr, Hyperrhiz: New Media Cultures, Leonardo, MATLIT (Materialities of Literature), Scholarly Research and Communication et ailleurs. Barber est le conservateur du Brautigan Library, une collection de manuscrits non publiés, et a été présenté sur This American Life. Voir son site web, pour plus d’informations et pour accéder à son travail.