Composing Audiovisually | Interview with Louise Harris

MF: Congratulations on your new book, Composing Audiovisually Perspectives on audiovisual practices and relationships. Could you give the ‘nutshell’ summary of what it’s about, just as a fast introduction for readers?

LH: The book is about what it means to compose in the audiovisual realm — so how do we approach composing when thinking about both sound and image simultaneously? How do we develop a language around that process? How do we teach it? And, crucially, how do we understand and discuss our responses to audiovisual works as audience members — what are we actually doing when we encounter an audiovisual work?

MF: What do you take to be the main venues for presentation and reception for audiovisual compositions of the kind that you discuss in the book?

LH: That’s tricky, and very much depends on the work being discussed! Many are screen-based, so could pretty much be (and are!) anywhere you can access them on screens. Others require concert performance — such as Nicole Lizee’s Hitchcock Etudes — or large, multichannel playback formats — such as Grayson Cooke’s Open Air — or site-specific realisation, such as Michele Spanghero’s Pebbles or Mariska de Groot’s Cinechine. Each of these works I argue are audiovisual compositions — they each represent a compositional process that is concerned with both what we see and what we hear as being of equal perceptual importance — but the format in which they exist is very diverse and often distinct to the individual work. I guess that’s one of the things I’m arguing in the book — that composing audiovisually is often more about process and approach than playback format.

MF: This next question might seem a bit pedantic, but I’ll ask it anyway because it’s intent is mainly practical. The academic archive is a vast data-place. What would be the key search terms that someone might use to find the most relevant literature connected to what you describe as audiovisual composition? Even for non-academics, everyone can access Google Scholar or Researchgate, and knowing exactly what to look for amongst the millions of sources can be a challenge, for anyone who may be wanting to expand their knowledge base.

LH: I often encourage my students to begin with visual music, as this is a good place to start in thinking about sound/image relationships on an even playing field. It’s also quite an established term/field now with a developing body of relevant literature, but still only really defines a collection of quite specific approaches. Much of the audiovisual work I discuss in the book comes from intermedia or fluxus traditions, so that’s another useful beginning, or from expanded cinema. I think the term ‘audiovisual composition’ is also beginning to have more currency, and there are relevant resources to be found under that heading now.

MF: What are the main tools & technologies that you see being used in university courses that teach audiovisual compositions skills and knowledge? And are these the same or different for professional practitioners?

LH: This often depends on the academic leading the programme 🙂 but often max, processing, touchdesigner (increasingly), adobe creative cloud, reaper, logic, ableton… I think there is some overlap with professional practice but equally at my University for example we try to use open source tools or tools that are more affordable to students as opposed to going for the industry standard stuff. Often the workflow and basic skills are the same/similar, so you’re still teaching applicable skills, just not necessarily in the same programmes.

MF: I have to admit to often being underwhelmed by the visual design of a lot of live video processing that might often accompany laptop orchestras, MIDI controllerist performances etc. The reason for this underwhelming experience is that I feel that there is often too little sense of visual culture or education behind these visuals which often look the same, because you get a lot of ‘variation of a shape’ effects where it’s just a simple geometric shape or squiggle randomly playing out parametric value shifts. There seems to be a lack of knowledge of visual art in general that would be useful to making these visuals more engaging, since the primary expertise is usually on the audio programming side. What kind of curricula do envision would make both the audio and visual components of some of these popular presentation formats a more complete or holistic experience? Or, are we always going to face the resistances of young students who think the old experimental stuff is irrelevant, because it’s old, and so they don’t need no art history?

LH: Here I’m reminded of Jeff Goldblum in Jurassic Park — “Yeah, but your scientists were so preoccupied with whether or not they could, they didn’t stop to think if they should”. I know exactly the kind of underwhelming performances you’re discussing, and rather than a lack of knowledge, familiarity, experience with visual art and vocabulary etc., more often than not I think these result from just thinking ‘oh, let’s add some visuals’ rather than thinking about what those visuals are there for — why they’re there, what they are doing in the work, and how they relate to the sound materials, particularly if the works began as audio-only. This is something I spend a lot of time talking to my students about — how do the shape, form, structure, motion, colour, timbre of the sound and image relate to one another? What is the actual relationship you’re articulating? There are a lot of visualisation systems that can turn audio into dancing particle systems or geometric forms, but unless you’ve thought through why a sound looks a certain way, and vice versa, you’re lost. That’s the foundation of what I describe in the book as thinking audiovisually — seeing the two as part of an interconnected creative artefact, not as separate perceptual entities.

MF: There is a lot of pressure on higher education units these days to frame their courses and programs in terms of ‘job relevant’ skills, e.g. in this case, maybe something like ‘creative technologies’ and so on. Have you found it difficult to teach audiovisual composition in an academic context, or is the climate more welcoming, or do you have to sometimes pitch this material as ‘motion graphics’ to administrators who may understand skills in a more simplified manner?

LH: Actually, it’s been ok — the main thing I run at UofG is our MSc in Sound Design and Audiovisual Practices, so the audiovisual component is right there in the name. I think we try to be really clear about what we do — and indeed what we don’t do — and what’s distinctive about the programme is that it teaches job relevant skills in a way that encourages creative exploration and experimentation. In the book I talk a little about the expectations students often have coming into the audiovisual composition module, but we specifically address these and spend some time discussing what audiovisual composition actually means, and how it relates to their broader practice. So I’m pretty lucky in that regard.

MF: What are your thoughts about sound and image relationships that may be informed by non-Western sources or aesthetics? I think in this globalizing world of ours, it’s essential to take some steps outside of the Western canon or tradition, but when you do, you are probably going to be a total novice or amateur about it. For example, I’ve assigned a reading to sound design students, “Aesthetic Suggestiveness in Chinese Thought: A Symphony of Metaphysics and Aesthetics”, as an instructor with pretty much zero academic training in Chinese art or philosophy. Yet I feel that one has to ‘take a stab’ at some kind of cross-cultural engagement, because the alternative is non-engagement, which seems irresponsible when the student body is so diverse today (e.g. my courses in Vancouver average 2/3rds East Asian students).

LH: First — that looks like a really fascinating article, so thanks for pointing me to it. I think you’re absolutely right, and I struggle with the best ways to go about this to be honest — in the book, I’ve tried to engage with artists whose heritage isn’t purely within the western tradition, but this is tricky (case in point — the artists were chosen from questionnaire responses, the questionnaire was initially disseminated to my immediate professional network and then sent on further but I suspect the majority of respondents were western). I guess all we can do is try — and acknowledge our naivety and lack of broader context. Particularly with my international students, I always encourage them to bring their own culture and heritage into what they do, and to use that as contextual touchstones when discussing their work — it’s a good way for me to expand my knowledge base, apart from anything else.

MF: Do you think research in this area is hampered by the text-bound nature of books and journals? I can’t say that I see a lot of interest in research publications that are truly media rich, since academic requirements typically send one to journals which usually publish PDFs behind paywalls. Even open-source journals format articles as PDFs or as HTML with minimal media content. There may be a link or few in an article that one can click on in the Notes section, but I’ve always found it a hindrance that one cannot write in the midst of rich media content, e.g. embedded Vimeo videos or Soundcloud links, or even color images, etc. (obviously this is a lot of what I do with this blog!). Should there be a ‘media rich’ academic movement that aims to use the full affordances of the web for a less logocentric approach to media research-creation?

LH: in short, yes! I struggled with this withthe book, but as you say this is the system we work within. Research catalogue is doing some interesting things here, but it’s tricky. I’m very open to suggestions on this, and it’s something I’ve discussed with students (particularly doctoral students pursuing practice based work who struggle to fit what they do into traditional print formats). I think it will come — and steps are being taken in that direction.

MF: Today, with the whole NFT craze and art streaming platforms like Loupe Art, there’s been a kind of revitalization of motion-based art, what some curators call ‘moving paintings’ that loop a moving visual of some kind over short durations. Crypto-investors and speculators have made digital art a hot commodity. Have you seen students interested in these developments at all? Have you thought about short form loop-based audiovisual compositions that might work in these new digital art online presentation contexts?

LH: Our students are definitely interested, yes, though more from a conceptual perspective than actually engaging with those forms as an art process. It feels a bit tangential to what we do, to be honest, so I’ve not given it too much thought (though will no doubt need to in the coming years!). I run a little research lab at the University called the Digital Departures Lab with Tim Barker (film scholar) and Sarah Cook (curator) and it’s something that’s starting to emerge as a research theme for us — though, to be honest it also feels like everything in that field is moving too quickly for academia to keep up with.

MF: Could you provide a succinct summary of ‘transperceptual attention’ for readers? That’s a core concept in your book.

LH:

The way I break down the term in the book, and my reasons for doing so, are as follows:

’Trans’ — as a prefix, meaning across, beyond or through

‘Perceptual’ — meaning the ability to perceive, become aware of or interpret through the senses

‘Attention — meaning the act or power of carefully thinking about, listening to, or watching someone or something: notice, interest, or awareness

Encountering audiovisual works is not the same perceptual experience as listening to a sound work, or viewing a visual work, but is instead a transperceptual experience, rooted in both of its individual sensory modes yet also moving across, beyond and/or through the spaces between those modes and the effects those modes have on one another. By using the term attention to describe this form of engagement, I am deliberately trying to side-step terms such as ‘watching’ or ’listening’, so loaded with preconceptions as sensory experiences, and trying to find a term that can speak to active engagement in more than one perceptual mode.

But you asked for a succinct version! So I guess, transperceptual attention is the act of attending to an experience that moves across and between perceptual modes (in this case encountering an audiovisual work) but also attending to the attending to that experience, making it an apperceptive act — considering not only the experience itself but the things we bring into that experience that inflect on how we encounter it.

MF: A lot of your discussion on composition focuses on what are often referred to as ‘formal’ elements (e.g. line, shape, color, etc.) in visual art. These elements, at the level of whole compositions, are also called ‘non-objective’ relative to its opposite, ‘representation,’ and the middle category ‘abstract’ (recognizable subject matter but highly formalized). Could you summarize your ideas about representational (or mimetic, or even semiotic) sound and visuals in relation to audiovisual composition?

LH: I think for me, in my own practice at least, this has shifted over time. When I began to make audiovisual works I deliberately avoided anything overtly representational — I remember making a 3D version of a piece built around simple circles, and once it was 3D I started to see them as bubbles, or marbles, or even berries, and it completely changed how I understood the audiovisual relationship. Over time, though, I think I’ve moved away from that a little and been less preoccupied with how representational/abstract/mimetic aspects of either the audio or visual materials are and more concerned with the ways in which they construct meaning in their combination — which I think is what transperceptual attention is about, as it’s concerned with individual meaning making, not through materials but through interpretation.

MF: Do you have any online audiovisual works that I can embed below, to give readers more direct access to your works?

LH:

https://vimeo.com/user4806953

Comments are closed.
Stay up to date

Subscribe to our newsletter to stay up to date with all the latest goings on at Optophonia.