Studies in Sound and Vibration

David Bobier, Director/Curator, VibraFusionLab

VibraFusionLab: an innovative centre for arts-based vibrotactile research and creative practice.

VibraFusionLab is a media arts centre based in London, Ontario that provides opportunities for the creation and presentation of multi-sensory artistic practice, partnering with other arts and technology-related organizations in order to achieve this. As an interactive creative media studio VibraFusionLab promotes and encourages the creation of new accessible art forms, including the vibrotactile, and focuses on inclusive technologies that have the potential of expanding art-making practices in the deaf, blind, disabled and hearing communities, and for creating more inclusive experiences for deaf, blind, disabled and hearing audiences.

Providing the opportunity to create compositions and expand artistic practices that are designed to be experienced as tactile stimuli (rather than acoustic sound), this art form is accessible to a greater diversity of artists and audiences alike. VibraFusionLab represents considerable potential in generating artistic development and innovative research in vibration and the tactile as an artistic modality. It furthers the desire of combining alternative language, communication and emotional strategies and experiences into artistic practice. In addition, the use of this type of interactive multimedia, multisensory approach provides those with different abilities to enjoy equal participation. It provides the potential for making various forms of artistic expression more accessible. It offers new and unique approaches for those with diverse abilities to produce art using an innovative tactile mode and opens alternative opportunities through artistic expression.

Over the last decade, consumers have witnessed a dramatic increase in the use of low-fidelity, discrete vibrotactile feedback to enhance or replace audio stimuli in entertainment systems. However, use of high-resolution, continuous vibrotactile displays remains quite uncommon. The Emoti-Chair is a high-resolution, continuous vibrotactile display system that may be driven by any type of audio signal and is purported to convey the emotional properties of sound through organized vibrotactile stimulation. The Emoti-Chair project was developed by the Inclusive Media and Design Centre, IMDC, (formerly Centre for Learning Technologies) at Ryerson University in Toronto, Ontario and expanded by Tactile Audio Displays Inc., (TAD) Toronto. The research concept was to develop a theatre chair for the purposes of presenting a tactile vibrational representation of the movie audio files.

The technology separates audio signals into discrete vibrotactile output channels (voice coils) that can be presented on the body to create a high-resolution audiotactile experience through direct connection with musical instruments, live sound and digital sound files.

A partnership between IMDC, TAD and VibraFusionLab resulted in this one-of-a-kind innovative studio space. Other collaborative projects include Tangled Arts + Disability (Toronto); Tangled London (London, ON); Project Re•Vision, REDLAB  (University of Guelph); Deep Listening Institute (Rensselaer Polytechnic Institute in Troy, NY); Department of Disability Studies at Kings College (University of Western University, London, ON); InterAccess and the Music Gallery (Toronto); Inter Arts matrix (Waterloo, ON); Centre3 (Hamilton); Mobile Media Lab, (Montreal); Together 2012, (London, UK); Art Gallery of Guelph; Adi Hollander (Amsterdam)

VibraFusionLab has hosted and worked with artists and musicians from Canada, United States, Holland and United Kingdom including Deirdre Logue, Graham Smith, Gordon Monahan, Marla Hlady, Ellen Moffat, Isabella Stefanescu, Cailen Dye, Rick Sacks, Dennis Siren, Jason Doell, Emilie Lebel, Gayle Young, Allison Cameron, Brent Lee, Jesse Stewart, Moe Clark, as well as, Deaf and/or disabled artists Lindsay Fisher, Lynx Sainte-Marie, Chisato Minamimura, Alison O’Daniel, Catherine McKinnon, Tiphaine Girault, Rachel Gadsden, Chantal Dequire, Tracey Salaway, Alexander Genievsky.

essays

Expanding The Artistic Experience

David Bobier

VibraFusionLab: Bridging Practices in Accessibility, Art and Communication

Curatorial Essay by David Bobier

VibraFusionLab (VFL) and I are pretty much synonymous and for the most part inseparable. That’s not to say I don’t do work on my own, I do, but much of my own creative practice is influenced by the goals of VibraFusionLab and much of the activities and projects out of VibraFusionLab mirror my own personal reflection and focus on accessibility in and for the arts.

I launched VibraFusionLab programming in early 2013 with the support of a 3-year Social Sciences and Humanities Research Council Creation grant under the coordination and in partnership with Dr. Deb Fels, Director of Inclusive Media and Design Centre (IMDC), Ryerson University. This followed a couple of years working on contract with IMDC, offering vibrotactile workshops in Toronto and London and drawing in a diversity of artists – deaf, hearing, disabled and non-disabled. My conviction, and with the enthusiastic support of Dr. Fels, was to transport some of their inclusive research technologies out of the academic setting and into a public creative arts setting as the first step in providing an accessible art making facility.

At the same time VibraFusionLab gained a critical partnership with Dr. Maria Karam, Director of Tactile Audio Displays Inc. (TAD), Toronto and a primary researcher on the original Emoti-Chair project at Ryerson. The Emoti-Chair project was developed by IMDC, (formerly Centre for Learning Technologies) at Ryerson University and expanded on by TAD. Through these continuous partnerships new audio/tactile systems have been created suited to the personalized needs of the artist. Disability is not generic and neither are the tools for the creation of art from the Deaf or disabled perspective.

Over the last decade, consumers have witnessed a dramatic increase in the use of low-fidelity, discrete vibrotactile feedback to enhance or replace audio stimuli in entertainment systems. However, use of high-resolution, continuous vibrotactile displays remains much less common. As such the Emoti-Chair vibrotactile display system may be driven by any type of audio signal and is purported to convey the emotional properties of sound through organized vibrotactile stimulation. The research concept was to develop a theatre chair for the purposes of presenting a tactile vibrational representation of the movie audio files. The technology separates audio signals into discrete vibrotactile output channels (voice coils) that can be experienced on the body (through chairs, pillows, wearables, hand-held shapes, floor platforms, etc.) to create a high-resolution audio-tactile experience through direct connection with live sound, digital or analog sound, visual imagery and with movement recognition systems.

Established as a media arts centre based in London, Ontario VibraFusionLab provides opportunities for the creation and presentation of multi-sensory artistic practice. As an interactive creative media studio promotes and encourages the creation of new accessible art forms, including the vibrotactile, and focuses on inclusive technologies that have the potential of expanding art-making practices in the deaf, blind, disabled, hearing and non-disabled communities and for creating more inclusive experiences for audiences from those communities. In focussing on a holistic reliance on the senses of sight, sound, smell and touch as acceptable modes of knowing, understanding and communicating VFL encourages the creation of new inclusive art forms and supports efforts to develop art practices of greater accessibility and audience participation. The Lab features several inclusively designed systems aimed at accommodating and inspiring artists, participants and audiences of all abilities.

By engaging with broader and more diverse communities VFL has maintained a path of exploration in challenging social perceptions of body abilities and limitations, deconstructing barriers associated with deafness and disability and of providing vital opportunities for artists of all abilities to work and share the creative process on an equal basis. By making resources available to artists of all abilities it’s mandate is to facilitate integrated programming by bringing together artists fully representative of all communities, cultures and abilities and to provide inclusive access to the experience, understanding, insight and enjoyment of art in all its disciplines. Through this exhibition VibraFusionLab continues an emphasis on the bridging of science, technology and the arts and the encouragment of multi-disciplinary, multi-sensory, multi-modality and accessible research and practice

The works selected for this exhibition specialize in the exploration of “vibrotactility” in technology, investigating it as a creative medium, with a capacity to combine visual, audio and tactile elements into a highly emotional and sensorial art practice. Viewers can expect wearable devices, and new approaches to art making that champion the senses beyond vision and hearing and to build new methods of communication and language. In considering accessible technologies as creative tools and mediums through VibraFusionLab the works of these seven artists explore the bridging of methods of communication and language and of interpreting or transforming one modality to another. They open the opportunity to engage in a multi-sensory approach and experimentation that allows for the transitioning and re-interpreting of content and experience from one medium to another.

This exhibition, VibraFusionLab: Bridging Practices in Accessibility, Art and Communication, brings together seven artists, all with an affiliation with VibraFusionLab over the past three years – four self-identified deaf and/or disabled and three non-disabled. It has always been the intent of VFL to be inclusive in it’s programming believing that greater accessibility can be beneficial to all and bridging ability differences within the shared creative space leads to greater insight, understanding, tolerance and expansion of the notion of art and art making as an inclusive practice and experience. The seven works in this exhibition highlight the tremendous artistic potential from the nature of these kinds of collaborative opportunities. All seven artists came with a total commitment and desire to explore greater accessibility and experience in their work and a willingness of giving over to expanded experiences of interactivity, play and the multi-modal experiences.

Marla Hlady has been the ‘investigator’ of the nature of sound for much of her career. From the reinvention of sound through familiar objects, to dramatic indoor and outdoor audio installations to the interior of walls and floors she emphasizes the wonders of experience, inquisitiveness and creation. She does this with a finely tuned sense of detail and craft, an intrigue of the familiar, a willingness to share in something both tangible and mysterious, a penchant for order and often with a slyness of humour that only enriches the viewers engagement. In Soundball(Dancehauling) there is all of this. The viewer/participant is invited to step onto a wooden box and step up onto a simple wooden podium-like platform thus elevating them to a position of superiority, awkwardness and uncertainty. Taking on the role of both conductor and musician, the soundballs are lifted, rotated with arm and hand movements and sound streams from within them and from below. The balls become sources of light and sound, the floor of the platform vibrates in time and the ‘dancehalling’ begins. With each rotation of participants, Hlady hands over the stage for the next performance and the installation responds anew to the creative whims of the individual(s) on top.

Gordon Monahan’s practice bridges disciplines of audio and multi-media installation, concert performance, theatre and sound art. He draws from elements of nature, science, technology and invention to create new experiences of phenomena and spectacle. His work is a masterful mix of hi and low tech, simple and sophisticated and at times evoking the world of the magician, the mad scientist, the mastermind with something up his sleeve because we are mesmerized, amused, bewildered, always entertained and at times, even soothed. In Cymbalism, we are clearly the audience to Monahan’s sound and ‘cymbalic’ theatre. The intentional placement of the chair positions the viewer centrally and when seated, the viewer’s eyes are at eye level with the four formally suspended cymbals. With the cymbals at sight level and the periodic sounds resonating from the cymbals that are reactivated by transmitted recordings of their own sounds, the stage is set. In this installation Monahan requires that the viewer/participant is patient and comfortably seated, for there are periods of silence and solemnity. Through our patience we are rewarded with a controlled symphony of sound and surprisingly immersed in vibration through the system attached to the back of the chair. By instantaneously feeling the vibration of the sounds produced through the vibrotactile system, the magic and wonder of the immersive begins.

In Ellen Moffat’s work, Small Sonorities: a pen, a sponge, we see a continuation of her ongoing interest and fascination in the underlying fundamentals of sound. In her installation a gentle, carefully choreographed sound composition emanates from two beautifully constructed wooden ‘tables’ that are built low to the floor. Vibration from the audio becomes visible through the synchronized movement of a sponge on a speaker in one component and in the other, we observe the slow rotational movement, as if charting time, of a simple calligraphic pen in a clear glass circular-shaped vase seated on a speaker. It is a deceptively simple presentation of sound made visible. Moffat doesn’t stop and that. The intentionally low presentation challenges the viewer out of a standardized human zone or position to the more familiar space of a child, wheelchair user or someone with restricted height. Additionally, the elegant wooden surface invites us to reach out and touch with the full gratification of ‘feeling’ the source of sound and motion. The resolution is a wonderfully satisfying experience of understanding sound through the properties of touch and movement. The use of natural materials and familiar objects and the aesthetics of beauty and elegance of the installation draws one from that of a distant viewer to an extension of the work. Through touch we become the physical receptor of the work and complete the skilled intentions of an artist that celebrates the viewers response and marvels in the possibilities of the shared immersive space.

Los Angeles-based Alison O’Daniel works in a world of sound differences and as a hard of hearing artist O’Daniel experiences and responds to the aural world differently. In her words “I grew up in a hearing world. Sometimes I feel like my hearing is so fine-tuned that I hear details that others don’t notice, like my imagination is opening up to fill in gaps where I’m at a loss. My experience ricochets between enjoying the solitude of muffled hearing-aid-less mornings to deep frustration at people’s unwillingness to be sensitive to missing an entire film or conversation or nuances of daily experiences and feeling ignorant and therefore isolated to a perpetual and profound state of observation and wonder. All of these experiences have made me sensitive to sound, to the loss of it, the abundance of it, how it impacts social situations, and the amazing possibilities in the aural world.” O’Daniel explores the uniqueness of her experiences through 3 video shorts selected for this exhibition as part of a large ongoing project called The Tuba Thieves. These shorts present jarring and diverse emotional landscapes – from a deaf club gathering at a loud and raucous rock bar, to the almost soundless movement of colourful kites against a perfectly serene blue sky to the interior of a truck trailer filled with plants that quiver and shake mysteriously. Her unique cinematic eye and intense attention to sound is enhanced through the addition of two vibrotactile cushions or pillows and a simple black, vibrationally-charged bench from which to view, hear and feel the richness of her work.

In the words of Lindsay Fisher “My work often looks at the construction of identity, the body, and notions of difference and ….at my facial difference as an element for study and how it can be used to re-invent my surrounding environment”. In her video In Fragments, Fisher critiques the notion of perfection as seen through the ‘crip’ lens. Seeing only their arms and hands we are taken through an ‘instructional guide’ as to the precise way of doing perfectly painted nails. However, the illusion of popular perfection is dismantled as we witness cripped hands struggling with the physical preciseness of the nail polish application and the altered voices that reflect the nature of their individual disabilities. The video is at once enchanting, humourous, diabolical and deeply reflective from the disability perspective. The charm on the surface is quickly taken over by the absurdity of societal misconceptions and the predominant notion of beauty and the body. In this presentation, the audio is channeled through transducers on the palm of a wooden replica of a hand and two small ‘vorbs’. The intention is by picking up the hand (referencing the hands in the video) or the orbs we discover the sound of the voices emanating from these additional components duplicated by the feel of the voices on our hands plus intentionally drawings reference to other forms of hand communication such as braille and sign language.

As an artist and activist Lynx Sainte-Marie’s creative and activist practice takes on many forms – poetry, performance, storytelling, song, multimedia installation. I feel confident in saying their art and activism is inseparable, bound with a passion to address and challenge issues of dominance and oppression while honouring and celebrating the marginalized and non-traditional. Influenced by their Jamaican diaspora background, Sainte-Marie immerses themselves in the gothic, often surreal forms of mythology specific to Jamaica and West Africa. Their experience of this history informs their artistic language and production. In their created environments, including Children of O, Sainte-Marie imagines a world where disability and spirituality are one in the same and the disabled, in their words, are the ‘sacred ones most connected to the spirits and the ancestors’, contradicting dominant western doctrine, while mirroring the traditions of many other indigenous cultures. In this installation Sainte-Marie invites us into their private and honouring space – an immersive multi-media time capsule of sight, sound and touch complete with a wheelchair accessible ramp. The floor within the ‘inter-realm’ responds to a separate audio of undersea sounds of the communication of whales. It is a sacred and ritualistic space with an altar laden with symbolic objects and metaphors for the future, where the viewer is placed face-to-face with a video of Sainte-Marie themselves. In a selection from the video they proclaims softly,

“…The sacred ones, their bodies were seen as devoid of thought and purpose
But we now know this is divine connection
any way that our bodies respond to the realms
visions, voices, pain
cosmic intervention between the ancestors, the spirits, and I
The ways that we receive greetings, messages
from beyond our skin
I sit by the altar, the offerings are hot and clean
I wear my hair long and white
I am filled, surrounded by my ancestors
Surrounded by the sacred ones throughout time
And I let the spirits come for me
My skin rocks and dances
And I am filled with love
This movement
My body belongs here and there
With you and with me
no mistake
no regret
No fear
no need for their sky technology
For our beloveds surrounds us
living, breathing, knowing,
in the sky

And I am free”

Originally commissioned for the ‘Pop Up Shop’, exhibition organized by Centre(3), Hamilton as part of the 2015 Viva! Pan Am Hamilton celebrations, Touch~Sound~Pulse by David Bobier examines the spectacle of sport and sport achievement through the lens of the Deaf and Disabled. The work draws attention to these communities and the defining lack of attention, support, access and participation in the heavily commercialized and worshiped world of sports. In using sports as a metaphor, he aims to equalize the appreciation, understanding and enjoyment of Deaf and Disabled Arts by drawing attention to alternative and inclusive sensory experiences in art making and art enjoyment. For this installation Touch~Sound~Pulse Bobier has developed a faux-projector instrument by incorporating a programmable music box. His ‘instruments’ work to transform language scripts (in this case braille) into audio, visual and vibrational manifestations through the use of strips of ‘punched’ music scores, LED lights and small magnifying lens. The vibrotactile system is installed directly into the back and seat of an antique wooden school desk, with an addition vibrotactile pillow available to hold for those with physical limitations. By using codified communication systems as the framework for engaging his audience Bobier questions and aims to demystify the grand spectacle of mainstream sports, dismantle the computerized, commodified and consumer-obsessed mindset inherent in creating spectacle and reimagine the notion of social inclusion and diversity through art. By re-contextualizing language scripts and codification systems, he envisions personalized, interactive and multi-sensory opportunities for all.

In reflecting on the social model of disability which identifies systemic barriers, negative attitudes and exclusion by society (purposely or inadvertently) that mean society is the main contributory factor in disabling people. It states that disability is caused by the way society is organised, rather than by a person’s impairment or difference. It looks at ways of removing barriers that restrict life choices for disabled people.

The coupling of Deaf and Disability Art with new creative technologies allows for greater embodied experiences in academic, community and studio-based activities and new opportunities of and for their intersection.
The hybrid work of the artists presented in this exhibition and the work of many other practitioners, both artists and researchers, being done across Canada and globally represents a significant force and movement away from the centre towards inclusive and experiential processes and results that can only be confirmed as “the last remaining avant garde movement” .

Let’s collectively celebrate who we are as individuals, what we have and how we can best contribute to society for the benefit of everyone!
Art for all!!!

Curator, David Bobier

1 http://www.tadsinc.com/
2 http://imdc.ca/ourprojects/emotichair
3 The use of the term ‘vorb’ is attributed to the artist Leslie Putnam in defining her vibrotactile system
developed at VibraFusionLab.
4 Attributed to Turner Prize nominee Yinka Shonibare during a panel discussion at The London Disability Arts Forum
(LDAF), in association with Tate Modern and the Office for Disability Issues, December, 2007

>>read more

Vibrotactile technology expands the dimension of touch.

Dr. Evan Hibbard

All my life, I’ve lived in a world contained by four senses as a Deaf individual. Sound was abstract, strange and existed within a dimension that I could not touch, see, smell, taste, or hear. I could see people responding and acting on phenomena, sound that I could not decipher. My earliest memory of being aware of this phenomenon was as a small child of four years old, sitting around restaurant table, watching my family open and close their mouths at each other. I mimicked them as I puzzled at how they did not use their eyes except for brief glances at faces. I struggled with finding right questions to ask to understand this social behavior. I was on the hunt to explore this phenomenon. I collected clues as a researcher collects data.

Few years later, my speech therapist pushed a button on a old tape player a recording of a firetruck siren while showing my classmates a photo of firetruck. My classmates smiled and said, “high sound” while I felt like a failure because I could not detect the sound. I strained my ears as fiddled with my awkward chunky hearing aid strapped around my torso. I swallowed my tears and used my eyes to follow the lively discussion about firetruck siren. When the lesson was over and their attention was diverted elsewhere, I poked at the tape player. I was puzzled how a siren could come from a tape player.

The therapist placed my hand on the tape player speakers as she played again the siren. I felt the speakers dance against my skin, my mouth dropped open. More questions popped into my brain, such as “why the speakers move like that?”, “does that mean sound makes things vibrate?”, and “can I feel all sound?”.  I was struggling with my command of spoken English, articulating more than two words at a time was beyond my ability at the time. I looked at her, saw her lips say, “you feel that?”. I nodded yes.

In a very short time after this experience, I was taken to a music performance of Buffalo Orchestra where I was given a balloon to hold while I sat in first row watching the musicians and feeling the skin of balloon dance against my fingertips. There were hundreds of music instruments on stage but I could not detect the difference between individual instruments from using the balloon alone. However, holding the balloon made the performance more real and created a doorway into the dimension of sound that I could step into.

My inquiring mind led me a career and doctorate in science research in which I found myself in collaborative research lab in which I was rubbing elbows with musicians, engineers, computer programmers, social scientists, and graphic designers. While I focused on video annotation software, SignLink Studio, which created tools for linking together web signed content, I touched on on-going research projects such as Emoti-Chair. Maria Karam who was working on her PhD asked me if I could try their prototype Emoti-chair. At the time, it was two bulky audio speakers strapped to my back with yellow straps, the kind used to strap camping gear to roof of a car. I was hooked up to wires as music was played through the speakers. I smiled and from the vibrations dancing on my back, I realized sound was divided between the speakers. I was enthralled, another way to explore sound through touch, instead of my hands, I could use skin on my back! I quickly asked them to keep me involved on their progress.

The speakers progressed from two to four placed between slats of a wooden Muskoka lawn chair. I leaned back and I grinned as the vibrations became more distinct. However, the whole chair vibrated so it created a sense of blur against my back. Eventually they created two columns of micro-speakers embedded in a folding chair, about 12 to 14. They again hooked me up to wires to record my skin and heartbeat response and had me watch except of scary movie, Stephen King’s Shinning and Sixth Sense. I had watched those previously but the experience of getting tactile input changed the movie watching experience. I felt my heart race and my skin pin-prick with sweat as the vibrations increased in tempo and quickly changed in its placement along my back.

I attended a concert in which “script” was written to be played through the Emoti-Chairs from the music instruments. I quickly decoded the vibrations dancing along my back to differentiate which individual instruments contributed to the vibration. I could close my eyes and feel the emotion conveyed by the vibrations. I felt like I was no longer standing at the doorway as I did with the balloon, but stepping into this new dimension and exploring it. As I explored, a new vibration pattern emerged that I could not match the instruments to. I scanned the musicians and I realized one of them was singing joyfully. I sat up in surprise, that I could not only differentiate music instruments but I could feel his voice and how it fit in with the patterns dancing on my back. I wanted to buy it and install it at my home. However, that was not possible for me due to financial constraints but this is on my wish list to add to my office in the future.

As I continued in my PhD research on inclusive technology and online media, I realized what it means to be truly accessible. True access is not just about coming in from the margins or access to content that hearing and able-bodied people enjoy, it is about creating inclusion in which I could explore emotion and meaning of the media, if scary or joyful.  For example, with Emoti-chair I could sit next to a hearing person and experience scary or joy at the same time through sense of touch to explore sound instead of trying to use something that didn’t work for me, sense of hearing. I realized this understanding could be applied to my research, to explore what it means to be truly visual from a Deaf person, to create inclusive online navigation. Instead of using text that referred to representation of spoken medium, I created graphics that had meaning for Deaf people that could cross over for hearing people, such as dyslexia.

The phrase that hearing people typically use to describe Deaf people’s experiences, “silent world”, “living in silence” etc., is not accurate. I have been Deaf my whole life, I never used the sense of hearing so I never lost it as I never had it to begin with. My four senses expand and I find meaning in those senses that sometimes, hearing people don’t experience. It is my thought with the vibrotactile technology, it would show hearing people what it means to use sense of touch the way we Deaf people do. For example, Deaf people use wooden surfaces to feel vibrations through their feet as they dance to music or be aware of cues such as is the washing machine on, or how much liquid is left in the container. The technology could do more than just showing hearing people how to explore their sense of touch, it could create inclusion by creating a new dimension in which both hearing and Deaf people could explore for expression of joy or scary.

I envision a future in which, someday, we can combine the vibrotactile technology with smart garments such as dance clothing (leotards) can be worn to dance to vibro-tactile input and people in the audience can sit in Emoti-chairs to feel the music.  As we have theme restaurants such as dining in the dark and being served by Blind staff, we could add this to restaurants, diners could sit in Emoti-chairs as they eat. This could create more innovation and variety in our environments that benefit everyone, especially for Deaf/disabled individuals. Deaf community would have more options for their entertainment and enjoyment of arts besides renting wooden floors, sub-woofers and DJ willing to crank the music up full blast. In addition, not just having more options, but also can feel included and equal to hearing people enjoying those things instead of being pushed to the margins as I was in that class when they were listening to firetruck sirens.

>>read more