Arduino-based Video Synth: An Open Source Interface

FALL 2011: V.07 N.02: CAA Conference Edition 2011

Sabine Gruffat
Assistant Professor of Digital Media, Department of Communication Arts
University of Wisconsin-Madison

Wayback Machine, 2010, Sabine Gruffat, installation with electronics, machine parts, CRT monitors, and two speakers, image: Sabine Gruffat.

Wayback Machine, 2010, Sabine Gruffat, installation with electronics, machine parts, CRT monitors, and two speakers, image: Sabine Gruffat.

The Arduino-based Video Synth [1] is influenced by early video artists like Nam Jun Paik and early computer animators like James and John Whitney.  They produced groundbreaking moving images by building their own machines or finding new ways to use old machines.  Pioneering video makers and electronic artists were improvisers, craftsmen, programmers and hackers.  For these early electronic artists, the creation of new forms of imagery necessitates that the artist engage with a variety of technologies and build his/ her own tools, often patching together and adapting electronic odds and ends to create monstrous analog and digital systems.  My interest in these ideas of the interface has resulted in several interactive installations I collectively describe as video synthesizers.  These are machines I built from scratch utilizing video and arcade game controls to synthesize analog video signals, and combine parts that are saved from the landfill or discovered in my attic.

The Interface
Whether by typing on a numbered keypad, clicking an optical mouse or wearing earphones, the way we interact with technology is physically understood and absorbed by the body.  In my art, I am interested in how the body and the mind operate in communication with machines and systems.  I believe interfaces are either a site of contention or agreement between the human and the machine.

Like many artists who work with found materials, my video synthesizers are built from objects and technologies that are culturally familiar, such as a game controller or a golf club.  Repurposing familiar objects allows me to comment on the value of these objects and to offer new possibilities for their use (value).

I have built three types of video synthesizers, but here I will speak about The Wayback Machine, a multi-screen video synthesizer I built using handmade electronics, recycled machine parts, and computer monitors.  The Wayback Machine represents the final result of several months of research involving analog and digital methods of signal synthesis.  I worked with Adrian Freed [2] of the Center for New Media and Audio Technologies at University of California, Berkeley, who programmed the first generation of digital (music) keyboards.  By combining Freed’s research in audio synthesis and my research on video signal synthesis, we were able to program a low-memory computer chip with an open-source electronics prototyping platform to produce a surprising range of frequencies and amplitudes resulting in abstract video images.

The Wayback Machine allows both children and adult visitors to the Children’s Museum to see and learn how electronic signals become sounds and images.  Children push and pull large mechanical levers that produce abstract electronic sounds and images.  The interfaces, old golf clubs and vacuum cleaners, invite museum visitors to contemplate the relationships between old and new objects and technologies.
Bike Box: Bicycle and Mobile Phone as Hybrid Instrument and Interface

Bike Box, 2010, Bill Brown and Sabine Gruffat, Locative media project, iPhone Application, iphones, Bicycles, and two speakers, image: Sabine Gruffat.

Bike Box, 2010, Bill Brown and Sabine Gruffat, Locative media project, iPhone Application, iphones, Bicycles, and two speakers, image: Sabine Gruffat.

Last summer, we launched a location-aware iPhone app we called Bike Box [2]. Participants were able to borrow bicycles equipped with speakers and a phone mount and ride around central Brooklyn.  The app allowed users to listen to a curated collection of site-specific audio content provided by a variety of audio artists.  Users could also record and contribute their own site-specific audio.

Open Space
Among the issues we grappled with while developing this project was how to use Smartphone technology to enhance rather than replace a user’s experience of physical space.  The intention of many apps is to bring the world to the user, rather than the other way around.  The tendency of such apps is to de-spatialize or trans-spatialize the user, allowing her to review, survey, or experience spaces without moving.  To put this in terms that Paul Virilio might appreciate, this is the tendency of an app – or mobile media in general – toward a spatio-temporal totalitarianism.  “If in preindustrial eras the low speeds of various vehicles structured and geometrized the social landscape,” Virilio writes in Bunker Archaeology, “… since the acquisition of high speeds… it is here, and not over there [emphasis added], that the critical is from now on played out.” [3] To encourage mobility, exploration, and discovery, to insist on the “here-ness” of spatial media, we designed the app to allow users to listen to audio tags only by physically encountering the tagged spaces.  For Bike Box, there is no content without negotiating physical space.  In addition, we designed the app so that there can be no overlap of audio tags.  Rather, a participant who adds a tag “owns” a 30-meter sphere of space.  Among other things, this encourages a centrifugal or expansive relationship to space, propelling participants ever onward and outward toward unexplored and “unclaimed” territories.

The meaning of the term “space” is a complex one.  On the one hand, one can argue that Bike Box opens a new space for communication, broadcast and social struggle, and on the other hand, this space may only be the virtual space of capitalism created for the purpose of being colonized.  Geo-locative space is absolute space, as David Harvey writes in his book Spaces of Global Capitalism: “[absolute space] is fixed and we record or plan events within its frame…Socially this is the space of private property and other bounded territorial designations.” [4]

Yet Bike Box uses a space that is still somewhat open, not yet occupied or regulated by the industry or the state.  For example, we did not submit our application to the Apple Store and were able to distribute it via the iPhone configuration tool without Apple’s benediction.  Mobile technology is still in its infancy and it is still possible to utilize this emerging technology as “temporary autonomous zone.” [5]

Bike Box, 2010, Bill Brown and Sabine Gruffat, iPhone application screengrab, image: Sabine Gruffat.

Bike Box, 2010, Bill Brown and Sabine Gruffat, iPhone application screengrab, image: Sabine Gruffat.

Mobile Phone Intimacy and Performance
Another tendency we hoped to work against is the notion of an app as a private and intimate experience.  Though users could listen to audio through headphones, we encouraged them to borrow one of the bikes we had equipped with speakers.  The bikes served as broadcasting units, allowing riders to share their listening experience with friends and passersby.  The speakers allowed the audio content to extend into the space.  For the few moments the audio tag plays through the speaker, it becomes spatialized – no longer just a response to or gloss on the tagged space, but a part of the experience of that space.

The performative acts inherent in participating in a mobile game or interacting with a portable networked computer in public space constitute a rich area for research.  This research could include such genres as flash mobs or even public cell phone use. [6] Should we begin looking at this expression as a type of performance?  It’s obvious that ubiquitous computing, mobile media, and invisible technologies like GPS and Radio-frequency Identification (RFID) are undermining concepts of public space and changing our notions of privacy.  By utilizing high-powered speakers, Bike Box emphasized this extension into public space.

Locative Media and the Local
The advent of a new technology usually presents the possibility to repackage old media.  This way, businesses can sell the same idea in a different package.  The profit motive aside, it behooves us to consider what new forms of communication and community are possible as a result of technologies such as Smartphones and the availability of location-aware data.  The new web standard (HTML-5) incorporates geolocation, meaning that it will be easier to target specific users in terms of their location.  In conceiving Bike Box, we were interested in poetic responses to locations, be they field recordings, fictional narratives or interpretive audio.  Participants were therefore enabled to contribute whatever audio they wished.  The content we culled was more diverse than we had imagined it could be.  For all participants, it seemed important that the audio be either produced locally or by a local person.  As geolocation becomes attached to identification, the concept of a local acquires new meaning and becomes an important ingredient in cultural narratives.

2. See Adrian Freed’s blog at (accessed April, 2, 2011).
3. Paul Virilio, Bunker Archeology, (Princeton Architectural Press: New Jersey, 2008).
4. David Harvey, Spaces of Global Capitalism: A Theory of Uneven Geographical Development, (Verso: London, 2006).
5. See Hakim Bey, T.A.Z.: The Temporary Autonomous Zone, Ontological Anarchy, Poetic Terrorism, (Autonomedia: New York, 1991).