Michael Schippling
Variations For

An interactive video driven suite of programs
which collaborate in creating their own choreography

Project Description

Variations For (VFor) extracts an articulated motion sequence (e.g., dance moves) from the activity of its audience, displays the sequence, and then produces variations. The variations are created by independent Generator processes who reach agreements on what elements to change at each step. The changes are passed around a circle of Generators with the results displayed at each stage. Modifications continue to be made until all the Generators agree on a final pattern -- in effect developing a collaborative choreography. Subsequent audience input will affect the patterns, which will eventually decay to a steady state when no more input is received.

VFor encompasses its audience in an interactive system where they provide input and the piece provides new impetus to elicit further response. It responds to a variety of inputs. Crowds walking past without stopping can produce motions reminiscent of waves of grain. An accomplished improv dancer might find an inspiring partner. In between, viewers can see a machine that thinks for itself.

The four main components are:

  1. Motion Capture using a video camera and image analysis software to extract motion sequences from a scene.
  2. Pattern Recognition to find repeatable patterns in the motion sequences. This will probably use a recurrent neural network however the details are part of VFor's development.
  3. Pattern Generation to create variations on the recognized patterns using stochastic finite state automata. The state machines contain a catalog of possible motion elements and use input from new and currently executing sequences to adjust their execution probability. This is the process used by my RoboCar Collective project.
  4. Output Display will be video generated using kinematic simulation software already available for commercial robot arms.

Given the temporal and financial constraints, output devices for the initial development will be video displays. However real-world articulated robot arms will be integrated into a later version of the system.

Artist's Bio

Michael Schippling's CV --
http://www.etantdonnes.com/TEXT/BioartSchip.pdf (also attached)

block diagram

Output Devices

The following are examples of commercial robotics products included for illustration purposes only. These are simple demos of robot arms and behaviors and are not indicative of Variation For's output or display technology.

maplesoft example
http://www.maplesoft.com/applications/view.aspx?SID=6850
claytex example
http://www.claytex.com/products/applications/robotics-and-mechatronics/
fanuc example
http://www.industrial-robots.com/fanuc/robot/r-2000ib-170cf
robai example
http://robai.com/robots/gamma-1500/


Theoretical Underpinning

Artistic merit

Interaction is a two way street. However Interactive Art is often merely responsive on the part of the art object (for more detail see my blog entry: Artistic Rendering). The key to interaction is communication. And communication between multiple systems opens the door to higher level adaptive and collaborative human-machine behaviors.

Generally technologically based (read: video, electronic, and digital) art presents pretty pictures or sounds at the behest of an audience with little control over the ultimate behavior. Instead I would like to reconnect the 1970s Systems Art paradigm, seen especially in the work of Jack Burnham and (pre-[and-post]-Guggenheim) Hans Haacke, to the world of Art-Science-Technology collaboration. Burnham and Haacke saw the limits of Modernist reduction and sought a larger view in the behavior of systems themselves. Unfortunately, at just that time, art turned to impressive technical displays or retreated into the conceptual, and this exploration of underlying systems withered on the vine.

As Complexity Science, and particularly Artificial Life, have revived alternative approaches in the sciences I want to revive Systems in art, re-exposing the alternatives that were lost when techno-art turned to spectacle. To do this I make things which are not just entertainments but have some small lives of their own, in their own worlds.

On a personal level this piece will allow me to develop new technologies to further my integration of collaborative machines into the web of the world. Hopefully the ideas and techniques developed in VFor will then enable other workers to forge further afield.

The LACMA Art and Technology project, culminating in 1971, was the last large scale project which encouraged artists and technologists, not to speak of scientists, to interact and even collaborate in working with the fabric of post-modern society. I'm glad to see it is being revived.


Dialog with technology

The common attraction-repulsion response to artificial beings is based on desire and fear. First, the desire for absolute control over our environment; and second, the fear that it will control us. I label the former desire, The Illusion of Control -- because achieving total control is ultimately a futile endeavor. And the latter fear, The Frankenstein Complex -- because we easily forget that Dr. Frankenstein's creature became a Monster only after it was mistreated at the hands of humanity.

While fear, anger, and disgust are necessary for biological evolution, the only reason that our mechanical progeny continue to be subject to these emotions is that we have not thought to provide them with alternatives. By constructing artificial beings which are not subject to biologically driven imperatives we can instead, for instance, give them the wish to become virtuoso dance performers. With this we encourage them to become truly creative and collaborative beings on their own terms.

I believe that, rather than leaving it to the military-financial-complex, artists should be at the forefront of this research program.


Public Engagement

Each component of the proposed system provides opportunities for independent demonstrations and discussions via workshops and artists' talks. I imagine at least four presentations, one for each major sub-system (motion capture, pattern generation, and final display), culminating with a session on the system's integration.

Since the initial development is entirely in software it can be open-sourced and shared with workshop participants and the internet audience at large, some of whom may be inspired to become VFor collaborators.

The machine learning processes used in this project have not disseminated into the art world to any appreciable degree. Concepts from this work may inspire other workers to pursue similar projects, and specific modules can be adapted to other systems thus seeding new projects.


Budget

Labor

I estimate a six month, full-time, development cycle. At about one month each, the milestones are:

  1. Detailed design and component selection
  2. Image capture and pattern recognition
  3. Pattern generation
  4. Output display graphics and animation
  5. System integration
  6. The last 80%: installation and tuning

I ask for a full-time-equivalent artist's stipend of $3000 per month ($18,000) which should cover travel and incidental expenses. I would also ask for an additional $2000 per month ($12,000) to compensate possible collaborators, bringing the labor budget to $30,000.

I would especially appreciate help in finding collaborators with experience in image processing, machine learning, robotics simulation, and animation software.

Equipment

Depending on the size of the output monitors used, the total hardware budget should be less than $10,000.

Total

The total requested expenditure is: $40,000.


Plan

I would prefer to spread this work over a longer period, perhaps working half time, giving each of the above components about a two month duration. However the deliverables listed below are estimated at a full-time-equivalent one-month-per-component rate:

MILESTONE START DATE FUNDS NEEDED
Design July 2014 $5000
Image recognition August 2014 $5000
Pattern generation September 2014 $5000
Output animation October 2014 $15000 (hardware purchase)
Integration November 2014 $5000
The last 80% December 2014 $5000










Notes

For the title

I am harking back to an earlier aleatoric experimental thread in Los Angeles' art history: John Cage's, Variations IV at the Feigen Palmer Gallery, 1965.

A little personal background

My small RoboCar Collective developed a, very limited, dance together over the period of about an hour using stochastic finite state automata to select "steps" at random and then share them with each other: Learning to Dance. This is the same technique I propose for VFor's pattern generators.

Some examples of my recent work can be found here: 2013 Kinetic Sculptures.