|The words "Sein und Zeit," the title of a book by M. Heidegger, have been adopted as the key words of this project. Distinct from their usage in the publication, they represent an attempt to consider what forms audio and visual information take when accompanied by a time axis, and how they are interrelated; this with respect to how humans appreciate information that is a reflection (phenomenon) of our computerized existence. By means of breaking up the continuity of an image, and placing a subject on a different time axis, this project attempts to expose a segmented time gap to an audience who see a continuous image that constitutes thirty still frames a second. It is also an attempt to explore the interactive potential of audio and visual information systems and physical beings that exist in real time, when the systems themselves are on a different time axis.|
|Mass and Media|
The performance piece "Mass and Media for Percussion" is the first
concrete result of the "Sein & Zeit" project, and at the same
time a prototype that will form a base for future developments.
|This performance work is firstly a concert piece. It consists of a
device that samples and scratches television signals broadcast at the
time depending on the drumming of the performer.
Television and video images are usually displayed at a speed of 30 frames a second. This is a "hand operated video replay device" that replays these images in keeping with the rhythm of the percussion instruments. If the percussionist strikes his instruments slowly the image replay slows, and if the percussionist strikes his instruments hard the images grow brighter. The audio that accompanies the images follows closely the progress of the image frames, which are themselves dependent on the actions of the percussionist. The setup of the percussion instruments revolves around bongo drums that have been assigned single frame advance and reverse functions, and other instruments that have been assigned functions such as changing the television channel, and starting and stopping the television broadcast sampling. The performer recites the piece adhering faithfully to the musical score. Towards the end of the performance the image memory, that is full of samples of the television broadcasts, is written over with images of the percussionist himself, upon which the percussionist scratches the images of himself by continuing to recite the music dictated by the musical score.
|The performance system comprises two computers. One is used solely
for digital image processing and the other is used for audio processing
and control of the whole system.
- A DV video sampling and playback software package called “videotronics” was developed especially for this project. Together with MIDI Control, this software is used on the image-processing computer to record and replay video signals, and for capturing and scratching digital signals in conjunction with the system control computer. It also controls the brightness of the images, which in turn depends on the volume of the drums.
- The audio-processing and system control computer is used firstly to read and replay the video, and accompanying audio signals. It is also used to simultaneously control the image-processing computer, and process the performance information (transformed into MIDI data) obtained from multiple microphones and drum triggers. The video and audio playback speed is unrelated to the original audio signals. Granular synthesis is used in the playback algorithms to facilitate playback of the signals from the percussion instrument triggers according to overall time progression.
The video recorder used for receiving the television broadcasts, changing channels and the image effecter are all controlled remotely using a MAX/MSP patch specially developed for this project.
|On the stage there is a large video image screen, a percussion kit, and PA speakers on the left and right of the stage. All other equipment including the computers is placed to one side of the percussion kit, underneath the screen. The performer controls the whole system during the performance and there is no separate operator. There is a PA mixer located in the central area of the auditorium.|
|Potential for development|
|The software and performance system developed for "Mass & Media"”
is not necessarily restricted to percussion instruments. It is a versatile
system with ample room for development. Examples of other uses include
the modern revival of live production using manually operated cameras
that have been used in the past, or a system that controls frame advance
according to the movement of dancers.
Amongst these possibilities, one that I would most like to see realized is the “Mass & Media Tele-presence version”. As the reader would understand from the system outline, all the performance date generated by the performer is digitalized, which opens the possibility for remote control, or control by multiple systems. That is, by simultaneously transmitting the images of the recital and all other performance data, it is possible to generate multiple versions of the work, all different from the original, via using local television in other countries around the globe. One of the special characteristics of this project is how, in principle, the time delay that usually occurs when using a network for transmission does not present a problem. The delay in transmission of image information when sending pictures can not be avoided, however due to delays in, or by purposely delaying the transmission of the MIDI information, it is possible to fully coordinate the local video materials with the movement of the percussionist contained in the video images transmitted from the performance "base."