Monica Fleischmann GMD - Forschungszentrum Informationstechnik GmbH Fleischmann@gmd.de		Wolfgang Strauss GMD - Forschungszentrum Informationstechnik GmbH Strauss@gmd.de
	Mette Ramsgaard Thomsen GMD - Forschungszentrum Informationstechnik GmbH Thomsen@gmd.de		Jasminko Novak GMD - Forschungszentrum Informationstechnik GmbH Novak@gmd.de

Following the goals of the eRENA project (electronic Arenas for Culture, Performance, Art and Entertainment) for user embodiment and group Interaction, building real and virtual environments as mixed reality, we have developed the e-MUSE system (electronic multi-user stage environment). e-MUSE is derived from an artistic point of departure: the installation 'Murmuring Fields'.

'Murmuring Fields' is an interactive communication space. Realised in part real and part virtual space, it is the framework for a Mixed Reality installation.
e-MUSE is the underlying platform for networked communication, interface, rendering and display organisation. It is a VRML (Virtual Reality Modelling Language) based multi-user system. By implementing VRML as a networked multi-user application we have opened the otherwise closed system of VRML. Splitting browser and interface we have enabled the use of non-standard interfaces. Because of the ease and flexibility of the underlying architecture, e-MUSE software can be used in different situations, for example the production of performative systems and inhabited TV environments.

Setting up the virtual stage.

Imagine a room furnished with data.
The notion of mixed reality, the augmentation of real and virtual space and the ability to be present in both spaces at the same time, is amplified through the notion of shared environment: a situation in which participants discover their interdependence in exploring, perceiving and creating the 'world'.

'Murmuring Fields' is a virtual room filled with data. Here the visitor can rely on the scale of the body and space while at the same time navigating through the 'furniture' of data. Data is organised spatially and revealed as the visitor navigates the space. As a mixed reality environment the visitor's exploration of virtual space is connected to real space and other participants' experiences. Building on the concept of virtual stages, participants are brought into a field of body language.

'Murmuring Fields' relies on the development of invisible and intuitive interfaces which facilitate full body interaction. Using real-time video tracking the visitor's intuitive exploration of real space is transferred to the virtual environment. The real space - a room of 10 by 10 metres - is mapped on to the virtual space. As the visitor's awareness increases the body interaction from one spatial environment becomes influenced by the other.

The visitor's body movement is represented as traces in virtual space. Treated as an extended body of communication, user representation is replaced with the notion of user enactment. The TRACE-avatar embodies movement and gesture as channels of communication. The guiding aspects are the abstraction of visual appearance and the procedural creation of traces connected to body movement in real time.

Developing a vocabulary of body language

Murmuring Fields' demonstrates a framework for an extended performance, overcoming the split between acting and observing. It moves away from the isolated actor connecting him with a multi-sensorial reality joined by other participants. When entering 'Murmuring Fields', the first thing visitors notice is the black and white murmuring sound space reacting on the movements of their bodies. The sounds are structured as a field of density. The virtual world is inhabited by statements of the philosophers and thinkers: Paul Virilio, Villem Flusser, Marvin Minsky and Joseph Weizenbaum. In the centre these statements overlap creating a collage of noise. Space is dense and filled. As one moves outwards the statements are broken first into words and then into single phonemes. The movement of the user triggers the sound files creating sound experiences. Travelling from the centre outwards is a travel from density to silence. Movement causes coding and de-coding, arranging and rearranging content. Philosophical navigation incites discovery and produces polyphone sound scapes. As people move around they develop a spatial relation between their memory and the virtual space.

Sound scape of 'Murmuring Fields'

e-MUSE: future systems
Our research focuses on patterns of communication, building on the concept of audience participation in theatre. The goal is to develop a series of ex- periments using e-MUSE as a flexible multi-user platform open for augmenting real and virtual space. Serving as a story board/script system, a person tracking system or a reversed networked virtual studio system, e-MUSE can be used to examine different situations involving extended space and performance. We are interested in new formats and concepts, especially in playful ITV applications on tele-learning and tele-operations. Presently we are working with e-MUSE in three different situations all searching the field of Mixed Reality: play, theatre and networked living.

WHAT DID WE LEARN ?

1. The biggest problem was trying out the concepts developed and getting a real feedback on how it would feel in the complete installation setup envisioned.
2. We found that it is necessary to make case studies on specific user groups as for instance children in play situations or theatre people.
3. Interactive nonlinear narration is difficult for a broad audience to understand. We find it important to research communication patterns people are familiar with. As with the description of behaviour in the Comedia del' Arte, it is necessary to create a basic vocabulary.
4. We need to evaluate and critique the notion of Mixed Reality in an arena of specialists, e.g. theatre performers, anthropologists, neurologists, so as to understand the emerging expressions in interactive systems.
5. We will create a workshop, 'Hotel Perception', retroprojecting previous media and media theories (Berthold Brecht: Radiotheory), theories on merging fields of art (John Cage: Speisesaal), and active observation of interactive communication in recent art work.
6. e-MUSE should be implemented on an internet server to enable artists to work with net-communities.

Monika Fleischmann is an Artist and the Head of Media Arts Research Studies, Wolfgang Strauss is an Artist and Interface Designer, Mette Thomsen is an Architect, and Jasminko Novak is a Computer Scientist. They are all from MARS:
Media Arts Research Studies Institute for Media Communication
GMD German National Research Center for Information Technologies
imk.gmd.de/mars

© 1998,1999,2000 i3net	-- Search i3net --	About the i3net web pages
Revised: 23 January, 1999.		Mail to webmaster@i3net.org