Augmented Space

[....]

Augmented Architecture

In the 1990s, computer hardware manufacturers and computer game industry drove the development of applications that use 3-D interactive virtual spaces such as computer games. While today's PC are already too fast for practically all the applications needed for a typical home or business user, real time rendering of the detailed simulated worlds still can use faster machines; it also requires special graphics cards. The industry therefore has a direct interest in continuously fueling the interest of the consumers in more and more "realistic" virtual spaces – because this is what justifies the sales of new computer hardware.

Augmented space research has the potential for many commercial, consumer and military applications, and thus it receives funding from diverse groups. Ultimately, it is probably of most concern to the huge telecom industry. So if the computer industry thrives on sales on new PCs and graphics boards needed to run latest computer games, the telecom industry is interested in selling new generations of cell phones and PDA which will provide multimedia, e-commerce, and wireless location services – and of course getting huge gains from charging the users for these services.

So much for economics. But what about the phenomenological experience of being in a new augmented space? What about its cultural applications? What about its poetics and aesthetics? One way to begin thinking about these questions is to approach the design of augmented space as an architectural problem. Augmented space provides a challenge and opportunity for many architects to rethink their practice, since architecture will have to take into account that layers of contextual information will overlay the built space.

But is this a completely new challenge for architecture? If we assume that the overlaying of different spaces is a conceptual problem not connected to any particular technology, we may start thinking about which architects and artists have already been working on this problem. To put this in a different way, overlaying dynamic and contextual data over physical space is a particular case of a general aesthetic paradigm: how to combine different spaces together. Of course electronically augmented space is unique since information is personalized for every user, since it can change dynamically over time, since it is delivered through an interactive multimedia interface, etc. Yet it is crucial to see it as a conceptual rather than just as a technological issue, as something that already was often a part of other architectural and artistic paradigms.

Augmented space research gives us new terms to think about previous spatial practices. If before we would think of an architect, a fresco painter, or a display designer working to combine architecture and images, or architecture and text, or incorporating different symbolic systems in one spatial construction, we can now say that all of them were working on the problem of augmented space: how to overlay layers of data over physical space. Therefore, in order to imagine what can be done culturally with augmented spaces, we may begin by combing previous cultural history for useful precedents.

To make my argument more accessible, I have chosen as my examples two well-known contemporary figures. Janet Cardiff is a Canadian artist who became famous for her "audio walks." She creates her pieces by following a trajectory through some space and narrating an audio track that combines instructions to the user ("go down the stairs"; "look into the window"; "go through the door on the right") with narrative fragments, sound effects and other aural "data." To experience the piece, the user puts on earphones connected to a CD player, and follows Cardiff's instructions [14]. In my view her "walks" represent the best realization of augmented space paradigm so far - even though Cardiff do not use any sophisticated computer, networking and projection technologies. Cardiff's "walks" show the aesthetic potential of overlaying a new information space over a physical space. The power of these "walks" lies in the interactions between the two spaces - between vision and hearing (what the user is seeing and what she is hearing), and between present and past (the time of user's walk versus the audio narration which like any media recording belongs to some undefined time in the past).

Jewish Museum Berlin by Daniel Liberskind can be thought of as another example of augmented space research. If Cardiff overlays a new data space over the existing architecture and/or landscape, Liberskind uses the existent data space to drive the new architecture he constructs. The architect put together a map that showed the addresses of Jews who were living in the neighborhood of the museum site before World War II. He then connected different points on the map together and projected the resulting net onto the surfaces of the building. The intersections of the net projection and the walls gave rise to multiple irregular windows. Cutting through the walls and the ceilings at different angles, the windows evoke many visual references: narrow eyepiece of a tank; windows of a medieval cathedral; exploded forms of the cubist/abstract/suprematist paintings of the 1910s-1920s. Just as in the case of Cardiff's audio walks, here the virtual becomes a powerful force that re-shapes the physical. In Jewish Museum the past literally cuts into the present. Rather than something ephemeral, an immaterial layer over the real space, here data space is materialized, becoming a sort of monumental sculpture.

White Cube as Cellspace

While we may interpret practice by selected architects and artists as having particular relevance to thinking of how augmented space can be used culturally and artistically, there is another way to link augmented space paradigm with modern culture. Here is how it works.

One trajectory which can be traced in the twentieth century art is from a two dimensional object placed on a wall towards the use of the whole 3D space of a gallery. (All other cultural trajectories in the twentieth century, this one is not a linear development; rather, it consists from steps forward and steps back, the rhythm which follows the general cultural and political outline of the twentieth century: highest peak of creativity in the 1910s-1920s, followed by a second, smaller peak in the 1960s). Already in the 1910s Tatlin's reliefs break the two-dimensional picture plane, exploding a painting into the 3D dimension. In the 1920s, Lissitzky, Rodchenko and others moved away from an individual painting / sculpture towards thinking of a whole white cube as one singular surface – yet their exhibitions activate only the walls rather than the whole space.

In the mid-1950s, assemblage legitimized the idea of an art object as a three dimensional construction (1961"The Art of Assemblage" MOMA exhibition). In the 1960s, minimalist sculptors (Carl Andre, Donald Judd, Robert Morris) and other artists (Eva Hesse, Arte Powera) finally start dealing with the whole of 3D space of a white cube. Beginning in the 1970s, installation (Dan Graham, Bruce Nauman) grows in importance to become in the 1980s the most common form of artistic practice of our times – and the only thing which all installations share is that they engage with 3D space. Finally, the white cube becomes a cube – rather than just a collection of surfaces.

What is the next logical step? For modern art, augmented space can be thought as the next step in the trajectory from a flat wall to a 3D space. For a few decades now artists have already dealt with the entire space of a gallery; rather than creating an object that a viewer would look at, they placed the viewer inside this object. Now, along with the museums, the artists have a new challenge: placing a user inside a space filled with dynamic, contextual data with which the user can interact.

Moving Image in Space: Video Installations as Laboratory for the Future

Before we rush to conclude that the new technologies do not add anything substantially new to the old aesthetic paradigm of overlaying different spaces together, let me note that the new technologically implemented augmented spaces have one important difference from Cardiff's walks, Liberskind's Jewish museum, and similar works – in addition to their ability to deliver dynamic and interactive information. Rather than overlaying a new 3-D virtual dataspace over the physical space, Cardiff and Liberskind overlay only a two-dimensional plane, or a 3-D path, at best. Indeed, Cardiff's walks are new 3-D paths placed over an existing space; rather than complete spaces. Similarly, in Jewish Museum Berlin Liberskind projects 2-D map onto the 3-D shapes of his architecture [15].

In contrast, GPS, wireless location services, surveillance technologies, and other augmented space technologies all define data space – if not in practice than at least in their imagination - as a continuous field completely extending over and filling in all of physical space. Every point in space has a GPS coordinate which can be obtained using GPS receiver. Similarly, in the cellspace paradigm every point in physical space can be said to contain some information that can be retrieved using PDA or a similar device. With surveillance, while in practice video cameras, satellites, Echelon (the set of monitoring stations which are operated by the U.S. and are used to monitoring all kinds of electronic communications globally), and other technologies so far can only reach some regions and layers of data but not others, the ultimate goal of the modern surveillance paradigm is to able to observe every point at every time. To use the terms of Borges's famous story, all these technologies want to make the map equal to the territory. And if, according to Michel Foucault's famous argument in Discipline and Punish, the modern subject internalizes surveillance, thus removing the need for anybody to be actually present in the center of the Panopticum to watch him/her, modern institutions of surveillance insist that s/he should be watched and tracked everywhere all the time.

It is important, however, that in practice data spaces are almost never continuous: surveillance cameras reach look at some spaces but not at others, wireless signal is stronger in some areas and non-existent in others, and so on. As Matt Locke eloquently describes this,

Mobile networks have to negotiate the archiecture of spaces that they attempt to inhabit. Although the interfaces have removed themselves from physical architectures, the radio waves that connect cell spaces are refracted and reflected by the same obstacles, creating not a seamless network but a series of ebbs and flows. The supposedly flat space of the network is in fact flat, puled into troughts and peaks by the gravity of archiecture and the users themselves [16].

This contrast between continuity of cellspace in theory and its discontinuity in practice should not be dismissed; rather, it itself can be the source of interesting aesthetics strategies.

My third example of already existing augmented space – electronic displays mounted in shops, streets, building's lobbies, train stations and apartments – follows different logic. Rather than overlaying all of the physical space, here data space occupies a well-defined part of the physical space. This is the tradition of the Alberti's window, and, consequently, post-Renaissance painting, cinema screen, and TV monitor. However, if until recently the screen usually acted as a window into a virtual 3-D space, in the past two decades of the 20th century it turned into a shallow surface in which 3-D images co-exist with 2-D design and typography. Live action footage shares space with motion graphics (titles), scrolling data (for instance, stock prices or weather) and 2-D design elements. In short, a Renaissance painting became a an animated Medieval illustrated book.

My starting point for the discussion of the poetics of thus type of augmented space will be the current practice of video installations that came to dominate art world in the 1990s. Typically, these installations use video or data projectors; they turn a whole wall or even a whole room into a display or a set of displays; thus rehearsing and investigating (willingly or not) the soon-to-come future of our apartments and cities when large and thin displays will become the norm. In the same time, these laboratories of the future are rooted in the past: the different traditions of "image within a space" of the twentieth century culture.

White Cube versus Black Box

Among different oppositions that have structured the culture of the twentieth century that we have inherited has been the opposition between an art gallery and a movie theatre. One was high culture; another was low culture. One was a white cube; another was a black box.

Given the economy of art production – one of a kind objects created by individual artists – twentieth century artists spent lots of energy experimenting with what can be placed inside the neutral setting of a white cube: breaking away from a flat and rectangular frame by going into the third dimension; covering a whole floor; suspending objects from the ceiling; and so on. In other words, if we are to make an analogy between an art object and a digital computer, we can say that in modern art both "physical interface" and "software interface" of an art object were not fixed but open for experimentation. In other words, both the physical appearance of an object and the proposed mode of interaction with an object were open for experimentation. Artists have also experimented with the identity of a gallery: from a traditional space of aesthetic contemplation to a place for play, performance, public discussion, a lecture, and so on.

In contrast, since cinema was an industrial system of mass production and mass distribution, the physical interface of a movie theatre and software interface of a film itself were pretty much fixed. A 35 mm image of fixed dimensions projected on a screen with the same frame ratio; dark space where the viewers were positioned in a set of rows; a fixed time of a movie itself. Not accidentally, when in the 1960s experimental filmmakers started to systematically attack the conventions of traditional cinema, these attacks were aimed at both its physical interface and software interface (along, of course, with the content). Robert Breer projected his movies on a board that he would hold above his head as he moved through a movie theatre towards the projector; Stan VanderBeck contrasted semi-circular tents for projection of his films; etc.

The gallery was the space of refined high taste while the cinema served to provide entertainment for the masses, and this difference was also signified by what was acceptable in two kinds of spaces. Despite all the experimentation with its "interface," gallery space was primary reserved for static images; to see the moving images the public had to go a moving theatre. Thus until recently, moving image in a gallery was indeed an exception (Duchamp's rotoscopes, Acconci's masturbating performance).

Given this history, the 1990s phenomena of omni-present video installation taking over the gallery spaces goes against the whole paradigm of modern art – and not only because installations bring moving images into the gallery. Most video installations adopt the same physical interface: a dark enclosed or semi-enclosed rectangular space with video projector on one end and the projected image on the opposite wall. From a space of constant innovation in relation to physical and software interface of an art object, a gallery space has turned into what for almost its century was its ideological enemy – a movie theatre, characterized by the rigidity of its interface.

Many software designers and software artists – from Ted Nelson and Alan Kay to Perry Hoberman and IOD – revolt against the hegemony of mainstream computer interfaces, such as the keyboard and mouse, GUI, or commercial Web browsers. Similarly, the best of video, or more generally, moving image installation artists, go beyond the standard video installation interface - a dark room with an image on one wall. Examples include Diana Thater, Gary Hill, Doug Aitken, as well as the very first "video artist" – Nam Juke Paik. The founding moment of what came later to be called "video art" was Paik's attack on physical interface of a commercial moving image – his first show consisted of television with magnets attached to them, and TV monitors ripped open of their enclosures.

The Electronic Vernacular

When we look at what visual artists are doing with a moving image in a gallery setting in comparison with these other contemporary fields, we can see that the white gallery box still functions as a space of contemplation, quite different from the aggressive, surprising, overwhelming spaces of a boutique, trade show floor, an airport, or a retail/entertainment area of a major metropolis [17]. While a number of video artists continue the explorations of 1960s "expanded cinema" movement by pushing moving image interfaces in many interesting directions, outside of a gallery space we can find at least as rich field of experiments. I can single out three areas. First, contemporary urban architecture - in particular, many proposals of the last decade to incorporate large projection screens into architecture which would project the activity inside, such as Rem Koolhaas 1992 unrealized project for the new ZKM building in Karlsruhe; a number of projects, also mostly unrealized so far, by Robert Venturi to create what he calls "architecture as communication" (buildings covered with electronic displays); realized archiectural/media installations by Diller + Scofilio such as Jump Cuts and Facsimile [18]; the highly concentrated use of video screens and information displays in certain cities such as Seoul and Tokyo or in Time Square in NYC; and finally, imaginary future architecture as seen in movies from Blade Runner (1982) to Minority Report (2002) which uses electronic sreens on the scale not possible today Second is the use of video displays in trade show design such as in annual SIGGRAPH and E3 Conventions. The third is the best of retail environments (I will discuss this in more detail shortly).

The projects and theories of Robert Venturi deserve a special consideration since for him an electronic display is not an optional addition but the very center of architecture in information age. Since the 1960s Venturi continuously argued that architecture should learn from vernacular and commercial culture (billboards, Las Vegas, strip malls, architecture of the past). Appropriately, his books Complexity and Contradiction in Architecture and Learning from Las Vegas are often referred to as the founding documents of post-modern aesthetics. Venturi argued that we should refuse the modernist desire to impose minimalist ornament-free spaces, and instead embrace complexity, contradiction, heterogeneity and iconography in our build environments [19]. In the 1990s he articulated the new vision of "Architecture as communication for information age (rather than as space for the Industrial Age)." [20] Venturi wants us to think of "architecture as iconographic representation emitting electronic imagery from its surfaces day and night." Pointing out at some of the already mentioned examples of the aggressive incorporation of electronic displays in contemporary environments such as Time Square in NYC, and also arguing that traditional architecture always included ornament, iconography and visual narratives (for instance, a Medieval cathedral with its narrative window mosaics, narrative sculpture covering the façade, and the narrative paintings), Venturi proposed that architecture should return to its traditional definition as information surface [21]. Of course, if the messages communicated by traditional architecture were static and reflected the dominant ideology, today electronic dynamic interactive displays make possible for these messages to change continuously and to be the space of contestation and dialog, thus functioning as the material manifestation of the often invisible public sphere.

Although this has not been a part of Venturi's core vision, it is relevant to mention here a growing number of projects where the large publicly mounted screen is open for programming by the public who can send images via Internet, or choose information being displayed via their cell phones. Even more radical is Vectorial Elevation, Relational Architecture #4 by artist Raffael Lozano-Hemmer [22]. This project made possible for people from all over the world to control a mutant electronic architecture (made from search lights) in a Mexico City's square. To quote from the statement of the jury of Prix Ars Electronica 2002 which awarded this project Golden Nica at Ars Electronica 2002 in Interactive Art category:

Vectorial Elevation was a large scale interactive installation that transformed Mexico City's historic centre using robotic searchlights controlled over the Internet. Visitors to the project web site at http://www.alzado.net could design ephemeral light sculptures over the National Palace, City Hall, the Cathedral and the Templo Mayor Aztec ruins. The sculptures, made by 18 xenon searchlights located around the Zócalo Square, could be seen from a 10-mile radius and were sequentially rendered as they arrived over the Net.

The website featured a 3D-java interface that allowed participants to make a vectorial design over the city and see it virtually from any point of view. When the project server in Mexico received a submission, it was numbered and entered into a queue. Every six seconds the searchlights would orient themselves automatically and three webcams would take pictures to document a participant's design [23].

Venturi's vision of "architecture as iconographic representation" is not without its problems. If we focus completely on the idea of architecture as information surface, we may forget that traditional architecture communicated messages and narratives not only through flat narrative surfaces but also through the particular articulation of space. To use the same example of a medieval cathedral, it communicated Christian narratives not only through it's the images covering its surfaces but also through its whole spatial structure. In the case of modernist architecture, it similarly communicated its own narratives (the themes of progress, technology, efficiency, and rationality) through its new spaces constructed from simple geometric forms – and also through its bare, industrial looking surfaces. (Thus the absence of information from the surface, articulated in the famous "ornament is crime" slogan by Adolf Loos, itself became a powerful communication technique of modern architecture).

An important design problem of own time is how to combine the new functioning of a surface as an electronic display with new kind of spaces that will symbolize the specificity of our own time [24]. While Venturi fits electronic displays on his buildings that closely follow traditional vernacular architecture, this is obviously not the only possible strategy. A well-known Freshwater Pavilion by NOX/Lars Spuybroek (1996) follows a much more radical approach. To emphasize that the interior of the space constantly mutates, Spuybroek eliminates all strait surface and strait angle; he makes the shapes defining the space appear to move; and he introduces computer-controlled lights that change the illumination of an interior [25]. As described by Ineke Schwartz, "There is no distinction between horizontal and vertical, between floors, walls and ceilings. Building and exhibition have fused: mist blows around your ears, a geyser erupts, water gleams and splatters all around you, projections fall directly onto the building and its visitors, the air is filled with waves of electronic sound." [26]

I think that Spuybroek's building is a successful symbol for information age. Its continuously changing surfaces illustrate the key effect of a computer revolution: substitution of every constant by a variable. In other words, the space which symbolizes information age is not a symmetrical and ornamental space of traditional architecture, rectangular volumes of modernism, or broken and blown up volumes of deconstruction – rather, it is space whose shapes are inherently mutable, and whose soft contours act as a metaphor for the key quality of computer-driven representations and systems: variability.

Learning from Prada
[.....]