Strategic Working Groups

New Media || 2005

“New Media” has received many different interpretations over the past few years, ranging from technical to artistic to cultural. The Mellon Strategic Group in New Media chose to consider “new media” as any process of information-transmission that has the potential to change various aspects of human culture and experience in some radical way. The group considered the invention of writing in ancient Mesopotamia, the invention of architectural scale drawings in 15th-century Italy, and the invention of the Morse Code and transmitter in 1832, to be examples of “old” new media. Today, new media encompass a wide range of converging technologies for the creation, representation, and communication of information, based on the paradigm of computation. They include the Internet, video games, computer-aided design, computer music, video surveillance, and mobile telephones, among other things. And they have already begun to shape modern culture, affecting the way people work, communicate, learn, behave, and conceive of themselves and their world. 

The impacts of new media have become a subject of study in many disciplines, which try to assess critically the technical, social, professional, aesthetic, or ethical values of their affordances and implications. These discussions, however, are typically confined within disciplinary boundaries, oblivious to similar discussions in other disciplines. Consequently, judgments about the costs and benefits of new media are determined by particular normative vocabularies, value systems, and disciplinary expectations. Moreover, in failing to study the conditions that enable the invention, production, and distribution of new media, humanists and engineers alike tend to universalize their accounts of a medium’s benefits and drawbacks, despite historical and ethnographic evidence that the same medium can be used and valued differently in different times and places.  The Mellon Strategic Group in New Media was assembled (“assemblage” being a paradigmatic model of socialization in the era of “new media”) to counteract the disciplinary boundaries that work to disable a full understanding of the theory, history, and practice needed to study and shape new media in productive ways. The goal was to engage humanists and social scientists (users and students of new media) in conversation with designers, artists, and engineers of new media to gain a more complex view of the affordances and impacts of new media on culture, social relations, values, institutions, and everyday life. The multi-disciplinary “assemblage”—members of which introduce themselves below—met weekly during the Fall 2005 semester, subjecting each other’s work and disciplinary perspective to lively and stimulating interrogation.

Overall, the discussions—and they were discussions rather than presentations, given that not a single participant was permitted to finish his or her prepared comments—could be described as a re-education in the ways we look at media: learning to see the way other disciplines see and describe the world. For example, Peter Lyman commented that at his recent visit to the Prague show at the Metropolitan Museum in New York, instead of just looking at the paintings he studied the architectural plans for the cathedral, having learned about the role of paper in the founding of architecture from Kalay’s earlier presentation. When he now hears music he thinks about the boundary between sounds and music, inspired by David Wessel’s presentation. He learned about the relationship between perception and computer code from Greg Niemeyer. As an ethnographer, he couldn’t help but watch how the group created its own culture of conversation, dialogue often breaking out within the first minute of the speaker’s presentation, everyone wanting to join the adventure. But perhaps it was the “adventure” in new media provided by Greg Niemeyer’s video game that best summarized participants’ experience: “voice”—that metaphor naming the capacity within any medium to afford an opportunity for expression and communication, for human agency—is indeed a “joystick.” 

About the Group Members

Yehuda Kalay (Architecture) kicked off the group’s discussions by examining the affordances and implications of new media on the discipline of architecture, by discussing two attitudes toward new media that are prevalent in architecture: that of forcing a square peg into a round hole—implying that the use of a new medium for the production of architecture is misdirected, or at least poorly fits the processes that have traditionally been part of an architectural design practice; or as a horseless carriage—in which the affordances of the new medium are described in obsolete and ‘backward’ terms. Both views describe the difficulty of understanding the meaning of introducing a new technology into an old practice—a lack of appreciation for the emerging potentials of technology to change the very task to which they are applied, an appreciation that requires both time and open-mindedness. The process where a ‘new’ medium like computation has changed the profession of architecture is, however, not unique: the invention of scale drawings in the 15th century provided similar affordances and had similar, wide-ranging implications. Scale drawings—a medium of representation—separated the process of constructing buildings from the process of their design. This separation allowed more people to be involved in the design process, which allowed for designing more complex buildings, and their critical appraisal before they were constructed. It opened the field to people who had no experience in construction—for example, noblemen—thus elevating the status architecture from a craft to a discipline, establishing along the way the position of Architect as distinct from Master Builder. It has also allowed for errors in transmitting the design from the architect to the contractor—a problem that has plagued the construction industry ever since the 15th century. Kalay argues that a similar process is underway today, due to the advent of digital representation and communication media. Computational design tools can be viewed through the lenses of the same two paradigms, and their potential impacts are equally misunderstood by the profession of architecture and by the society it serves.

Alva Noë (Philosophy), raised the question of “perceptual presence”: can there be novel experiences? how can technology and media shape our investigation of this question? how does the answer to this question determine what media are and can be? These questions have puzzled philosophers and cognitive scientists for a long time: how do we understand the world (situations and things) which only show up for us in the form of (perceptually mediated) experiences? How, for example, can a voluminous 3D object such as a tomato get experienced as fully present when our perceptual view of it can never be more than partial? In what does our sense of the visual presence of the tomato’s backside consist, if we cannot actually see it? Noë proposed that perceivers have a special kind of skill-based access to (for example) the partially hidden portions of the things they see. In particular, perceivers are in a position now to understand, in an intimate, bodily kind of way, that by moving their eyes and bodies they can bring hidden portions of things into view. The back of the tomato can be present now in our visual experience even though it is not now seen because we now have sensorimotor skill-based access to it. In general, the world of situations and things can show up for us in experience thanks to the fact that we have the knowledge and skills to take hold of it. We can experience it because we comprehend it. One consequence of this view is that there can be no experience beyond the limits set by our skills and knowledge. There can be no novel experiences. For a novel experience to get experienced it must be made familiar, that is, deprived of its novelty, understood in terms of what is already mastered. Noë argues that this set of issues raise a special problem for the theory of new media, for one might think that new media are or provide new environments, new contexts, and thus, new possibilities for knowledge, thought, and, ultimately, experience.

Matthew Tiews, associate director of the Townsend Center for the Humanities, discussed the impact of the telegraph on the cultural and discursive practices of the mid-nineteenth century. While the telegraph is not generally considered under the rubric of “new media,” its novelty in the nineteenth century can provide an interesting perspective on media that we think of as new today. Tiews was particularly interested in the unintended consequences—social and linguistic—of a medium whose functional goals seem straightforward. The discussion ranged from the efflorescence of encryption technologies around communications media, to the key role played by access restrictions—both legal and economic—in determining a medium’s impact. Perhaps most important was the continued pressure put on the question, “What is a medium?”. Can the telegraph, generally considered a medium of long-distance communication, actually be considered a “medium,” since it was never a vehicle for original composition? Or is it simply a technological intrusion that affects other media—spurring the creation of artificial languages, for instance? Unfortunately, Tiews left Berkeley half-way through the semester and so was unable to continue with the group’s discussions.

Rather than offer (only) critical reflections on the impact of new media on the visual arts, Greg Niemeyer (Art Practice) presented a new media artifact in its own format. He asked the fellow participants to play “Organum Playtest,” a game he developed with Hellman Family Funds. In this collaborative game, the voices of three players act as joysticks. He then presented the computer code for the game, and discussed how the program related to the game’s performance and to the  player’s experience. Participants illuminated the difference between program and performance by asking how much of the performance could vary before the artists intentions would be altered. This question pointed out the importance of making programming accessible even to those who seek to critique new media. “The purest expression of a new media project is in its code” Niemeyer commented, “yet it is the least accessible.” After the seminar, Peter Lyman smiled at Niemeyer and said “That was pretty courageous, to show the code.” Yes, it does take courage to stand naked before other disciplines.

John Canny (Electrical Engineering and Computer Science) commented that new media lie at a nexus of technology, arts, and social science. One point of contact for all these traditions is the idea of “mediation” by a tool, text, performance, video, mix, game, or “experience.” While notions of mediation have diverged with these disciplines in the 20th century, there is a great opportunity now for reunification. Canny began with an analysis of the turn-of-the-20th century “pragmatist” tradition, which nurtured an extraordinary body of interdisciplinary ideas. These include the roots of phenomenology and semiotics, of social constructivist learning, and many contemporary approaches to social research. Taking an interdisciplinary (pragmatist) approach to mediation today allows researchers to build testable theories from a diverse but coherent set of ideas from many fields. One influential and interdisciplinary articulation of mediation was given by Lev Vygostsky in the 1910s. Another of Vygotsky’s “meta-theories” was the “genetic method,” which recurs in different flavors in many fields. In particular, Canny discussed how evolutionary development is a generic explanation for the mysterious mathematical “blueprint” (a power law) of many social phenomena. A final powerful idea articulated by Vygotsky is “language as symbolic action.” This idea manifests in strikingly similar ways in theater, education, psychology, and sociology. Drawing on these three meta-theories, one can paint a general but quite sharp explanation of human action. This explanation is radically different from traditional scientific models, and is much more compatible with humanist and social science perspectives, but is no less “scientific” as far as invoking testable hypotheses. This perspective opens up a host of possibilities for research, by opening doors between the core fields of New Media.

David Wessel (Music) raised the question “What does it mean for a signal or symbol-generating mechanism to function as a medium?” He turned to music for an answer. Computer technology has provided musicians with unprecedented control of sound. Not only can traditional attributes like pitch and rhythm be manipulated, but a myriad of audio attributes can be specified and systematically varied. Timbral features, spatial qualities like location and extent, as well as invented attributes like “speechiness” or “effortfullness” are all available for manipulation. Do such attributes of sound all have equal status from a perceptual point of view? Can each of them function is such a way so as to carry form? Drawing on cognitive science, in particular the work of Roger Shepard, Stephen McAdams, and Michael Kubovy, Wessel presented criteria with which one can decide whether or not an attribute can bear form. His stressed the role of perceptual invariance of patterns under transformation. In vision, shape is invariant under a variety of geometric transformations. Pitch is a form-bearing attribute, because it affords transposition in that melodies remain perceptually invariant if the ratios among the frequencies in them remain constant, a characteristic of the musical scale. Wessel then proceeded to demonstrate that timbre—if properly represented—can carry form, whereas spatial features of sound—like location—are problematic as form-bearing attributes. In closing remarks, addressing the work of Alva Noë, he speculated on the link between perception and bodily action and its role in the media of music.

Peter Lyman (School of Information) spoke about his research team’s MacArthur Foundation-sponsored ethnographic research about how kids are creating their own cultures of communication using new media. He talked about kids’ use of messaging on cell phones to create a new sense of emotional place: transcripts of a messaging exchange between kids that seemed to make no sense when read as a dialogue or narrative, unless one knows where the kids were located in space, which made it possible to infer the emotional context of the words. He discussed Internet places like MySpace, where nearly every student in a San Francisco high school has a blog; and FaceBook, the most used Internet information resource used by Berkeley students. Lyman commented that the traditional methods and theories developed to investigate the face to face world don’t really work online, requiring us to think about whether media—mediated communications—are evolving into fundamentally new cultures and organizations, and what kind of social theory can account for kids’ use of new media. For example, online places like MySpace can be interpreted in terms of identity and community, or examined as a new kind of economic relationship between kids and corporations. It is notable that corporations have online behavior under surveillance in astonishing ways; online lives that kids think about as private are actually public in unprecedented ways.  What kids might call online play or expression is, at the same time, appropriated by online software as intellectual property.

Nancy Van House  (School of Information) is concerned with understanding both how people incorporate new media into their ongoing practice and how they develop new practices, new practices which then affect the design of new media. She related two areas of her research concerning new media: trust and credibility in epistemic communities; and private image creation and sharing, specifically personal photography using camera phones. Epistemic communities’ established practices of information dissemination and evaluation, including publication, are being made visible and challenged by new media. The discipline of Science and Technology Studies (STS) provides valuable methodologies for studying the use of new media, she suggested, because of its emphasis on the interpretive flexibility of technology. She then elaborated these claims by discussing her research on the use of camera phones as a medium of personal memory.  The various uses of camera phones, and the meanings assigned to these uses, emerge in relation to embedded understandings of personal photography, but also have the potential to alter uses and meanings of personal photographs not taken with a camera phone.  That a new medium like the camera phone can affect not only the history of photography but also the understanding of memory suggests more generally the value investigating both continuity and innovation in the uses of new media. She described the seminar as particularly valuable for introducing her to understandings of photography in the humanities, and helping her to see how the analytical approaches of the humanities and empirical social science research can play complementary roles in understanding the meaning of photographs, photographic activity, and, by extension, other new media.

Celeste Langan (English) focused on the relevance of poetry—specifically, Romantic poetry—to the study of new media. The relation between new media and the large category of “literature” has often been imagined to involve chiefly the status of the book, even of the novel (“Has the novel been supplanted by film, television, and computer screen, by newer, more ‘interactive,’ forms of narration?”). To ask ‘what is the medium of poetry?’ is to remind ourselves that language is a medium, a medium that can use several “technologies”—speech, handwriting (one thinks of Blake), and print. Romantic poets—who famously indulged in fantasies of unmediated, “immediate” experience—were also writing during what has been called “the first age of information,” in which print appeared to have made obsolete poetry’s oral-formulaic mode of transmitting cultural memory. But by producing such experiments as Lyrical Ballads—by asking readers to “hear” the oral tradition of balladry through the visual medium of the printed lyric—Romantic poets demonstrated a revisionary axiom of media theory: that the capacities of a medium to enrich or impoverish sensory experience—to be “cool” or “hot” in McLuhan’s terms—are not determined by properties inherent in the technology or “program”; rather, those properties are made indeterminate by the uses to which they can be put. By experimenting with the “digital” technology of meter, Romantic poets also remind their readers that “orality” does not precede but rather describes “the technologizing of the word.” Her chief example was Coleridge’s “Christabel,” which situates the language of poetry in the middle of (as a medium between) ‘mere’ animal speech and the absolute metricality of the bell-tower clock. Alva Noë was kind enough to “read aloud” and so to demonstrate, in the (indeed slight) imperfection of his “performance” of the poem’s conclusion, how Coleridge’s metrical “program” requires recursive reading.

Natalia Brizuela (Spanish & Portuguese), is interested, on the one hand, in the development of different technologies of reproduction during the nineteenth and twentieth century in Latin America, and, on the other hand, in the material and aesthetic conditions of and for new media in Latin America. She has been exploring the diverse techniques of visualization that lie at the root of the emergence of photography, and the spaces that such techniques—understood as anxieties—opened up. That is, why and how are photographs made? What does one see when one looks at a photograph? Why is 19th-century photography in particular so overwhelmingly obsessed with photographing nature, and more specifically, territory? Is there a relationship between photography and map-making? Can photography, especially early photography, be considered analogous to map-making? In order to explore these questions, in her presentation to the group Brizuela delved into a comparison between some of the earliest examples of photography in the Brazilian Empire at the beginning of the nineteenth century and the sixteenth-century mapping adventure of New Spain—also a project of Empire—known as the Relaciones Geograficas. Brizuela was interested in what seemed to be, after a semester of meetings, a possible common denominator amongst both practitioners and theorists of new media: new media—whether old or contemporary—as intricately related to the desire for new spatial configurations and explorations.

Sergey Dolgopolskii (Near Eastern Studies), who specializes in Talmudic Studies and Continental Philosophy, came to the group with a strategic question: “Is media always technology?” Inspired by Heidegger’s famous claim that the essence of technology is nothing technical, he wanted to explore the relationship between technological and humanitarian aspects of the process of mediation. By the end of the semester, his question morphed into a surely no less strategic, but perhaps both broader and sharper set of questions: How should we approach newness of new media? Should the historical-chronological line of thinking be automatically considered the only or even the most effective measure of newness of media? Does the notion of experience, either historical or ontological, mark the only region, in which newness of new media can be construed?  He drew on two examples of potential alternatives to conceptualizing media as a progressive historical sequence or as a matter of experience. A counterpart to experience might be Galileo’s notion of “experiment” or the 15th-century Spanish Rabbi Y. Canpanton’s method of experimental speculation (iyyun) in the book of the Talmud (later also known as the method of pilpul). An alternative to chronological sequence might be Eisenstein’s ontological notion of montage as “the set-up of all things”; historical editing of the Talmud is treated in modernist theories of Talmudic redaction as if it were montage. As experiment is not the same as experience, so also montage is not the same as either representational content or viewers’ dispositions that montage allows to emerge in experience. Rather, such mediation intrinsically involves a play of different and even heterogeneous registers, which therefore are to be studied by methods proper to heterogeneity of their matter.

Richard Rinehart (Berkeley Art Museum/Pacific Film Archive), summed up the impact of digital media on museums with the two-part phrase, “digitally remembering and remembering digital.” Digitally remembering refers to museums, art organizations and other cultural heritage institutions such as libraries and archives using digital media as a tool to execute their mission of long-term social memory. A specific example of the impact of digital media in this area is the effect of heightened tension surrounding intellectual property law in the digital age and the museum’s conflicted role as both public steward and moral guardian of culture. Remembering digital refers to the fact that many of the cultural artifacts, or primary evidence, that museums now preserve are themselves digital. A specific example in this area is the preservation of digital art that has museums rethinking what aspects of an art work are important to preserve when faced with a choice between retaining original materials or original behavior, and artistic intent. From his interaction with this group, Rinehart took away the idea that the way museums construct social memory could be informed by the ways that people cognitively construct artifacts that make up their world-picture using their senses, how societies construct dynamic social spaces using social software and cell phones, how architects translate ideas into buildings, and how musicians use specific aspects of sound to construct media—topics brought to the discussion by other participants.

Camilo Salazar (undergraduate student, Philosophy), who heard all of the discussions in his capacity as the group’s research assistant, commented that after Alva Noë’s presentation he could not think of everyday experience without asking himself the questions “Can we have novel experiences?” “Is this a novel experience?” Such questions—along with Sergey Dolgopolskii’s question “Is new media always technology?”—transfigured his approach to understanding the ideas both of “newness” and “media,” which in turn has enriched his approach to film and digital video.