Creativity Support Tools for and by the New Media Arts Community

 

Pamela Jennings Human Computer Interaction Institute, School of Art Carnegie Mellon University Pittsburgh, PA 15213 412-268-5273 pamelaj@andrew.cmu.edu

Elisa Giaccardi Center for LifeLong Learning and Design (L3D) University of Colorado Boulder, CO 80309-0430, USA  303-492-4147 elisa.giaccardi@colorado.edu

 

 

INTRODUCTION

The new media arts are a particularly fertile domain for the development of creativity support tools that both supplement creative practices and contribute valuable research methodologies for other disciplines. Many parallel research concerns of new media art practitioners and researchers are found in the Human Computer Interaction and software engineering communities, including: education technology, computer supported collaborative work, data visualization, database architecture, and tools development research in pervasive computing, tangible interfaces, emotion and context aware interaction, and so on. New media arts practitioners and researchers should be regarded as valuable contributors not only as users needing better creativity support tools (CST) to enhance their own creative process, but also as the designers of experimental and innovative creativity support tools capable of providing insights and indications for:

 

1.        Categorizing what features of the interface and what system components engender and satisfy the requirements of these multiple forms of creativity. Section 1, Creativity Support Tools development for New Media Arts Curriculum presents examples of tools that have been designed for new media art production, and potential tool features, that could empower the creative potentials of practitioners across all fields of creative production.

2.        Defining the range of potential “creativities” (Sternberg, 2005) for which new technologies and tools may be developed.  Section 2, Research-In- Practice, introduces works by new media artists that have developed creativity support tools that are used by a large user-base of artists and other professionals as a result of their own creative practices. Sometimes this is a primary outcome, often a residual effect from developing robust tools for one’s own art practice that can withstand a variety of user interaction and manipulation.  This section also presents cases of new media arts research-in-practice that hold the similar broad reaching potentials.

3.        Developing more comprehensive and appropriate evaluation methodologies grounded in the “research-in-practice” approach. Section 3, Policy Making and New Media Arts, is a brief overview of international policies for new media arts practices.  Many of these policies recognize the innovative potentials for effecting research in science and technology including and beyond the arts. 

 

This document, generated from presentations and conversations at the NSF Creativity Support Tools Workshop held in Washington D.C. in June, 2005, presents example cases of the contributions that new media art pedagogy, practice and research can provide in the development of support tools that promote the situated, affective and social aspects of creativity.

 

1. CST DEVELOPMENT FOR NEW MEDIA ARTS CURRICULUM

Education technology research by information technology, education, and public policy researchers has grown tremendously as the Internet has become the de facto platform for the dissemination of information, and platform for community-based collaborations. From component based authoring environments to cognitive tutors, much of the research in this area has been funded by the National Science Foundation to support STEM (Science, Technology, Engineering, and Mathematics) disciplines. Our tendency to separate the STEM disciplines from creativity, culture and humanities sets forth a pattern of missed opportunities to develop new technologies that could help to solve research problems for disciplines that present alternative perspectives on data acquisition, analysis, and manipulation. In recent years new media arts practices have entered the mainstream.  Many international universities are actively creating new media arts programs geared towards artists that both use and create technologies (Jaimes & Jennings, 2004). NSF has recently funded a few research initiatives to develop curriculum modules that integrate computer science and new media arts (Integration Digital Media Curriculum Development (NSF DUE- 0340969) and the Digital Media Curriculum Development Project (NSF DUE-0127280). Pedagogical practices in new media arts are based on problem solving through open exploration of conceptual ideas that sometimes conform to, but mostly challenge, the intended functionalities of technology-based tools. This pedagogical practice of open tools usage sits in contrast to the typical computer science curriculum that is based on learning through constrained problem solving.  In the later case, students are typically given assignments that have a limited number of acceptable solution variations. The open and indeterminate nature of new media arts practice presents unique opportunities for developers of creativity support tools to incorporate complex programming abstractions, relational databases, integrated search functionalities, and scaffolded interfaces as a means to create more flexible creativity support tools for new media arts students and practitioners. (Bransford & Brown, 1999)

 

1.1. NEW MEDIA ARTS CURRICULUM CASE #1:

Design of Appropriate Tools Features for Open Ended Creative Production

 

Fig. 1 ­ 3. Human Computer Interaction and Interaction Design students working with electronics in the Physical Computing: Wearables offered by the School of Art at CMU.

 

Students in the Introduction to New Media Arts class at Carnegie Mellon University were given an assignment to create a sound self-portrait five minutes in length.  The main rule was a restriction on the length of a sound bite to ten seconds.  This rule was given to encourage the exploration of composition by editing multiple sound layers.  PEAK, a sound editing application, was selected because of its relative easy to use and happens to be the introductory application installed on all of our computers.  An advanced sound editing application, though more flexible, was not introduced at this stage to give students, of all levels, a sense of efficacy that would be difficult with a more advanced application.  PEAK is also an application that replaces Sound Edit Pro, a Macintosh application from the early 1990’s. Both applications are simple to use but have a couple of profound differences.  Sound Edit Pro enabled artists to approach sound editing in a much more fluid and conceptual manner with an interface that did not assume that the end product was to be a perfectly balanced stereo recording.  Thus it did not put constraints on how the user could combine multiple layers of sound across multiple sound channels.  PEAK, on the other hand comes with more sound synthesis filters and plug-ins than any practitioner may ever want to use.  It has savvy widgets for editing within a sound channel, but has very limited flexibility for mixing more than two mono tracks at a time, making the combination of multiple sound layers an unnecessarily arduous task.  The students persevered through the assignment and produced wonderfully complex sound compositions, because of their deep interest in contemporary sound culture. Benefits they gained from working with this inflexible interface included the realization that working with computer-based tools requires, focus, patience, and the willingness to fail often before producing a satisfactory product. However, they could have learned these lessons with an application better attuned to alternative interpretations of assumed use by its developers.  This would be an interface with features for the beginning professional that respect her developing aesthetic voice. 

 

1.2. NEW MEDIA ARTS CURRICULUM CASE #2:

Tools to Support Mixed Skills Level Classes

Upon completion of the compulsory introductory classes in new media arts, the Carnegie Mellon University art student can enroll in intermediate and advance elective classes that range from video and sound production and animation, to interactive programming and physical computing.  These classes are also very popular for students in the Human Computer Interaction Institute, Electrical and Computing engineering, Entertainment Technology Center, College of Humanities and Social Sciences, and School of Design.  This mix of students enables opportunities for cross-disciplinary learning and collaboration.  The classes attract a broad range of students for several reasons.  Students learn specific technology-based tools for creative arts and interactive design prototypes.  Students can explore these techniques in a curriculum that encourages freedom of exploration and expression.  Many of our computer science and engineering students enjoy these classes because they can engage in open-ended learning, which is generally a different approach than the typical exercise or lab-based technology-learning pedagogy.

 

The broad range of students’ skills in these classes presents a few creativity support tool development opportunities The first is the design of tools that can accommodate a range of skills­ from beginner to advance ­ with proper scaffold features and functionalities for each user level. The second is the development of multi-level help modules that can assist students and instructors in facilitating a broad range of questions and possible solutions. For example, it would be wonderful to have a cognitive tutor that notices that a student with beginning skills levels continuously gets a syntax error in her code which she is having difficulty correcting.  After analyzing and finding no errors in the code structure, the help module recommends that she check her spelling.  Another example if a student wants multiple sprites on the stage to design a Tetris game like interface should he be instructed to create each sprite as a unique entity ­ a long and tedious process that uses a minimum of code, or should he delve into property lists and object oriented programming ­ far more efficient but requires a basic knowledge of programming.  Or, should he be encouraged to implement sorting and advanced search algorithms that allow him to develop an intelligent self-playing game.  These are not hypothetical examples, but the range of students I have seen in new media art classes taught in the School of Art at Carnegie Mellon University and surely a case that is replicated at other universities.

 

1.3    NEW MEDIA ARTS CURRICULUM CASE #3:

Holistic Presentation Tools

Students taking interaction design classes are often required to work in groups to solve a particular design /human computer interaction interface problem.  They are typically given a hypothetical situation to ideate, visualize, and present.  Students are encouraged to select very creative means to explore and represent their ideas including text, graphics, photographs, video, theater, 3D animation, sound, programming examples, etc….  It is the student team’s task to come up with methods to formalize their needs analysis research, aggregate and negotiate their ideas and produce a visual or functional demo of the interface, application, or gadget.  Unfortunately, all of the effort in this work is often reduced to a power point presentation for class-wide discussion and critique.  Here is a tremendous opportunity to develop a tool, or set of interoperable tools to support all the requirements as described above without forcing the final presentation to be formatted into bullet point lists.   

 

Fig. 4 ­ 7. Storyboard excerpts from Critical Interaction Design class.  Students involved were from Human Computer Interaction, Architecture, Entertainment Technology Center and School of Design at Carnegie Mellon University.

 

2. “RESEARCH-IN-PRACTICE”

Creativity Support Tools for and by the New Media Arts Community

 

The new media arts are characterized by what is usually referred to as “research-in-practice”: an experimentalism and reflexivity that bring artists to link creative research and practice in a “highly responsive, iterative process where new insights are fed back quickly into the development process” (Candy & Edmonds, 2005). The new media arts express a risk-taking and subversive attitude, ultimately seeking cultural acts through which to provide society with entry points for change (for example, in the very definition of what is creativity and how it can be supported). Stephen Wilson’s “Information Arts: Intersections of Art, Science and Technology” (Wilson, 2002) is an encyclopedic archive of arts, technology and science collaborations exploring new ideas that challenge and contribute alternative perspectives on research practice and tools development across nearly every category of science and technology research from microbiology to nanotechnology to augmented reality.     Fig. 8 . David Rokeby, Very Nervous System (1990); Fig. 9. Paul Kaiser with Bill T. Jones, Ghost Catching (1999); Fig. 10. Screen shot of Cycling 74 MAX / Jitter interface; Fig. 11. Char Davies, navigation technologies developed for Osmose virtual environment (1995).   Examples of innovative tools created within the new media arts community include, David Rokeby's "Very Nervous System", a computer vision system used by many installation artists and stage performers. Miller Puckett's MAX, distributed by Cycling 74 and his open source version Pure Data (PD) has opened the door to real-time audio and video synthesis and analysis as well as controlling external equipment for theatrical performances for students and professionals working in a variety of media-based fields.  Char Davies influence on the graphical user interface and aesthetic filters for SoftImage 3D rendering software represents another interesting case illustrating how novel ideas from the new media arts have influenced the aesthetics of mass media and Hollywood cinema, as well as physical navigation of virtual environments.   Paul Kaiser’s work with choreographer Bill T. Jones and computer programmer Shelley Eshkar has produced new techniques for real-time motion capture and visual processing. Donna Cox’s visualizations of the universe have aided school children and scientist to understand phenomena like the “big bang.”   We live in a culture that tends to separate research and acquisition of new knowledge into two general camps, applied technical research and aesthetic and social research.  The HCI community has made great progress in reuniting social and technical inquiry. Inclusion of new media arts practices and research presents the opportunity to not only integrate aesthetic inquiry with the socio-technical platform of HCI, but also deliver an influential impact on domains outside of the arts, for example, investigating the relationships between metadata, multimedia content, and culture; developing novel forms and tools for interaction with data; understanding the influence of different narrative traditions on data collection and presentation and on the design of novel forms of digital representations that extend beyond the pervasive WIMP model (Jaimes & Jennings, 2004).

 

2.1. CASE STUDIES

This section presents innovative principles and interface features created by new media artists that have impact on the future development of creativity support tools within and outside of the new media arts research and practice aligned with pervasive, tangible and collaborative screen-based development methods. These principles and features support: (a) temporal, spatial and conceptual distribution across multiple interaction spaces; (b) emotion and context aware interaction to nourish participation in the creative process; (c) use of generative elements to evoke surprise and provoke user reactions; and (d) integration of applied research and production methods from art, design and technology-based fields.

 

2.1.1. NEW MEDIA ART RESEARCH IN PERVASIVE COMPUTING

Because creative activities take place in a context in which interactions with other people and artifacts are essential contributors (Harrington, 1990; Mockros & Csikszentmihályi, 1999; Fischer et al., 2005), some new media artists have started to think of creativity support tools as distributed structures that mutually reinforce both individual and social creativity. This approach implies a shift from the idea of tool—or set of tools—to the notion of a socio-technical architecture deeply interwoven with the physical environment and social fabric of local communities, based on mobile and ubiquitous computing, and focused on the transmission of data and information among different interaction spaces. This line of inquiry, which we might call pervasive creativity, appears to be a relevant context for investigating and promoting situated and distributed aspects of creativity, particularly in relation to temporal, spatial and conceptual distribution across multiple interaction spaces.

 

INNOVATIVE CREATIVITY SUPPORT TOOL CASE #1:

THE SILENCE OF THE LANDS
Developed at the Center for LifeLong Learning & Design (L3D), University of Colorado, Boulder by Elisa Giaccardi and Hal Eden in collaboration with Gianluca Sabena, Politecnico di Torino, Italy.

 

Case #1: Research Description

The Silence of the Lands (SOL) is a combined social game and an information-gathering tool, inspired by the vision and principles of the EDC (Arias et al., 2000) and the emerging use of pervasive computing. SOL supports the collection, interpretation, and visualization of Global Positioning Systems data that have been recorded directly from the members of a local community in order to address some of the societal problems in the definition of policies for the protection and enjoyment of natural quiet (Fig. 12). In its initial application, these data are "ambient sounds" and represent subjective interpretations of the "soundscape" of urban or natural settings that affect everyday life. By means of social participation and engagement, the project promotes a model for preservation, experience, and renewal that empowers the active and constructive role of local communities in the process of interpretation of natural quiet. This model embodies an approach to interaction design (viz., metadesign, see Giaccardi & Fischer, 2005) as a form of cultural intervention aimed to support creative and sustainable solutions to complex societal problems.

 

SOL enables people with different, sometimes competing visions to communicate and coordinate their different knowledge and perspectives about natural quiet. This is accomplished by using sounds (rather than words) as the conversation pieces of a social game about preservation and enjoyment of natural quiet in urban or natural settings. The goal is to create a living space inside the local community by engaging participants in the recording and mapping of their own, experienced soundscape and in the construction of an idealized, virtual one. In order to support the social dialogue and the soundscapes’ collaborative design, the project combines pervasive computing and tangible interfaces in a socio-technical architecture of disinct, but integrated interaction spaces.

 

 

Fig. 12. Overview of the socio-technical architecture of SOL: A combination of multiple interaction spaces and social practices mediated by pervasive computing and tangible interfaces: (1) satellite (GPS signal); (2) participants (sound walks); (3) mobile interface (sound catching and geo-referencing); (4) antenna (wifi connection); (5) server (database management); (6) web interface (visualization and description of individual soundscapes); (7) tangible interface (face-to-face interaction and collective soundscape interpretation).

 

By providing different entry points, promoting the different properties of each interaction space, and supporting different interaction roles over a sustained period of time, such an architecture aims to: (a) empower the creative interaction between current and future interpretations engendered by collaborative design, (b) enable participation and collaboration that fits more naturally with existing social practices and the way in which people act and interact with their local environment, and (c) support processes of social awareness and informal learning. The collective conversation produced by the collaborative design of the participants is expected to create an “affective geography” of natural quiet and transform such an abstract concept into a living entity that changes according to current and future interpretations. In this way, The Silence of the Lands not only increases sensitivity and social awareness about ambient sounds, but it also provides a tool for the visualization of collective perception and public trends.

 

Case #1: Design Principles

Experimental design principles for pervasive creativity deriving from the new media arts suggest that—in order to activate the collective stock of ideas and visions that belong to an environmental setting—tools and spaces must be woven into the existing social fabric and physical environment of the urban setting or community by means of mobile and ubiquitous computing.

 

The Silence of the Lands addresses the design of creativity support tools from an “ecological” perspective: that is, as a set of multiple tools and interaction spaces promoting the environmental setting as a “creative milieu” (Cliche et al., 2002) and composing a distributed, socio-technical infrastructure capable of mediating and linking the ideas, visions, people, places, production processes, and values that pertain to a specific environmental setting.

 

According to initial studies on public authoring (Silverston & Zoe, 2004), place-based content facilitates memory, association, and connotation. Furthermore, the shift from the single-desktop tool to multiple tools and interaction spaces (separated physically but seamlessly integrated virtually) promotes the integration of individual and social creativity. The flowing and manipulation of data throughout multiple interaction spaces (including the natural environment) enables users to promote not only spatial and temporal distribution, but also the distribution of ideas and visions (Fischer et al., 2005). Moreover, it produces what we might call information enrichment, that is, the engaging possibility of collecting and reinterpreting both individual and collective data over a sustained period of time, according to the different properties of the space with which a user is interacting and through which data is traveling (Fig. 13).

Fig. 13. Data Flow in SOL: Ambient sounds are collected from the natural environment by means of handheld devices. Each sound is linked to the user that collected it and is associated with GPS data (which determine its location in space and time). Sounds and walks (i.e. the paths followed by participants during recording sessions) are stored on the web server and visualized on the web site as individual soundscapes, one for each participant. On the web, users can access and manage their individual soundscapes (eventually modifying and changing them). They can also visualize the collective soundscape resulting and growing from the overlap of all individual soundscapes. Such a collective soundscape represents the starting point for participants in the community to collaborate on the creation of the virtual soundscape (i.e. the ideal soundscape). In the public space, both old and new participants can interact with the collective soundscape by means of tangible interfaces. Each public session produces a temporary soundscape, reflecting the understanding and creativity of the people that participated in that session. All temporary soundscapes are then composed in a historical soundscape on the basis of purposely-designed algorithms. A visualization of the historical soundscape is provided both on the web site of the project and in the public interactive space.

Case #1: Tools Development and Evaluation Methodology

This result is obtained by combining direct experience, cognitive mapping, and face-to-face interaction; that is, by combining: (a) data catching (individual sound collection and geo-referencing by mobile computing); (b) data description (individual soundscape management by web tools); and (c) data interpretation (collective interaction and social negotiation by tangible interfaces in a public space).

Sounds are geo-referenced and visualized on a GIS map as evolving matrices of audio objects aimed to reveal areas of dissension, consensus, and uncertainty by means of color-coded attributes and descriptors (Fig. 14 and Fig. 15).

 

Figg. 14-15. Web visualization of the collective soundscape and color-coded audio objects at different zoom levels.

 

2.2. NEW MEDIA ART RESEARCH IN TANGIBLE SOCIAL INTERFACES

Design thinking, which has been categorized into available design, design, and re-design, is integral to the development of meta-cognitive and meta-linguistic abilities. (New London Group, 1996)  Re-design, the most transformative of the categories, supports the generation of new knowledge from current discourse by supporting the process of inquiry, discourse and negotiation.  This process can be facilitated by convivial tools that enable “users to invest the world with their meaning, to enrich the environment with the fruits of their vision and to use them for the accomplishment of a purpose they have chosen” is a method by which to incorporate the transformative process of re-design (Illich, 1973). In our increasingly media-rich environment, marked by pervasive and ubiquitous computing and wireless devices, practices in new media culture are no longer limited to screen-based, audiovisual and interactive media content but address the wider social, urban and global context of the information environment, through novel approaches to process-based networked projects.  Many new media artists have taken on the challenge to design systems that foster “the diversity of the public actors and terrains and…develop strategies [for] articulating the new public domains that connect physical urban spaces and potential public sphere of electronic networks.” (Broeckmann, 2000) Convivial systems, such as the tangible social interface, encourage users to be actively engaged in computer-mediated open generative systems, designed to support intersubjective experiences that encourage, provoke and support debate, discussion for the construction of new knowledge and understanding in our shared worlds. Intersubjectivity is a theoretical concept used to understand how individuals can interact and produce consensual interpretations about a shared experience which can be other people, objects, or events (Thompson, 2001).

 

INNOVATIVE CREATIVITY SUPPORT TOOL CASE #2

CONSTRUCTED NARRATIVES

Principle Investigator: Pamela Jennings, Assistant Professor School of Art and the Human Computer Interaction Institute, Carnegie Mellon University.

 

Fig. 16. Potential target user audience for the Constructed Narratives project in an airport waiting lounge , Zurich, Switzerland.  Fig. 17.  Demonstration of the Constructed Narratives project  at the Kiasma Museum for Contemporary Art in Helsinki, Finland 2004.

 

Case #2: Research Description

The Constructed Narratives project is a tangible social interface (TSI) ­ a physical interface designed to enable users to collaboratively construct and negotiate their social and knowledge networks based upon their unique preferences and user profiles. This on-going project is comprised of a set of physical blocks that when connected form an open topology network.  Construction patterns in the emerging collaboratively built structure are tracked and analyzed.  This analysis  is used to seed a search for text which is revealed in a 3D screen-based navigable replica of the physical structure.  The collaboratively built construction is a socio-technical architecture, similar in goals to projects described earlier in this document, built from the development and repurposing of information technologies to explore physical environments and the social networks among people they support.  This computer supported collaborative play project is being designed for public spaces to enable dialogue between builders (users) in environments where such communicative acts are less likely to occur.   The project, and overall research inquiry, was inspired by the principle investigators countless hours watching people watch people in international airports and wondering how information technologies could be used to  make communicative connections between people who are co-located in a public space.  Though inspired by airports, the project is envisioned for any public space where a large number of people are facilitated -- from an informal science center to a Cineplex. 

 

Constructed Narratives is a platform being developed as a common-ground mediator that incorporates play and problem solving as a means for enhancing informal learning and knowledge networking in situations or about relational topics between participants that are unlikely to happen without a mediator to prompt contact.  A key principle for learning, as articulated by the National Research Council Committee on Developments in the Science of Learning, is the ability for an individual “to engage in the mental work of making inferences,” as a means to make relationships between available information for resolving an inquiry, problem, or task. (Bransford & Brown, 1999) The Constructed Narratives system prompts the builder to incorporate inferential problem solving techniques to understand and manipulate the relationships between the physical construction and the text output.  The builders’ actions of arranging and rearranging the physical blocks artifacts supports a process of empowerment where the builder negotiates structural solutions simultaneously  with her collaborator  and the topic of discussion as revealed through the semantic layer.  The builders are co-constructing a world in which they have ultimate design authority.  This is a world in which they are the very material of which that that world is made.  Topics of discussion are prompted by a text layer to the construction made visible in a 3D navigable screen-projection of the physical construction.  The semantic layer is determined by an underlying software engine that examines in real-time the emerging construction, ownership of the blocks and a few simple questions each builder answers prior to game play. Topics of the semantic layer include the relationship of each builder to other’s at the construction table, (e.g. self-identity, origins, environments of work and play and belief constructs,) or a domain specific topic such as environmental science, issues and their relationships to communities familiar to the builders. 

 

Case #2: Design Principles

The design of the Constructed Narratives block is based on George Stiny's shape grammars, a computational design methodology. Stiny’s was greatly influenced by Froebel’s Kindergarten Gifts philosophy of learning through play for his design methodology.  Constructed Narratives also references and a lineage of research based on the work of Architect Jonathon Frazer and his Universal Constructor generative system. (Stiny 1980, Frazer, 1995; Jennings, 2005b) The Constructed Narratives research project was developed by a team of Carnegie Mellon University students from eight  schools on campus including the School of Art, Human Computer Interaction Institute, Electrical and Computer Engineering, Computer Science, School of Drama, School of Design, Cognitive Science, and Information Systems Management.  Working on this project continues to be an interdisciplinary and collaborative effort requiring each research team member to quickly become adept with negotiating their discipline specific knowledge-base while learning new technology platforms for the development of the system. Several types of boundary objects have been used as tangible aids in the exploration of ideas and development of system solutions, as illustrated in figure 18 -21.  As Gerhard Fischer has noted in his research, the boundary object is a mediator that enables the exploitation of problem solving opportunities afforded by the “symmetry of ignorance” in an interdisciplinary research team (Fischer, 1999).  They provide a means to support dialogue between team members who may not be accustomed to the discipline specific language of other team members.

 

A selection of boundary objects used for the design and development of the Constructed Narratives blocks, hardware and software systems.

Fig.18. First wood block boundary objects used to understand computational implications inherent in the block shape grammar design;

Fig. 19.Cardboard prototype used for design and development;

Fig. 20 & 21. CAD design of the block using stereolithography rapid prototyping methods.

 

Development concepts from extreme programming have been used not only for software design, but experience, artifact and hardware design. Multiple iterations of sketches, scaled drawings and physical models led to the design of the Constructed Narratives block. The fluidity of this process enabled us to draw upon the technique that could best answer the design or development question of the moment. Simple drawings and quick cardboard mockups were the most useful in aiding the brainstorming process. A set of cardboard prototypes became an indispensable boundary object for project development across form factor, hardware, and software and experience design issues. Over time, various protocol system codes were etched on the cardboard prototypes for understanding the complex dynamic network.  These cardboard prototypes with layers of penciled in notes served as crucial guides for designing and testing the integrity of the software and hardware systems. The development of scaled drawings with exact measurements and scaled physical models were important to the integration of hardware components. Functional and aesthetic design requirements were negotiated through iterative brainstorming and experimentation that supported a process of adding, substituting and removing design elements to invent an optimized solution. Circuit board diagrams of various fidelities where drawn by hand and computer aided programs to understand the theoretical and applied functions of internal circuits. Whiteboards and electronic reports and updated API files were used to develop complex software communication protocols and the relationships of those protocols to hardware and block form factor design (Jennings, 2005a).

 

Case #2: Tools Development and Evaluation Methodology

The boundary objects mentioned above have lead to the first iterative development of the Constructed Narratives hardware and software interface.  The system is divided into four development areas including experience, tangible interface, and hardware and software architecture design to create an open-topology network that tracks emerging collaborative design patterns and the identification of the builder who is responsible for the placement of each block in the construction in real-time.

 

Figg. 23-27.  Screen images from the Constructed Narratives real time Virtual Build Application.  The series of images show a progressive construction with the physical blocks recorded and mirrored in a navigable virtual environment.  Recognized patterns in the physical construction are found by the pattern search engine.  The pattern data is used to form the parameters for a semantic search in a computer based lexicon of the English language.  The found words are printed onto the blocks in the virtual environment. 

 

The software architecture includes the system interfaces and communication protocol between the tangible interfaces and database resources from the builder’s profile application; the host application which keeps a dynamic graph of all data as the construction emerges; the semantic engine which applies a series of rules and iterative searches to find recognized patterns in the data that is used to define the type of word search to perform a linked lexicon of the English language, and the virtual build application which is responsible for printing the results into a 3D real-time navigable environment.   The design of the tangible block interface and enabling hardware has also been an articulated process for the design of an open-topology twisted pair network with a block interface that has 40 degrees of freedom for possible connection surfaces.  A pattern search optimization algorithm was developed to prioritize search effort and reduce search times to an acceptable response rate for a real-time interactive feedback.

 

2.3. NEW MEDIA RESEARCH IN COLLABORATIVE SCREEN-BASED APPLICATIONS

Emotion and context aware interaction is particularly important for the development and nourishing of co-creative activities among the participants in a collaborative application. Main motivational paths to co-creative activities—i.e. activities in which the construction and sharing of personally meaningful artifacts among participants is the creative result of a collective process—are emotionally driven, and often such activities are engendered by the context and collection of interactions among participants that are molded without any central guidance toward specific objectives or determined strategies (Giaccardi, 2005). This line of inquiry, which we might call affective creativity, appears to be a relevant context for investigating and promoting the emotional aspects of creativity not only in the framework of the creative practices, but also in relation to the development of creativity support tools for domains that exhibit a high degree of task uncertainty and self-organization, like for example the humanities (Pejtersen, 1980). Studies on collaborative applications for screen-based visual interaction (Giaccardi, 2004; Giaccardi, 2005) have revealed some design principles and interface features for affective creativity that are based primarily on a shift in how time, space, and environment excitations are perceived by users.

 

INNOVATIVE CREATIVITY SUPPORT TOOL CASE #3

Poietic Generator And Open Studio

The Poietic Generator and Open Studio represent instances of collaborative applications for screen-based visual interaction that exemplify, by means of different interface features, the same general principles for affective creativity.

 

Case #3: Research Description

 

The Poietic Generator (http://poietic-generator.net) is a distributed application by Olivier Auber. Its title refers to the idea of “poiesis”, which, according to Plato in the Symposium, converts anything that we consider from non-being to being. In practice, the Poietic Generator enables a large number of people across the world to participate in real time in the emergence of an ephemeral and ever-changing image. This virtual image is the result of many local images, which are adjoining and do not overlap. Participants can join or leave the collective drawing process at any time; each new connection or disconnection causes the automatic rescaling of all the local images contained in the resulting global one. Once launched, the program continuously offers a double view of the drawing process (Fig. 28). The first view shows the current state of the global image, and it is the same for each participant. The second view shows each participant an enlargement of the local image associated with him or her. Both the abstract and minimalist character of individual signs (reduced to the scale of pixels) and the non-imposition of definite forms of expression or narrative force participants’ subjective interpretation to high levels of dynamism. Collective interaction produces here an uninterrupted sequence of abstract or figurative shapes that can be observed and modified at will by any of the participants, but not globally controlled (Fig. 29).

Olivier Auber, Poietic Generator (1997-2005): Fig. 28. Local images (on the right) and resulting global image (on the left). Fig. 29. Some results of visual interaction.

 

Open Studio (http://draw.artcontext.net/) is a Java-based drawing system by Andy Deck that concurrently links users up to a single pictorial surface, and allows them to participate in the creation of a graphic animation (Fig. 30). Once connected, participants can choose whether to start interacting from scratch, by drawing on the surface of Open Studio in its current state, or to retrace the older, archived drawings. Anything a participant plays, draws, or edits on his or her applet surface is automatically shared by the other participants and added to the history of Open Studio (Fig. 31).

 

These various opportunities of interaction produce multiple and overlapped spaces of real and recorded time. Because it is impossible to identify one participant from another only on the basis of his or her drawing activity, the user does not know whether the strokes and marks appearing on the canvas are recorded or drawn in real time. Some participants will be “real” and some will be rather “phantoms”. However, regardless of when the action took place, drawing tools have been designed to be expressive and reactive to participants’ movements (speed, direction, curving, and so on). Lines, marks, and strokes convey a persistent visual and “bodily” quality that questions the nature of participants’ presence, and time linearity as well.

 

Andy Deck, Open Studio (1999): Fig. 30. View of the tools and canvas shared by all the participants. Fig. 31. Some results of visual interaction.

 

Case #3: Design Principles

Experimental design principles for affective creativity deriving from the new media arts suggest that—in order to support context and emotion aware interaction and induce co-creative activities—the computational environment must enable the dynamic embodiment of users’ activities and intentions. This means, for example, that in the Poietic Generator and Open Studio a user’s embodiment does not take place through the figurative representation of a user’s body (e.g. an emoticon or an anthropomorphic avatar), but as a visualization of participants’ spontaneous activities. As indicated by Albrechtsen et al. (Albrechtsen et al., 2001), an understanding of embodiment in terms of direct perception-action in the graphical interface is particularly important for “loosely coupled domains”, i.e. domains characterized by “a high degree of task uncertainty” and “a high degree of freedom and diversity of cognitive control among the actors involved” (viz. self-organization) like, for instance, hospitals and libraries.

 

In the cases presented here, the visual embodiment supported by the Poietic Generator and Open Studio enables users to experience the computational environment as both the world in which they can manifest and express themselves and also a source of sensory-motor and emotional excitations. Their embodiment in the computational environment takes the form of the visual language performed to express themselves and communicate with each other: their body is defined by the way in which their manifest themselves through the use of marks, colors, and other different kinds of visual elements. For example, the local image in the Poietic Generator, or the individual painting in Open Studio, constitutes the contours of a users’ body. However, it is only by acting and reacting, affecting the others and in turn being affected by the visual events produced by other participants, that users manifest themselves and identify meaningful structures. Such a relational setting provides a social and dynamic context for the evolution of the interaction process that allows participants to spontaneously negotiate their common goals and creative processes.

Design principles for affective creativity can be summarized as following (Giaccardi, 2005):

 

§         Space must be perceived and experienced as a proximal field (interface features must be designed for people to interact with each other in a “physical” and “intimate” way, rather than simply to locate them in the same or a different place);

§         Time must be perceived and experienced as a network of intentionalities (interface features must be designed for people to determine and recognize chains of actions and meaningful events over time, rather than simply to define whether they are interacting synchronously or asynchronously);

§         Spatial and temporal interface features together must enable the formation of affective bodies (flexible representations defined by the ways in which users manifest themselves) and relational settings (environments where actions are embodied by these representations);

§         The interplay between the opportunities for action provided by the information system and the external representations of cognitive activities carried out by means of the system must support the emergence of environment excitations collectively interpreted as meaningful structures through loops of perception and action among participants. See (Giaccardi, 2005) for a conceptualization of affordance, externalization and mediator in this context.

Case #3: Tools Development and Evaluation Methodology

Generally speaking, the questions raised by these applications can be summarized in: (a) can embodiment—intended not as the presence of agent-like characters on the screen, but as the level of our interdependency in perception and action with the world mediated by the tool (as just described in the Poietic Generator and Open Studio)—be the measure of a “creative milieu”? Can engagement—intended as our level of activity and motivation—be a measure of creative performance? The answers come from a phenomenon-based approach, grounded on the integration of different kind of data and descriptions (objective, subjective and empathic) as suggested by Francisco Varela and his colleagues (Varela & Shear, 1999; Roy et al., 1998). Such a methodology has been applied in the evaluation of the Poietic Generator and Open Studio as creativity support tools (Giaccardi, 2004; Giaccardi, 2005) and has produced the identification of above mentioned design principles.

 

3.   POLICY MAKING AND NEW MEDIA ARTS: PROMOTING TECHNOLOGY DEVELOPMENT, CULTURAL DEVELOPMENT, AND THE SUSTAINABILITY OF MICRO-ECONOMIES

The potential legacy and impact of new media arts on cultural development and local micro-economies (Florida, 2005) is being recognized by both creative industries and non-profit organizations.  During the past decade, several initiatives in the form of international research collaborations, interdisciplinary symposiums and conferences, and new government and foundation sponsored funding opportunities have been recognizing the important role that creativity, and in particular arts driven creativity, plays in the development of new technologies and information rich applications (Makela, et.al, 2004; Mitchell, 2003; Jennings, 2000).

 

In light of contemporary creative practices, policy-making has revised the notion of artistic creativity. Even though artistic creativity is usually described as a rules-breaking process leading to innovative visions of the individual artists, the focus is shifting to recognize the diverse and sustainable collective stock of “intangible assets” (“creative milieu”) that are created by artists and arts collectives influencing and changing the way in which the greater society incorporates information technology tools in the daily work, play, and educational activities. (Cliche et al. 2002)

 

The progressive approaches to policy-making in several of the initiatives listed in the appendix of this paper, the “Helsinki Agenda: Strategy Document on International Development of New Media Culture Policy.” and listed elsewhere in the Creativity Support Tools report have been designed to support and nourish with sufficient infrastructure the environmental settings and resources compatible to STEM based initiatives to support the development of creative innovation in the new media arts research-in-practice. (Makela, et.al. 2004; See the Helsinki Agenda principles in appendix A).

 

CONCLUSIONS

New media artists are not only “creative people”, intended as users of tools capable of producing creative work (“people that do creative things”). Often artists, as well as scientists, are also the creators of their own tools. In the case of the new media arts, they are the designers of creativity support tools for others to engage in an interactive experience or be originally creative in the production of a “co-authored” work. The same pattern of developing innovative interfaces for others to use can be found in other practices or disciplines, and new media arts add a piece to the big picture of how to develop support tools for the multiple forms of individual and social creativity expressed in different domains.

 

We encourage the HCI and software engineering community to look at new media artists not only as consumers but also as peers, and to treasure their potential contribution by establishing a thoughtful and vibrant dialogue, aimed to create long-term support for:

 

1.        Transdisciplinary educational programs focused on technology and science inquiry and innovation through creative practice;

2.        Experimental processes and practices in the new media arts that relate to the public space, discourse, and development of new technologies, tools, and transdisciplinary knowledge;

3.        Research networks bringing together local, regional and global constituents (individuals, organizations, funding agencies, corporations, etc.) to share information, form alliances, and develop best practices in new media arts research.

 

APPENDIX

 

A. The Helsinki Agenda principles

1.        Art practice and research in new media is a key generator of new knowledge in arts, science, technology, communication and education.

2.        Art practice and research in new media inform the dialogue between practitioners, researchers, creative industries and the public.

3.        New media practices have developed forms and protocols of knowledge sharing and access based on principles of openness, collaboration and creative freedom.

4.        New media practitioners can revitalize museums, archives and other heritage contents by allowing for greater public access, public renditions and imaginative readings.

5.        New media artists create transformative cultural experiences that inspire communities and individuals and expand the scope of creative industries and technology development.

6.        New media cultural practice informs larger social policies. By enabling and establishing deeper, as well as more pervasive modes of contemporary communication systems these practices lead to richer possibilities of social, inter-generational and inter-cultural communication, participation and access in our increasingly complex and multi-cultural societies.

 

B. International new media arts funding policy initiatives

 

Canada

•             Image, Text, Sound, Technology Strategic Grant (ITST)
Social Sciences and Humanities Research council of Canada Supports new media initiatives that are networks, consortia or conferences and workshops

•             Canada Council with Natural Sciences and Engineering Research Council
Provides research funding for joint initiatives by new media artists and scientists juried through both councils.

•             National Research Council (NRC) Artists-in-Residency program
Places artists into the NRC’s extensive laboratory system for two year research and creation experience

•             CANARIE Applied Research in Interactive Media (ARIM)  http://www.canarie.ca/funding/arim/guidelines.html
Supports innovative, collaborations amongst participating organizations, focus on areas of advanced networking such as grid computing, a method of using resources distributed across a network to create the tools that allow a community of users to share network based cultural expression and experiences.  

•             Hexagram
http://www.hexagram.org/spip/index_en.php3
The mission of Hexagram is to promote and support research, creation and transfer in media arts and technologies. The challenge is stimulating since the goal is to build a real bridge between artist/researchers and users.

 

United Kingdom

•              Arts and Science Research Fellowships
www.interdisciplinary.org.uk
Fellowships for artists collaboration with science and technology researchers

•              Engineering and Physical Science Research Council programs support artistic engagement with technology

•              NESTA (National Endowment for Science Technology and the Arts)
www.nesta.org.uk/insidenestal/hwf_learning.html
Lottery money allocated to support talented individuals working in innovative ways; 500 new media projects funded.

 

Australia

•             Australia Network for Art and Technology
http://www.anat.org.au/
Scientific Serendipity initiative jointly supported by Australia Council and Department of Industry Science and Technology

•             Synapse Strategy www.synapse.net.au
ANAT, ARC, university research centers, Commonwealth Science and Industrial Research Organization (CSIRO) and industry engage the nexus between art and science at the very point where these collaborations fuse complex social and political issues of the 21st century.

 

Netherlands

•             Virtual Platform
http://www.virtueelplatform.nl/article-1024.86.html
The Virtual Platform is a network for policy and cooperation in the field of new media and ‘living culture’ in the Netherlands. Its aim is to further the free development and application of ICT and free access to ICT within culture in general and in the arts in particular.

 

International

•             UNESCO 2004 Digital Art Award
http://portal.unesco.org/culture/en/ev.php-URL_ID=22405&URL_DO=DO_TOPIC&URL_SECTION=201.html
Aims to promote digital art as an innovative and artistic reflection on the information society.  It forms a special category of the UNESCO prize for the Promotion of the Arts, rewarding young emerging artists for outstanding creative achievements.

•             Creative Crossings http://www.elasticspace.com/2004/04/creative-crossings
Arts Council England, m-cult Finland, Banff Centre for the Arts, Canada to support Canadian, UK and Finnish artists, researchers, technologist who are exploring issues of ethics, meta data structure, cultural analysis, participatory design, creating alternate modalities and methods in wireless applications.

 

REFERENCES:

 

Albrechtsen, H., Andersen, H.H.K., Bødker, S., & Pejtersen, A.M. (2001) Affordances in activity theory and cognitive systems engineering, Report No. Risø-R-1287(EN), Risø National Laboratory, Centre for Human-Machine Interaction, Roskilde, Denmark.

Arias, E., Eden, H., Fischer, G., Gorman, A., & Scharff, E. (2000) “Transcending the Individual Human Mind: Creating Shared Understanding Through Collaborative Design”, ACM Transactions on Computer-Human Interaction, 7:1, pp. 84-113.

Bransford, J. Brown, A. (1999), How People Learn: Brain, Mind, Experience, and School, Panel on the Developments in the Science of Learning, National Research Council, National Academies Press.

Broeckman, A. (2000) “Public Spheres and Network Interfaces in Vectorial Elevation,” in Rafael Lozano-Hemmer (Ed.) Relational Architecture No. 4, Cultura y la Artes Mexico, Mexico City.

Cliche, D., Mitchell, R., & Wiesand, A. (2002) Creative Europe: On Governance and Management of Artistic Creativity in Europe, ERICarts, Bonn.

Cox, Donna. http://www.ncsa.uiuc.edu/People/cox/

Davies, Char. http://www.immersence.com/

Edmonds, E., Candy, L., Fell, M., Pauletto, S., & Weakley, A. (2005) "The Studio as Laboratory: Combining Creative Practice and Digital Technology Research," International Journal of Human Computer Studies (Special Issue on Creativity and Computational Support), in press.

Arts Council of Finland (2004) "New Media Arts and Culture - Policy and Practice in Dialogue," ARSIS Magazine, 3(4), available at: http://edmund.taiteenkeskustoimikunta.fi/download/ARSIS+verkko+3-04.pdf?lngDoc_id=1534

Fischer, G. (1999) “Symmetry of Ignorance, Social Creativity and Meta-Design,” in L. Candy and E. Edmonds (eds), Proceedings of the Conference "Creativity & Cognition 1999", ACM Press, pp 116-123.

Fischer, G. (2002) "Beyond 'Couch Potatoes': From Consumers to Designers and Active Contributors," FirstMonday (Peer-Reviewed Journal on the Internet), available at: http://firstmonday.org/issues/issue7_12/fischer/

Fischer, G., Giaccardi, E., Eden, H., Sugimoto, M., & Ye, Y. (2004) "Beyond Binary Choices: Integrating Individual and Social Creativity," International Journal of Human-Computer Studies (Special Issue on Creativity and Computational Support), p. (in press).

Florida, R. (2005) The Flight of the Creative Class: The New Global Competition for Talent, Harper Business, New York.

Frazer, John (1995), An Evolutionary Architecture, Architectural Association Publications, London.

Giaccardi, E. (2004) “Principles of Metadesign: Processes and Levels of Co-Creation in the New Design Space”, PhD Thesis, University of Plymouth, UK.

Giaccardi, E. (2005) "Mediators in Visual Interaction: An Analysis of the ‘Poietic Generator’ and ‘Open Studio’," Journal of Visual Languages and Computing (Special Issue on Context and Emotion Aware Visual Interaction), in press.  

Giaccardi, E., Eden, H., & Sabena, G. (2005) "The Silence of the Lands: Interactive Soundscapes for the Continuous Rebirth of Cultural Heritage," CUMULUS 2005 - Pride & Pre-Design: The Cultural Heritage and the Science of Design, Lisbon, Portugal, pp. 163-168.

Giaccardi, E., & Fischer, G. (2005) "Creativity and Evolution: A Metadesign Perspective." In Proceedings of the European Academy of Design (EAD-6) Conference, Bremen, Germany, 29-31 March, available at: http://l3d.cs.colorado.edu/~gerhard/papers/ead06.pdf

Harrington, D. (1990) "The Ecology of Human Creativity: A Psychological Perspective." In M. Runco, & R. Albert (Eds.), Theories of Creativity, Sage, Newbury Park, CA, pp. 143-169.

Kaiser, Paul. http://www.openendedgroup.com/

Makela, T., Tarkka, M., Czegledy, N., Diamond, S. Donovan, A., Ferran, B., Jennings, P., Moller M., Samola, J., Schwarz, M., Sengupta, S., Le Sourd M. (2004) “The Helsinki Agenda: Strategy Document on International Development of New Media Culture Policy”, IFACCA Online Press, available at http://www.ifacca.org/files/040916Helsinki_agenda_final.pdf

Illich, I. (1973) Tools for Conviviality, Harper & Row, New York.

Jaimes, J. & Jennings, P. (2004) “ACM Multimedia Interactive Art Program: An Introduction to the Digital Boundaries Exhibition,” in Proceedings of ACM Multimedia 2004, ACM Press, New York, pp. 979-980.

Jennings, P. (2000) New Media Art | New Funding Models, The Rockefeller Foundation, available at: www.rockfound.org/Documents/ 528/New_Media_Arts_New_Funding_Models.pdf.

Jennings, P.( 2005a) “Tangible Social Interfaces: Critical Theory, Boundary Objects and Interdisciplinary Design Methods,” in Proceedings of Creativity and Cognition 2005, ACM Press, New York, pp. 17 ­186.

Jennings, P. (2005b) “Constructed Narratives a Tangible Social Interface,” in Proceedings of Creativity and Cognition 2005, ACM Press, New York, pp. 263-266.

Mitchell, W. J., Inouye, A. S., & Blumenthal, M. S. (2003) Beyond Productivity: Information Technology, Innovation, and Creativity, The National Academies Press, Washington, DC.

Mockros, C. A., & Csikszentmihályi, M. (1999) "The Social Construction of Creative Lives." In A. Montuori, & R. E. Purser (Eds.), Social Creativity, Hamption Press, Inc., Cresskill, NJ, pp. 175-218.

Pejtersen, A.M. (1980). “Design of a Classification Scheme for Fiction Based on an Analysis of Actual User-Librarian Communication and Use of the Scheme for Control of Librarians’ Search Strategies”. In O. Harbo & L. Kajberg (Eds.) Theory and Application of Information Research, Taylor Graham, London, UK, pp. 146-159.

Puckette, Miller. http://crca.ucsd.edu/~msp/

Rokeby, David. http://homepage.mac.com/davidrokeby/home.html

Roy, J. M., Petitot, J., Pachoud, D., & Varela, F. J. (1998) Naturalizing Phenomenology: Current Issues in Contemporary Phenomenology and Cognitive Science, Stanford University Press, Stanford, CA.

Silverstone, R., & Sujon, Z. (2004) Summarizing the Urban Tapestries Social Research: Experimenting with Urban Space and ICTs, available at: http://www.proboscis.org.uk/urbantapestries/socialresearch.htm.

Sternberg, R. J. (2005) "Creativity or Creativities?" International Journal of Human Computer Studies (Special Issue on Creativity and Computational Support), in press.

Stiny, G. (1980) “Kindergarten Grammars: Designing with Froebel’s Building Gifts,” in Environment and Planning B, Vol. 7, pp. 409-462.

Thompson, E. (2001), Empathy and consciousness, in Journal of Consciousness Studies, Vol. 8, No. 5­7, pp. 1­32.

Varela, F. J., & Shear, J. (1999) "First-Person Methodologies: What, Why, How?" In F. J. Varela, & J. Shear (Eds.), The View From Within: First-Person Approaches to the Study of Consciousness, Imprint Academic, Thorverton, UK and Bowling Green, OH, pp. 1-14.

Wilson, S. (2002) Information Arts: Intersections of Art, Science and Technology, MIT Press, Cambridge, MA.