Creativity Support Tools for and by the New Media Arts Community
Human Computer Interaction Institute,
Center for LifeLong Learning and Design (L3D)
The new media arts are a particularly fertile domain for the development of creativity support tools that both supplement creative practices and contribute valuable research methodologies for other disciplines. Many parallel research concerns of new media art practitioners and researchers are found in the Human Computer Interaction and software engineering communities, including: education technology, computer supported collaborative work, data visualization, database architecture, and tools development research in pervasive computing, tangible interfaces, emotion and context aware interaction, and so on. New media arts practitioners and researchers should be regarded as valuable contributors not only as users needing better creativity support tools (CST) to enhance their own creative process, but also as the designers of experimental and innovative creativity support tools capable of providing insights and indications for:
1. Categorizing what features of the interface and what system components engender and satisfy the requirements of these multiple forms of creativity. Section 1, Creativity Support Tools development for New Media Arts Curriculum presents examples of tools that have been designed for new media art production, and potential tool features, that could empower the creative potentials of practitioners across all fields of creative production.
2. Defining the range of potential “creativities” (Sternberg, 2005) for which new technologies and tools may be developed. Section 2, Research-In- Practice, introduces works by new media artists that have developed creativity support tools that are used by a large user-base of artists and other professionals as a result of their own creative practices. Sometimes this is a primary outcome, often a residual effect from developing robust tools for one’s own art practice that can withstand a variety of user interaction and manipulation. This section also presents cases of new media arts research-in-practice that hold the similar broad reaching potentials.
3. Developing more comprehensive and appropriate evaluation methodologies grounded in the “research-in-practice” approach. Section 3, Policy Making and New Media Arts, is a brief overview of international policies for new media arts practices. Many of these policies recognize the innovative potentials for effecting research in science and technology including and beyond the arts.
This document, generated from presentations and conversations at the NSF Creativity Support Tools Workshop held in Washington D.C. in June, 2005, presents example cases of the contributions that new media art pedagogy, practice and research can provide in the development of support tools that promote the situated, affective and social aspects of creativity.
1. CST DEVELOPMENT FOR NEW MEDIA ARTS CURRICULUM
Education technology research by information technology, education, and public policy researchers has grown tremendously as the Internet has become the de facto platform for the dissemination of information, and platform for community-based collaborations. From component based authoring environments to cognitive tutors, much of the research in this area has been funded by the National Science Foundation to support STEM (Science, Technology, Engineering, and Mathematics) disciplines. Our tendency to separate the STEM disciplines from creativity, culture and humanities sets forth a pattern of missed opportunities to develop new technologies that could help to solve research problems for disciplines that present alternative perspectives on data acquisition, analysis, and manipulation. In recent years new media arts practices have entered the mainstream. Many international universities are actively creating new media arts programs geared towards artists that both use and create technologies (Jaimes & Jennings, 2004). NSF has recently funded a few research initiatives to develop curriculum modules that integrate computer science and new media arts (Integration Digital Media Curriculum Development (NSF DUE- 0340969) and the Digital Media Curriculum Development Project (NSF DUE-0127280). Pedagogical practices in new media arts are based on problem solving through open exploration of conceptual ideas that sometimes conform to, but mostly challenge, the intended functionalities of technology-based tools. This pedagogical practice of open tools usage sits in contrast to the typical computer science curriculum that is based on learning through constrained problem solving. In the later case, students are typically given assignments that have a limited number of acceptable solution variations. The open and indeterminate nature of new media arts practice presents unique opportunities for developers of creativity support tools to incorporate complex programming abstractions, relational databases, integrated search functionalities, and scaffolded interfaces as a means to create more flexible creativity support tools for new media arts students and practitioners. (Bransford & Brown, 1999)
1.1. NEW MEDIA ARTS CURRICULUM CASE #1:
Design of Appropriate Tools Features for Open Ended Creative Production
Fig. 1 3. Human Computer Interaction
and Interaction Design students working with electronics in the Physical Computing:
Wearables offered by the
Students in the Introduction
to New Media Arts class at
1.2. NEW MEDIA ARTS CURRICULUM CASE #2:
Tools to Support Mixed Skills Level Classes
Upon completion of the
compulsory introductory classes in new media arts, the
The broad range of students’
skills in these classes presents a few creativity support tool development
opportunities The first is the design of tools that can accommodate a range of
skills from beginner to advance with proper scaffold features and
functionalities for each user level. The second is the development of
multi-level help modules that can assist students and instructors in
facilitating a broad range of questions and possible solutions. For example, it
would be wonderful to have a cognitive tutor that notices that a student with
beginning skills levels continuously gets a syntax error in her code which she
is having difficulty correcting. After
analyzing and finding no errors in the code structure, the help module
recommends that she check her spelling.
Another example if a student wants multiple sprites on the stage to
design a Tetris game like interface should he be instructed to create each
sprite as a unique entity a long and tedious process that uses a minimum of
code, or should he delve into property lists and object oriented programming
far more efficient but requires a basic knowledge of programming. Or, should he be encouraged to implement
sorting and advanced search algorithms that allow him to develop an intelligent
self-playing game. These are not
hypothetical examples, but the range of students I have seen in new media art
classes taught in the
1.3 NEW MEDIA ARTS CURRICULUM CASE #3:
Holistic Presentation Tools
Students taking interaction
design classes are often required to work in groups to solve a particular
design /human computer interaction interface problem. They are typically given a hypothetical situation
to ideate, visualize, and present. Students
are encouraged to select very creative means to explore and represent their
ideas including text, graphics, photographs, video, theater, 3D animation,
sound, programming examples, etc…. It
is the student team’s task to come up with methods to formalize their needs
analysis research, aggregate and negotiate their ideas and produce a visual
or functional demo of the interface, application, or gadget. Unfortunately, all of the effort in this work
is often reduced to a power point presentation for class-wide discussion and
critique. Here is a tremendous opportunity
to develop a tool, or set of interoperable tools to support all the requirements
as described above without forcing the final presentation to be formatted
into bullet point lists.
Fig. 4 7. Storyboard excerpts from Critical
Interaction Design class. Students
involved were from Human Computer Interaction, Architecture,
Fig. 4 7. Storyboard excerpts from Critical
Interaction Design class. Students
involved were from Human Computer Interaction, Architecture,
Creativity Support Tools for and by the New Media Arts Community
The new media arts are characterized by what is usually referred to as “research-in-practice”: an experimentalism and reflexivity that bring artists to link creative research and practice in a “highly responsive, iterative process where new insights are fed back quickly into the development process” (Candy & Edmonds, 2005). The new media arts express a risk-taking and subversive attitude, ultimately seeking cultural acts through which to provide society with entry points for change (for example, in the very definition of what is creativity and how it can be supported). Stephen Wilson’s “Information Arts: Intersections of Art, Science and Technology” (Wilson, 2002) is an encyclopedic archive of arts, technology and science collaborations exploring new ideas that challenge and contribute alternative perspectives on research practice and tools development across nearly every category of science and technology research from microbiology to nanotechnology to augmented reality.
Examples of innovative tools created within the new media arts community include, David Rokeby's "Very Nervous System", a computer vision system used by many installation artists and stage performers. Miller Puckett's MAX, distributed by Cycling 74 and his open source version Pure Data (PD) has opened the door to real-time audio and video synthesis and analysis as well as controlling external equipment for theatrical performances for students and professionals working in a variety of media-based fields. Char Davies influence on the graphical user interface and aesthetic filters for SoftImage 3D rendering software represents another interesting case illustrating how novel ideas from the new media arts have influenced the aesthetics of mass media and Hollywood cinema, as well as physical navigation of virtual environments. Paul Kaiser’s work with choreographer Bill T. Jones and computer programmer Shelley Eshkar has produced new techniques for real-time motion capture and visual processing. Donna Cox’s visualizations of the universe have aided school children and scientist to understand phenomena like the “big bang.”
We live in a culture that tends to separate research and acquisition of new knowledge into two general camps, applied technical research and aesthetic and social research. The HCI community has made great progress in reuniting social and technical inquiry. Inclusion of new media arts practices and research presents the opportunity to not only integrate aesthetic inquiry with the socio-technical platform of HCI, but also deliver an influential impact on domains outside of the arts, for example, investigating the relationships between metadata, multimedia content, and culture; developing novel forms and tools for interaction with data; understanding the influence of different narrative traditions on data collection and presentation and on the design of novel forms of digital representations that extend beyond the pervasive WIMP model (Jaimes & Jennings, 2004).
2.1. CASE STUDIES
This section presents innovative principles and interface features created by new media artists that have impact on the future development of creativity support tools within and outside of the new media arts research and practice aligned with pervasive, tangible and collaborative screen-based development methods. These principles and features support: (a) temporal, spatial and conceptual distribution across multiple interaction spaces; (b) emotion and context aware interaction to nourish participation in the creative process; (c) use of generative elements to evoke surprise and provoke user reactions; and (d) integration of applied research and production methods from art, design and technology-based fields.
2.1.1. NEW MEDIA ART RESEARCH IN PERVASIVE COMPUTING
Because creative activities take place in a context in which interactions with other people and artifacts are essential contributors (Harrington, 1990; Mockros & Csikszentmihályi, 1999; Fischer et al., 2005), some new media artists have started to think of creativity support tools as distributed structures that mutually reinforce both individual and social creativity. This approach implies a shift from the idea of tool—or set of tools—to the notion of a socio-technical architecture deeply interwoven with the physical environment and social fabric of local communities, based on mobile and ubiquitous computing, and focused on the transmission of data and information among different interaction spaces. This line of inquiry, which we might call pervasive creativity, appears to be a relevant context for investigating and promoting situated and distributed aspects of creativity, particularly in relation to temporal, spatial and conceptual distribution across multiple interaction spaces.
INNOVATIVE CREATIVITY SUPPORT TOOL CASE #1:
THE SILENCE OF THE LANDS
Developed at the Center for LifeLong Learning & Design (L3D),
Case #1: Research Description
The Silence of the Lands (SOL) is a combined social game and an information-gathering tool, inspired by the vision and principles of the EDC (Arias et al., 2000) and the emerging use of pervasive computing. SOL supports the collection, interpretation, and visualization of Global Positioning Systems data that have been recorded directly from the members of a local community in order to address some of the societal problems in the definition of policies for the protection and enjoyment of natural quiet (Fig. 12). In its initial application, these data are "ambient sounds" and represent subjective interpretations of the "soundscape" of urban or natural settings that affect everyday life. By means of social participation and engagement, the project promotes a model for preservation, experience, and renewal that empowers the active and constructive role of local communities in the process of interpretation of natural quiet. This model embodies an approach to interaction design (viz., metadesign, see Giaccardi & Fischer, 2005) as a form of cultural intervention aimed to support creative and sustainable solutions to complex societal problems.
SOL enables people with different, sometimes competing visions to communicate and coordinate their different knowledge and perspectives about natural quiet. This is accomplished by using sounds (rather than words) as the conversation pieces of a social game about preservation and enjoyment of natural quiet in urban or natural settings. The goal is to create a living space inside the local community by engaging participants in the recording and mapping of their own, experienced soundscape and in the construction of an idealized, virtual one. In order to support the social dialogue and the soundscapes’ collaborative design, the project combines pervasive computing and tangible interfaces in a socio-technical architecture of disinct, but integrated interaction spaces.
12. Overview of the socio-technical architecture of SOL: A combination of
multiple interaction spaces and social practices mediated by pervasive computing
and tangible interfaces: (1) satellite (GPS signal); (2) participants (sound
walks); (3) mobile interface (sound catching and geo-referencing); (4) antenna
(wifi connection); (5) server (database management); (6) web interface (visualization
and description of individual soundscapes); (7) tangible interface (face-to-face
interaction and collective soundscape interpretation).
Fig. 12. Overview of the socio-technical architecture of SOL: A combination of multiple interaction spaces and social practices mediated by pervasive computing and tangible interfaces: (1) satellite (GPS signal); (2) participants (sound walks); (3) mobile interface (sound catching and geo-referencing); (4) antenna (wifi connection); (5) server (database management); (6) web interface (visualization and description of individual soundscapes); (7) tangible interface (face-to-face interaction and collective soundscape interpretation).
By providing different entry points, promoting the different properties of each interaction space, and supporting different interaction roles over a sustained period of time, such an architecture aims to: (a) empower the creative interaction between current and future interpretations engendered by collaborative design, (b) enable participation and collaboration that fits more naturally with existing social practices and the way in which people act and interact with their local environment, and (c) support processes of social awareness and informal learning. The collective conversation produced by the collaborative design of the participants is expected to create an “affective geography” of natural quiet and transform such an abstract concept into a living entity that changes according to current and future interpretations. In this way, The Silence of the Lands not only increases sensitivity and social awareness about ambient sounds, but it also provides a tool for the visualization of collective perception and public trends.
Case #1: Design Principles
Experimental design principles for pervasive creativity deriving from the new media arts suggest that—in order to activate the collective stock of ideas and visions that belong to an environmental setting—tools and spaces must be woven into the existing social fabric and physical environment of the urban setting or community by means of mobile and ubiquitous computing.
The Silence of the Lands addresses the design of creativity support tools from an “ecological” perspective: that is, as a set of multiple tools and interaction spaces promoting the environmental setting as a “creative milieu” (Cliche et al., 2002) and composing a distributed, socio-technical infrastructure capable of mediating and linking the ideas, visions, people, places, production processes, and values that pertain to a specific environmental setting.
According to initial studies on public authoring (Silverston & Zoe, 2004), place-based content facilitates memory, association, and connotation. Furthermore, the shift from the single-desktop tool to multiple tools and interaction spaces (separated physically but seamlessly integrated virtually) promotes the integration of individual and social creativity. The flowing and manipulation of data throughout multiple interaction spaces (including the natural environment) enables users to promote not only spatial and temporal distribution, but also the distribution of ideas and visions (Fischer et al., 2005). Moreover, it produces what we might call information enrichment, that is, the engaging possibility of collecting and reinterpreting both individual and collective data over a sustained period of time, according to the different properties of the space with which a user is interacting and through which data is traveling (Fig. 13).
Fig. 13. Data Flow in SOL: Ambient sounds are collected from the natural
environment by means of handheld devices. Each sound is linked to the user
that collected it and is associated with GPS data (which determine its location
in space and time). Sounds and walks (i.e. the paths followed by participants
during recording sessions) are stored on the web server and visualized on
the web site as individual soundscapes, one for each participant. On the web,
users can access and manage their individual soundscapes (eventually modifying
and changing them). They can also visualize the collective soundscape resulting
and growing from the overlap of all individual soundscapes. Such a collective
soundscape represents the starting point for participants in the community
to collaborate on the creation of the virtual soundscape (i.e. the ideal soundscape).
In the public space, both old and new participants can interact with the collective
soundscape by means of tangible interfaces. Each public session produces a
temporary soundscape, reflecting the understanding and creativity of the people
that participated in that session. All temporary soundscapes are then composed
in a historical soundscape on the basis of purposely-designed algorithms.
A visualization of the historical soundscape is provided both on the web site
of the project and in the public interactive space.
Fig. 13. Data Flow in SOL: Ambient sounds are collected from the natural environment by means of handheld devices. Each sound is linked to the user that collected it and is associated with GPS data (which determine its location in space and time). Sounds and walks (i.e. the paths followed by participants during recording sessions) are stored on the web server and visualized on the web site as individual soundscapes, one for each participant. On the web, users can access and manage their individual soundscapes (eventually modifying and changing them). They can also visualize the collective soundscape resulting and growing from the overlap of all individual soundscapes. Such a collective soundscape represents the starting point for participants in the community to collaborate on the creation of the virtual soundscape (i.e. the ideal soundscape). In the public space, both old and new participants can interact with the collective soundscape by means of tangible interfaces. Each public session produces a temporary soundscape, reflecting the understanding and creativity of the people that participated in that session. All temporary soundscapes are then composed in a historical soundscape on the basis of purposely-designed algorithms. A visualization of the historical soundscape is provided both on the web site of the project and in the public interactive space.
Case #1: Tools Development and Evaluation Methodology
This result is obtained by combining direct experience, cognitive mapping, and face-to-face interaction; that is, by combining: (a) data catching (individual sound collection and geo-referencing by mobile computing); (b) data description (individual soundscape management by web tools); and (c) data interpretation (collective interaction and social negotiation by tangible interfaces in a public space).
Sounds are geo-referenced and visualized on a GIS map as evolving matrices of audio objects aimed to reveal areas of dissension, consensus, and uncertainty by means of color-coded attributes and descriptors (Fig. 14 and Fig. 15).
Figg. 14-15. Web visualization of the collective soundscape and color-coded audio objects at different zoom levels.
2.2. NEW MEDIA ART RESEARCH IN TANGIBLE SOCIAL INTERFACES
Design thinking, which has been categorized into available design, design, and re-design, is integral to the development of meta-cognitive and meta-linguistic abilities. (New London Group, 1996) Re-design, the most transformative of the categories, supports the generation of new knowledge from current discourse by supporting the process of inquiry, discourse and negotiation. This process can be facilitated by convivial tools that enable “users to invest the world with their meaning, to enrich the environment with the fruits of their vision and to use them for the accomplishment of a purpose they have chosen” is a method by which to incorporate the transformative process of re-design (Illich, 1973). In our increasingly media-rich environment, marked by pervasive and ubiquitous computing and wireless devices, practices in new media culture are no longer limited to screen-based, audiovisual and interactive media content but address the wider social, urban and global context of the information environment, through novel approaches to process-based networked projects. Many new media artists have taken on the challenge to design systems that foster “the diversity of the public actors and terrains and…develop strategies [for] articulating the new public domains that connect physical urban spaces and potential public sphere of electronic networks.” (Broeckmann, 2000) Convivial systems, such as the tangible social interface, encourage users to be actively engaged in computer-mediated open generative systems, designed to support intersubjective experiences that encourage, provoke and support debate, discussion for the construction of new knowledge and understanding in our shared worlds. Intersubjectivity is a theoretical concept used to understand how individuals can interact and produce consensual interpretations about a shared experience which can be other people, objects, or events (Thompson, 2001).
INNOVATIVE CREATIVITY SUPPORT TOOL CASE #2
Pamela Jennings, Assistant Professor School of Art and the Human Computer
Fig. 16. Potential target user audience
for the Constructed Narratives project in an airport waiting lounge ,
Case #2: Research Description
The Constructed Narratives project is a tangible social interface (TSI) a physical interface designed to enable users to collaboratively construct and negotiate their social and knowledge networks based upon their unique preferences and user profiles. This on-going project is comprised of a set of physical blocks that when connected form an open topology network. Construction patterns in the emerging collaboratively built structure are tracked and analyzed. This analysis is used to seed a search for text which is revealed in a 3D screen-based navigable replica of the physical structure. The collaboratively built construction is a socio-technical architecture, similar in goals to projects described earlier in this document, built from the development and repurposing of information technologies to explore physical environments and the social networks among people they support. This computer supported collaborative play project is being designed for public spaces to enable dialogue between builders (users) in environments where such communicative acts are less likely to occur. The project, and overall research inquiry, was inspired by the principle investigators countless hours watching people watch people in international airports and wondering how information technologies could be used to make communicative connections between people who are co-located in a public space. Though inspired by airports, the project is envisioned for any public space where a large number of people are facilitated -- from an informal science center to a Cineplex.
Constructed Narratives is a platform being developed as a common-ground mediator that incorporates play and problem solving as a means for enhancing informal learning and knowledge networking in situations or about relational topics between participants that are unlikely to happen without a mediator to prompt contact. A key principle for learning, as articulated by the National Research Council Committee on Developments in the Science of Learning, is the ability for an individual “to engage in the mental work of making inferences,” as a means to make relationships between available information for resolving an inquiry, problem, or task. (Bransford & Brown, 1999) The Constructed Narratives system prompts the builder to incorporate inferential problem solving techniques to understand and manipulate the relationships between the physical construction and the text output. The builders’ actions of arranging and rearranging the physical blocks artifacts supports a process of empowerment where the builder negotiates structural solutions simultaneously with her collaborator and the topic of discussion as revealed through the semantic layer. The builders are co-constructing a world in which they have ultimate design authority. This is a world in which they are the very material of which that that world is made. Topics of discussion are prompted by a text layer to the construction made visible in a 3D navigable screen-projection of the physical construction. The semantic layer is determined by an underlying software engine that examines in real-time the emerging construction, ownership of the blocks and a few simple questions each builder answers prior to game play. Topics of the semantic layer include the relationship of each builder to other’s at the construction table, (e.g. self-identity, origins, environments of work and play and belief constructs,) or a domain specific topic such as environmental science, issues and their relationships to communities familiar to the builders.
Case #2: Design Principles
The design of the Constructed
Narratives block is based on George Stiny's shape grammars, a computational
design methodology. Stiny’s was greatly influenced by Froebel’s Kindergarten
Gifts philosophy of learning through play for his design methodology. Constructed
Narratives also references and a lineage of research based on the work
of Architect Jonathon Frazer and his Universal Constructor generative
system. (Stiny 1980, Frazer, 1995; Jennings,
2005b) The Constructed Narratives
research project was developed by a team of Carnegie Mellon University students
from eight schools on campus including
the School of Art, Human Computer Interaction Institute, Electrical and Computer
Engineering, Computer Science, School of Drama, School of Design, Cognitive
Science, and Information Systems Management. Working on this project continues to be an interdisciplinary
and collaborative effort requiring each research team member to quickly become
A selection of boundary objects used for the design and development of the Constructed Narratives blocks, hardware and software systems.
Fig.18. First wood block boundary objects used to understand computational implications inherent in the block shape grammar design;
Fig. 19.Cardboard prototype used for design and development;
Fig. 20 & 21. CAD design of the block using stereolithography rapid prototyping methods.
Development concepts from extreme programming have
been used not only for software design, but experience, artifact and hardware
design. Multiple iterations of sketches, scaled drawings and physical models
led to the design of the Constructed
Narratives block. The fluidity of this process enabled us to draw upon the
technique that could best answer the design or development question of the
moment. Simple drawings and quick cardboard mockups were the most useful in
aiding the brainstorming process. A set of cardboard prototypes became an
indispensable boundary object for project development across form factor,
hardware, and software and experience design issues. Over time, various
protocol system codes were etched on the cardboard prototypes for understanding
the complex dynamic network. These
cardboard prototypes with layers of penciled in notes served as crucial guides
for designing and testing the integrity of the software and hardware systems.
The development of scaled drawings with exact measurements and scaled physical
models were important to the integration of hardware components. Functional and
aesthetic design requirements were negotiated through iterative brainstorming
and experimentation that supported a process of adding, substituting and
removing design elements to invent an optimized solution. Circuit board
diagrams of various fidelities where drawn by hand and computer aided programs
to understand the theoretical and applied functions of internal circuits.
Whiteboards and electronic reports and updated API files were used to develop
complex software communication protocols and the relationships of those
protocols to hardware and block form factor design (
Case #2: Tools Development and Evaluation Methodology
The boundary objects mentioned above have lead to the first iterative development of the Constructed Narratives hardware and software interface. The system is divided into four development areas including experience, tangible interface, and hardware and software architecture design to create an open-topology network that tracks emerging collaborative design patterns and the identification of the builder who is responsible for the placement of each block in the construction in real-time.
Figg. 23-27. Screen
images from the Constructed Narratives real time Virtual Build Application.
The series of images show a progressive construction with the physical
blocks recorded and mirrored in a navigable virtual environment.
Recognized patterns in the physical construction are found by the pattern
search engine. The pattern data is
used to form the parameters for a semantic search in a computer based lexicon
of the English language. The found
words are printed onto the blocks in the virtual environment.
Figg. 23-27. Screen images from the Constructed Narratives real time Virtual Build Application. The series of images show a progressive construction with the physical blocks recorded and mirrored in a navigable virtual environment. Recognized patterns in the physical construction are found by the pattern search engine. The pattern data is used to form the parameters for a semantic search in a computer based lexicon of the English language. The found words are printed onto the blocks in the virtual environment.
The software architecture includes the system interfaces and communication protocol between the tangible interfaces and database resources from the builder’s profile application; the host application which keeps a dynamic graph of all data as the construction emerges; the semantic engine which applies a series of rules and iterative searches to find recognized patterns in the data that is used to define the type of word search to perform a linked lexicon of the English language, and the virtual build application which is responsible for printing the results into a 3D real-time navigable environment. The design of the tangible block interface and enabling hardware has also been an articulated process for the design of an open-topology twisted pair network with a block interface that has 40 degrees of freedom for possible connection surfaces. A pattern search optimization algorithm was developed to prioritize search effort and reduce search times to an acceptable response rate for a real-time interactive feedback.
2.3. NEW MEDIA RESEARCH IN COLLABORATIVE SCREEN-BASED APPLICATIONS
Emotion and context aware interaction is particularly important for the development and nourishing of co-creative activities among the participants in a collaborative application. Main motivational paths to co-creative activities—i.e. activities in which the construction and sharing of personally meaningful artifacts among participants is the creative result of a collective process—are emotionally driven, and often such activities are engendered by the context and collection of interactions among participants that are molded without any central guidance toward specific objectives or determined strategies (Giaccardi, 2005). This line of inquiry, which we might call affective creativity, appears to be a relevant context for investigating and promoting the emotional aspects of creativity not only in the framework of the creative practices, but also in relation to the development of creativity support tools for domains that exhibit a high degree of task uncertainty and self-organization, like for example the humanities (Pejtersen, 1980). Studies on collaborative applications for screen-based visual interaction (Giaccardi, 2004; Giaccardi, 2005) have revealed some design principles and interface features for affective creativity that are based primarily on a shift in how time, space, and environment excitations are perceived by users.
INNOVATIVE CREATIVITY SUPPORT TOOL CASE #3
Poietic Generator And Open Studio
The Poietic Generator and Open Studio represent instances of collaborative applications for screen-based visual interaction that exemplify, by means of different interface features, the same general principles for affective creativity.
Case #3: Research Description
The Poietic Generator (http://poietic-generator.net) is a distributed application by Olivier Auber. Its title refers to the idea of “poiesis”, which, according to Plato in the Symposium, converts anything that we consider from non-being to being. In practice, the Poietic Generator enables a large number of people across the world to participate in real time in the emergence of an ephemeral and ever-changing image. This virtual image is the result of many local images, which are adjoining and do not overlap. Participants can join or leave the collective drawing process at any time; each new connection or disconnection causes the automatic rescaling of all the local images contained in the resulting global one. Once launched, the program continuously offers a double view of the drawing process (Fig. 28). The first view shows the current state of the global image, and it is the same for each participant. The second view shows each participant an enlargement of the local image associated with him or her. Both the abstract and minimalist character of individual signs (reduced to the scale of pixels) and the non-imposition of definite forms of expression or narrative force participants’ subjective interpretation to high levels of dynamism. Collective interaction produces here an uninterrupted sequence of abstract or figurative shapes that can be observed and modified at will by any of the participants, but not globally controlled (Fig. 29).
Olivier Auber, Poietic Generator (1997-2005):
Fig. 28. Local images (on the right) and resulting global image (on the left).
Fig. 29. Some results of visual interaction.
Open Studio (http://draw.artcontext.net/) is a Java-based drawing system by Andy Deck that concurrently links users up to a single pictorial surface, and allows them to participate in the creation of a graphic animation (Fig. 30). Once connected, participants can choose whether to start interacting from scratch, by drawing on the surface of Open Studio in its current state, or to retrace the older, archived drawings. Anything a participant plays, draws, or edits on his or her applet surface is automatically shared by the other participants and added to the history of Open Studio (Fig. 31).
These various opportunities of interaction produce multiple
and overlapped spaces of real and recorded time. Because it is impossible
to identify one participant from another only on the basis of his or her drawing
activity, the user does not know whether the strokes and marks appearing on
the canvas are recorded or drawn in real time. Some participants will be “real”
and some will be rather “phantoms”. However, regardless of when the action
took place, drawing tools have been designed to be expressive and reactive
to participants’ movements (speed, direction, curving, and so on). Lines,
marks, and strokes convey a persistent visual and “bodily” quality that questions
the nature of participants’ presence, and time linearity as well.
Andy Deck, Open Studio (1999):
Fig. 30. View of the tools and canvas shared by all the participants.
Fig. 31. Some results of visual interaction.
Case #3: Design Principles
Experimental design principles for affective creativity deriving from the new media arts suggest that—in order to support context and emotion aware interaction and induce co-creative activities—the computational environment must enable the dynamic embodiment of users’ activities and intentions. This means, for example, that in the Poietic Generator and Open Studio a user’s embodiment does not take place through the figurative representation of a user’s body (e.g. an emoticon or an anthropomorphic avatar), but as a visualization of participants’ spontaneous activities. As indicated by Albrechtsen et al. (Albrechtsen et al., 2001), an understanding of embodiment in terms of direct perception-action in the graphical interface is particularly important for “loosely coupled domains”, i.e. domains characterized by “a high degree of task uncertainty” and “a high degree of freedom and diversity of cognitive control among the actors involved” (viz. self-organization) like, for instance, hospitals and libraries.
In the cases presented here, the visual embodiment supported by the Poietic Generator and Open Studio enables users to experience the computational environment as both the world in which they can manifest and express themselves and also a source of sensory-motor and emotional excitations. Their embodiment in the computational environment takes the form of the visual language performed to express themselves and communicate with each other: their body is defined by the way in which their manifest themselves through the use of marks, colors, and other different kinds of visual elements. For example, the local image in the Poietic Generator, or the individual painting in Open Studio, constitutes the contours of a users’ body. However, it is only by acting and reacting, affecting the others and in turn being affected by the visual events produced by other participants, that users manifest themselves and identify meaningful structures. Such a relational setting provides a social and dynamic context for the evolution of the interaction process that allows participants to spontaneously negotiate their common goals and creative processes.
Design principles for affective creativity can be summarized as following (Giaccardi, 2005):
§ Space must be perceived and experienced as a proximal field (interface features must be designed for people to interact with each other in a “physical” and “intimate” way, rather than simply to locate them in the same or a different place);
§ Time must be perceived and experienced as a network of intentionalities (interface features must be designed for people to determine and recognize chains of actions and meaningful events over time, rather than simply to define whether they are interacting synchronously or asynchronously);
§ Spatial and temporal interface features together must enable the formation of affective bodies (flexible representations defined by the ways in which users manifest themselves) and relational settings (environments where actions are embodied by these representations);
between the opportunities for action provided by the information system and the
external representations of cognitive activities carried out by means of the
system must support the emergence of environment excitations collectively
interpreted as meaningful structures
through loops of perception and action among participants. See (Giaccardi,
2005) for a conceptualization of affordance, externalization and mediator in
Generally speaking, the questions raised by these applications can be summarized in: (a) can embodiment—intended not as the presence of agent-like characters on the screen, but as the level of our interdependency in perception and action with the world mediated by the tool (as just described in the Poietic Generator and Open Studio)—be the measure of a “creative milieu”? Can engagement—intended as our level of activity and motivation—be a measure of creative performance? The answers come from a phenomenon-based approach, grounded on the integration of different kind of data and descriptions (objective, subjective and empathic) as suggested by Francisco Varela and his colleagues (Varela & Shear, 1999; Roy et al., 1998). Such a methodology has been applied in the evaluation of the Poietic Generator and Open Studio as creativity support tools (Giaccardi, 2004; Giaccardi, 2005) and has produced the identification of above mentioned design principles.
POLICY MAKING AND NEW MEDIA ARTS: PROMOTING TECHNOLOGY
DEVELOPMENT, CULTURAL DEVELOPMENT, AND THE SUSTAINABILITY OF MICRO-ECONOMIES
POLICY MAKING AND NEW MEDIA ARTS: PROMOTING TECHNOLOGY DEVELOPMENT, CULTURAL DEVELOPMENT, AND THE SUSTAINABILITY OF MICRO-ECONOMIES
The potential legacy and
impact of new media arts on cultural development and local micro-economies (
In light of contemporary creative practices, policy-making has revised the notion of artistic creativity. Even though artistic creativity is usually described as a rules-breaking process leading to innovative visions of the individual artists, the focus is shifting to recognize the diverse and sustainable collective stock of “intangible assets” (“creative milieu”) that are created by artists and arts collectives influencing and changing the way in which the greater society incorporates information technology tools in the daily work, play, and educational activities. (Cliche et al. 2002)
The progressive approaches to policy-making in several of the initiatives listed in the appendix of this paper, the “Helsinki Agenda: Strategy Document on International Development of New Media Culture Policy.” and listed elsewhere in the Creativity Support Tools report have been designed to support and nourish with sufficient infrastructure the environmental settings and resources compatible to STEM based initiatives to support the development of creative innovation in the new media arts research-in-practice. (Makela, et.al. 2004; See the Helsinki Agenda principles in appendix A).
New media artists are not only “creative people”, intended as users of tools capable of producing creative work (“people that do creative things”). Often artists, as well as scientists, are also the creators of their own tools. In the case of the new media arts, they are the designers of creativity support tools for others to engage in an interactive experience or be originally creative in the production of a “co-authored” work. The same pattern of developing innovative interfaces for others to use can be found in other practices or disciplines, and new media arts add a piece to the big picture of how to develop support tools for the multiple forms of individual and social creativity expressed in different domains.
We encourage the HCI and software engineering community to look at new media artists not only as consumers but also as peers, and to treasure their potential contribution by establishing a thoughtful and vibrant dialogue, aimed to create long-term support for:
1. Transdisciplinary educational programs focused on technology and science inquiry and innovation through creative practice;
2. Experimental processes and practices in the new media arts that relate to the public space, discourse, and development of new technologies, tools, and transdisciplinary knowledge;
3. Research networks bringing together local, regional and global constituents (individuals, organizations, funding agencies, corporations, etc.) to share information, form alliances, and develop best practices in new media arts research.
1. Art practice and research in new media is a key generator of new knowledge in arts, science, technology, communication and education.
2. Art practice and research in new media inform the dialogue between practitioners, researchers, creative industries and the public.
3. New media practices have developed forms and protocols of knowledge sharing and access based on principles of openness, collaboration and creative freedom.
4. New media practitioners can revitalize museums, archives and other heritage contents by allowing for greater public access, public renditions and imaginative readings.
5. New media artists create transformative cultural experiences that inspire communities and individuals and expand the scope of creative industries and technology development.
6. New media cultural practice informs larger social policies. By enabling and establishing deeper, as well as more pervasive modes of contemporary communication systems these practices lead to richer possibilities of social, inter-generational and inter-cultural communication, participation and access in our increasingly complex and multi-cultural societies.
B. International new media arts funding policy initiatives
Sound, Technology Strategic Grant (ITST)
Social Sciences and Humanities Research council of Canada Supports new media initiatives that are networks, consortia or conferences and workshops
Provides research funding for joint initiatives by new media artists and scientists juried through both councils.
Council (NRC) Artists-in-Residency program
Places artists into the NRC’s extensive laboratory system for two year research and creation experience
Applied Research in Interactive Media (ARIM) http://www.canarie.ca/funding/arim/guidelines.html
Supports innovative, collaborations amongst participating organizations, focus on areas of advanced networking such as grid computing, a method of using resources distributed across a network to create the tools that allow a community of users to share network based cultural expression and experiences.
The mission of Hexagram is to promote and support research, creation and transfer in media arts and technologies. The challenge is stimulating since the goal is to build a real bridge between artist/researchers and users.
Arts and Science
Fellowships for artists collaboration with science and technology researchers
• Engineering and Physical Science Research Council programs support artistic engagement with technology
(National Endowment for Science Technology and the Arts)
Lottery money allocated to support talented individuals working in innovative ways; 500 new media projects funded.
Network for Art and Technology
Scientific Serendipity initiative jointly supported by Australia Council and Department of Industry Science and Technology
ANAT, ARC, university research centers, Commonwealth Science and Industrial Research Organization (CSIRO) and industry engage the nexus between art and science at the very point where these collaborations fuse complex social and political issues of the 21st century.
The Virtual Platform is a network for policy and cooperation in the field of new media and ‘living culture’ in the
Digital Art Award
Aims to promote digital art as an innovative and artistic reflection on the information society. It forms a special category of the UNESCO prize for the Promotion of the Arts, rewarding young emerging artists for outstanding creative achievements.
Arts Council England, m-cult Finland, Banff Centre for the Arts, Canada to support Canadian, UK and Finnish artists, researchers, technologist who are exploring issues of ethics, meta data structure, cultural analysis, participatory design, creating alternate modalities and methods in wireless applications.
Albrechtsen, H., Andersen, H.H.K., Bødker, S., & Pejtersen, A.M.
(2001) Affordances in activity theory and
cognitive systems engineering, Report No. Risø-R-1287(EN), Risø National
Laboratory, Centre for Human-Machine Interaction,
Bransford, J. Brown, A. (1999), How People Learn: Brain, Mind, Experience, and School, Panel on the Developments in the Science of Learning, National Research Council, National Academies Press.
Broeckman, A. (2000) “Public Spheres and Network Interfaces in
Vectorial Elevation,” in Rafael Lozano-Hemmer (Ed.) Relational Architecture No. 4, Cultura y la Artes
Cliche, D., Mitchell, R., & Wiesand, A. (2002) Creative Europe: On Governance and Management of Artistic Creativity in
Cox, Donna. http://www.ncsa.uiuc.edu/People/cox/
Davies, Char. http://www.immersence.com/
Arts Council of Finland (2004) "New Media Arts and Culture - Policy and Practice in Dialogue," ARSIS Magazine, 3(4), available at: http://edmund.taiteenkeskustoimikunta.fi/download/ARSIS+verkko+3-04.pdf?lngDoc_id=1534
Fischer, G. (1999) “Symmetry of Ignorance, Social Creativity and Meta-Design,” in L. Candy and E. Edmonds (eds), Proceedings of the Conference "Creativity & Cognition 1999", ACM Press, pp 116-123.
Fischer, G. (2002) "Beyond 'Couch Potatoes': From Consumers to Designers and Active Contributors," FirstMonday (Peer-Reviewed Journal on the Internet), available at: http://firstmonday.org/issues/issue7_12/fischer/
Fischer, G., Giaccardi, E.,
Frazer, John (1995), An Evolutionary Architecture, Architectural Association
Giaccardi, E. (2004)
“Principles of Metadesign: Processes and Levels of Co-Creation in the New
Design Space”, PhD Thesis,
Giaccardi, E. (2005) "Mediators in Visual
Interaction: An Analysis of the ‘Poietic Generator’ and ‘Open Studio’,"
Journal of Visual Languages and Computing (Special
Issue on Context and Emotion Aware Visual Interaction), in press.
Giaccardi, E., Eden, H.,
& Sabena, G. (2005) "The Silence of the Lands: Interactive Soundscapes
for the Continuous Rebirth of Cultural Heritage," CUMULUS 2005 - Pride & Pre-Design: The Cultural Heritage and the
Science of Design,
Giaccardi, E., & Fischer, G. (2005) "Creativity and Evolution: A Metadesign Perspective." In Proceedings of the European Academy of Design (EAD-6) Conference, Bremen, Germany, 29-31 March, available at: http://l3d.cs.colorado.edu/~gerhard/papers/ead06.pdf
Harrington, D. (1990)
"The Ecology of Human Creativity: A Psychological Perspective." In M.
Runco, & R. Albert (Eds.), Theories
of Creativity, Sage,
Kaiser, Paul. http://www.openendedgroup.com/
Makela, T., Tarkka, M.,
Czegledy, N., Diamond, S. Donovan, A., Ferran, B.,
Illich, I. (1973) Tools for Conviviality, Harper &
Jaimes, J. & Jennings,
P. (2004) “ACM Multimedia Interactive Art Program: An Introduction to the
Digital Boundaries Exhibition,” in Proceedings
of ACM Multimedia 2004, ACM Press,
Mitchell, W. J., Inouye, A. S., &
Blumenthal, M. S. (2003) Beyond
Productivity: Information Technology, Innovation, and Creativity, The
National Academies Press,
Mockros, C. A., & Csikszentmihályi, M. (1999) "The Social Construction of Creative Lives." In A. Montuori, & R. E. Purser (Eds.), Social Creativity, Hamption Press, Inc., Cresskill, NJ, pp. 175-218.
Pejtersen, A.M. (1980).
“Design of a Classification Scheme for Fiction Based on an Analysis of Actual
User-Librarian Communication and Use of the Scheme for Control of Librarians’
Search Strategies”. In O. Harbo & L. Kajberg (Eds.) Theory and Application of Information Research, Taylor Graham,
Puckette, Miller. http://crca.ucsd.edu/~msp/
Rokeby, David. http://homepage.mac.com/davidrokeby/home.html
Silverstone, R., & Sujon, Z. (2004) Summarizing the Urban Tapestries Social Research: Experimenting with Urban Space and ICTs, available at: http://www.proboscis.org.uk/urbantapestries/socialresearch.htm.
Sternberg, R. J. (2005) "Creativity or Creativities?" International Journal of Human Computer Studies (Special Issue on Creativity and Computational Support), in press.
Stiny, G. (1980) “Kindergarten Grammars: Designing with Froebel’s Building Gifts,” in Environment and Planning B, Vol. 7, pp. 409-462.
Thompson, E. (2001), Empathy and consciousness, in Journal of Consciousness Studies, Vol. 8, No. 57, pp. 132.
Varela, F. J., & Shear, J. (1999) "First-Person Methodologies: What, Why, How?" In F. J. Varela, & J. Shear (Eds.), The View From Within: First-Person Approaches to the Study of Consciousness, Imprint Academic, Thorverton, UK and Bowling Green, OH, pp. 1-14.
Wilson, S. (2002) Information Arts: Intersections of Art, Science and Technology, MIT