Creativity

Appearing in: Proceedings of 2nd International Conference on Computational Creativity, 2011

By David Kirsh

Abstract

Contemporary choreography offers a window onto creative processes that rely on harnessing the power of sensory systems. Dancers use their body as a thing to think with and their sensory systems as engines to simulate ideas non-propositionally. We report here on an initial analysis of data collected in a lengthy ethnographic study of the making of a dance by a major choreographer and show how translating between different sensory modalities can help dancers and choreographer to be more creative.

Introduction

The design process of ‘making’ a modern choreographic work offers insight into two creative processes much in need of understanding.

1. Distributed creativity: the mechanisms by which team members harness resources to interactively invent new concepts and elements, and then structure things into a coherent product;
2. Embodied cognition: the mechanisms by which creative subjects think non-propositionally, using parts of their own sensory systems as simulation systems, and in the case of dancers, using their own (and other’s) bodies as active tools for physical sketching.

The close study of both of these processes bears di- rectly on the goal of developing new theoretical models of creativity. It relocates creativity from a within-the-mind process to a more socio-technical process involving re- sources and other people; and it recognizes the importance that bodies and sensori-motor systems – both non-verbal and perhaps sub-rational elements – play in creative cognition. In this paper we consider only process two: the role of embodied cognition in creativity.

Why choreography? Usually, creative processes fall short of their potential because variance in ideas is not managed well. The generative phase of creation is closed down too early, or it runs dry of its own accord.

(1) The choreographer observed in this study (henceforth WM) has developed techniques for keeping the process open longer and for maintaining substantial variance among the dancers despite the urge for group think and convergent behavior.

(2) He has also developed techniques for exploiting the coding language of sensory systems, of both himself and his dancers, to create new movement ideas.

WM is a remarkably successful choreographer. His track record raises some obvious questions about his creative process. In particular, how does WM help his dancers to:

• break their personal signature? Each dancer has a standard repertoire of moves and styles of moving. How can they be pushed beyond their personal repertoire?
• be creative for longer periods — to stay in a creative phase at full intensity for longer? Dancers can be creative in bursts that issue in phrases that last for 20 or 30 secs. What can a choreographer do to lengthen a dancer’s period of creativity from 20 or 30 secs to 60 or even 70 secs?
• sustain long term creativity? Typical brainstorming sessions can be successful for a few hours, or occasionally for a day. What methods can keep a dancer at near peak levels for weeks at a time?
• prevent premature crystallization. Creativity requires a period of openness, followed by winnowing and narrowing of options. The danger is that ideas that seem good will be accepted before newer, even more radical ideas are proposed. How does the choreographer strike the right balance between keeping a process open and closing it?

Methodology

To study these and other fundamental questions about creative cognition we pursued a mixed methodology of close ethnographic observation, experimental study and computational analysis.

To understand the choreographic process we videotaped all scheduled interactions between choreographer and dancers during the time they worked together over thirty work days to create a new dance that premiered at a major dance venue in London a week after its completion.

Creative Cognition in Choreography

David Kirsh
Cognitive Science Department University of California, San Diego La Jolla, CA 92093, USA kirsh@ucsd.edu

Five high definition video cameras were placed on the studio walls, and, whenever possible, two standard video cameras were placed on the ceiling. Written notes about the process were taken in real-time. During the first three week phase of ‘making’, fifteen students took notes; during the second phase a single experienced ethnographer took notes. The choreographer was interviewed for between forty and sixty minutes on digital video each morning and night most days. The dancers were also interviewed. At the end of each rehearsal, four dancers were selected and interviewed for thirty minutes each. Our aim with the dancers was to have them reflect on specific elements of the rehearsal that day. Whenever possible we had them verbally describe their experience during the day and then show us through movement what they meant. We also reviewed all notebooks.

Coding: To code the video we used ELAN, a free software system developed by the Max Planck Institute for Psycho- linguistics. ELAN was designed for studying gesture and small-scale interactions.

We developed our coding system iteratively. On the basis of interviews and common sense we started out with a vocabulary for obvious communicative phenomena: for example, WM talking to one, two three… all the dancers, WM gesturing, making certain non-linguistic sounds. We included other gross actions related to directing movement: touching or positioning dancers, WM showing the movements he wants, the use of props such as projected images or shared photos, joint attention.

As our collection of instances of these phenomena grew we compared them for differences and began defining new coding predicates to differentiate or qualify them. For instance, when we looked more closely at sonifications (sounds WM would make to help communicate the shape, emphasis or dynamics of a phrase) we became interested in the relationship between the onset time of sonification and gesture, and then in the relationship between gestural form and a sonification’s sound pattern.

In another case, we became interested in a phenomenon that dancer’s call marking. From interview we learned about this practice, learned how to recognize it, and then through further interview, from close study of video and having dancers mark for specific purposes, we began to look for behavioral indicators of different types of marking, (3). We found for instance that marking is very different when its goal is to coordinate grips in duets and when its goal is to help a dancer consolidate a movement just taught. The longer we work on our corpus the more our coding scheme grows and specializes.

One Type of Dance Creativity: One specific problem WM sets for himself, as reported in interview, is to create dance where human bodies move in ways never before seen. In the past, he derived inspiration from studying motor disorders such as ataxia, and from observing the way the heart and other organs move when revealed in open-heart surgery. But most often, he relies on a collection of techniques for harnessing sensory simulators – for recruiting the power of embodied cognition based in the senses and in the elasticity of the body. It is these techniques we consider here.

Embodied Creativity

From earlier work on this topic (4), we discovered that dancer and choreographer regularly use their bodies as things to think with. They spend much of their time thinking non-propositionally. When trying to create new movement forms they use their bodies as a cognitive medium, much the way a graphic artist uses drawing as a cognitive medium or a violinist uses the sound emanating from his violin as a cognitive medium. Just as an artist or musician develops a close coupling with their tools — pencil and paper for the artist, violin for the violinist — so a dancer must have a tight control relation between body-as-tool and body-as-display-medium. Embodiment bears on dance the way instruments bear on artistic or musical product. Change the instrument and you may change the form or style of the output. So too in dance, changing the body-as-tool, say by making parts of it rigid or spasmodic, leads to a change in form and style of dance. This places the mechanics of the body front and central in the generation of dancerly movement.

Additionally, from earlier work (3,4) we found that both choreographer and dancer rely on imagery in the visual, somato-sensory, tactile, and motor systems to create novel movement. The choreographer explicitly gives his dancers tasks that require them to shift between modalities. For instance, he might ask them to imagine that their bones are made of firm rubber, or that they should imagine the feeling of being attacked. Their task is to translate those feelings into movements.

One reason to see this process of simulating in one sensory modality and then translating to another modality as embodied cognition is that it relies on each modality having its own way of coding input, and ‘concepts’. Although embodied cognition, as a scientific expression, has different meanings, (5,6) a common element across most versions is that cognitive processes are grounded in modality specific brain systems. The way we acquired concepts through sight, sound, touch, and so on, continues to affect our understanding of those concepts, long after they have been abstracted from specific senses. The idea of running is abstract. But we ground our understanding of that idea in the physical activity of running which we experienced when running. Embodied cognition, then, can be understood as a form of computation, distinct from familiar symbol manipulation or connectionist computation, wherein parts of the body, or parts of a sensory system, are harnessed to simulate some process. By simulating that process a subject understands it.

For instance, the mirror neuron system is sometimes cited as an example of embodied cognition because it is thought to explain how a subject can imbue meaning to the actions that someone else performs. By personally simulating in their own motor or visual cortex, the planning and other processes related to executing those actions themselves (7) they understand what it is like to perform that action. Thus, when subjects see another person pouring a cup of tea, their brains respond by activating many of the same parts of cortex as would be activated were they pouring tea themselves. Some psychologists have argued that fainter versions of these same activations occur whenever a subject understands a sentence about pouring tea, and that this activation is what grounds much linguistic understanding. (8)

In dance, the tenets of embodied cognition may explain how dancers invent ‘dancerly’ movements. Often WM will task the dancers with ‘solving’ a choreographic problem. An example problem is to imagine what it’s like to have a rigid rod connected to your shoulder. The rod is pushed and pulled. To solve this problem a dancer works with a partner some distance away. That partner is notionally holding the rod and moving it. The dancer then generates mental imagery associated with the movement of the rod. Most of this imagery will be about the somatic or kinesthetic feelings of being pushed and pulled.

The pattern of somatic or kinesthetic priming these images creates serves to bias the next somatic or kinesthetic images in the dancers imagination. The priming defines a weighting function over somatic or kinesthetic image continuations. It is obvious that without a body or neural system capable of image continuations there would be no causal basis for priming and hence image continuations. It would be impossible to link or translate a given somatic state into motor movement continuations. No body no motor movement continuations. The upshot is that a dancer’s capacity to relate somatic or kinesthetic images to motor dispositions can be used to help him or her create interesting movements and also judge their aesthetic quality. By interpreting their movement through the lens of one or more sensory modalities other than movement control per se, they are able to judge whether the movement looks right visually, feels right somatically and kinesthetically, or whether it captures a sound right. This form of cognition is both embodied and non-propositional. (9,10).

Here is another choreographic problem that may clarify the method. A timeworn choreographic task is to ask a dancer to ‘paint’ a contour, say Manhattan’s skyline. Dancers would never use their hands alone as the paint brush — that is too simple and boring — they use different body parts. For example, they might start with their elbow, continue the contour line with their head, then move to their hip or foot. This process involves several modalities because the visual modality is required to imagine the contour, and if the dancer has feelings attached to parts of the contour then whatever modality is tied to emotional feeling will be a factor too. For instance, a dancer may believe that people have jumped off the empire state building so he or she may have a special feeling about that part of the contour. As the different parts of the body trace the different parts of the contour the feeling in one of these modalities — somato-sensory, visual, emotional — can be used to judge the movement’s aesthetic virtue. The creative process here is: generate in one modality, map to another, test in a third.

This suggests that there are two distinct types of embodied cognition at play.

• using the body as a medium to think in – dancers don’t think in words they think physically, through their bodily form;
• using sensory systems as non-propositional systems to think in – dancers don’t think in words or propositions, but in visual, tactile or somato-sensory forms.

Although this makes it sound as if embodied cognition is a continuous or analog process relying on a body’s elasticity or a sensory system’s simulation capacity, it is important to recognize that these thinking processes are still representational. They are representational but they are so tied to the properties of the underlying medium (muscle, tendon and bone, body control mechanisms, sensory modalities and sensory simulators) that the cost of embodying the representation is significant. The cost of creating a representation or simulation, of sustaining it and transforming it depends on the cost structure of the neural system implementing the representation. Properties of the underlying neural system show through. Accordingly, for a given person it might be easier to run imagery in his or her visual system than in their somato-sensory system. For another person it might be easier to run somato-sensory imagery. It is also likely that differences in the ease of generating modality specific imagery will depend on the content of the image. It’s easy to visually imagine entering into a sphere through a small opening but harder to imagine what it might feel like to enter a self healing sphere, where you use your hands to open a hole and then step in and seal it up.

This idea, that the cost structure of cognition changes with the medium of cognition is central to our approach to creativity.

In non-propositional systems, where the structures to be created are not interpreted as being true or false, an ‘idea’ can be shifted around by moving it from code to code, system to system, each system making it possible to discover different things. Thus, in architecture, a domain rife with image based representations, an idea that starts as a sketch on paper, where certain issues are worked out, may be transformed when the architect tries to model his sketch in three dimensions in foam-core or wood. Each medium teaches the architect something different.

Sometimes you have to encode an idea in a different form or medium to appreciate its strength or weakness most clearly. The same applies to music. A piece of music that sounds one way when played on a violin may sound quite another way when played on a tuba. Each instrument may stimulate the composer to notice new aspects of his original ‘germ’ idea, or to derive new associations, or to ‘infer’ new ideas. Each encoding is situated in a different energy landscape of closeness.

Ironically, the special power of embodied thinking in dance, then, is the power of representation everywhere. If an ‘idea’ can be encoded in one representational system easily, or worked out easily there, it can then be translated into another representational system where it might have been difficult to discover initially. Once encoded in that new representational system, though, it has a form that carries new possibilities and makes it easier to discover new connections. A problem stated in geometry may be hard to solve in classical geometric representations, but once translated into an algebraic representation it is easy. Once solved algebraically it can be translated back to geometry. This is the huge power of representational systems. Each representational system operates with its own metric of inferential distance. Two ideas that are close in one may be distant in another and vice versa.

In a simple experiment designed to test the value of moving between different representation systems we compared the performance of subjects who were allowed to move scrabble tiles with those whose tiles were fixed. Because of differences in the way people manipulate letters mentally and letters (tiles) physically, they are likely to stumble on good sequences by simple physical rearrangement that would be hard for them to find mentally. Mental rearrangements follow least energy paths in a lexical or phonological landscape, while physical rearrangement is sensitive to how easy or hard it is to move the tiles. The state spaces are different and hence the trajectories through those spaces will be different.

Thus, because ‘letters’ that are physically close are often different than those that can be phonologically or lexically close it is probable that by physically moving tiles new ideas will occasionally be stimulated. During the movement phase there will be moments when phonologically implausible sequences are visually present and therefore considered momentarily. This potentially increases the number of combinations reviewed.

In a like way, bodies, sensory systems and artifacts each constrain different energy landscapes of possibilities. The trick is to know how to harness their comparative virtue.

Sensory simulators

To explore the idea that sensory simulation can be used as a filter on goodness, we need some definitions. Sensory systems operate with a sensory code. The code need not be symbolic; it only need be able to encode different states. It may be an analog code. Having a code makes it possible to talk of a sensory system having an expressive power — its full state space — and to talk about trajectories through this state space. Assume, further, that sensory states can be classified into equivalence classes, such as those associated with a smell, taste, visual shape, body feeling, or movement; and also that there are contingency tables specifying probability measures between these equivalence classes. Regularities in experience have trained our sensory systems to ‘expect’ certain pathways. These pathways become primed whenever states that lead to them are activated.

Because our senses encode different aspects of the world each is informative, and contains bits of information the others do not. Hence each sensory system supports different priming pathways. Events that seem ‘natural’ or obvious in one sensory system may seem unnatural or completely unobvious in another. We can think of this on analogy with numerical representational systems. To decide whether the number 30,163 is divisible-by-7 takes some computation. In the base 7, however, 30,163 is represented as 153,640, and here it is completely obvious that it is divisible-by-7, just as it is obvious that 97,230 in base 10 is divisible-by-10. It is transparent. See (10, 11). In the somato-sensory system, a dancer may immediately recognize graceful movements. What feels graceful, however, may not always look graceful, since the encoding of a movement in the visual system is so different than its somato-sensory encoding. This is even more obvious when we consider impossible movements. What the motor system deems impossible may be quite different than the visual system.

One potentially interesting consequence of this account is that it explains how humans can think non-propositionally. They think in their sensory systems. They simulate outcomes, and they control the simulation process in non-propositional thought much the way that they control propositional thought by controlling auditory images of linguistic elements.

Moreover, because of the different encoding properties of sensory systems dancers are able to reach ‘conclusions’ in some sensory systems that are hard to reach in others. It is sometimes easier to think in one modality than another. I believe that when a dancer visualizes an object — say a reptile slithering around a chair — and then transforms the visual experience into a movement they are first trying to draw creative insight from a visual solution before moving to a bodily solution. They visually imagine themselves slithering before feeling themselves moving and then finally moving. They transform between sensory media.

Multi-modal translation

The choreographer relies heavily on this sort of modality translation to stimulate movement ideas in his dancers. He does this in two ways. First, he personally uses a broad range of modalities to communicate with his dancers — modalities to direct or guide them. Second, he assigns them ‘choreographic’ tasks that require imagining scenarios or processes and then translating these into interesting movement.

We have already sonification, as a vehicle for shaping movement. We observed WM sometimes ‘saying’ things like “Yah ooh ehh” to communicate the shape of a movement. He used sound to shape dynamic form or perhaps to communicate feeling or attitude. The choreographer also uses tactile and kinesthetic imagery as a creative stimulus, either by touching the dancers and then asking them to draw tactile or kinesthetic inferences from the dynamics of his touch, or by speaking to them, and assigning each a cognitive task that requires them to recruit their tactile and kinesthetic imagery abilities. This is an instance of the second method, the general technique of inventing new shape through cross-modality problem solving. (12). Here is another example of that.

In one task we observed, a dancer conjured the visual image of a massive bell gonging. She then transformed that moving image into a new structure, the kinetic feel of moving body parts as if those parts are connected to the heavy bell, or perhaps the feeling of rocking the bell. See figure 2 where we show a snapshot of a video we annotated based on an interview with a dancer. The dancer seems to be comparing the feel of the body movement to the visual or perhaps conceptual structure of wrapping one’s hands and legs around a heavy bell and moving it, honor- ing its inertia. This is interesting for what it shows us about using visualization to unleash individual creativity.

Conclusion

I have briefly reviewed some methods a noted choreographer uses with his contemporary dance company to reliably generate novel dance phrases. Choreography is a revealing domain to study creativity because the process often lasts over many weeks and requires both choreographer and dancer to generate countless candidate ideas, then select and refine them. We found by careful ethnographic analysis that WM relies heavily on modality translation as a generative technique. He often assigns his dancers tasks that require them to imagine what something feels like kinesthetically, or to imagine what something would look like, or smells like, or feel like in an emotional sense, then to translate this to movement. At other times he communicates with his dancers non-linguistically and relies on their ability to translate his gestures, touches, sounds or sights to movement relevant forms. I argued that this is a successful method for creativity because it harnesses the power of multiple representation systems.

Acknowledgements

I gratefully acknowledge the help of Dafne Muntanyola in helping with the organization of this study and her thoughtful comments throughout. Funding for this work is from the NSF CreativeIT program, grant. IIS-1002736.