ABSTRACT
We are developing a
prototype storytelling robot for use with children in rehabilitation. Children
can remotely control a large furry robot by using a variety of body sensors
adapted to their disability or rehabilitation goal. In doing so, they can teach the robot to act out emotions (e.g.
sad, happy, excited) and then write stories using the storytelling software and
include those emotions in the story. The story can then be "played"
by the remote controlled robot, which acts out the story and the emotions. We believe that this robot can motivate the
children and help them reach their therapy goals through therapeutic play,
either by exercising muscles or joints (e.g. for physically challenges
children) or by reflecting on the expression of emotions (e.g. for autistic
children). We use an innovative design methodology involving children as design
partners.
Keywords:
Therapeutic play, robot, children, user interface, design process,
rehabilitation
INTRODUCTION
As advances in technology continue, people with
disabilities are benefiting from greater availability of assistive
technology. Assistive technology has
been traditionally credited for increasing independence, improving the quality
of life and providing an overall boost in self-esteem for people with
disabilities by making the most of an individual’s abilities. Assistive technology, when strategically
applied, may also provide therapeutic benefits. Hilker et al. (1995)
believes the rationale for this concept is based on an overlap between
assistive technology use and physical therapy interventions. Our work pushes that theory, by making use
of assistive technology as a motivating environment to improve basic functions
associated with physical therapy goals.
Video games have been
used in the past for a variety of therapeutic and rehabilitation situation
(Griffiths 1997) and several studies - mainly set in a clinical environment -
have shown how those games can be used as a mean to achieve desired functional
goals (e.g. Vilozni, et al. 1994). The
rich literature that describes the use of puppetry as a therapeutic,
educational, and social tool (e.g., White, J.& Allers, C. T.,1994, Vidler,
1972; Currant, 1985; Carter, 1987; Caputo, 1993) also shows the benefits of
technology. For example researchers
have found that the use of puppets as a mediating technology can produce more
expressive communication patterns among children who are in crisis or have
on-going emotional, behavioral or academic challenges. Technology can be used by a child to
confront what may be too painful or difficult to express otherwise. By telling a “story” with these props,
children can “play out” their feelings just as an adult may “talk out” their
difficulties with a therapist (Cochran, 1996).
Here we introduce a
storytelling robot technology for therapeutic play with the hope that it will
provide a richer environment useful for children with a variety of challenges,
as well as the long term motivation needed to reach children's therapeutic
goals.
The partners
- This work is
the result of the collaboration of a university laboratory and a small startup
company specializing in adaptive technology.
The University of Maryland Human-Computer
Interaction Laboratory (HCIL) has done extensive work in interaction design and
evaluation with a unique program focusing on the design of children's
technology (Druin, 1999a; Druin, 1999b).
One of the more unique projects to come out of this work has been a
prototype robot for storytelling (Druin, et al., 1999; Montemayor et al.,
2000), which demonstrated the benefits of working with children as active
partners in the design of new technology.
This storytelling robot was originally designed for all children,
without consideration for special challenges.
Children using the storytelling software could write stories and select
emotions from a list of pre-programmed emotions (e.g. happy, sad,
surprised). The story could then be
"played" by the remote controlled robot, which told the story
(playing recorded or synthesized speech) and acted out the emotions at the
appropriate time in the story.
The collaboration with AnthroTronix is broadening
the scope of our storytelling robot research to emphasize designing tools for
the purpose of children's rehabilitation.
AnthroTronix is a startup company; part of the
Rehabilitation Engineering Research Center led by The Catholic University of
America and the National Rehabilitation Hospital. AnthroTronix designs and develops robotics and virtual reality
interfaces to support children’s rehabilitation. The company has developed a
system that augments human function by providing personal augmentation devices
(PADs), controlled by physiological signals, for the purpose of navigating and
manipulating the external environment under their control. The system utilizes a variety of sensors
connected to detect input (or “gestures”) from the user. Software is then used to assess and process
this input, and to map the input to appropriate outputs. The information fed back to the user depends
on the intended outcome. For example,
the systems can be used to train the user to learn input-output mappings, to
enhance performance, and to assess performance.
This paper first reports on a pilot study using a
video game with AnthroTronix assistive technology, which illustrates the
potential effectiveness of therapeutic play and some of its limitations. Then we describe how we extended the
storytelling robot to enable children to directly control the robot, to
"teach" the robot how to act out the emotions. This was accomplished using AnthroTronix
assitive technology, allowing for the easy adaptation of sensors to support
children's abilities or challenges. We believe that a robot can provide a strong
motivation for children to reach their therapy goals by exercising the
appropriate physical ability or emotional expression.
USING VIDEO GAME
TECHNOLOGY AS THERAPEUTIC TOOL – A CASE STUDY
Our work with a storytelling robot was motivated in
part by the promising results of a case study using AnthroTronix assistive
technology with an off-the-shelf videogame.
A study was conducted by Rinaldi with children who
have mild to moderate cerebral palsy (Rinaldi, 2000). Four children were
monitored using a SuperNintendo video game system. The game was used with adaptive inputs, which challenged the
participants’ isolated motor control and range of motion. For example, instead
of pressing a button to make a game character jump over an obstacle, children
had to bend their arm beyond a certain threshold. The input sensors were
designed to provide alternative switch access by allowing the switch threshold
to be adjusted as the child improves or changes.
The games were used the children at home during a
12-week period. Therapists gave the
children exercises and the use of the game was recorded. Strength and range of motion were recorded
at 6 weeks interval along with other qualitative information. The findings
indicate an overall improvement in range of motion with game use but no changes
in strength. The parents indicated that
the response to this type of therapy was positive. Everyone enjoyed having the system at home and felt it was safe
and easy to use. This case study
highlights the benefit of therapeutic interventions that are administered in a challenging,
familiar and fun environment that is integrated into a child's daily
activities.
Feedback from children and parents made it clear
that using the Nintendo game broke the routine therapy exercises but also
suggests that the initial excitement might fade after a few weeks. This of course highlights the need for
longer studies but it was conjectured that two factors played a role in the
decline of interest after 12 weeks.
First, an arm movement is slower than the press of a button, which
resulted in poor performance in the game, a great disappointment for the
children. Second, it was conjectured
that the repetitiveness and the complete lack of creativity in the Nintendo
activity might be a hurdle in any attempt to alleviate the boredom of daily
therapeutic exercise.
This leads us to believe that a creative
storytelling robot might provide a long-term motivation that is needed to help
children in rehabilitation.
DESIGNING A ROBOT FOR THERAPEUTIC PLAY
The pre-existing technology - PETS - Personal Electronic
Teller of Stories:
At the University of Maryland, we built a robot
called PETS: A Personal Electronic Teller of Stories (see Figure1). Children
can build a robotic pet by simply snapping together special robotic animal
parts (e.g., dog paws, a fish tail, duck feet ). After a robotic pet is built,
children can tell stories with the My
PETS software, giving their robot emotions (e.g., excited, sad, lonely) to
act out throughout their story (Druin et al., 1999; Montemayor et al., 2000). A
7-year old girl in Maryland created the example story below.
The initial PETS robotic
animal parts were built with LEGO bricks covered in fur, feathers, felt, etc.
Further refinements have led to a skeletal structure built from metal, plastic,
and polycarbonate materials. These have
been covered in a foam outer-shell with brightly colored felt and fur. Each animal part can be snapped into place
on the body and is also plugged into a plugbox embedded in the animal’s torso.
This plugbox is an interface to a Handyboard controller also in the animal’s
body, which controls servos and motors, and can read inputs from sensors that
are attached throughout the robot’s body. This controller has a serial
connection to a Macintosh computer. In the Macintosh, the application software
layer, My PETS, takes a story written
by children, translates and transfers it to the system software layer that
resides in the Handyboard.
Overall design
methodology
The name, concept, and the
development of the PETS prototypes came about because of children. We develop
new technologies for children, with children in an “intergenerational design
team.” This current team consists of
children (ages 7-11 years old) and adult professionals, with experience in computer
science, education, art, and robotics. Our research goals include creating new
technologies as well as understanding better ways to give children a voice in
the technology development process. An
outcome of our research has been to develop the methodology of “cooperative
inquiry,” research methods that support the partnership of children and adults
in developing new technologies (Druin, 1999a).
For this new part of the
project our design team is also comprised of rehabilitation staff. The early phase of the design has been
conducted solely with our existing team of child design partners. Children with disabilities will join the
team later as informants during the refinement of the design and of course as
testers of the refined prototypes. One
of our goals is to design a product that will have the potential of being both
a commercial toy for all children and a rehabilitation tool for children with
special challenges through the addition sensors and the development of the
rehab methodology.
Feedback from
the design sessions with the children partners
From the design
sessions with our child design partners emerged several findings:
First it was made
clear that the storytelling robot was a very powerful tool to engage children
in discussing emotions, their expression or the risks of misinterpreting
expressions of emotions, leading us to believe that PETS could be used as a
rehabilitation tool for children with autism.
Second, despite the
great excitement created by the robot when used by the children, one of the
most obvious limitations of the initial prototype was that the emotions—or any
other movement of the robot—had to be programmed and “pre-stored” in the
software in order to be used by the children for their stories. Originally, software was created to control
the robot directly via a graphical user interface. But it soon became obvious
that a more natural way to operate the robot was to enable children to teach
the robot through their own body motions.
A first design session
was organized during which children tested a car remotely controlled with an
instrumented glove. The children all
enjoyed the direct control of the car (comparing it to the buttons or
joysticks), and liked the absence of wires between the car and the glove. On the other hand, they also found the glove
uncomfortable but most importantly found the proposed mapping between the body
movement and the car movement very confusing.
They had to bend their wrist up or down to go forward or backward, and
spin their wrist right of left to turn, but constantly confused and combined
the movements which send the car in seemingly random direction. This session clearly highlighted the need
for a direct correspondence between body motion and the motion of the
robot. Ideas proposed by the team
included:
·
the need to use wireless control to avoid
tangles of cables between the sensors and the computer;
·
embedding the sensors in agreeable soft
accessories such as bracelets or hair bands;
·
and decorating those accessories with elements
that resembled the object or robot part being controlled in order to clarify
the connection between sensors and actuators (e.g. a miniature hand might
decorate the sensor that controls the hand of the robot, a wheel might decorate
the sensor chosen to turning the car)
Based on these ideas,
a new prototype interface was developed with sensor controls of the
storytelling robot. Sensors (accelerometers) were imbedded in two armbands - to
control the 2 arms of the robot- and in a hat to control the head. An additional sensor (built in a shoe sole)
allows the entire robot to spin when foot pressure is applied to the front of
the shoe (this is clearly not the best way to control a spin but additional
refinements will be made to this in the near future).
Early feedback from
about 3 hours of trials suggest that our design team children are able to control
the robot and that excitement of using the robot for storytelling was
heightened by the increased sense of control over the robot. An unexpected observation is that both
children and adults’ attention is strongly attracted to the robot - and not to the
person controlling the robot. After a
few adjustments and corrections of the control system, children were able to
control individual movements of the robot, while the randomness of the
movements of the robot—when the children was not precisely controlling the
robot—gave it a compelling liveliness.
The controls can be
recalibrated to match the amplitude of the child’s movements and can also be
reassigned. For example, the arm bands
can be inverted to control the arm side directly (kid left arm to robot left
arm) or in a mirror fashion (kid left arm to robot right arm) which seemed a
better initial setup. Of course the
sensors could also be used on other body parts to match specific rehabilitation
needs (e.g., it could be placed on a foot to exercise an ankle while
controlling the robot.)
User Scenarios
Our original prototype is
very rudimentary. More controls need to
be added and more flexibility is needed in matching sensors and robot movements
before we can successfully work with children with disabilities. Wireless
connections from the computer to the sensors are needed, the size of the robot
needs to be reduced to facilitate a robot’s motion on a table or floor, but we
can start develop scenarios of use.
Below are 2 examples and we are now working with professional
rehabilitation experts to develop other strategies and games.
1- Physical therapy
Max has
cerebral palsy. To learn to feed
himself he needs to do a lot of supination and pronation exercises to
strengthen his arms. Today’s standard
of care makes him repeat the same exercise 100 times. Five-year old Max finds
the exercises extremely boring and often refuses to finish them. But today the
therapist tells him “Do you have a pet? Let's tell a story about it together...
and the robot will act out the story for you.”
As the child start talking about the story the therapist equips the
child with an elbow pad embedded with input devices and shows the gestures that
will make the robot act “happy” of
“confused” when appropriate in the story. For example, arm extensions could be used to make the robot move
around the space, and arm rotation to turn it around. Max is now engaged in a creative endeavor and is motivated to do
the exercises. After much practice and many revisions the story is recorded on
video to take home. Next time Max will
learn to teach the robots new tricks, associated with new movements; eventually
he could control the robot remotely from home and tell the story to the
children in the waiting room.
2- Emotional Expression for Children with Autism
Sue is greeted by his
therapist, “Sue, are you interested in telling a story today?” Together they
pull out the robot toolkit. The
therapist asks, "What character do you want to tell a story with
today?" And Sue answers, “Benny,”
as she pulls out the bird wings, purple horns, and flying saucer. Slowly Sue
builds her stuffed robot caracter. Then
she moves to the computer and begins to write a story, “My name is Benny and I
am SAD.” She presses the “tell me a
story” button and the robot tells the story and acts sad. The therapist asks Sue, “You know, I'm not
sure the robot looks really sad. Can you teach the robot to really look
sad?” Then she takes out their robot
sensors and they attach them to Sue’s arm and leg. Sue then moves toward the
wall. The Therapist asks, “Is that the
only way to look sad?” Together they
talk about what it means to be sad and try different movements and expressions. Late in the session, the therapist asks,
“Sue, can we work on making Benny look happy?
I think I'm tired of sad.” Sue then goes to the computer and writes a
story about Benny, “My name is Benny and I am HAPPY. Benny likes his therapist.”
FUTURE WORK
We are now revising
our prototype and it is our goal to have at least one of the above scenarios
tested during summer. We anticipate
that by Fall 2000 (the time of the conference), we will be revising the robot
hardware and software to respond to the needs of rehabilitation elicited during
the summer testing sessions.
We anticipate producing
a robot toolkit that includes a base robot, accessories for customizing the
robot (paws, feathers, sound), a rich set of input devices to control the robot
adapted to the range of abilities of children in rehabilitation. Software will also be needed to monitor
usage. After the usual cycle of formative evaluation and iterative design, we
will develop a methodology for rehabilitation staff to provide therapeutic
play, including training, examples of use, preparation, and storytelling
techniques. Measuring the effectiveness of such therapeutic play with thorough
scientific evaluations is our main challenge for the coming year.
DISCUSSION
We believe that this
new type of technology has the potential to help motivate children to reach
their therapy goals through therapeutic play.
It has been argued that for therapeutic treatment to be successful it
has to take into account the dynamic of the family (Ross & Thomas, 1993).
Storytelling appears to offer a solid motivating direction as it is a typical home
activity (with relatives, siblings or friends). Play can give children with disability a sense of competence and
control over environmental circumstances. It can also help a child learn new
skills.
We have built a
working prototype, scenarios have been developed to illustrate how the
prototype could be used and we are ready for testing with challenged children.
We believe that our innovative design methodology involving participatory
design and children as design partners will lead to a product that will truly
excite children's interest as well as support the rehabilitation experience.
Acknowledgements
We thank our adult and child design partners for their
countless hours of work and design feedback. We also want to thank the Maryland
Industrial Partnership program and National
Institute on Disability and Rehabilitation Research, U.S. Dept. Education
grant #H133E980025. All opinions are those of grantees.
References
Caputo, R. A. (1993). Using puppets with students
with emotional and behavioral disorders.
Intervention in School and Clinic,
29 (1), pp. 26-30.
Carter, S. (1987). Use of puppets to treat traumatic
grief: A case study. Elementary School Guidance and Counseling,
21(3), pp. 210-215.
Cochran, J. L. (1996). Using play and art therapy to
help culturally diverse students overcome barriers to school success. School Counselor, 43, pp. 287-298.
Currant, N. (1985). The expansive educational value
of puppets. Academic Therapy, 21(1),
pp. 55-60.
Druin, A. (1999a). Cooperative
Inquiry: Developing New Technologies for Children with Children, Proceedings of Conference on Human Factors
in Computing Systems (CHI'99), Pittsburgh, PA, USA, May 15-20, ACM, New
York, 592-599
Druin, A. (ed.) (1999b). The design of children’s technology.
San Francisco, CA: Morgan Kaufmann
Druin, A., Montemayor, J., Hendler, J., McAlister,
B., Boltman, A., Fiterman, E., Plaisant, A., Kruskal, A., Olsen, H., Revett,
I., Schwenn, T.P., Sumida, L. and Wagner, R., (1999). Designing PETS: A
Personal Electronic Teller of Stories Proceedings of Conference on Human Factors
in Computing Systems (CHI'99), Pittsburgh, PA, USA, May 15-20, ACM, New
York, 326-329
Griffiths, M (1997). Video games and clinical
practice: issues, uses and treatments. Br
J Clin Psychol Nov;36, 639-41
Herman-Hilker, S L; Hilker, D F; Levine, S P. (1995). Achievement of physical therapy goals as a result of
assistive technology use. RESNA, pp. 59-61
Montemayor, J., Druin, A., and Hendler, J., (2000). PETS: A Personal
Electronic Teller of Stories , in A. Druin & J. Hendler
(Eds.) Robots for Kids: New Technologies
for Learning. Morgan Kaufmann, San Francisco, CA
Rinaldi, M. C. (2000). Using Adaptive Video Game
Technology as Therapeutic Tool, Master of
Biomedical Engineering Thesis, The Catholic University of America School of
Engineering.
Ross, K; Thomson, D. (1993). An
evaluation of parents' involvement in the management of their cerebral palsy
children.
Physiotherapy, August 1993, Volume 79, no. 8, 561-565.
Vidler, V. (1972). Use of puppets to reach the
emotionally disturbed. Instructor, 81(9).
Vilozni D; Bar-Yishay E; Gur I; Shapira Y; Meyer S; Godfrey S (1994)
Computerized respiratory muscle training in children with Duchenne muscular
dystrophy. May;4(3), 249-55
White, J. & Allers, C. T. (1994). Play therapy
with abused children: A review of the literature. Journal of Counseling and Development, 72, pp. 390-394.
