About PF. Magic

Description from the original PF. Magic website. SOURCE

PF. Magic, from its toy- and game-filled offices in San Francisco's Multimedia Gulch, is known for its unique and innovative interactive characters, games and digital toys. Since its start in 1991, the Company has developed several award-winning, cutting-edge products.

The Company's management team brings a wealth of experience in the consumer electronics, video game, entertainment and multimedia businesses. With world-class expertise in interactive character development, 3-D game design, and on-line/networkable game development, PF.Magic focuses its creative and technical resources on the creation of unique entertainment. PF.Magic develops for CD-ROM based platforms including PC and Macintosh computers, as well as for the new generation of home entertainment systems such as the Sony Playstation and the Sega Saturn.

In the Fall of 1995, the Company unveiled its latest creative innovation Computer Petz with the intro-duction of Dogz, the first-ever pet to live in the world of its owners computer. The best-selling Dogz created a new category of computer interactivity, one in which real life on the computer is no longer a simulation. In June of 1996, PF.Magic grew the Computer Petz line with the introduction of Catz. Catz brought a new Petz purrsonality to the desktop, and added a new element to Computer Petz multiple-character interaction with the introduction of a mouse to drive the Catz wild.

The newest additions to the Computer Petz line Oddballz were released November 1996. These new and hilarious Computer Petz are definitely digital life at its wackiest! With the launch of Oddballz, PF.Magic also launched the first-ever, online Collectible Program, making Oddballz available for collection through partnerships with leading sites around the World Wide Web.

PF.Magic has joined forces with entertainment software powerhouse, Virgin Interactive Entertainment, under Virgins Affiliate Label program, ensuring broad retail distribution of the Computer Petz line. Dogz and Catz quickly rose to the top selling slots on leading international computer retail charts, including Software Etc., Babbages, and Electronics Boutique. The two have sold over 200,000 copies in the U.S., and remain top sellers in the United Kingdom and Asia. As of October, almost 1 Million people have visited the Computer Petz Web sites (www.dogz.com, www.catz.com), as a part of the cyber adoption program.

Other award-winning products released by PF.Magic, include Ballz, PaTaank and Max Magic. Ballz, the first truly 3-D fighting game for the Sega Genesis, Super Nintendo and 3DO platforms, received rave reviews from the critics for its innovative 3-D game play as well as its unique character design. PaTaank, the high-energy, point-of-view, 3-D pinball thrill-ride for the 3DO player, received praise by gaming critics and won several awards for innovative 3-D game play. Max Magic, the worlds first electronic magic kit, represented a breakthrough in interactive 3-D character design. Developed on the CD-I format for Philips Interactive Media, Max Magic received numerous prestigious awards for its uniquely intuitive way of entertaining and teaching children and adults about magic.

PF.Magic also worked with AT&T for two years to design and develop a voice and data modem peripheral for videogame and personal computer systems, and to create valuable new system software for online/ networked multiplayer games.

PF.Magic is a privately held company. Equity investors include Robertson Stephens & Company and AT&T.

How The Night Trap Led To Petz

Snippet about Petz from an interview with Rob Fulop. SOURCE

So how did PF Magic end up creating the Petz series?

RF: PF Magic was based on a piece of hardware we called the Edge. The Edge was a piece of hardware that you hooked up to your Genesis and the telephone and you can play games. The idea was that you could play Street Fighter with your friend across town. That's what it was about.

So AT&T put up the money, and Sega was a partner, and that was our big thing. We were going to make this system and make games for it. That was the idea that threw up PF Magic. But then AT&T pulled out because, again, the machine was too expensive to build. So now we were left with about half of our funding and we just had to figure something out, which was when we came up with Petz.

What was the inspiration for Petz?

RF: There are two vectors that created the path. Vector number one was I had made a game called Night Trap, which had been released at that point by this other company, Digital Pictures. And it was becoming very controversial because we had shown imagery of girls being dragged off by monsters and there was a big political scandal around Night Trap. To me personally, it was kind of silly.

It was also deeply embarrassing, because friends of mine, my parents and my girlfriend, didn't really get games. All they knew was that a game that I had made was on TV, being talked about as being bad for kids. And, you know, Captain Kangaroo came on TV and said, "This is bad for kids." It was horrible.

At the time the Captain Kangaroo show, every kid watched it. He was just an icon of kids' television. I watched him, I think he was on every Saturday morning. Anyway, they dragged him up and he said something bad about Night Trap and it was just embarrassing. I fell out with my girlfriend about it, because I thought it was completely bullshit criticism.

But I decided that the next game I made was going to be so cute and so adorable that no one could ever, ever, ever say that -- it was, like, sarcastic -- what's the cutest thing I could make? What's the most, you know, sissy game that I could come out with?

Then at the same time it was in December, and I would always go at the end of the year to see Santa Claus at department stores. You go to Santa Claus and he'll tell you exactly what's going to sell that Christmas. He knows better than anyone, because all the kids talk to him. His job all day is to ask kids what they want. So if you want to know what's going to sell this year, just go and talk to Santa Claus and this guy will tell you exactly what's going to sell. I did that every year.

So we go into Macy's, talk to Santa, and actually he goes, "It's still the same, the most popular thing that kids ask for every year is a puppy. For the last 50 years." So the two just came together. It was make a puppy. That was really how the idea came out.

And then we also had this technology. The first game we made with 3D was called Ballz. It was a fighting game and all the characters were made out of spheres. You know, the sphere is the only thing that wherever you put the camera, it's still a sphere. You don't need to store different images of the sphere. You just need to know the sphere has a radius of five and is blue and you can create it.

So if you build the whole character out of spheres, all you have to do is keep track of is how the spheres connect. So we had a patent on that, we built the whole world using that. Then we made it into a pet. We built it into a dog. People loved it.

It doesn't sound like it was inspired by any previous games at all…

RF: It wasn't; there was no computer pets before that. But about the same time, though, Japan had come out with the Tamagotchi toy. The two were kind of built at the same time. I don't think we even knew about each other. It's kind of interesting when designers think along the same lines, though. Ours was on the computer, theirs was on a little hand held thing.

It just struck a nerve. People liked it. People really liked it. I mean, it's a very obvious thing. It's a puppy dog. The way we sold it was, we'd give it away. We sold it the same way real puppies are sold. We give it to you for 10 days and ask for it back. Give a puppy to a kid and ask him for it back five days later, just go ahead see what he says. In sales this is called "the puppy dog close."

We gave you five days' worth of food, and if you want more food you've got to call us and we give you a whole lifetime supply for 20 bucks. It's not little puppies that don't cry, you know. They'd whimper and cry and then you have to delete it and it asks, "Do you want to delete me?." I mean who can say yes?

No, you sit there with it whimpering and your little puppy would say, "Do you want to delete me?" And who can delete it? It's cruel, it's a little puppy and you won't feed it. It's impossible to say yes, so we sold a lot of them. We sold a ton of them.

It was completely viral, and then after a couple of weeks your pet would have a puppy, and then you could keep it yourself or give it to someone else. You put your floppy in and picked your puppy, and give it to a friend, and so there is another spawning. That was the way it works; it was great.

The virtual pet has become quite an established genre now, Nintendogs, etc.

RF: At this point yes. Nintendogs sure. Petz is still a brand. Ubisoft still puts it out every year. They're not as lifelike as they used to be, because they became more realistic. If you see our original dog, it wasn't realistic at all but it was really likeable. It was cute, it was loving. The ones now look like real dogs but because they spend so much bandwidth on making it look real it's hard to make the thing do a lot of different things. It's a trade off.

What do you think it is that attaches people to these virtual pets?

RF: It's that they need you. You're needed. So you come home and this thing is like, in your face. It needs you. If you're not there you know that it's going to be very unhappy -- or it could die, right? Isn't that why you get attached to plants and puppies and kittens? I don't think a virtual pet is any different, we just create them. And when you come home, what do they do?

Jump up and down, somersaults…

RF: Right, and you love that, right? So that's what we get our puppy to do. Whenever you move your mouse he'd follow your mouse. They can't wait to be petted. When you put them out they come running over, right? That was their biggest excitement. Basically I think people are attached to those things to be needed, which is why a baby is the ultimate.

Should Virtual Petz Die?

From an article published on dogz.com. SOURCE

INTERACTIVE VIRTUAL PETS - A LIFE AND DEATH ISSUE

Should Virtual Pets Die?

The release of Bandai's "Tomagotchi" and Tiger Electronics' "Giga Pets" on American toy store shelves this month brings a serious mortality question to the kennel - "to be or not to be?" The life expectancy of these virtual pets is dependent on how long their hearts, or batteries, keep pumping - usually only about 100 days. Can the children of America handle the trauma of a premature ending to their relationships with their virtual furry friends? Does it have to end this way? Is there any solution to this imminent epidemic of heartbreak?

Fortunately, the answer is a resounding "yes." PF.Magic, a pioneer in interactive pet technology, has brought happiness to children and adults around the globe for the past two years with its "Computer Petz" line. Computer Petz, which have sold more than 750,000 units since 1995, include "Dogz," "Catz" and wacky creatures called "Oddballz." These Petz live on your computer desktop, and like their Tamagotchi and Giga Pet descendants, develop based on the amount of love and attention they receive from their owners. Artificial Intelligence serves as these Computer Petz' virtual brain, and they offer all the benefits of pet ownership without the hassles of cleaning up, daily feedings or pet cemeteries.

Most importantly, Computer Petz have mastered what we can only fathom - immortality. It's true, all Computer Petz live forever, or as long as the computer desktop on which they live keeps ticking. No dog years or nine lives here, scrappy, because there's simply no need. Imagine everlasting life and health - or alternatively - the needless pain, suffering and sadness that accompanies every purchase of a "here today, gone tomorrow" pet key chain. The choice is yours. Put an end to the madness.

PF. Magic, from its toy and game-filled offices in San Francisco's "Multimedia Gulch," is known for its interactive living characters and innovative entertainment products. Since its start in 1991, the Company has developed several award-winning, cutting-edge products. In the Fall of 1995, PF.Magic introduced the best-selling Computer Petz line with Dogz. In 1996, Catz leaped onto the scene followed by those wacky Oddballz. More than 750,000 Computer Petz have been adopted around the globe and are still alive and kicking. PF.Magic is privately held.

Computer Petz Web site is located on the World Wide Web at www.pfmagic.com.

Socially Intelligent Virtual Petz

Paper for the 1997 Socially Intelligent Agents AAAI Fall Symposium. SOURCE

Socially Intelligent Virtual Petz
Adam Frank, Andrew Stern, Ben Resner

PF.Magic
501 2nd St, Suite 400
San Francisco, CA 94107
+1 415 495 0400

Abstract

We have developed a series of lifelike computer characters called Virtual Petz. These socially intelligent agents live on your PC computer desktop. The Petz are autonomous characters with real-time layered 3D animation and sound. Using a mouse the user moves a hand-shaped cursor to directly touch, pet, and pick up the characters, as well as use toys and objects. Virtual Petz grow up over time and strive to be the user's friends and companions. They have evolving social relationships with the user and each other. To implement these agents we have invented hybrid techniques that draw from improvisational drama, cartoons, classical AI and video games. As of June 1997 over a million copies of virtual Petz have been sold around the world.

Introduction

Traditional digital interactive entertainment has centered around characters for years. Most entertainment products contain computer characters that represent the user, and the user puppeteers the character in its environment. In Petz products, the characters are completely autonomous, the user is represented as a hand cursor, and the pets perceive the user as just another socially intelligent autonomous character. In this way the Petz and the user are on essentially equal footing.

The virtual Petz concept has been a good way for us to explore the complicated issues in creating interactive, real-time autonomous characters. Choosing dogs and cats as our agents helps us in a few ways. People understand what pets are and essentially know how to interact with them. This is critical for producing believable characters. Pets are also a good choice because they are expected to behave in ways that we can successfully implement. They have relatively simple gestures, language and cognitive abilities. Also, animated dogs and cats have well established cartoon archetypes. We base the look and behavior of the Petz on these archetypes.

The virtual Petz experience is non-goal oriented. We allow users to explore the characters and their toys in any order they like. This freedom allows users to socialize with the Petz in their own way and at their own pace. This also encourages users to come up with their own interpretation of their pet's feelings and thoughts. Often the scope of their interpretations exceeds what we originally planned for.

Developing Relationships with Virtual Petz

The goal of the Petz is to build an intimate relationship with user. Therefore the pet's primary motivation is to receive attention and affection. They feed off of this interaction. Without it they become lethargic, depressed, and if ignored long enough, they will run away.

The most direct way the user shows affection is through petting. Using the left mouse button users can pet, scratch and stroke. The pet immediately reacts in a variety of ways depending on what spot on its body is being petted, how fast, and how the pet feels at the time. We call this direct interaction. The fact that you can touch the character is a very effective way of building an intimate relationship. It also goes a long way in creating an illusion of life.

We've tried to make the Petz have equal footing in their relationship with the user. The toys and objects in the pet's environment have direct object-like interaction for both the user and the Petz. Petz have full access to the toy shelf, and if they really want something, they will just go and get it themselves. This helps express the unpredictability and autonomous nature of the Petz. It also requires users to share control of the environment with the Petz. The act of sharing the space is a further strengthens the owner-pet relationship.

By picking up and using a toy, the user can initiate play. For example, throwing a ball may initiate a game of fetch, or holding a tugtoy in front of a pet may initiate a game of tug-of-war. Similarly a pet can get its own toy and bring it to the user to initiate play. This cooperative decision-making helps build the relationship.

We have created a variety of personalities - playful terriers, grumpy bulldogs, hyper Chihuahuas, finicky Siamese cats, lazy Persian cats, aggressive hunter cats, timid scaredy cats and so on. Additionally each pet has its own individual likes and dislikes, spots and body coloration, and personality quirks. Users get to play with a pet and see if they like them before deciding to adopt. Once adopted, the user gives them a name. This individual variation allows the user to develop a unique relationship with a particular pet. Every owner-pet relationship is different.

Communicating with Virtual Petz

Petz communicate with the user in a variety of ways. Emotion can be expressed through different facial expressions (eyebrows, mouth, ears), styles of movement and body language (sad walks, happy trots, various postures, a variety of tail motions), and sounds (excited playful barks and meows, sad whines and whimpers, yelps of pain, etc). They get fat when they eat too much and get skinny when hungry.

Petz communicate their intentions and desires through their actions. We have created a broad base of behavior for the Petz -- they can eat, sleep, play, attack, groom, hiss, explore and so on. When a pet wants attention they may start barking or meowing. If hungry they may go searching for food, hunting mice, or start begging. When they are upset or scared they may run and hide, act aggressively, or cower and shiver.

It is important to note that the only way the user can understand what the Petz are feeling is to interpret their actions and physical cues, in the same way an audience interprets an actor's performance. We do not display bar graphs or text messages describing the pet's internal variables, biorhythms or emotional state. By forcing a natural interpretation of the pet's behavior, we don't break the illusion of a relationship with something alive.

The user can communicate to their pet in a variety of ways. In addition to petting users can gesture to their Petz. By double-clicking in an empty space on the screen the hand cursor animates a "come here" motion and plays a whistling sound. Petz tend to stop what they're doing and come to the user's cursor.

Users can train their Petz to do tricks for treats. When holding a treat, users can make the hand cursor do directional hand motions (up, down, left, right) to which a pet might jump, sit, lie down, rollover, and so on. If you shake the treat over a toy, a pet may attempt a trick using that toy, such as balancing on a ball or fetching.

In general users can reward or discipline any behavior. Methods of rewarding include a giving treats, feeding, petting, or presenting its favorite toy. Discipline is achieved by squirting a pet with a water spray bottle. (This is the SPCA-approved method of discipline.) In this way users can modify a pet's overall behavior through positive and negative reinforcement.

The personalities of each pet can change under extreme circumstances. For instance, if any pet is underfed, it will begin hunting and acting aggressively. If a pet is overfed, it will become fat and lazy. Depending on how the user treats their pet, each pet's personality can slide along a broad spectrum of personalities.

Relationships between Virtual Personalities

In our newest product, Petz II, released in October 1997, multiple characters can interact with the user and each other at the same time. When two Petz first meet each other, typically as puppies or kittens, they cautiously investigate each other. As they wrestle and play they form relationships and attitudes towards one another. Some will become best friends and will often groom each other, playfully nip at each other, sleep side by side, bring each other treats, and so on. Other pets may become enemies and will fight, chase and terrorize each other throughout their lives. A dominant dog might steal food and toys from a submissive dog.

Adult Petz tend to nurture and protect younger Petz. They will comfort them if they get scared and carry them to a pillow to sleep. Younger Petz will follow the adults around and attempt to copy their behavior. However once adolescent, a pet will form an adult relationship with the other adults.

Most Dogz and Catz tend to dislike each other. Typically they will growl, bark, hiss, fight and chase each other. However if they are introduced to each another at a young age and grow up together, they may become friends.

The interplay between the variety of personalities gives rise to many dramatic situations. For example, a tired old dog won't be able to go to sleep when a young playful kitten is bouncing around him. The dog may try to take a nap after distracting the kitten with a bowl of food, but the kitten's short attention span defeats this strategy.

Catz are always on the lookout for mice. If the user takes out the cheese and puts it in front of the mouse hole, the mouse may sneak out for a nibble. The cat will immediately begin stalking it for an attack. If the user then brings out a dog, a three-way chase ensues. This usually ends with the dog chasing the cat away and the mouse getting a free meal.

Even a loving relationship between two buddies can be upset when the user pays more attention to one pet than the other. The jealous pet will become angry and upset and may even act aggressively to the user or the other pet. However the two Petz will eventually make up and return to their friendly relationship.

Should Virtual Petz Die?

Petz begin as puppies or kittens and age over time. Users witness their pet develop and can affect what kinds of personalities they turn out to have. However at a certain point the Petz stop aging and do not die.

This has been an issue which our design team has fought over back and forth. Some argued that if the pet eventually dies it makes the lifecycle more important. Immortality may devalue the experience by making the user's time with their pet less precious.

On our website we recently conducted an informal poll asking users if they thought virtual Petz should die. 65% of the responses said they should not die. 30% said they would like to be able choose if their Petz should die. Only 5% said they wanted their Petz to eventually die. After long discussions with our marketing department, it was decided that the disadvantages of death outweighed the advantages.

Instead of death we decided that the only way a user can lose their pet is if they severely mistreat them. After a few days of ignoring a pet, it will actively begin whining for attention. If the user does nothing to remedy this situation, the pet begins acting more and more neglected, even doing "naughty" behaviors to get attention. Eventually the pet will simply run away and not be available to the user anymore. Because the user's actions have consequences, we have added some value to the relationship.

Implementation Principles

To create these socially intelligent agents on a personal computer, we had to develop new display techniques. Our high-degree of interactivity and lifelikeness requires the use of real-time 3D animation. Pre-rendered animation, however beautiful it may be, cannot be as responsive, dynamic, or as varied as a we require. But to deliver this real-time animation on a PC, drastic optimizations on the display level are required. Our proprietary method of achieving this is unique. We chose to err on the side of frame rate versus a photo-realistic image that moves poorly.

The Petz products use a network of basic 3D body animation which has a high resolution of branch points. For example, from any basic pose such as standing or sitting, the pet can transition to any other pose. As the pet's general body animation moves across this network, additional animation is layered on top. This allows the pet to look in all directions no matter what its body doing. Also we layer breathing, blinking, meowing, head shaking and even character specific posture. In fact the look of the each breed and their ages are layers. The subsequent number of permutations is enormous and therefore passes a critical threshold in human perception. This method greatly enhances our ability to create an illusion of life.

Using this display technology we have developed many behavior mechanisms that operate simultaneously. There are small mechanisms to control blinking and breathing and a variety of cognitive mechanisms to keep track of multiple goals, different plans to execute each goal, multiple emotions towards other characters and objects, attitudes towards other characters and objects, and personality-specific quirks.

In general to achieve a real, unpredictable, non-repetitive quality in these mechanisms, we use constrained randomness. For example, there are rules for each individual pet that determine which toys it most likes to play with, but there is some randomness in that decision. This results in the possibility that a pet might play with any toy, not just the ones it likes the most. A lot of development time is spent tweaking the constraints on the randomness to achieve a good balance between consistent personality and unpredictability. We are focusing on the user's perception of the character and trying not to over-engineer the artificial intelligence.

When making consumer software there are major constraints on both design and implementation. Our project has to be used by grandmothers and four-year olds and run on very slow, old computers, on multiple operating systems, and still perform as a seamless, real-time experience. These constraints force us to design with a bottom-up approach - always paying the most attention to what the final user will perceive. We focus our energies on what we can show and not waste valuable time implementing an elaborate system that we cannot express onscreen.

Conclusion

The Petz products are an experiment that turned out to be relatively successful. The product is equally popular among girls and boys, kids and adults. This is unusual in consumer entertainment software. In the program, users can take a bitmap "photo" of their Petz which they can save to disk. We encourage users to e-mail their photos to us, which we post on our website. We've had a tremendous response, receiving thousands of photos, some with personal stories attached and some with elaborately painted scenes of their Petz on Mars, or dressed up in tuxedos, or windsurfing, or flying World War II planes. We receive many letters from these kids describing their relationships with their individual Petz. Some kids we know get up especially early every day to feed them. Some of the stories describe feelings and complex behavior that we never put in the product. It seems that through combining a direct interaction interface, strong and recognizable personalities, and some fundamental social intelligence, an intimate owner-pet relationship can arise.

We've found that developing socially intelligent agents requires people with interdisciplinary skills from both technology and the arts. Until recently, building lifelike artificially intelligent characters has been treated as a computer science problem. Character development, in any medium, is fundamentally an artistic process which combines elements of drama, storytelling and expressive action. Lifelike computer characters are ultimately perceived through the filter of human interpretation. No computer character can be perceived as truly alive and intelligent unless its builders always keep in mind what the final user will perceive.

Of course, there is a tremendous amount to explore in this area and we realize there is much more work to be done.

Acknowledgements

We thank Rob Fulop, John Scull, Jonathan Shambroom, Alan Harrington, Ted Barnett, Jeremy Cantor, and the rest of the PF. Magic Petz team.

Creating Emotional Relationships With Virtual Characters

Chapter in Emotions in Humans and Artifacts. SOURCE

Creating Emotional Relationships With Virtual Characters
Andrew Stern

1: Introduction

During the workshop on "Emotions in Humans and Artifacts" participants presented research addressing such questions as, how do emotions work in the human mind? What are emotions? How would you build a computer program with emotions? Can we detect the emotional state of computer users? Yet as the discussions wore on, instead of providing some answers to these questions, they persistently raised new ones. When can we say a computer "has" emotions? What does it mean for a computer program to be "believable"? Do computers need emotions to be "intelligent", or not?

These scientific, engineering and philosophical questions are fascinating and important research directions; however, as an artist with a background in computer science and filmmaking, I found my own perspective on emotions in humans and artifacts generally left out of the discussion. I find the burning question to be, what will humans actually do with artifacts that (at least seem to) have emotions? As a designer and engineer of some of the first fully-realized, "believable" interactive virtual characters to reach a worldwide mass audience, Virtual Petz and Babyz (PF Magic 1995, Mindscape 1999), I know this is a question no longer the domain of science fiction. Today millions of people are already encountering and having to assimilate into their lives new interactive emotional "artifacts."

Emotions have been a salient feature of certain manmade artifacts, namely stories and art, long before the scientific study of emotion began. Looking to the past we see a tradition of humans creating objects in one form or another that display and communicate emotional content. From figurative painting and sculpture to hand puppets and animated characters in Disney films such as Snow White, manmade artifacts have induced emotional reactions in people as powerful and meaningful as those generated between people themselves.

Now we are at point in the history of story and art where humans can create interactive artifacts, using the computer as a new medium. We can now write software programs that can "listen" to a human, process what it just "heard", and "speak" back with synthetically generated imagery and sound, on machines that are already in the households of millions of families around the world. The new urgent challenge for artists and storytellers is to create interactive artifacts that can induce the same emotional reactions and communicate emotional content as traditional non-interactive stories and art have done. In fact because the computer theoretically can be programmed to tailor the experience to an individual, it could become the most powerful medium of all for creating affective stories and art.

The question becomes, how do you do that? How do emotionally powerful stories and art "work" anyway? Alas, creating an artifact that produces a meaningful emotional reaction in a human is considered an "art" itself. Although techniques and advice on the artistic process have been published - works such as The Art of Dramatic Writing by playwright Lajos Egri, The Illusion of Life by Disney animators Thomas and Johnston, Letters to a Young Poet by poet Rainer Rilke - the act of creating emotionally powerful artifacts is by and large considered elusive, mysterious and unquantifiable. Even in art school the typical approach is to teach students to imitate ("master") traditional styles and techniques, after which it is hoped the student will be ready to "find their own style," which sometimes never happens.

Naturally it is difficult to discuss the art of creating emotionally powerful artifacts in the context of a scientific workshop, in the way one would approach a computer science or engineering problem - which helps explain the general reluctance to research the topic, and the not-so uncommon attitude among artificial intelligence researchers that the topic is "mushy", ill-formed, or worst of all, unimportant. Art and entertainment is considered to be fun, not a serious pursuit. This view is short-sighted. On the contrary stories and art are among the most serious and meaningful pursuits we have. We communicate ideas and experiences to each other in this way. The fact that the problem is, to a degree, mushy and unquantifiable, makes it all the more challenging and difficult to undertake.

This paper puts forth virtual characters as an emerging form of manmade artifact with emotional content. Virtual characters go by a few other names in the AI research community, such as believable agents, synthetic actors and synthetic personalities. These are embodied autonomous agents that a user can interact with in some fashion, animated as real-time graphics or built as physical robots, which appear to have personality, emotion and motivation, designed to be used in art or entertainment. Over the past decade several media labs have been exploring the issues involved in building virtual characters (such as Bates et al 1992, Blumberg 1995, Perlin 1995 and Goldberg 1997, Hayes-Roth 1996, Elliot et al 1997). Some groups have designed architectures and implemented prototypes that have been demonstrated at academic conferences. But it should be made clear that in the business of creating emotional artifacts, in the final analysis, prototypes and demos are not enough. The point of creating emotionally powerful experiences, whether interactive or not, is to induce a reaction in an audience, in "users." These creations must be experienced by the general public to serve the purpose for which they were created in the first place. Until this happens the work created in closed-door media labs is ultimately incomplete.

The public has been consuming interactive entertainment for two decades now, in the form of software products from the videogame and computer game industry. Unfortunately the experiences offered in these games are mostly juvenile, primarily focused on fighting, shooting, racing and puzzle-solving, and the virtual characters offered in them are most often shallow, one-dimensional cardboard-cutouts. Such games can be emotionally powerful experiences for those who play them, but they do not appeal to the majority of the population. Very few successful pieces of interactive entertainment or art have been made with emotional content for a mass audience - that is, the kind of "personal relationship" stories that books, theater, television and movies offer, or the kind of "high art" exhibited at museums and art shows (Stern 1999a, Mateas 1999).

This remainder of this paper suggests new ways to employ virtual characters to create emotionally powerful interactive experiences, using our Virtual Petz and Babyz projects as case studies. The techniques used to create these projects will be presented and discussed, with emphasis on the importance of design. Finally we will attempt to address the question of what it could mean for a human to have an emotional relationship with a virtual character.

2: Do you feel it? The case for emotional relationships

Animation and artificial intelligence technologies for creating real-time interactive virtual characters are currently being researched and developed in academic labs and industry companies. We are told that soon we will have virtual humans that look photorealistic, with behavior driven by some degree of AI. Many are working with the intention that these characters will become functional agents, at our command to perform a variety of complicated and menial tasks. They will learn our likes and dislikes, able to autonomously communicate and negotiate with others. And they will become teachers in the virtual classroom, always ready and willing to answer our questions.

At first glance it seems natural that adding emotions to these virtual characters should greatly enhance them. After all, real people have emotions, so virtual human characters should have them too. Some researchers in neuroscience and psychology point to emotion as an important factor in problem solving capabilities and intelligence in general (Damasio 1994). As Marvin Minsky put it, "the question is not whether intelligent machines can have emotions, but whether machines can be intelligent without any emotions" (Minsky 1985). Virtual characters may very well need emotions to have the intelligence to be useful.

But when thinking in terms of a virtual character actually interacting with a user, is emotional behavior really appropriate for these types of applications? In real life it is arguable that interactions with "functional agents" (e.g., waiters, butlers, secretaries, travel agents, librarians, salespeople) are often best when emotions are not involved. Emotional reactions can often be irrational, illogical and time-consuming, which work against the efficient performance of tasks. Of course in any transaction, politeness and courtesy are always appreciated, but they hardly qualify as emotional. We expect teachers to be a bit more personable and enthusiastic about their material than a travel agent, but do we want them to get angry or depressed at us?

While emotions may be required for intelligence, I would argue that the most compelling interactions with virtual characters will not be in the area of functional agents. If a user encounters a virtual character that seems to be truly alive and have emotions, the user may instead want to befriend the character, not control them. Users and interactive virtual characters have the potential to form emotional relationships with each other - relationships that are more than a reader's or moviegoer's affinity for a fictional character in a traditional story, and perhaps as meaningful as a friendship between real people. By an emotional relationship we mean a set of long-term interactions where the two parties pay attention to the emotional state of the other, communicate their feelings, share a trust, feel empathetic, and establish a connection, a bond.

2.1: Virtual friends

The recent success of several "virtual pet" products, popular among both kids and adults, offers some support for this idea. The most sophisticated of these characters are animated on the computer screen, such as Dogz and Catz (PF.Magic 1995 - 1998), and Creatures (Grand et al 1997), but some are displayed on portable LCD keychain toys or even embodied as simple physical robots, such as Tamagotchi (Bandai 1996), Furby (Tiger Electronics 1998), and Aibo (Sony 1999). Users "nurture" and "play" with these pets, feeding them virtual food, petting them with a virtual hand, and generally giving them attention and care lest they runaway or die. While there can be some blurring into the domain of videogames, in their purest form virtual pets are not a game, because they are non-goal-oriented; it is the process of having a relationship with a virtual pet that is enjoyable to the user, with no end goal of winning to aim for.

As of this writing, there have been no completed formal studies of virtual pets; a study is currently underway by (Turkle 1999). In our experience with the Dogz and Catz products, our anecdotal evidence suggests that depending upon the sophistication of the virtual character, the emotional relationship that a user can form with it ranges anywhere from the attachment one has to a favorite plant to the bond between a master and their dog. Children are the most willing to suspend their disbelief and can become very attached to their virtual pets, playing with and feeding them everyday. It is precisely those irrational, illogical and time-consuming emotional interactions that may hamper a functional agent that are so engaging and entertaining here. (Please refer to the Appendix of this paper to read real customer letters we have received about Petz.) Only a few Petz users, mostly technology-oriented adult men, have requested that their Petz be able to perform functional tasks such as fetching e-mail.

What is most interesting about the phenomenon of virtual pets are not the toys and software themselves - some of which have minimal interactivity and little or no artificial intelligence driving them - but the fact that some people seem to want to form emotional relationships with them. Some appear quite eager to forget that these characters are artificial and are ready and willing to engage in emotional relationships, even when some of the virtual pets offer little or no reward or "warmth" in return. This offers some promise for the public's acceptance of the concept of a more advanced virtual friend.

As commercially successful as these virtual pets are, it seems likely that emotional relationships at the level of favorite plants or pets will be far easier to accomplish than the level of friendship between two adults. An owner-to-pet relationship dynamic is much simpler than a person-to-person one, and much less communication is required between the two parties. Most importantly, the relationship is inherently unequal. An owner of a real pet chooses (even purchases!) their real-life cat or dog. Therefore the act of buying a virtual pet as a toy or piece of software does not violate the hierarchy of a real-world owner-to-pet relationship.

As people we do not get to choose which other people will be friends with us. Friends, by definition, choose to be friends with one another. Therefore, even if we create an interactive virtual character that can perform all the behaviors required for an emotional relationship between human adults, as a manmade artifact that can be bought and sold, could a "true" friendship could be formed at all? This is an open question that invites exploration.

2.2: Interactive stories

Stories have long been our primary way to observe and understand our emotional relationships. If stories could be made interactive - where users could immerse themselves in virtual worlds with characters they could talk to, form relationships with, touch and be touched by, and together alter the course of events, literally creating a new story in real-time - then we would have a new form of interactive entertainment that eclipses videogames. Like traditional stories from books, theater, television and movies, an interactive story would be affecting and meaningful, but made all the more personal because the user helped shape it and create it (Stern 1998).

Virtual characters programmed to simulate the dynamics of emotional relationships could be used as starting points for creating interactive stories. In her recent book, Hamlet on the Holodeck: The Future Of Narrative in Cyberspace, Janet Murray suggests that interactive virtual characters "may mark the beginning of a new narrative format" (Murray 1997). As a first step in this direction, instead of relying heavily on planning to generate story plots as some previous story researchers have done (such as Meehan 1976, Pemberton 1989, Turner 1994), a developing and ongoing emotional relationship itself could serve as a narrative. For example a user and a virtual character could meet and get to know each other, begin to develop trust for one another, and perhaps (accidentally or not) violate that trust in some way, causing the relationship to take a downturn. The relationship could progress from there in many ways, perhaps recovering, ending, cycling between highs and lows - much like real-life relationships. There are several traditional stories that follow this pattern, such as "boy meets girl, boy and girl fall in love, boy loses girl", and so on.

2.3: Interactive art

Autonomous interactive virtual characters are only just beginning to make their way into installation and performance art. The goal of Simon Penny's 1995 "Petit Mal" was, according to the artist, "to produce a robotic artwork which is truly autonomous; which was nimble and had 'charm'", to give "the impression of being sentient" (Penny 1997). Petit Mal was a tall, thin robot on bicycle wheels which could quietly and gently move about in a room, able to sense when it was approaching walls or people. Celebrated video artists Lynn Hershman and Bill Viola have begun experimenting with combining video imagery of people and some simple interactivity. Mark Boehlen and Michael Mateas recently exhibited "Office Plant #1", a robot plant that will bloom, wither and make sounds in response the mood of your office environment (Boehlen and Mateas 1998). Other efforts include the large, destructive autonomous robots from the industrial performance art of Survival Research Labs, the RoboWoggles (Wurst and McCartney 1996), and the robot installations of Alan Rath. The potential for exploration and discovery in this area seems untapped and wide open.

3: I need your love: Virtual dogs, cats and babies

Recognizing a dearth of consumer software with characters that displayed emotions or personality, the startup company PF.Magic was formed in 1992 with the mission to "bring life to entertainment." We wanted to break out of the mold of traditional videogames (e.g., flight simulators, sports games, running-jumping-climbing games, shooters, puzzle games) to create playful new interactive experiences with emotional, personality-rich characters.

3.1: Dogz and Catz

By 1995 the personal computer was powerful enough to support the real-time animation we felt was required for a convincing virtual character. The first Dogz program, originally conceived by company co-founder Rob Fulop and created by Adam Frank and Ben Resner, was a simple idea: an animated virtual dog that you could feed and play with. As a product it was very risky; an emotional relationship as the sole basis of an interactive experience had never been done before. It was unknown at the time if anyone would pay money to interact with a virtual character in this way.

The program quickly generated interest from a wide range of customers male and female, kids and adults, which is typically unheard of in entertainment software. We followed up Dogz with a companion product, Catz, establishing the explicit design goal to create the strongest interactive illusion of life we could on a PC. To imbue the Petz characters with personality and emotion we began cherry-picking techniques from computer animation and artificial intelligence, to construct what eventually became a powerful real-time animation engine tightly integrated with a goal-based behavior architecture.

The Virtual Petz characters are socially intelligent autonomous agents with real-time 3-D animation and sound. By using a mouse the user moves a hand-shaped cursor to directly touch, pet, and pick up the characters, as well as use toys and objects. Petz grow up over time on the user's computer desktop and strive to be the user's friends and companions. The interaction experience is non-goal oriented; users are allowed to explore the characters and their toys in any order they like within an unstructured yet active play environment. This freedom allows users to socialize with the Petz in their own way and at their own pace. This also encourages users to come up with their own interpretation of their pet's feelings and thoughts. To date the Virtual Petz products have sold over two million copies worldwide.

The goal of the Petz characters is to build an emotional relationship with user. Their behaviors are centered around receiving attention and affection. They feed off of this interaction. Without it they become lethargic, depressed, and if ignored for long enough, they will run away.

The most direct way the user can show affection to the Petz is through petting. By holding down the left mouse button users can pet, scratch and stroke with a hand cursor; the Petz immediately react in a variety of ways depending on what spot on their body is being petted, how fast, and how they feel at the time. Users can also pick up the characters with the right mouse button and carry them around the environment. We found that being able to (virtually) touch and hold the characters to be a very effective way of building emotional relationships and creating the illusion of life.

The Petz have equal footing in their relationship with the user. The toys and objects in their environment have direct object-like interaction for both the user and the characters. Petz have full access to the toy shelf, and if they really want something, they have the freedom to get it themselves. This helps express the unpredictability and autonomous nature of the Petz. It also requires users to share control of the environment with them. For example, by picking up and using a toy the user can initiate play. Throwing a ball may initiate a game of fetch, or holding a tugtoy in front of a pet may begin a game of tug-of-war. Similarly, a pet can get its own toy and bring it to the user to initiate play. The act of sharing control of the environment and cooperative decision-making helps further strengthen the relationship.

We have created a variety of personalities - playful terriers, grumpy bulldogs, hyper Chihuahuas, lazy Persian cats, aggressive hunter cats, timid scaredy cats and so on. Each individual character has its own likes and dislikes, spots and body coloration, and personality quirks. Users get to play with individual Petz to see if they like them before deciding to adopt. Once adopted, the user gives them a name. This individual variation allows the user to develop a unique relationship with a particular character. Each owner-pet relationship has the potential to be different.

3.2: Babyz

Our newest virtual characters, Babyz, released in October 1999, have the same non-goal-oriented play and direct interaction interface as the Petz. The user adopts one or more cute, playful babies that live in a virtual house on the computer. The Babyz have a similar cartoony style as the original Petz, but have more sophisticated, emotive facial expressions, and some simple natural language capability. Babyz vary in size, shape and personality, and in some ways appear to be smarter and cleverer than real one-year-old babies would be. They want to be fed, clothed, held and nurtured, but are also quite playful and mischievous. Users can think of themselves as their parent or babysitter, which ever they feel most comfortable with.

The user can nurture a Babyz character by holding and rocking it, tickling it, feeding it milk and baby food, putting on a fresh diaper, giving it a bubble bath, soothing it if it gets upset, giving it medicine if it gets sick, laying it down to sleep in a crib, and so on. Play activities include playing with a ball, blocks, baby toys, music, dancing and singing, and dress-up. Through voice recognition Babyz can understand and respond to some basic spoken words (such as "mommy", "baby", "yes", "no", "stop that"), and can be read simple picture books.

The Babyz characters develop over time, appearing to learn how to use toys and objects, learning to walk and to speak a baby talk language. In this version of the product, they will always be babies - never progressing beyond a stumbly walk and simple baby talk. (All behaviors are pre-authored, with the user's interaction unlocking them over time, to create the illusion that the Babyz are learning.) If the user has more than one baby adopted, they can interact and form relationships with one another. Babyz can be friends and play nicely together or engage in long-term sibling rivalries. The program becomes an especially entertaining and chaotic experience with three active Babyz all getting into mischief at the same time (Stern 1999b).

3.3: Behaviors to support emotional relationships

To allow for the formation of emotional relationships between the user and the Petz and Babyz characters, we built a broad base of interactive behaviors. These behaviors offer the characters and the user the means of communicating emotion (or the appearance of emotion) to each other. This section will detail these interactions and behaviors, specifying in each the emotions we intended to be perceived by the user. Our hope is that by having the characters express emotion in a convincing and life-like way, the user will instinctively feel empathetic; and at the same time, if given the means to express their own emotions in return, users will feel like they are connecting to the characters on an emotional level.

Affection. Users can express affection to the characters by touching them and holding them with their mouse-controlled hand cursor. For Petz the touching is petting; for Babyz it is tickling. Petz express affection to the user by sweetly barking, meowing or purring, licking and nuzzling the hand cursor, and bringing the user a toy. Babyz will smile, giggle and laugh, coo, act cute, and say "mama" or "dada" in a loving voice. When perceiving a lack of affection, Petz will howl and yowl, sounding lonely; Babyz will cry and say "mama" or "dada" in a sad tone of voice. The intent is for the user to perceive the feelings of love, warmth, happiness and loneliness in the characters.

Nurturing. Users can feed, clothe and give medicine the characters. Petz express the need to be nurtured by acting excited when food is brought out, begging, acting satisfied and grateful after eating, or disgusted when they don't like the food. Babyz may hold out their arms and ask for food, whine and cry if hungry or need a diaper change, and may throw and spit up food they don't like. Users are meant to perceive feelings of craving, satisfaction, pleasure, gratefulness, dislike and disgust.

Play. By picking up a toy users can initiate play with one or more of the characters, such as a game of fetch or building blocks. Petz or Babyz may join the user's invitation to play, or get a toy of their own and begin playing by themselves or each other, waiting for the user to join them. A character may react if the user is ignoring them and instead playing with another character. Emotions intended to be perceived by the user include excitement, boredom, aggressiveness, timidity, laziness, and jealousy.

Training. Users can give positive and negative reinforcement in the form of food treats, water squirts (for Petz), and verbal praise or discipline to teach the characters to do certain behaviors more or less often. Petz or Babyz are programmed to occasionally act naughty, to encourage users to train them. During these behaviors users are meant to perceive the emotions of feeling rewarded, punished, pride, shame, guilt, and anger.

3.4: Effective expression of emotion

None of the aforementioned behaviors would seem believable to the user unless the characters effectively expressed convincing emotions. We found all of the following techniques to be critical for successful real-time emotion expression in virtual characters.

Emotion expression in parallel with action. During any body action (such as walking, sitting, using objects, etc.) Petz and Babyz characters can display any facial expression or emotive body posture, purr or cry, make any vocalization or say any word in any of several emotional tones. This allows a baby character to look sad and say it wants while it crawls towards the user. Catz can lick their chops and narrow their eyes as they stalk a mouse. Characters can immediately sound joyful when the user tickles their toes. We found if a virtual character cannot immediately show an emotional reaction, it will not seem believable. Timing is very important.

Emotion expression at regular intervals. Programming the characters to regularly pause during the execution of a behavior to express their current mood was a very effective technique. For example, while upset and crawling for a toy that it wants, Babyz may stop in place and throw a short tantrum. Or just before running after a ball in a game of fetch, Dogz may leap in the air with joy, barking ecstatically. These serve no functional purpose (in fact they slow down the execution of a behavior), but they contribute enormously to the communication of the emotional state of the character. Additionally, related to Phoebe Sengers' concept of behavior transitions (Sengers 1998), when Petz or Babyz finish one behavior and are about to begin a new one, they pause for a moment, appearing to "stop and think" about it, look around, and express their current mood with a happy bark or timid cower.

Emotion expression through customization of behavior. Some behaviors have alternate ways to execute, depending on the emotional state of the character. Mood may influence a character's style of locomotion, such as trotting proudly, galloping madly in fear, or stalking menacingly. A hungry character may choose to beg for food if lazy, cry and whine for food if upset, whimper if afraid, explore and search for food if confident, or attack anything it sees if angry. The greater the number of alternate ways a character has to perform a particular behavior, the stronger and deeper the perceived illusion of life.

Prioritization of emotion expression, and avoidance of thrashing. It is possible for a character to have multiple competing emotions, such as extreme fear of danger simultaneous with extreme craving for food. We found it to be most believable if "fear" has the highest priority of all emotion expression, followed by "craving" for food and then extreme "fatigue". All other emotions such as "happiness" or "sadness" are secondary to these three extreme emotional states. It is also important that characters do not flop back and forth between conflicting emotions, else their behaviors appear incoherent.

Theatrical techniques. Our characters are programmed to obey several important theatrical techniques, such as facing outwards as much as possible, looking directly outwards into the eyes of the user, and carefully positioning themselves relative to each other ("stage blocking"). If the user places a character offscreen, behind an object or at an odd angle, the characters quickly get to a visible position and turn to face the user. If two characters plan to interact with one another, such as licking each other's noses or giving each other an object, they try to do this from a side view, so the user can see as much of the action and emotion expression as possible in both characters. Similar techniques were identified in the context of virtual characters in (Goldberg 1997).

3.5: Animation and behavior architecture

Animation and behavior are tightly integrated in the Petz and Babyz architecture. An attempt was made during software development to construct a clean modular code structure; however both time constraints and practical considerations forced us to at times adopt a more ad hoc, hackish approach to implementation.

The lowest level in the architecture is an animation script layer where frames of real-time rendered 3-D animation are sequenced and marked with timing for sound effects and action cues, such as when an object can be grabbed during a grasping motion. Above this is a finite state machine that sequences and plays the animation scripts to perform generic but complicated low-level behaviors such as locomotion, picking up objects, expressing emotions, and so on. The next level up is a goal-and-plan layer that controls the finite state machine, containing high level behaviors such as "eat", "hide" or "play with object". Goals are typically spawned as reactions to user interaction, to other events in the environment, or to the character's own internal metabolism. Goals can also be spawned deliberately as a need to regularly express the character's particular personality or current mood. At any decision point, each goal's filter function is queried to compute how important it is for that goal to execute under the current circumstances. Filter functions are custom code in which the programmer can specify when a goal should execute. Part of the craft of authoring behaviors is balancing the output of these filter functions; it is easy to accidentally code a behavior to happen far too often or too seldom for believability.

Alongside instantiated goals are instantiated emotion code objects such as "happy", "sad" and "angry". Emotions have filter functions much like goals, but can also be spawned by the custom logic in states or goals. These emotion code objects themselves can in turn spawn new goals, or set values in the character's metabolism. For example, the filter function of a pet's "observe" goal may be activated in reaction to the user petting another pet. The "observe" goal is written to spawn a "jealousy" emotion if the other pet is a rival and not a friend. The "jealousy" emotion may in turn spawn a "wrestle" goal; any fighting that ensues could then spawn additional emotions, which may spawn additional goals, and so on. Goals are constantly monitoring what emotion objects are currently in existence to help decide which plans to choose and how they should be performed; states monitor emotions to determine which animations, facial expressions, and types of sound to use at any given moment.

Note that by no means did we implement a "complete" model of emotion. Instead, we coded only what was needed for these particular characters. For example, the Babyz have no "fear" emotion, because acting scared was not necessary (or considered entertaining) for the baby characters we were making. Fear was necessary however for the Petz personalities such as the scaredy cat. The emotion lists for Petz and Babyz varied slightly; it would have been inefficient to try to have both to use the same exact model.

At the highest level in the architecture is the "free will" and narrative intelligence layer. This is custom logic that can spontaneously (using constrained randomness) spawn new goals and emotions to convey the illusion that the character has intent of its own. This code is also keeping track of what goals have occurred over time, making sure that entertaining behaviors are happening regularly. It keeps track of long-term narratives such as learning to walk, sibling rivalries and mating courtship.

4: Feeling holistic: The importance of design

In creating an interactive emotional artifact, even the best animation and artificial intelligence technology will be lost and ineffective without a solid design. In this section we discuss the importance of the overall design of an interactive experience to ensure that a virtual character's emotions are effective and powerful to the user.

Concept and context. The type of characters you choose and the context you present them in will have a great impact on how engaging and emotionally powerful the interactive experience is. Judging by the confusing and poorly thought-out concepts in many pieces of interactive entertainment today, we feel this is a design principle too often ignored. In our products were careful to choose characters that people immediately recognize - dogs, cats, and babies - which allow users to come to the experience already knowing what to do. They immediately understand that they need to nurture and play with the characters.

Even though Petz and Babyz are presented in a cartoony style, we were careful to keep their behavior in a careful balance between cartooniness and realism. This was important to maintain believability and the illusion of life; if the Petz stood up on their hind legs and began speaking English to one another, users would not have been able to project their own feelings and experiences with real pets onto the characters. One of our maxims was "if Lassie could do it, our Petz can do it." That is, the Petz can do a bit more than a real dog or cat would normally do, but nothing that seems physically or mentally impossible.

A very important design principle in Petz and Babyz for supporting emotional relationships is that users play themselves. Users have no embodied avatar that is supposed to represent them to the characters; the hand cursor is meant to be an extension of their real hand. The characters seem to "know" they are in the computer, and they look out at the user as if they actually see them. There is no additional level of abstraction here; you are you, and the characters are the characters. This is akin to a first-person versus third-person perspective. If the user had an avatar that they viewed from a third-person perspective, the other characters would be required to look at that avatar, not at the user directly, thereby weakening the impact of their emotional expression.

Direct, simple user interface. Petz and Babyz are almost completely devoid of the typical user interface trappings of most interactive entertainment products. To interact with the characters, users operate the hand cursor in a "natural", direct way to touch and pick up characters and objects. No keyboard commands are required. All of the objects in the virtual world are designed to be intuitively easy to use; you can throw a ball, press keys on a toy piano, open cabinets and so on.

Of course this simplicity limits the amount of expressivity offered to the user. We cannot make objects and behaviors that require more complicated operation, such as a holding a baby and a milk bottle at the same time. While we could program some obscure arbitrary keyboard command sequence to accomplish this, we have chosen not to in order to keep the interface as pure and simple as possible. To allow the user more expressivity we would be required to add more intuitive interface channels, such as a dataglove or voice recognition. In fact instead of typing words to your Petz and Babyz (which you would never do in real-life of course), the latest versions of the products allow you to speak some basic words to the characters.

In general we feel that user interface, not animation or artificial intelligence technology, is the largest impediment for creating more advanced virtual characters. With only a mouse and keyboard users are very constrained in their ability to naturally their express emotions to virtual characters. When interacting with characters that speak outloud, users should be able to speak back with their own voice, not with typing. Unfortunately voice recognition is still a bleeding-edge technology. In the future we look forward to new interface devices such as video cameras on computer monitors that will allow for facial and gesture recognition (Picard 1997).

Natural expression. When trying to achieve believability we found it effective for characters to express themselves in a natural way, through action and behavior, rather than through traditional computer interface methods such as sliders, number values, bar graphs or text. In Petz and Babyz the only way the user can understand what the characters seem to be feeling is to interpret their actions and physical cues, in the same way an audience interprets an actor's performance. We do not display bar graphs or text messages describing the characters' internal variables, biorhythms or emotional state. By forcing a natural interpretation of their behavior, we don't break the illusion of a relationship with something alive.

Favor interactivity and generativity over a high resolution image. In Petz and Babyz we made a tradeoff to allow our characters to be immediately responsive, reactive and able to generate a variety of expressions, at the expense of a higher resolution image. Surprisingly most game developers don't make this tradeoff! From a product marketing perspective, a beautiful still-frame is typically considered more important than the depth and quality of the interactive experience. Of course there is a minimum level of visual quality any professional project needs, but we feel most developers place far too much emphasis on flashy effects such as lighting, shading and visual detail (i.e., spectacle), and not enough emphasis on interactivity and generativity.

Purity versus "faking it": take advantage of the Eliza effect. The "Eliza effect" - the tendency for people to treat programs that respond to them as if they had more intelligence than they really do (Weizenbaum 1966) - is one of the most powerful tools available to the creators of virtual characters. As much as it may aggravate the hard-core computer scientists, we should not be afraid to take advantage of this. "Truly alive" versus "the illusion of life" may ultimately be a meaningless distinction to the audience. 99% of users probably won't care how virtual characters are cognitively modeled - they just want to be engaged by the experience, to be enriched and entertained.

5: Conclusion

This paper has put forth virtual characters as a new form of emotional artifact, and the arrival of emotional relationships between humans and virtual characters as a new social phenomenon and direction for story and art. The design and implementation techniques we found useful to support such emotional relationships in the Virtual Petz and Babyz projects have been presented. We will conclude with some final thoughts on what it could mean for a person to have an emotional relationship with a virtual character.

Are relationships between people and virtual characters somehow wrong, perverse, even dangerous - or just silly? Again we can look to the past to help us answer this. Audiences that read about or see emotional characters in traditional media - painting, sculpture, books, theater, television and movies - have been known to become very "attached" to the characters. Even though the characters are not real, they can feel real to the audience. People will often cry when the characters suffer and feel joy when they triumph. When the written novel first appeared it was considered dangerous by some; today we find that this is not the case. However television, a more seductive medium than the novel, has certainly captured the free time in the lives of many people. Some consider the effect of television and videogames on children's development to be a serious problem; the media has even reported a few outrageous stories of people going to death-defying lengths to take care of their virtual pet Tamagotchis. Designers should be aware that manmade characters have the potential to have a powerful effect on people.

Why create artificial pets and humans? Isn't it enough to interact with real animals and people? From our perspective on making Virtual Petz, this was not the point. Our intent was not to replace people's relationships with real living things, but to create characters in the tradition of stuffed animals and cartoons. And while some people are forming emotional relationships with today's virtual characters, by and large they are still thought of as sophisticated software toys that try to get you to suspend your disbelief and pretend they are alive. However as we move towards virtual human characters such as Babyz, the stakes get higher. As of this writing we have not yet gotten feedback from the general public on their feelings and concerns about Babyz.

Also, the characters made so far have been "wholesome" ones, such as dogs, cats and babies, but one could easily imagine someone using these techniques to create characters that could support other types of emotional relationships, from the manipulative to the pornographic. Inevitably this will happen. Of course the promise and danger of artificial characters has long been an area of exploration in literature and science fiction, ranging from friendly, sympathetic characters such as Pinocchio and R2D2 to more threatening ones such as Frankenstein and HAL9000.

As virtual characters continue to get more life-like, we hope users keep in mind that someone (human) created these virtual characters. Just as an audience can feel a connection with the writer, director or actor behind a compelling character on the written page or the movie screen, a user could potentially feel an even stronger connection to the designer, animator and programmer of an interactive virtual character. For the artist, the act of creating a virtual character requires a deep understanding of the processes at work in the character's mind and body. This has always been true in traditional art forms, from painting and sculpting realistic people to novels to photography and cinema, but it is taken to a new level with interactive virtual characters. As the artist you are not just creating an "instantiation" of a character - a particular moment or story in the character's life - you are creating the algorithms to generate potentially endless moments and stories in that character's life.

People need emotional artifacts. When the public gets excited about buzzwords like "artificial intelligence" or "artificial life", what they are really asking for are experiences where they can interact with something that seems alive, that has feelings, that they can connect with. Virtual characters are a promising and powerful new form of emotional artifact that we are only just beginning to discover.

Acknowledgements

Virtual Petz and Babyz were made possible by a passionate team of designers, programmers, animators, artists, producers and testers at PF.Magic / Mindscape that include Adam Frank, Rob Fulop, Ben Resner, John Scull, Andre Burgoyne, Alan Harrington, Peter Kemmer, Jeremy Cantor, Jonathan Shambroom, Brooke Boynton, David Feldman, Richard Lachman, Jared Sorenson, John Rines, Andrew Webster, Jan Sleeper, Mike Filippoff, Neeraj Murarka, Bruce Sherrod, Darren Atherton and many more. Thanks to Robert Trappl and Paola Petta for organizing such a fascinating and informative workshop.

Appendix: Real customer letters

I had a dog that was a chawawa and his name was Ramboo. Well he got old and was very sick and suffering so my parents put him to sleep. Ever since then I have begged my parents for a new dog. I have wanted one soo bad. So I heard about this dogz on the computer. I bought it and LOVE it!!! I have adopted 9 dogs. Sounds a bit to much to you ehhh? Well I have alot of free time on my hands. So far everyday I take each dog out one by one by them selves and play with them, feed them, and brush them, and spray them with the flee stuff. I love them all. They are all so differnant with differant personalitys. After I take them out indaviually then I take 2 out at a time and let them play with me with each other. Two of the dogs my great Dane and chawawa dont like to play with any of the other dogs but each other. This is a incrediable program. I had my parents thinking I was crazy the other night. I was sitting here playing with my scottie Ren and mutt stimpy and they where playing so well together I dont know why but I said good dog out loud to my computer. I think my parents wondered a little bit and then asked me what the heck I was doing. But thankz PF.Magic. Even though I cant have a real dog it is really nice to have some on my screen to play with. The only problem now is no one can get me away from this computer, and I think my on-line friendz are getting a little mad cause im not chatting just playing fetch and have a great time with my new dogz. Thanks again PF. magic. I love this program and will recomend it to everyone I know!!!!!!!

I am a teacher and use the catz program on my classroom PC to teach children both computer skills and caring for an animal. One of the more disturbed children in my class repeatedly squirted the catz and she ran away. Now the other children are angry at this child. I promised to try and get the catz back. It has been a wonderful lesson for the children. (And no live animal was involved.) But if there is any way to get poor Lucky to come homze to our clazz, we would very much appreciate knowing how to do it. Thanks for your help, Ms. Shinnick's 4th grade, Boston, MA

Dear PF. Magic, I am an incredible fan of your latest release,Petz 3,I have both programs and in Janurary 1999,my cherised Dogz Tupaw was born. He is the most wonderful dogz and I thank you from the bottom of my heart, because in Janurary through to the end of April I had Anorexia and i was very sick. I ate and recoverd because i cared so much about Tupaw and i wanted to see him grow up. I would have starved without you bringing Petz 3 out. Please Reply to this,it would mean alot to me. Oh,and please visit my webpage,the url is http://www.homestead.com/wtk/pets.html. Thankyou for releasing petz 3,Give your boss my best wishes, Sincerily, Your Number One Fan, Faynine

I just reciently aquired all your Petz programs and I think they are great! I really love the way the animals react. I raised show dogs and have had numerous pets of all kinds in my life and making something like this is great. I am a school bus driver and have introduced unfortunate kids to your program. Children who not only can they not afford a computer but they can't afford to keep a pet either. This has taught them a tremendous amount of responsibilty. I am trying to get the school to incorporate your programs so as to give all children a chance to see what it is like to take care of a pet. It might help to put a little more compassion in the world. Please keep me updated on your newest releases. Thanks for being such a great company. Nancy M. Gingrich

Dear PF.Magic, Hello! My name is Caitlin, and I'm 10 years old. I have Dogz 1 and Catz 1, as well as Oddballz, and I enjoy them all very much. Just this morning was I playing with my Jester breed Catz, Lilly. But I know how much better Petz II is. For a while, I thought I had a solution to my Petz II problem. I thought that if only I could get Soft Windows 95 for $200, that would work. Well, I took $100 out of my bank account (by the way, that's about half my bank account) and made the rest. I cat-sit, I sold my bike, and I got some money from my parents. Anyway, I really, really love animals (I'm a member of the ASPCA, Dog Lovers of America, and Cat Lovers of America) but I can't have one! That's why I love Petz so much! It's like having a dog or cat (or alien for that matter) only not. It's wonderful! I have a Scrappy named Scrappy (Dogz), Chip named Chip (Dogz), Bootz named Boots (Dogz), Cocker Spaniel named Oreo (Dogz), Jester named Lilly (Catz), and Jester named Callie (Catz). And then every single Oddballz breed made. =) I don't mean to bore you as I'm sure this letter is getting very boring. I would love SO MUCH to have Petz II. I really would. (At this point in the letter I'm really crying) I adopted 5 Catz II catz at my friend's house, but I go over to her house so little I'm sure they'll run away. I'd hate for them to run away. Is there anything I can do? I love my petz, and I'm sure they'd love Petz II. Thank you for reading this. Please reply soon. ~*~ Caitlin and her many petz ~*~

My husband went downtown (to Manchester) and found Catz for sale, and having heard so much about it he bought it on the spot. He put it on his very small laptop and came back from one of his business trips saying, "How many Dutchmen can watch Catz at once on a little laptop on a Dutch train?" The answer was TEN. I asked if any of them said, "Awww," the way we all did, but he said they all walked off saying it was silly. I bet they ran out to buy it anyway, though! Yours, Mrs. H. Meyer

Dear Sirs, Just wanted to thank-you for the pleasure my petz have brought me. I am paralyzed from the neck down yet your program has allowed me too again enjoy the pleasure of raising my own dogz. I have adopted 5 so far. I love them equally as if they were real. Thanks again

References

Bandai. 1996. Tamagotchi keychain toy. Website URL: http://www.bandai.com

Bates, J. 1992. The Nature of Characters in Interactive Worlds and The Oz Project. Tech Report CMU-CS-92-200, School of Computer Science, Carnegie Mellon University.

Bates, J. 1994. The Role of Emotion in Believable Agents. Technical Report CMU-CS-94-136, School of Computer Science, Carnegie Mellon University, Pittsburgh, PA.

Blumberg, B. 1997. Multi-level Control for Animated Autonomous Agents: Do the Right Thing… Oh, Not That… In Creating Personalities for Synthetic Actors, R. Trappl and P. Petta (Eds.), Springer-Verlag, New York.

Boehlen, M. and Mateas, M. 1998. Office Plant #1: Intimate space and contemplative entertainment. Leonardo, Volume 31 Number 5: 345-348.

Damasio, A. 1994. Descartes' Error: Emotion, Reason and the Human Brain. Putnam Publishing Group, New York.

Dautenhahn, K. 1997. Ants Don't Have Friends - Thoughts on Socially Intelligent Agents. In Proceedings of the 1997 AAAI Fall Symposium, Socially Intelligent Agents, Menlo Park: AAAI Press.

Elliot, C.; Brzezinski, J.; Sheth, S.; and Salvatoriello, R. 1997. Story-morphing in the Affective Reasoning paradigm: Generating stories automatically for use with ``emotionally intelligent'' multimedia agents. Institute for Applied Artificial Intelligence, School of Computer Science, Telecommunications, and Information Systems, DePaul University.

Egri, L. 1946. The Art of Dramatic Writing. Simon & Schuster, New York.

Frank, A.; Stern, A.; and Resner, B. 1997. Socially Intelligent Virtual Petz. In Proceedings of the 1997 AAAI Fall Symposium, Socially Intelligent Agents, FS-97-02, pp. 43-45. Menlo Park: AAAI Press.

Frank, A.; and Stern, A. 1998. Multiple Character Interaction Between Believable Characters. In Proceedings of the 1998 Computer Game Developers Conference, pp. 215-224. Miller Freeman, San Francisco.

Goldberg, A. 1997. IMPROV: A System for Real-Time Animation of Behavior-Based Interactive Synthetic Actors. In Creating Personalities for Synthetic Actors, R. Trappl and P. Petta (Eds.), Springer-Verlag, New York.

Grand, S.; Cliff, D.; and A. Malhotra. 1997. Creatures: Artificial Life Autonomous Software Agents for Home Entertainment. In Proceedings of the First Intl. Conference on Autonomous Agents, Minneapolis, ACM Press, pp. 22-9.

Hayes-Roth, B.; van Gent, R.; and Huber, D. 1997. Acting in Character. In Creating Personalities for Synthetic Actors, R. Trappl and P. Petta (Eds.), Springer-Verlag, New York.

Kline, C; and Blumberg, B. 1999. The Art and Science of Synthetic Character Design. 1999 Convention of the Society for the Study of Artificial Intelligence and the Simulation of Behavior (AISB), Edinburgh, Scotland.

Loyall, A. 1997. Believable Agents: Building Interactive Personalities, Ph.D. Dissertation, Carnegie Mellon University, Pittsburgh. CMU-CS-97-123.

Mateas, M. 1999. Not Your Grandmother's Game: AI-Based Art and Entertainment. In Proceedings of the 1999 AAAI Spring Symposium, Artificial Intelligence and Computer Games, SS-99-02, pp. 64-68. Menlo Park: AAAI Press.

Meehan, J. 1976. The metanovel : writing stories by computer. Ph.D. Dissertation, Dept. of Computer Science, Yale University.

Minsky, M. 1985. The Society of Mind. Simon & Schuster, New York.

Murray, J. 1997. Hamlet on the Holodeck: The Future of Narrative in Cyberspace. The Free Press, New York.

Pemberton, L. 1989. A Modular Approach to Story Generation. In 4th Conference of the European Chapter of the Association for Computational Linguistics, pp. 217-224.

Penny, S. 1997. Embodied Cultural Agents: At the Intersection of Robotics, Cognitive Science and Interactive Art. In Proceedings of the 1997 AAAI Fall Symposium, Socially Intelligent Agents, FS-97-02, pp. 43-45. Menlo Park: AAAI Press.

Perlin, K. 1995. Real-time responsive animation with personality. IEEE Transactions on Visualization and Computer Graphics.

PF Magic / Mindscape. 1995 - 1999. Dogz and Catz: your Virtual Petz, Babyz. Website URLs: http://www.petz.com and http://www.babyz.net

Picard, R. 1997. Affective Computing. MIT Press.

Reilly, W. 1996. Believable Social and Emotional Agents. Ph.D. Thesis. Technical Report CMU-CS-96-138, School of Computer Science, Carnegie Mellon University, Pittsburgh, PA.

Rilke, R. 1934. Letters to a Young Poet. W. W. Norton and Company, New York.

Sengers, P. 1996. Socially Situated AI: What It Means and Why it Matters. In Proceedings of the 1996 AAAI Symposium, Entertainment and AI/A-Life. Menlo Park: AAAI Press.

Sengers, P. 1998. Do the Thing Right: An Architecture for Action-Expression. In Proceedings of the Second Intl. Conference on Autonomous Agents, pp. 24-31. ACM Press, New York.

Sony Electronics. 1999. Aibo robot dog. Website URL: http://www.world.sony.com/robot/get/meet.html

Stern, A.; Frank, A.; and Resner, B. 1998. Virtual Petz: A Hybrid Approach to Creating Autonomous, Lifelike Dogz and Catz. In Proceedings of the Second Intl. Conference on Autonomous Agents, pp. 334-5. ACM Press, New York.

Stern, A. 1998. Interactive Fiction: The Story is Just Beginning. IEEE Intelligent Systems, Volume 13, No. 5, pp. 16-18. Los Alamitos: IEEE Computer Society.

Stern, A. 1999a. AI Beyond Computer Games. In Proceedings of the 1999 AAAI Spring Symposium, Artificial Intelligence and Computer Games, SS-99-02, pp. 77-80. Menlo Park: AAAI Press.

Stern, A. 1999b. Virtual Babyz: Believable Agents with Narrative Intelligence. In Proceedings of the 1999 AAAI Fall Symposium, Narrative Intelligence. Menlo Park: AAAI Press.

Thomas, F; and Johnston, O. 1981. Disney Animation: The Illusion of Life. Abbeville Press, New York.

Tiger Electronics. 1998. Furby toy. Website URL: http://www.furby.com

Turner, S. 1994. The Creative Process: A Computer Model of Storytelling and Creativity. Lawrence Erlbaum Associates.

Turkle, S. 1999. Ongoing virtual pet study. Website URL: http://web.mit.edu/sturkle/www/vpet.html

Weizenbaum, J. 1966. Eliza. Communications of the ACM, 9:36-45.

Wurst, K. and McCartney, R. 1996. Autonomous Robots as Performing Agents. In Proceedings of the 1996 AAAI Symposium, Entertainment and AI/A-Life. Menlo Park: AAAI Press.

PF.Magic, 501 2nd St, Suite 400, San Francisco, CA 94107 | tel: +1 415 495 0400

Multiple Character Interaction Between Believable Characters

CGDC 1998 presentation paper by Adam Frank and Andrew Stern. SOURCE

Multiple Character Interaction
Between Believable Characters

PF.Magic
501 2nd St, Suite 400
San Francisco, CA 94107
+1 415 495 0400
http://www.pfmagic.com
http://www.netcom.com/~apstern

Introduction

Computer games have always had characters. In a typical game the player controls the "main character" to fight or compete against one or more computer-controlled "non-player characters". The game's "artificial intelligence" (AI) controls the behavior of the non-player characters in an effort to out-maneuver, out-shoot or out-smart the player.

But even to this day, with few exceptions, computer characters are really no more than shallow cardboard-cutouts. While they are beginning to look more realistic, they still act in very limited and often predictable ways, with almost no variety in their behavior. They tend to do and say the same things over and over no matter how hard the player tries to get them to react differently.

Fortunately, as the power of the personal computer grows, it is becoming more and more possible to create truly lifelike and believable autonomous characters. In this paper we first propose what lifelike, believable characters should be, and design principles involved in making them. The first multiple character interaction we discuss is between the user and a single lifelike computer character. Next we discuss what should happen when multiple lifelike computer characters interact with each other, and what new kinds of interactive experiences for the user can result from this.

We use the Virtual Petz we developed at PF. Magic to demonstrate these key points. The original Dogz, initially released in 1995, was the world's first virtual pet, followed soon by Catz, Oddballz and the new Dogz II and Catz II. Virtual Petz are autonomous real-time 3D characters that live on your PC computer desktop, grow up over time, and strive to be the user's friends and companions.

What are Lifelike, Believable Characters?

These days most people think the way to make a computer character more "real" is to make it look more photorealistic, and to give it a better AI. Since almost all interactive entertainment software products are games, the typical AI is designed to figure out the best move to make, such as the next punch or kick move, where to run to and shoot from, and so on. But figuring out the best next move is an incomplete definition of AI. When people talk about wanting great AI, or great "artificial life" for that matter, what they really want is the experience of interacting with something truly alive.

When would a player believe a computer character is truly alive? What constitutes a convincing, interactive "illusion of life"? It seems that the best way to define what truly alive means is to study real living "characters" - that is, animals and people.

- Real living things are very reactive and interactive. Given this, when users physically interact with a lifelike computer character in its environment, the character should be aware of what is happening and react immediately. If a character reacts in a way that makes no sense, or doesn't react at all, then the illusion of life is broken, and the character cannot be considered believable. Also, real living things don't always react the exact same way each time! A believable character should not appear to be repeating itself verbatim every time the user interacts with it. Along the same lines, if the user is given little or no way to interact with the character in the first place, the whole interface itself does not support believability.

- Real living things are able to communicate. Most animals, and people especially, have several ways to communicate with each other. These include gestures, actions, body language, sounds, and spoken or written language. To be truly believable, users and computer characters should be able communicate with each other in a natural, lifelike way. If a user comes face-to-face with on-screen human character, they really should be able to speak to communicate, not type.

- Real living things are very expressive and able to show a broad range of emotions, desires and personalities. A believable character wouldn't just fight or shoot, it would have a broad base of behavior - the ability to eat, sleep, play, chase, explore, groom, build, and so on. Lifelike characters should be emotional as well. Depending on what is happening in their lives they would act happy, sad, angry, afraid, jealous, and so on. They may get energetic or tired, hungry or full, sick or healthy, and may tend to be grumpy, hyper, sweet, or finicky. All of these emotions, moods and personalities must be expressed by how the computer character behaves, animates and sounds. Even if these things are being kept track of in the character's mind, if they're not expressed, they're effectively not there.

- Real living things are able to learn, adapt and change over time. Believable characters should be able to pay attention to what is going on in their environment and learn from it. They should have attitudes and feelings towards objects and other characters, and act appropriately upon these attitudes. They should form relationships with the other characters in the environment, including the user. As a believable character grows and ages over time, its attitudes and relationships will change, and it will gain experience and knowledge.

If you follow what's happening these days in academic media research laboratories, the concept of believability is now a key design goal for many synthetic character research projects, such as the Oz Project at CMU (1), Improv at NYU (2), Synthetic Characters at MIT Media Lab (3), the Virtual Theater Project at Stanford (4), and so on. Refer to my personal web page for a complete listing (5).

Examples of Believable Characters: Virtual Petz

Clearly, making truly lifelike believable characters is a difficult problem. At PF.Magic we created the Virtual Petz characters using the above criteria, incorporating as much of each component of believability as we possibly could. As far as we can tell, the Petz are the first consumer software products to address all of these criteria.

The most direct way the user can interact with the Petz is through petting. Using a mouse the user moves a hand-shaped cursor to pet, scratch and stroke the character. The pet immediately reacts in a variety of ways depending on where its body is being petted, how fast, and how the pet feels at the time. By right-clicking the user can pick up and carry a pet. We call this direct interaction. The fact that you can touch the character is a very effective way of building an intimate relationship and creating an illusion of life.

The toys and objects in the pet's environment have direct object-like interaction for both the user and the Petz. By picking up and using a toy, the user can initiate play. For example, throwing a ball may initiate a game of fetch, or holding a tugtoy in front of a pet may initiate a game of tug-of-war. Similarly a pet can get its own toy and bring it to the user to initiate play. This cooperative decision-making helps build the owner-pet relationship.

We have created a variety of personalities - playful terriers, grumpy bulldogs, hyper Chihuahuas, finicky Siamese cats, lazy Persian cats, aggressive hunter cats, timid scaredy cats and so on. Additionally each pet has its own individual likes and dislikes, spots and body coloration, and personality quirks. Users get to play with a pet and see if they like them before deciding to adopt. Once adopted, the user gives them a name. This individual variation allows the user to develop a unique relationship with a particular pet. Every owner-pet relationship is different.

Petz communicate with the user in a variety of ways. Emotion can be expressed through different facial expressions (eyebrows, mouth, ears), styles of movement and body language (sad walks, happy trots, various postures, a variety of tail motions), and sounds (excited playful barks and meows, sad whines and whimpers, yelps of pain, etc.). They get fat when they eat too much and get skinny when hungry.

Petz communicate their intentions and desires through their actions. We have created a broad base of behavior for the Petz -- they can eat, sleep, play, attack, groom, hiss, explore and so on. When Petz want attention they may start barking or meowing. If hungry they may go searching for food, hunt mice, or start begging. When upset or scared they may run and hide, act aggressively, or cower and shiver.

The user is given a few specific hand gestures that Petz may respond to. By double-clicking in an empty space on the screen the hand cursor animates a "come here" motion and plays a whistling sound. Petz tend to stop what they're doing and come to the user's cursor. Users can also pick up and wiggle food treats in different directions to train their Petz to jump, sit, rollover, somersault, and so on. Additionally, users can reward or discipline any behavior. Methods of rewarding include a giving treats, feeding, and petting, while discipline is achieved by squirting a pet with a water spray bottle. In this way users can modify a pet's overall behavior through positive and negative reinforcement. Additionally, the personalities of each pet can change under extreme circumstances. For instance if any pet is underfed it will begin hunting and acting aggressively. If a pet is overfed, it will become fat and lazy. So depending on how their user treats them, their behavior can slide along a broad spectrum of personalities.

A User-Perception Based Approach

It is important to note that the only way the user can understand what the Petz are feeling is to interpret their actions and physical cues, in the same way an audience interprets an actor's performance. We do not display bar graphs or text messages describing the pet's internal variables, biorhythms or emotional state. By forcing a natural interpretation of the pet's behavior, we don't break the illusion of a relationship with something alive.

Again, "real" AI or A-Life should be defined as the illusion of life. In entertainment software, all that matters is the strength of this illusion. It does not matter what internal techniques are used or how sophisticated the programming is. Lifelike believable characters are ultimately perceived and judged through human eyes. All that matters is what you see and believe on-screen. This should be the standard by which any believable characters are held to - does the audience believe that the characters are alive. Simply making a computer character look photorealistic won't work - without the interactivity, communication, expressiveness and adaptability, it's just a mannequin.

Design Principles for Making Lifelike Believable Characters

Certainly there are many ways to go about making lifelike computer characters. In the process of working on Virtual Petz we have come up with a few design principles that have worked well for us.

- Choose characters you can successfully implement. For Virtual Petz, choosing dogs and cats as our characters helps us in a few ways. People understand what pets are and essentially know how to interact with them. This is critical for producing believable characters. Pets are also a good choice because they are expected to behave in ways that we can successfully implement. They have relatively simple gestures, language and cognitive abilities. Also, animated dogs and cats have well established cartoon archetypes. We base the look and behavior of the Petz on these archetypes.

- Frame the experience in a looser, more character-centric way. Instead of the typical goal-oriented computer game, the Virtual Petz experience is non-goal-oriented. We allow users to explore the characters and their toys in any order they like. This freedom allows users to socialize with their Petz in their own way and at their own pace. This also encourages users to come up with their own interpretation of their pet's feelings and thoughts. Often the scope of their interpretations far exceeds what we originally planned for.

Note that we have focused almost all of our energy on creating rich characters and not on creating a sophisticated environment. The Petz come to the user's environment - the desktop - not the other way around. We only built the objects and toys that were needed to support the interaction with the characters. By doing this we are making the Petz the stars of the show.

- Design your animation to be as dynamic and responsive as possible. A high degree of interactivity and lifelikeness truly requires the use of real-time 3D animation. Pre-rendered animation, however beautiful it may be, cannot be as responsive, dynamic, or varied as required for lifelikeness and believability. One should choose to err on the side of frame-rate versus a photorealistic image that interacts poorly.

The Petz products use a network of basic 3D body animation which has a high resolution of branch points. For example, from any basic pose such as standing or sitting, the pet can transition to any other pose. As the pet's general body animation moves across this network, additional animation is layered on top. This allows the pet to look in all directions no matter what its body doing. Also we layer breathing, blinking, meowing, head shaking and even character-specific posture. In fact the look of the each breed and their ages are layers. The subsequent number of permutations is enormous and therefore passes a critical threshold in human perception. This method greatly enhances our ability to create an illusion of life.

- Don't over-engineer the behavior logic. The Petz behavior is controlled by a variety of mechanisms that operate simultaneously, each with varying complexity. There are small mechanisms to control blinking and breathing, biorhythm mechanisms to keep track of things like energy and hunger, and a variety of cognitive mechanisms to keep track of multiple goals, different plans to execute each goal, multiple emotions towards other characters and objects, attitudes towards other characters and objects, and personality-specific quirks. In general we've been able to reduce complexity and still retain a high-degree of lifelikeness and freshness with some relatively simple techniques. To achieve a real, unpredictable, non-repetitive quality in these mechanisms, we often use constrained randomness. For example, there are rules for each individual pet that determine which toys it most likes to play with, but there is some randomness in that decision. This results in the possibility that a pet might play with any toy, not just the toys it likes the most. A lot of development time is spent tweaking the constraints on the randomness to achieve a good balance between consistent personality and unpredictability. Again we are focusing on the user's perception of the character and being careful to not create a behavior system that models more than can be expressed.

Interaction Between Multiple Lifelike Computer Characters

Now that we've talked about what believable characters are, how should two lifelike computer characters interact with each other? Again let's look how real living things behave. Animals and people perceive and react to one another in all sorts of ways -- they look each other over; approach and investigate one another; communicate using gesture, body language, sounds or speech; directly contact one another by touching, holding, fighting, hugging, mating, and so on. A believable character should be able to do all of these things.

On a conceptual level, the two are forming a relationship with each other. Believable characters have attitudes and feelings about one another - how much they like each other, how much they know about each other, what they can gain from each other. This relationship evolves over time as they interact and events happen. If two characters help and support each other, they should become friends; likewise if they hurt or abuse another, they should become enemies. The characters' individual personalities should influence the ways and rate at which the relationship can change - a very timid character may quickly distrust another at the slightest provocation, whereas a happy-go-lucky character would need to be severely abused to begin distrusting others.

Again we'll use Petz as an example. In our newest products, Dogz II and Catz II, where two Petz can interact with the user and each other at the same time, we divided different levels of multiple character interaction into three categories.

- Looking but not touching. A lot of meaningful interaction can happen without any physical contact between characters. For the Petz this included looking at each other, investigating each other, following each other, barking or meowing at each other, ignoring or snubbing each other, and stand-offs like growling and hissing at each other. A lot of information about how two characters feel about one another can be expressed this way. These interactions were easiest to implement because they didn't require perfect timing synchronization between the two characters.

- Playing with a common object. This is similar to looking-but-not-touching, but now includes a common object which the interaction is focused around. For the Petz this included things like playing fetch together, chasing mice together, playing frisbee together, sharing food and water bowls, bringing each other toys and food, spraying each other with the water spray bottle, and so on. Again this doesn't require perfect synchronization between the characters because they never physically contact each other.

- Direct contact. This includes wrestling, fighting, tug-of-war, grooming each other, carrying each other, bumping into each other, licking noses, and mounting each other. These behaviors required physically aligning the bodies of two characters in space and time so they contacted each other properly, and then drawing their bodies so they accurately meshed in 3D. These types of behaviors are perhaps the most high-impact, because they are so lifelike and intimate, but also the most time-consuming and hardest to implement.

Developing Relationships

Using the above multiple character interactions, we implemented all sorts of different behaviors to allow Petz to express their relationships between each other and the user. Several different categories of relationships can be identified.

- Buddy relationships. When two Petz really like each other, they play a lot together, bring each other food and toys, groom each other, follow each other, lick noses, sleep side-by-side, and so on. Buddies act happy when they first see each other and act sad when they part. They will get angry at the user if the user abuses their buddy. If a pet considers the user their buddy, they will be affectionately lick the user's hand cursor, always come when called, obey commands and gestures, etc.

- Enemy relationships. When two Petz hate each other, they growl and hiss at each other, wrestle, fight, steal toys from each other, scare each other, spray each other with the spray bottle, and so on. Enemies are happy when the user disciplines the other, and are happy when the other goes away. One get jealous if the other receives attention and affection from the user. If a pet considers the user their enemy, they will act naughty, ignore the user, bark angrily, and even eventually runaway.

- Nurturing relationships. These are typical between adult Petz and younger Petz. Adult Petz will comfort and protect younger Petz if they get scared and carry them to a pillow to sleep. They may bring them food to eat or toys to play with. Younger Petz may follow the adults around and attempt to copy their behavior. Adult Petz may even try to teach younger Petz how to do tricks.

- Neutral relationships. Sometimes two Petz don't have strong feelings for each other one way or another, and they basically play independently of each other. However the user can often encourage two neutral Petz to play together, or taunt them to get them to fight. Depending on the user treats the two Petz, she can try to push them into a buddy or enemy relationship.

The Emergence of Story

Once you create characters that are very reactive, interactive, communicative, expressive and adaptive, exciting things begin to happen. You can suspend your disbelief and become immersed in the illusion of life. But even beyond the feeling that you are interacting with something truly alive, we have found that the interplay between a variety of rich archetypal personalities can give rise to many dramatic situations. Small "stories" seem to emerge as these complex synthetic characters act out their innate personalities.

For example, a tired old bulldog won't be able to go to sleep when a young playful kitten is bouncing around him. Because the dog has a nurturing relationship towards the kitten, he will bring the kitten some food, which then allows him to lie down for a nap. However the kitten, by its own nature and personality, has a short-attention span and quickly returns to jumping around the bulldog. The bulldog expresses his frustration at being tired, but because the kitten is gesturing for attention, he responds by bringing the kitten a toy. Note that the bulldog was never explicitly programmed to "distract" the kitten so he could sleep - but if you look at these "naturally" unfolding events as a whole, it comes together as a vignette about a dog trying to distract a kitten so he can sleep.

Catz are always on the lookout for mice. If the user takes out the cheese and puts it in front of the mouse hole, the mouse may sneak out for a nibble. The cat will immediately begin stalking it for an attack. If the user then brings out a dog, a three-way chase ensues. The situation usually ends with the dog chasing the cat away and the mouse getting a free meal. This has the look and feel of a live Tom and Jerry cartoon.

Even a loving relationship between two buddies can be upset when the user pays far more attention to one pet than the other. The jealous pet will become angry and upset and may even act aggressively to the user or the other pet. However, given time the two Petz may eventually make up and return to their friendly relationship. Over the long term this can appear to the user as a "stormy" relationship.

On the PF. Magic website6 there is a bulletin board where users have the chance to post stories and photos about their Petz. We've had a tremendous response. We've received thousands of stories from people describing their relationships with their individual pets. Some kids we know get up especially early every day to feed them. Most of the stories describe feelings and complex behavior that seem to emerge from playing with the Petz. Here's an example:

Our family just adopted our new puppy (named Spike) a few days ago. He is a bull dogz. We let Spike out all the time, and he is already part of our family along with our (non-computer dog, a doberman), who is being neglected and becomming very spiteful, because Spike is a new puppy and requires more of our time. We have had to be careful, about letting them spend to much time together, for Spike is still a puppy and wants to play and our doberman is extremely jealous and continues to pick fights. We are concerned that our doberman will hurt Spike. When it is feeding time (almost every time we walk into the room, if it was up to Spike) we have to put down a bowl of food for each dog. Luckly they can not reach each others bowls. We are afraid to leave Spike out of his play pen for to long of a period of time, without constant monitoring, for both their tempers seem to be short. We do believe Spike and our doberman will become long lasting friends, as they grow-up together, because they have the same personalities, and are both good guard dogs, and very good pets..... If anyone out their has a bull dogz, Spike would love to hear from you, as our cant read, please send a photo and dog biscuit to sweetpea@inreach.com

These kinds of results are promising for the future of interactive story. It makes sense that this is happening because the core foundation of any story is character. Character is even more important than plot, because without characters, no plot can be acted out. Only by first focusing on making lifelike believable characters will you have the building blocks for creating interactive stories. (Note one should not call this interactive storytelling, because the user is an active participant in creating the story.) A new book by Janet Murray called Hamlet on the Holodeck: The Future of Narrative in Cyberspace goes into great detail about these ideas. (7)

Conclusion

Computer games have yet to incorporate very lifelike, believable characters. Once this finally happens, interactive entertainment will no longer be just the domain of today's "gamers". Little girls and grandparents and 50-year-old businessmen will become interested in computer games when they can interact with engaging characters. In the long term, our industry will expand to rival movies and television as the primary source of entertainment for the majority of people.

With Petz we've already seen signs that this is possible - they are widely popular among girls and boys, kids and adults. As you know this is almost unheard of in entertainment software. People want virtual characters that they can form emotional relationships with. Not only do they want it, but they expect it. The virtual pet craze is just the beginning.

We've found that developing lifelike, believable characters requires people with interdisciplinary skills from both technology and the arts. Until recently, building lifelike artificially intelligent characters has been treated as a computer science problem. Character development, in any medium, is fundamentally an artistic, aesthetic decision-making process. Lifelike computer characters are ultimately perceived through the filter of human interpretation. No computer character can be perceived as truly alive and intelligent unless its builders always keep in mind what the final user will perceive.

Acknowledgements

We thank Rob Fulop, John Scull, Alan Harrington, Jonathan Shambroom, Peter Kemmer, Ted Barnett, Jeremy Cantor, and the rest of the PF. Magic Petz team.

References

1) The Oz Project, Carnegie Mellon University. URL: http://www.cs.cmu.edu/afs/cs.cmu.edu/project/oz/web/oz.html

2) Improv, NYU Media Research Laboratory. URL: http://www.mrl.nyu.edu/improv

3) Synthetic Characters Group, MIT Media Laboratory. URL: http://lcs.www.media.mit.edu/people/bruce

4) Virtual Theater Project, Stanford University. URL: http://www-ksl.stanford.edu/projects/CAIT/

5) Andrew Stern. Virtual Characters, Virtual Pets, Artificial Life, Artificial Intelligence, Interactive Story. URL: http://www.netcom.com/~apstern

6) PF. Magic. Dogz and Catz, Your Virtual Petz. URL: http://www.petz.com

7) Janet Murray. Hamlet on the Holodeck: The Future of Narrative in Cyberspace. New York: The Free Press, 1997.