,
,
,
,
) in which represents a set of goals
in disjunctive normal form, is the cost matrix,
is the belief network,
, is a set of possible actions, and
is personality which is a mapping from (,,) to .
=10000
S.S. Nemani and V.H. Allan
Department of Computer Science,
Utah State University
Logan, Utah 84322-4205. E-mail: allanv@cs.usu.edu
Negotiation between humans involves reacting to offers that are made as well as making initial offers to proposals. In the business world, software agents are able to make offers and counter-offers as well as evaluate whether a proposal should be accepted. Can software agents be programmed to mimic human behavior so well that humans benefit from observing the automated behavior? Using NAMS (Negotiating Agents and Marital Stability), a couple who is considering marriage (whom we will designate as Alice and Bob) use the tool in the following way. One agent is programmed with personality data from the Alice, while another agent is programmed with personality data from Bob. Agents negotiate a decision for a specific scenario.
An important component of this research is to model emotions. We use a two dimensional matrix in which the columns correspond to one of eleven fundamental emotions (interest, joy, surprise, sadness, anger, disgust, fear, shyness, shame, guilt, and love) and the rows correspond to cognitive activators of emotion (appraisal, attribution, memory and anticipation). The emotional state can be computed from two pieces: the permanent (trait matrix) in which changes occur gradually and the volatile (emotion matrix) which registers the reaction to current stimulus.
The matrices represents an emotional state. Our algebra of emotion allows us to model self-generating emotions (such an interest or anger), have history-sensitive interactions, and accommodate gender differences. Matrix multiplication allows us to combine various emotions, even ones that may be hierarchical.
This research asks the question, ``Can software agents be programmed to mimic human behavior so well that humans benefit from observing the automated behavior?'' The idea is influenced by the classic Turing Test proposed in the 1950's [27]. In an effort to define intelligent behavior, Turing suggested a test in which a machine is asked to perform cognitive tasks via a teletype. An interrogator tries to determine whether it is a machine or a human being who is answering the questions. In this research, we are not trying to convince anyone they are interacting with a human being. Rather, the goal is to let the agents demonstrate the role of personality attributes in negotiation. We term this tool NAMS (Negotiating Agents and Marital Stability).
A couple who is considering marriage (whom we will designate as Alice and Bob) use the tool in the following way. One agent is programmed with personality data from Alice, while another agent is programmed with personality data from Bob. Agents negotiate a decision for a specific scenario. Since we are trying to model the way in which the partners negotiate, the specific scenario is not important. For illustrative purposes, we will assume that the partners will negotiate a joint activity. Research in the area of marital stability indicates that it is not so much how a couple differs that is important, as there will always be differences. The key issue is in how they deal with those differences [12]. From a counseling viewpoint, the goals of NAMS are two-fold (a) to help individuals make appropriate marital decisions, and (b) to motivate an individual to change destructive habits. From a computer science viewpoint, the goal of NAMS is to study the mechanisms whereby emotion can be modeled reliably. The study of emotions is important for a variety of reasons [10]. First, human machine interactions with agents are benefited. Second, emotions are an abstraction of a complex internal state than can efficiently be communicated. Third, emotions can limit the search space of appropriate responses.
The couple could observe agents in negotiation to answer such questions as:
When parties collaborate, they work together to find a solution that satisfies both of their needs. While each person will likely not get things exactly as he/she desired, they get enough of their desires to feel valued. Collaboration requires a respect for the needs and desires of the other party. Often, collaborative solutions leave parties better off than if they had competed and won. This is true for several reasons. First, in interpersonal conflicts, the winner may actually lose. For example, the loser may agree to watch a movie, but be poor company. Second, the synergy of two people's ideas could be better than what either would propose alone. Third, competing with people usually deteriorates the quality of the relationship in other areas. Fourth, finding a common solution might enable a couple to grow together rather than apart. For example, a partner that agrees to consider a type of activity that he/she had not previously enjoyed may find the activity to be one that they can enjoy together. However, competition must also be modeled to achieve a realistic interaction.
In our system, an agent is modeled as a tuple (,,,,) in which represents a set of goals
in disjunctive normal form, is the cost matrix,
is the belief network,
, is a set of possible actions, and
is personality which is a mapping from (,,) to .
The cooperation problem is made more difficult as agents usually have incomplete information about the environment and the consequences of their actions [1]. Information about the coordination task needs to be gathered and incrementally updated. The agent's belief about the other agent's goals and motivation is stored in a belief network. The ability to negotiate will be improved by self-adaptive decision making strategies and policies which control them [14].
An example of a conversation plan is given in Figure 1. Behavior norms can also be used to enhance the decision making capability. The duration of the negotiation may also motivate a different priority function for selecting a possible disjunct.
Figure 1: Conversation Plan. Legend. Received messages are marked with an initial arrow.
Messages which are sent from the agent are marked with a trailing arrow.
Our system is implemented using the JADE (Java Agent Development Environment) Multi Agent system [2, ]. JADE is an appropriate agent tool in that it provides a powerful distributed platform and agent framework with complete conformance to FIPA specifications. It also gives us the ability to use various content languages for message passing. We use MySQL as a knowledge database to store likes, calendar information, and factual data, which helps the agent respond in an accurate manner (e.g., I do not like sports. I am busy Friday. That movie is rated ``R''; I would rather not go to R-rated movies.)
Of first marriages, roughly 40-50% will end in divorce under current trends [22]. Thus, divorce affects a significant portion of society. Marital problems are associated with lower work productivity [8]. Adults and children are at risk for greater mental and physical problems [6].
Children living with a single parent are more likely to have limited activity due to illness and higher rates of disability [7]. The combination of illegitimate births, marital conflict, and divorce leave children at greater risk for alienation, anti-social behavior, health issues, and poverty [25, ]. Relationships with high marital quality are positively related to self-esteem and happiness, but negatively correlated to poor health, depression, and anxiety [17]. These stakes are great indeed. As with Computational Organization Theory [4], the costs of modeling and researching relationships before a commitment is made using NAMS are far less than counseling real participants after poor decisions have been made.
Several studies indicate that, with high degrees of accuracy, researchers can predict which marriages will end in failure from information gathered before the couple marries [5, , 11, , ]. While it is generally considered poor practice for a therapist to make specific predictions about whether a specific couple will divorce, it is common for therapists to tell a couple that they are at substantially greater risk for divorce. Usually the highest rate of prediction accuracy comes from observing how a couple interacts [13]. Thus, a tool like NAMS is particularly important not only because it serves as a diagnostic aid for the counselor, but that interaction behavior can be demonstrated and modified so the couple can draw their own conclusions. In counseling, some emphasis has shifted to the criticality of who you choose to marry, but other researchers contend that couples can indeed change patterns of behavior to lower their risk of divorce. NAMS addresses both aspects.
Many of the studies in place are designed to measure problems of an existing marriage, but here the same idea is used for couples considering marriage to help them avoid problems that have caused other marriages to fail [12].
It is reported that couples argue most about money and children, but some researchers believe that the specific topic of the argument is not nearly as important as how they argue [20]. Thus, couples fall into the pattern of arguing about how they argue. People who feel they are in control of their lives have lower levels of depression [17], underlining the importance of negotiation between couples.
In one study in which personality is used to predict marital stability [19], personality data from teacher inquiries and personality tests was collected at ages 8, 14, and 27. Thus, it appears that personality data is seen to be present at early ages, but possibly subject to change. Marital dissolution is often predicted when the ratio of positive to negative behaviors during conflict resolution is less than one [24]. Similarly, stability is indicated when the ratio of positive to negative behavior is five or more. Stability is more likely when the husband is more willing to accept influence from the wife and when both make an effort to reduce his conflict-induced stress [13].
Through NAMS, couples see that some patterns of interaction always lead to failure. Women are more likely to start conflict discussions and therefore can often control the argument by not escalating from neutral to a negative conversation [13]. The negotiation will be guided by current research in the area such as social exchange theory and interdependence theory which help to identify goals and interaction strategies. Positive affect models such as humor, affection, and interest de-escalates marital conflict [13]. Men are more likely to withdraw from a conversation in the presence of negativity [13].
Some researchers [13] have found that the active listening model is confrontational in that it expects people to be empathetic in the presence of negativity and occurs infrequently during conflict resolution. Gottman [13] explains ``We are led to the hypothesis that the active listening model may be expecting a form of emotional gymnastics from people, who at that moment in their relationship, are somewhat emotionally disabled by conflict.''
Humans are social and emotional beings by nature. Since our agents are fundamentally designed to mimic human behavior, modeling emotions is a core aspect of our research. Emotions are interesting to model for a variety of reasons. Not only are they complex, but they are also dynamic, varying both episodically and longitudinally. A negative event can trigger an emotional response that may dissipate within a short time. Other emotions evolve slowly due to maturity, experience, or cumulative history. Some emotions are actually self-generating. For example, a person may become angry, with cause, but then remain angry because anger feeds on itself. Some emotions fade quickly, while others persist long after the original cause has been pushed from the forefront. For example, surprise may be short-lived, while guilt may fester. In addition, once an emotion is activated, the emotion itself becomes a driving force in subsequent actions.
Our research represents an emotional state in the form of a matrixi, E, (Figure 2) in which the rows represent the various fundamental emotions and the columns represent the cognitive activators. Thus, each emotional component is modeled by four cognitive activators, and eleven such components represent the emotional state of an agent. Personality traits, T, can be represented in the same matrix form.
Figure 2: Matrix representation of the Emotional State of an Agent
Human actions can be computed as responses to emotions. The final choice of response in any given situation not only depends on the personality of the agent, but also the present emotional state displayed by the agent. Izard defines emotional state as ``a particular emotion process of a limited duration'' and an emotional trait as a ``tendency of the individual to experience a particular emotion with frequency in his or her day-to-day life'' [15].
Types of emotion are inherently infinite in number, varying in intensity, duration, and range. Yet, most psychologists and researchers, over the years, have come to recognize a set a emotions called fundamental emotions. For example, fear can be used to represent terror, startle and many other variations of this general emotion. For this research, the set of fundamental emotions are identified as interest, joy, surprise, sadness, anger, disgust, fear, shyness, shame, guilt and love. [15].
The various causes of emotions are called activators. Differential Emotions Theory [15, 23] specifies that cognitive and non-cognitive activators exist in any individual. Non-cognitive activators can be neural (such as hormones, drugs, or expressive behavior) or affective (such as pain, gender, or fatigue). Cognitive activators on the other hand are conscious or sub-conscious mental processes that cause emotion [21]. In our research, we focus on the cognitive activators: appraisal, attribution, memory and anticipation[15].
Appraisal is the process of evaluating the situation or event, based on the facts, likes, and dislikes of the individual. Appraisal is what the individual instinctively feels when he/she comes to know of the occurrence of some event or situation. It is based purely on the personal tastes of the individual and does not involve any past experiences or other emotional constraints [15].
Situations are generally the result of some actions. The responsibility for these actions is attributed to someone [23]. The resulting emotions are attribution emotions. For example, if Bob does something praiseworthy (according to Alice) then there results in an emotion of approval for Bob, which can result in joy and love. Some of the common attribution emotions are admiration, reproach, shame, and guilt.
Memory is the root of many emotions. It involves access and evaluation of the history of events related to the present situation or event. The resulting emotions are memory emotions. Unpleasant memories can lead to sadness and depression that in turn can trigger other unpleasant memories. Similarly happy memories can trigger emotions of joy and interest.
Anticipation involves the expectation of some event. For example, when Alice and Bob discuss plans to attend the ballet next weekend, this could trigger anticipation in Alice about the event. The anticipation results in a feeling of joy for Alice, though the event is yet to occur.
Consider a sample scenario of a conversation between Alice and Bob.
Appraisal: Bob asks Alice if she would like to go to the new horror flick in town. Alice apprises the request and shows a lack of interest because she does not like horror flicks. Her appraisal gives rise to some surprise as she knows Bob is not enthusiastic about horror flicks either.
Memory: Further, Alice remembers that Bob has not shown any interest in her request to go to a ballet last weekend, which gives rise to anger. Alice is touched that Bob has made the move for reconciliation after yesterday's disagreement, and so this feeling of Love negates any feeling of anger that memories of the last weekend bring.
Attribution: Alice attributes a feeling of sadness to Bob as a result of the disagreement the day before. She also involuntarily attributes some shame and guilt in the disagreement of the previous day.
Anticipation: Alice has been anticipating a reconciliation move from Bob. She also feels joy at the tone of his request.
Once a percept (in our case a response/inquiry) is received from Bob. Alice's emotional state is updated and a response is formed based on factual information from the knowledge database, Alice's model of Bob's emotional state, Alice's current emotional state, and Alice's traits.
Fundamental emotions are interdependent and are interrelated in dynamic and relatively stable ways [15]. Some emotions are organized in a kind of hierarchical relationship (e.g., attention may graduate to surprise), while there is an apparent polarity between other emotions like joy and sadness. Still other sets of emotions have fairly regular relationships (e.g. interest may oscillate with fear when exploring the unknown) [15]. Our model represents these fundamental emotions as discrete and independent components. While this does restrict the model to an extent, most of the inter-relationships observed can be modeled (in the short term) as linear transformations on the emotion matrix. The usage of the inherent power of matrix algebra is the strength of this model. Non-linear changes occur in a separate logical transformation.
We model emotion in two pieces, which change independently. The permanent (trait matrix, T) allows for stable emotions which change gradually over time, while the volatile (emotion matrix, E) responds to current stimulus. One could classify the trait matrix as containing inherited attributes (computed from outside sources) and the reactive matrix as containing synthesized attributes (computed from inner sources).
Gmytrasiewicz and Lisetti [10] use a set of transformations, which they call emotional transformations, on tuples resulting in changed spaces in the parameter sets. We use emotional transformations, based on Gmytrasiewicz work, to effect the following transformations:
Combining of emotions:
For example,
if Bob successfully suppresses fear arising out of past experiences,
that is expressed via pre-multiplication of Bob's emotion matrix,
, by a combining matrix, C. Note that the superscript b indicates the
owner of the matrix.
= C
Reducing: Reducers effect cognitive activators of an emotion. For example, fatigue may cause present stimulus to be ignored and emotions related to memory to be enhanced. This transformation can be performed by doing a post-multiplication of E with an appropriate transformation matrix, A.
E' = EA
Narrowing Narrowing involves zeroing out rows or columns to reduce the number of different emotions to respond to. This is particularly important in creating an abstraction, both for efficiency and for capturing the essence of an emotional state. Such a reduced matrix in Bob may represent emotions that Alice can not appreciate and thus disables.
Filtering: Filtering involves disabling (or manipulating) certain emotions due to personal beliefs. For example, if an agent believes that women are not allowed to be angry or men are not allowed to show fear, the agent may block those emotions from their response. In addition, cognitive activators can be suppressed. This can be performed by both pre and post multiplying the emotion matrix with appropriate matrices.
E' = CEA
Comparing:
Comparison is done by matrix subtraction in order to model emotions that have not been acted upon
(e.g., pent up emotions).
These emotions which are not acted upon are prime
candidates for a later ``blow up''. For example, when Bob totally ignores
some emotions when he responds to a percept, then that emotion that is
ignored is carried over to later conversations. It might also alter his
emotional traits. 
is Bob's public emotional state (the state acted
upon). 
is Bob's internal emotional state (what he really feels).
The pent-up emotions are stored in the disparity matrix, 
which is computed as:
= -
This model of emotions allows other analysis. When Alice models Bob's emotions with a reduced matrix, there will be times when she is surprised at what he does. This ``jump'' in what she expected him to do and what he does is caused, partially, because she is working with a reduced model of him. We could explore the problems in a relationship caused because we are working with a simplified model. Thus, we may learn that Alice needs some sensitivity training in understanding components of Bob's emotions.
In our system, the percept is inquiry/response. The percept consists not only
of the locution (what is actually said) but the illocution (its core meaning). Thus,
the speech act consists of a tuple (Alice, Bob, L,
), which signifies
that Alice communicates to Bob with the locution L along with her Emotion matrix ,
in which the subscript, p, indicates that it is a public version of the
Emotion matrix and the superscript, a, indicates that it represents Alice's emotions.
The public version is distinguished from Alice's internal emotions (
)
in that it allows Alice to convey a different version of her emotions that she
actually feels.
When Bob receives the tuple, he uses the locution and
Bob's personal belief model of Alice (
) to translates the emotion matrix into his
version of Alice's emotion.
= Personalize(Bob,, ),
where in the superscript b refers to the ownership of the
belief and the superscript a refers to the subject of the belief.
The next step is to choose a response tuple. Thus, the function FormResponse returns a
tuple as specified below, where 
is the trait matrix for Bob:
(Bob, Alice, L', ) = FormResponse(Alice, Bob,L, , )
Note, the response contains the emotion Bob chooses to convey.
However, may be substantially different from Bob's internal emotion
. This difference between the expressed emotion (public) and the private
emotion (internal) is termed the disparity matrix
and is computed as a weighted difference in the emotion matrices, where m and n
represent the relative importance of the various matrices.
= m - n
This difference matrix is the source of additional emotion for Bob. It represents unexpressed emotion. It can be a positive factor (e.g., Bob is proud he did not express anger) or a negative factor (e.g., Bob is angry that he can not respond candidly).
Finally, when Alice receives the response tuple, she compares the emotional content of the message she receives (the perlocution) with what she expected (the illocution). The difference in what she expected to receive and what she did receive is also a trigger of emotion (e.g. Alice is joyful that Bob accepted the proposal, angry that he responded so happily to her rude comment, or surprised that he consented to attend the ballet).
Consider a sample scenario of a conversation between Alice and Bob. Alice's trait matrix is
represented as 
and Bob's trait matrix as . Now consider that Bob asks Alice, if she
could go to the new horror flick playing at the multiplex. This request could result in the following
emotional state, , for Alice. Note that each emotion is represented as an integer value in
the range 0-100.
Interest: Once Alice hears this request from Bob, there is an inevitable occurrence of interest for various reasons. Interest could be generated in appraisal, as Alice is interested in going to the movies in general. Interest could also be generated as a result of the knowledge of the movie, and memories of her previous movie going experiences with Bob. The anticipation of good time could also cause interest in Alice. These are represented in the first row of the matrix.
Joy: Alice could feel joy on appraisal, as she realizes that she does have the free time to go to the movies. She could remember her last experience of a horror movie from the same production and feel additional joy. While she also attributes some of the joy to Bob as he has suggested the idea. And finally, the anticipation of the enjoyment could itself cause more joy for Alice.
Surprise: The request from Bob could also have produced some surprise as they had a disagreement the night before and Bob was trying to make up. Also the surprise would be fueled from the memories of last night's disagreement.
Surprise and Anger: Alice also attributes some sadness due to Bob's behavior from last night. Sadness is also triggered from the memories of the fight. Similarly, Alice feels anger by attributing the previous nights events as Bob's mistake and also from the memories of the event.
Disgust and Fear: On the other hand, the movie itself might trigger memories of disgust (due to memories of horror scenes from previous movies). Alice might also anticipate feelings of disgust once she returns from the movie. Similarly, Alice could experience mild feelings of fear, from the appraisal of the request, memories of past horror movies and anticipation of the movie to be seen.
Guilt: Alice could also feel guilt from memories of her role in the previous night's disagreement. She would also attribute some of this guilt to herself, for her behavior.
Love: Finally, Alice could feel love for Bob as she appraises his offer. She would also remember their previous movie going experiences as well as attribute good feelings with Bob for his breaking of the ice. Also, she would anticipate a night of sweet reconciliation and this would result in further love towards Bob.
To fill the emotional matrix with numbers, we need to first realize what events might result in what emotions. While the person's basic preferences (like Alice's preference for horror movies etc.) play an important role, there are other more fundamental factors involved too. For example, interest generally results from a change of scene, an animation, novelty or imagery. These factors help us determine the emotions resulting from a situation or process (like appraisal or anticipation). While we have attempted to explain how the matrix is obtained, the calculation of the specific numbers is beyond the scope and space available for this article.
Once the emotional state of Alice () is computed into a matrix, then Alice's desires, goals, and
belief network (,,) are applied on .
Alice decides that since Bob had taken the initiative to break the ice, it is not appropriate for her to
display her anger over last night. This decision is reached from the set of beliefs,
, held by her as
well as her personality, . So, a transformation on is performed, to reflect this decision by
Alice.
In this context since we need to manipulate the cognitive activators of an emotion (anger), we
need to perform a Pre-multiplication with an appropriate 11x11 transformation matrix. Let the
transformation matrix be C:
The emotional state, , when pre-multiplied with C will result in a transformed emotional state
where the anger emotion is decreased to 20% of its original value, with the remaining
emotions intact i.e. = C*. The resultant matrix
is:
Similarly, If Alice decides not to let her anticipation be visible to Bob, and goes ahead to suppress it then it can me modeled using a post-multiplication with an appropriate 4x4 transformation matrix, A. If the anticipation is to be suppressed to 30% of its original value then A would be:
The emotional state, ', when post-multiplied with A, will result in a transformed emotional state
'' where the anticipation is suppressed to 30% of its original value, with the remaining cognitive
activators intact i.e. 
= A*. The resultant matrix
is:
After Alice's response is computed,
matrix subtraction is performed to acquire the pent-up or suppressed
emotions, which have a direct effect on the personality, emotional traits and the general
emotional state of a person. Lots of pent up emotions tend to lead to a ``blow up'' after some time.
Subtracting the transformed matrix () from the original emotional matrix ()
can perform compute this disparity matrix, 
.
So, = - '' i.e.
As can be observed the precipitate matrix contains mostly the suppressed anger and anticipation. This matrix would further be processed with transformations to place a priority on some emotions over others. For e.g. suppressed anger has a tendency to stay pent-up for a later ``blow-up'' than pent-up anticipation which would generally dissipate after some time.
Finally, once the emotional state has been computed and appropriate transformations have been
performed to reflect the beliefs, desires and goals of the agent, the response is prepared. The
response includes the rational decision that is taken based on the personality mapping
(,,) 
and also
the emotional state that was computed above.
When agents are asked to mimic human behavior, emotions and personality must be modeled. Each communication is modeled with a locution utterance and a illocution emotion matrix. An emotion matrix is computed from volatile and trait component matrices which are updated separately. Each matrix contains data representing not only the eleven fundamental emotions, but the cognitive activators. Using matrix algebra, these matrices are massaged to create a variety of effects: narrowing, filtering, enhancing, combining, and comparing.