Today, we live in an era in which we can communicate via computers more than ever before. While novel social networks and emerging technologies help us transcend the spatial and temporal constraints inherent to in-person communication, the trade-off is a loss of natural expressivity. While empathetic interaction is already challenging in in-person communication, computer-mediated communication makes such empathetically rich communication even more difficult. Are technology and intelligent systems opportunities or threats to more empathic interpersonal communication?
My future research vision is to build computational systems that facilitate understanding and empathy. Realizing empathy is suggested not only as a way to communicate with others, but also to design products for users and facilitate creativity (Lim, 2013). In this position paper, I suggest a framework to breakdown empathy, introduce each element, and show how computing, technologies, and algorithms can support (or hinder) certain elements of the empathy framework.
2. Empathy Framework
Empathic interactions involve two roles — the empathizer and the empathizee. Figure 1 shows the framework of empathy.
Empathy is often reduced to a problem in terms of one person needing to empathize with another. However, the other — the person who eventually receives empathy — is a significant stakeholder in this relation, and is often called the empathizee (Gladstein, 1987). Some anthropologists view empathy as depending on what empathizees are “willing or able to tell about themselves.” . Seeing the dynamic as dyadic emphasizes the importance of the person to be empathized with, placing that person on par with the person displaying empathy.
(Self-expression) How can we help empathizees with computing? The empathizee does not need to be present to the empathizers and can instead be “deeply heard from sound or video recording or perhaps even from written expression, an artistic work, or another expressive product (Barrett-Lennard, 1981).” This highlights the need for the empathizee to be afforded an expressive medium, including and beyond in-person communication — any kind of creative (or even artistic) practice through which one can express their thoughts and emotions may suffice. This can include simply computational supports in creative practice; namely writing, art, music, research, and other novel expressive media. Computational tools for self-expression can help constitute an empathic attention set (Barrett-Lennard, 1981).
(Self-reflection) Self-reflection is a process that may need to be followed by self-expression. It is the exercise of introspection, coupled with the willingness to learn about oneself, in order to help achieve self-awareness. Therefore, self-reflection is important to understanding what needs to be expressed. Furthermore, self-reflection is something that can support empathizers, as an understanding of themselves leads to enhancements in the ability to empathize with others (Böckler et al., 2017). Therefore, computational systems that promote self-reflection can enhance people’s abilities to empathize with others.
Psychologists typically consider empathy as an individual ability — the empathizer’s ability — to share others’ feelings by observing or learning about their emotional state (Decety and Lamm, 2006). Wiseman finds that there are four defining attributes of empathy: 1) perspective taking (“see the world as others see it”), 2) non-judgmental, 3) understanding another’s feelings, and 4) communicating that understanding (Wiseman, 1996).
(Perspective Sharing) Being able to take another’s perspective — a cognitive function — is an essential part of empathy. As people often express empathy as “being in someone else’s shoes”, the process of realizing empathy emphasizes sharing the visual perspective of the other. Therefore, perspective taking is the one of the essential elements in empathizing with others. I believe emerging mixed reality technologies can be used to support perspective taking, as with augmented reality and virtual reality — one can literally render the other’s perspective. For example, virtual reality can put a empathizer in an empathizee’s specific situations (e.g., homelessness) (Herrera et al., 2018). Researchers have used AR/VR technologies to facilitate empathic interaction between remote collaborators (Lee et al., 2016, 2017). Lastly, one needs to distinguish empathic response from own reactions drawing on prior experience and triggered by the empathizee’s perspectives. This includes recognizing other qualities that may have affected how the empathizee works, which may include the empathizee’s environment, personal traits, and other relevant contextual information. Alternatively, it may include unfolding information over the temporal dimension that empathizers may have not access to in real time (Lee, 2018). To that end, the empathizee will benefit from context-aware computing environments that can supplement expressiveness in regard to their experience (Schilit et al., 1994).
(Non-judgmental) Being non-judgmental is an area in which computing can help empathizers, as computers is not judgmental by default without any algorithm injected by human. Human-to-human communication can be mediated in a way that is less judgmental. For example, asynchronous textual communication enabled by computers can promote empathic communication by blocking visual cues that could have prompted judgment in in-person face-to-face conversation. Communication may also be moderated by algorithms to keep the conversation neutral before transmission.
(Recognizing Emotions) Emotion recognition is a long-standing topic in the field of affective computing and natural language processing. For example, speech and facial expressions have been analyzed for computers to understand the emotions of others. Therefore, existing research in these fields can support empathetic conversation, particularly when these modalities available in in-person communication are not available in other types of computer-mediated communication. Thus, enriching the communication medium may be a means of restoring or augmenting emotional recognition in computational media. Other types of communication can be similarly augmented with emotion recognition techniques.
(Communicating Understanding) Empathizers communicating back to empathizees to express empathy is a vital element of the empathy cycle (Barrett-Lennard, 1981). In computer-mediated empathy, as empathic interaction can happen without empathizees realizing it, awareness of empathy can help, closing the loop of empathy. In addition, empathizees being aware of the received empathy is often followed by further, deeper conversation on a subject that can further reinforce empathy.
3. Computational Empathy and Computational Thinking
In the suggested framework, empathizers and empathizees do not necessarily need to be human. Either end can be replaced with computers, machines, or algorithms. The case of the empathizer being a computer opens up a new field of computational empathy in which we make computers empathize with humans (Paiva et al., 2017). This is largely is the most relevant scenario in the existing field of affective computing. However, additional arguments made for both empathizers and empathizees remain valid. For example, as a human empathizee, we need to express ourselves in ways that computers can easily understand.
By contrast, computers can be empathizees, where a human must understand a computer’s intentions. This is relevant to computational thinking, an essential skill for the immediate future, in which humans collaborate and interact with computers ubiquitously (Wing, 2006). Again, as with computers as empathizees, it is important to be able to express computer algorithms in human-comprehensible ways, which is relevant to the recent initiative of explainable artificial intelligence (Gunning, 2017).
- The empathy cycle: refinement of a nuclear concept.. Journal of counseling psychology 28 (2), pp. 91. Cited by: §2.1, §2.2.
- Know thy selves: learning to understand oneself increases the ability to understand others. Journal of Cognitive Enhancement 1 (2), pp. 197–209. External Links: Cited by: §2.1.
- Human empathy through the lens of social neuroscience. The scientific World journal 6, pp. 1146–1163. Cited by: §2.2.
- What it all means. In Empathy and Counseling, pp. 173–189. Cited by: §2.1.
- Explainable artificial intelligence (xai). Defense Advanced Research Projects Agency (DARPA), nd Web. Cited by: §3.
- Building long-term empathy: a large-scale comparison of traditional and virtual reality perspective-taking. PloS one 13 (10), pp. e0204494. Cited by: §2.2.
- Improving collaboration in augmented video conference using mutually shared gaze. In Proceedings of the 27th International Conference on Artificial Reality and Telexistence and 22nd Eurographics Symposium on Virtual Environments, pp. 197–204. Cited by: §2.2.
- Liveness in interactive systems. External Links: Cited by: §2.2.
- A remote collaboration system with empathy glasses. In 2016 IEEE International Symposium on Mixed and Augmented Reality (ISMAR-Adjunct), pp. 342–343. Cited by: §2.2.
- Realizing empathy: an inquiry into the meaning of making. Seung Chan Lim. Cited by: §1.
- Empathy in virtual agents and robots: a survey. ACM Transactions on Interactive Intelligent Systems (TiiS) 7 (3), pp. 11. Cited by: §3.
- Context-aware computing applications. In 1994 First Workshop on Mobile Computing Systems and Applications, Vol. , pp. 85–90. External Links: Cited by: §2.2.
- Computational thinking. Communications of the ACM 49 (3), pp. 33–35. Cited by: §3.
- A concept analysis of empathy. Journal of advanced nursing 23 (6), pp. 1162–1167. Cited by: §2.2.