Designing for Co-located and Virtual Social Interactions in Residential Care

Designing for Co-located and Virtual Social Interactions in Residential Care

Francisco Ibarra
University of Trento
Trento, Italy Francesca Fiore
University of Trento
Trento, Italy Marcos Baez
University of Trento
Trento, Italy
Tomsk Politechnic University Fabio Casati
University of Trento
Trento, Italy
Tomsk Politechnic University

In this paper we explore the feasibility and design challenges in supporting co-located and virtual social interactions in residential care by building on the practice of reminiscence. Motivated by the challenges of social interaction in this context, we first explore the feasibility of a reminiscence-based social interaction tool designed to stimulate conversation in residential care with different stakeholders. Then, we explore the design challenges in supporting an assisting role in co-located reminiscence sessions, by running pilot studies with a technology probe. Our findings point to the feasibility of the tool and the willingness of stakeholders to contribute in the process, although with some skepticism about virtual interactions. The reminiscence sessions showed that compromises are needed when designing for both story collection and conversation stimulation, evidencing specific design areas where further exploration is needed.

  Designing for Co-located and Virtual Social Interactions in Residential Care

Francisco Ibarra University of Trento Trento, Italy Francesca Fiore University of Trento Trento, Italy Marcos Baez University of Trento Trento, Italy Tomsk Politechnic University Fabio Casati University of Trento Trento, Italy Tomsk Politechnic University

  Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the owner /author(s).

Copyright held by the owner/author(s).

DIS’18 Companion, June 9–13, 2018, , Hong Kong

ACM 978-1-4503-5631-2/18/06.

Author Keywords

Reminiscence; Social interactions; Older adults; Residential care; Design recommendations

ACM Classification Keywords

H.5.m. [Information Interfaces and Presentation (e.g. HCI)]: Miscellaneous


Failing to maintain meaningful connections and to stay socially active is a growing concern in residential care [?]. Transitioning to residential care requires adjustments difficult to assimilate, by both older adults and their families [?]. In turn, this requires nursing homes (NH) to devise special activities that promote social integration and a sense of community. Despite these efforts, social isolation and loneliness still plague residential care [?, ?], with devastating effects on the quality of life of older older adults [?].


In this paper we explore the feasibility of reminiscence technology to support and stimulate co-located and virtual interactions in residential care. While the idea of reminiscence technology is not novel (see [?] for a review), and previous work have taught us valuable lessons on how to facilitate usage by older adults [?, ?], stimulate cognitive functions [?, ?], collect memories and support storytelling [?, ?, ?], little attention has been paid to creating opportunities for social interactions and promoting social integration in residential care. Motivated by recent needfinding studies [?, ?], pointing to the potential of reminiscence technology to support NH social activities, we conducted semi-structured interviews and pilot reminiscence sessions to validate the concept of a reminiscence application that builds on the social functions of reminiscence therapy to support (and create opportunities for) co-located and virtual interactions.

Design concept

The concept, shown in Figure LABEL:figure:collage, resulted from visits, interviews, and focus groups in nursing homes and prior needfinding studies [?]. A central figure in this process is the helper, who will be in charge of handling the device and assisting older adult residents by guiding the activities described next:

• Digitizing pictures. The family contributes with pictures related to the resident and with tags (place, date, people), to create an archive that can later be enriched with stories. Pictures can be added as older adults enter the nursing home, or continuously during their stay.

• Reminiscing and collecting stories. During visits, family members use pictures to collect stories. The goal is to make visits more interesting, not only to collect information. Therefore, pictures and stories can be revisited at any time, shared with family and friends online, thus generating memories and stimulating conversation. Nursing home staff can also assist or conduct the activity itself.

• Engaging in online interactions. Feedback and comments on the stories shared are displayed in a format that facilitates consumption by the resident. Residents can also access pictures and stories from others (friends) in the NH, potentially leading to face-to-face interactions.

• Connecting with peers. The stories and tags are used to find common life points among residents. These common aspects are used to create mutual awareness, and to provide information about similar interests and affinity with certain topics so as to support animation activities.

The above process describes a social interaction model that aims at stimulating social interactions in nursing homes, via co-located interactions in reminiscence sessions, virtual interactions through shared stories, and facilitating social activities by finding common life points among residents. The design is built on the opportunities of co-located presence, and in particular, visits by family members and social activities in NHs. The co-located setting poses two main research questions: i) how to effectively support co-located social interactions, and ii) if and how helpers (family members, NH staff, volunteers) can cooperate in the process.


To validate the concept we conducted two separate studies in 4 nursing homes in northern Italy. The studies were approved by the Ethical Committee of the University of Trento.


Study 1. Evaluating feasibility and perceived value

In this study we evaluated the willingness and ability of family members (FM) and NH staff to participate in tool-driven reminiscence sessions (see Figure LABEL:figure:collage) as well as the perceived potential of the tool to achieve the intended benefits and the importance of such benefits. The study was structured as sessions with users in which we presented, via a storyboard, the concept, main supported activities and expected benefits of the tool. Using a printed mockup ( we then presented each activity in more detail, requesting feedback on the ability and willingness to perform them and the perceived value (5-point Likert scale). After evaluating activities, we requested participants to rate (on a 5-point Likert scale) the importance of the objectives addressed by the tool. For this study we recruited 27 participants (21 females, 15 relatives) but excluded 3 FMs who did not complete the questionnaires.

Study 2. Prototyping co-located reminiscence sessions

We prototyped the experience of co-located social interactions in reminiscence sessions among resident-relative pairs. With the help of the NHs, we recruited 3 diverse resident-relative pairs (see Table LABEL:tab:table1) who volunteered to participate. Prior to the study, FMs were asked to share 10 old pictures from their older adult relatives which we later used to prepare the supporting tool for the reminiscence session.

The session was designed to have FMs as "helpers", using an iPad tablet. We used Google Forms to prototype the story collection and reminiscence session (Figure LABEL:figure:form). The form was designed to emphasize pictures and request picture information (e.g. place, date, people, activity). The session started with demographic information completed by the FM on the tablet, which also served as a training and tutorial on tablet gestures and navigation. The FMs then joined their older relatives in a 25 minutes reminiscence session, where they were asked to go through the pictures and engage their relatives in storytelling while collecting picture information. Two researchers were present, one guided the activities and another observed and took notes.


Feasibility and perceived value

We evaluated all activities for feasibility and perceived value as agreement with the statements "I will be able to perform this activity on my own" and "This activity helps to accomplish the objectives described", respectively. As seen in Figure LABEL:figure:feasible (A), all participants said they would be able to "browse" and "collect" stories. Only sharing with "family" (83%) and "NH residents" (58%) received at least one response score below neutral. For perceived value (B), "digitizing" and "collecting" had only positive scores. Again, (virtual) sharing with "family" (83%) and "other residents" (66%) had lower (though overall still positive) scores.

Participants perceived the objectives addressed by the tool as important or very important. As seen in Figure LABEL:figure:feasible (C), this was the case particularly for "stimulating memory" (100%), conversations and collecting stories. Making residents aware of the interest generated by their stories (83%) and fostering bonding among residents (87%) received a few more neutral scores, all from FMs. NH staff on the other hand saw these objectives as very important. As for the perceived usefulness of the tool for the resident or for themselves, the response was quite high for participants (91%) but lower for NH residents (71%). NH staff was generally very positive while some FMs were more skeptical in relation to the usefulness of the tool for their relatives.

Co-located reminiscence session

Overall, participants had a good time during the reminiscence session (GUESS [?] Enjoyment subscale 6.6/7 for P2 and P3, and 7/7 for P1). Participants did not experience problems in operating the tablet or touch-typing.

margin: margin:

Elicitation was mostly prescriptive, heavily based on the form questions, which seem to deter participants from coming up with more engaging questions to generate conversation. We identified well defined information elicitation strategies: verification questions (Who is in the picture? It is <Person>, right? [Yes]), which gave little space for symmetric conversations, leading questions, (Who is this? Which aunt?) in which the FM provided clues in a clear effort to facilitate recall, and recall questions (Who is this person?), seeking a direct answer from the OA. Independently of the strategy, it was clear that the FMs already had the answer to these questions, and in some cases filled them out directly without asking the participant. The use of the tablet in this context switched from show and type, and behaviors ranged from controlling (FM) to more shared (FM-OA).

The sessions were effective in collecting information. All participants completed information for 4 pictures within the 25 minutes allowed, leaving no fields blank. However, this focus on information collection hindered social interactions. While FMs were typing, OAs were mostly silent. This silence occurred both when relatives where discussing the picture first and typing later, and when they where asking for information and typing simultaneously.

Storytelling emerged naturally, with picture aspects (people, place) triggering specific memories, even when residents were not able to recall the information for that particular picture (e.g., when revealed that <Person> was in the picture, P1 would describe the person "<Person> moved to <City>.. he had 5 children.."). In P2, involving a subject with memory recall issues, the FM would try (unsuccessfully) a question, and then tell the related story to the resident. The resident in this case would not tell stories but leave impressions ("The happiest day of my life"). Conversations were more natural when pursuing the topic of conversation that triggered a story. Because of the design of the form, participant tended to follow a script which made it less natural. Interestingly, FMs recurred to few reactions in response to pictures (e.g., appraisal, curiosity) to engage older adults.

Discussion and Conclusion

The results point to a tool that is considered useful and that addresses objectives considered as very important – in particular stimulating memory and conversations. Results also indicate feasibility and willingness of FM and NH staff to play an active role, especially in co-located activities.

In terms of design, we learned that i) the interface should motivate more symmetric interactions, avoiding scripted conversation, ii) tagging should be separated from the reminiscence session (reduced, or brought up only when necessary), as FMs are able to provide most of the information; iii) the tool should allow for non-linear navigations, following elements of interest based on the flow of conversation.

As ongoing work we are exploring design alternatives to support specific co-located tasks and the use of AI to support story elicitation, to reduce the need for manual tagging, which has recently shown exciting results [?, ?].


This project has received funding from the EU Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No 690962. This work was also supported by the “Collegamenti” project funded by the Province of Trento (l.p. n.6-December 13th 1999).


  • 1
  • 2 Arlene Astell, Norman Alm, Gary Gowans, Maggie Ellis, Richard Dye, Jim Campbell, and Phil Vaughan. 2009. Working with people with dementia to develop technology: The CIRCA and Living in the Moment projects. PSIGE Newsletter 64 (2009).
  • 3 Abir K Bekhet and Jaclene A Zauszniewski. 2012. Mental health of elders in retirement communities: is loneliness a key factor? Archives of psychiatric nursing 26, 3 (2012), 214–224.
  • 4 Beverly H Brummett, John C Barefoot, Ilene C Siegler, Nancy E Clapp-Channing, Barbara L Lytle, Hayden B Bosworth, Redford B Williams Jr, and Daniel B Mark. 2001. Characteristics of socially isolated patients with coronary artery disease who are at elevated risk for mortality. Psychosomatic Medicine 63, 2 (2001), 267–272.
  • 5 Valentina Caforio, Marcos Baez, and Fabio Casati. 2017. Viability of Magazines for Stimulating Social Interactions in Nursing Homes. In IFIP Conference on Human-Computer Interaction. Springer, 72–81.
  • 6 Francisco Ibarra, Marcos Baez, Francesca Fiore, and Fabio Casati. 2017. Stimulating Conversations in Residential Care Through Technology-Mediated Reminiscence. In IFIP Conference on Human-Computer Interaction. Springer, 62–71.
  • 7 Amanda Lazar, Hilaire Thompson, and George Demiris. 2014. A systematic review of the use of technology for reminiscence therapy. Health education & behavior 41, 1_suppl (2014), 51S–61S.
  • 8 Diana TF Lee, Jean Woo, and Ann E Mackenzie. 2002. A review of older people’s experiences with residential care placement. Journal of advanced nursing 37, 1 (2002), 19–27.
  • 9 Hung-Chi Lee, Ya Fang Cheng, Szu Yang Cho, Hsien-Hui Tang, Jane Hsu, and Chien-Hsiung Chen. 2014. Picgo: designing reminiscence and storytelling for the elderly with photo annotation. In Proceedings of the 2014 companion publication on Designing interactive systems. ACM, 9–12.
  • 10 Hung-Chi Lee and Jane Yung-jen Hsu. 2016. PicMemory: Enriching Intergenerational Family Interaction and Memory Collection. In Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems. ACM, 3715–3718.
  • 11 Nasrin Mostafazadeh, Ishan Misra, Jacob Devlin, Margaret Mitchell, Xiaodong He, and Lucy Vanderwende. 2016. Generating natural questions about an image. arXiv preprint arXiv:1603.06059 (2016).
  • 12 Fredrica Nyqvist, Mima Cattan, Lars Andersson, Anna K Forsman, and Yngve Gustafson. 2013. Social capital and loneliness among the very old living at home and in institutional settings: a comparative study. Journal of aging and health 25, 6 (2013), 1013–1035.
  • 13 Mikki H Phan, Joseph R Keebler, and Barbara S Chaparro. 2016. The Development and Validation of the Game User Experience Satisfaction Scale (GUESS). Human factors 58, 8 (2016), 1217–1247.
  • 14 Anne Marie Piper, Nadir Weibel, and James Hollan. 2013. Audio-enhanced paper photos: encouraging social interaction at age 105. In Proceedings of the 2013 conference on Computer supported cooperative work. ACM, 215–224.
  • 15 Ponnusamy Subramaniam and Bob Woods. 2016. Digital life storybooks for people with dementia living in care homes: an evaluation. Clinical interventions in aging 11 (2016), 1263.
  • 16 Kristine Theurer, W Ben Mortenson, Robyn Stone, Melinda Suto, Virpi Timonen, and Julia Rozanova. 2015. The need for a social revolution in residential care. Journal of aging studies 35 (2015), 201–210.
  • 17 Kenneth Tran, Xiaodong He, Lei Zhang, and Jian Sun. 2016. Rich image captioning in the wild. In Computer Vision and Pattern Recognition Workshops (CVPRW), 2016 IEEE Conference on. IEEE, 434–441.
  • 18 David West, Aaron Quigley, and Judy Kay. 2007. MEMENTO: a digital-physical scrapbook for memory sharing. Personal and Ubiquitous Computing 11, 4 (2007), 313–328.
Comments 0
Request Comment
You are adding the first comment!
How to quickly get a good reply:
  • Give credit where it’s due by listing out the positive aspects of a paper before getting into which changes should be made.
  • Be specific in your critique, and provide supporting evidence with appropriate references to substantiate general statements.
  • Your comment should inspire ideas to flow and help the author improves the paper.

The better we are at sharing our knowledge with each other, the faster we move forward.
The feedback must be of minimum 40 characters and the title a minimum of 5 characters
Add comment
Loading ...
This is a comment super asjknd jkasnjk adsnkj
The feedback must be of minumum 40 characters
The feedback must be of minumum 40 characters

You are asking your first question!
How to quickly get a good answer:
  • Keep your question short and to the point
  • Check for grammar or spelling errors.
  • Phrase it like a question
Test description