What does ChatGPT know about dementia?

Post by Jill Dosso

This blog post summarizes results from the peer-reviewed journal article “What does ChatGPT know about dementia? A comparative analysis of information quality” published in the Journal of Alzheimer’s Disease, 2023

The need for online information about Alzheimer’s Disease and dementia

Persons living with dementia and their care partners often wish for access to more and better information about living with the condition [1–3]. While healthcare providers are a valued resource for this information, their time is limited, and access can be challenging. At least 40% of older adults seek health information online [4], though this number may be much higher in some populations [5,6]. Online information about dementia exists on many virtual platforms, including social media, and varies widely in quality [1–3,7–10].

A recent analysis identified a number of barriers to online information access for persons living with dementia: information is targeted towards care partners and medical practitioners, rather than persons with lived experience; information can be pessimistic and hard to decipher; information is inaccurate or overly simple; and information is untrustworthy [11]. There is a clear demand for easily accessible, accurate dementia information that can be customized to a variety of user information needs.

ChatGPT: a new information source

ChatGPT (Conversational Generative Pre-training Transformer) is an online tool launched by OpenAI in November 2022. Users can engage in typed back-and-forth dialogue with the system through a web browser. Unlike a typical search engine query like a Google search, the platform retains information across an interaction, which creates a more natural and conversational experience. The “machinery” of ChatGPT is a generative artificial intelligence, meaning it uses machine learning models to create novel and data-driven content. These models have been trained on a dataset created from online materials and refined through human feedback [12,13].  The exact content of this dataset has not been publicly released.

How does ChatGPT stack up as a source of online information about dementia?

In a recent study, our research team at the Neuroscience, Engagement, and Smart Tech Lab at Neuroethics Canada asked how ChatGPT compared to other sources of online information about dementia. To create a set of questions that real users would likely have about dementia, we collected Frequently Asked Questions from the webpages of three national dementia organizations in Canada, USA, and Mexico. We posed these questions to ChatGPT-3.5 in April 2023. Responses from ChatGPT were evaluated using a standard tool previously developed by the NEST lab to assess the quality of online health information [14].

Strengths of ChatGPT-3.5: We found that ChatGPT, like the Alzheimer’s organizations, provided generally accurate information, directed users to bring their questions to a physician, and did not endorse commercial products.

Strengths of Alzheimer’s organization websites: Organizations were more likely than ChatGPT to state the limits of scientific evidence explicitly and produced more readable responses (i.e., responses had a readability score corresponding to a lower grade level). They were also more likely to link to local, specific, and actionable resources for support.

Conclusion

This research represents one snapshot of behaviour from a generative artificial intelligence tool: ChatGPT-3.5. This platform, and others, will continue to change over time and may produce different responses with different prompts or in languages other than English.

This work can support:

1. Persons living with dementia and their care partners in screening potential sources of dementia information online;
2. Healthcare providers as they advise persons living with dementia and their care partners; and
3. Non-profit providers of dementia support services as they create helpful resources for their communities.

There is an ethical imperative to include persons with lived experiences of dementia in the creation of technologies to support them [15]. These perspectives are critically important for tools at the intersection of generative artificial intelligence and digital health. Understanding the online information available to these families is a first step in prioritizing their needs and perspectives in technology research and development.

---

Jill Dosso, PhD is a Postdoctoral Fellow in the Neuroscience, Engagement, and Smart Tech (NEST) lab at the University of British Columbia and BC Children’s Hospital. In her work, she studies the perspectives of persons with lived experience on emerging technologies to support brain health across the lifespan.

---

References

[1]          Allen F, Cain R, Meyer C (2020) Seeking relational information sources in the digital age: A study into information source preferences amongst family and friends of those with dementia. Dementia 19, 766–785.

[2]          Montiel-Aponte MC, Bertolucci PHF (2021) Do you look for information about dementia? Knowledge of cognitive impairment in older people among their relatives. Dement Neuropsychol 15, 248–255.

[3]          Washington KT, Meadows SE, Elliott SG, Koopman RJ (2011) Information needs of informal caregivers of older adults with chronic health conditions. Patient Educ Couns 83, 37–44.

[4]          Yoon H, Jang Y, Vaughan PW, Garcia M (2020) Older Adults’ Internet Use for Health Information: Digital Divide by Race/Ethnicity and Socioeconomic Status. J Appl Gerontol 39, 105–110.

[5]          Levy H, Janke AT, Langa KM (2015) Health Literacy and the Digital Divide Among Older Americans. J Gen Intern Med 30, 284–289.

[6]          Tam MT, Dosso JA, Robillard JM (2021) The impact of a global pandemic on people living with dementia and their care partners: analysis of 417 lived experience reports. J Alzheimers Dis 80, 865–875.

[7]          Robillard JM (2016) The Online Environment: A Key Variable in the Ethical Response to Complementary and Alternative Medicine for Alzheimer’s Disease. J Alzheimers Dis 51, 11–13.

[8]          Robillard JM, Johnson TW, Hennessey C, Beattie BL, Illes J (2013) Aging 2.0: Health Information about Dementia on Twitter. PLOS ONE 8, e69861.

[9]          Robillard JM, Illes J, Arcand M, Beattie BL, Hayden S, Lawrence P, McGrenere J, Reiner PB, Wittenberg D, Jacova C (2015) Scientific and ethical features of English-language online tests for Alzheimer’s disease. Alzheimers Dement Diagn Assess Dis Monit 1, 281–288.

[10]        Robillard JM, Feng TL (2016) Health Advice in a Digital World: Quality and Content of Online Information about the Prevention of Alzheimer’s Disease. J Alzheimers Dis 55, 219–229.

[11]        Dixon E, Anderson J, Blackwelder D, L. Radnofsky M, Lazar A (2022) Barriers to Online Dementia Information and Mitigation. In CHI Conference on Human Factors in Computing Systems ACM, New Orleans LA USA, pp. 1–14.

[12]        Introducing ChatGPT, Last updated November 30, 2022, Accessed on November 30, 2022.

[13]        Forbes, The Next Generation Of Large Language Models, Last updated February 7, 2023, Accessed on February 7, 2023.

[14]        Robillard JM, Jun JH, Lai J-A, Feng TL (2018) The QUEST for quality online health information: validation of a short quantitative tool. BMC Med Inform Decis Mak 18, 87.

[15]        Robillard JM, Cleland I, Hoey J, Nugent C (2018) Ethical adoption: A new imperative in the development of technology for dementia. Alzheimers Dement 14, 1104–1113.

Emotions and stigma: Social robots from the perspectives of older adult end-users

Post by Jaya Kailley

This blog post summarizes results from the peer-reviewed journal article “Older adult perspectives on emotion and stigma in social robots” published in Frontiers in Psychiatry (2023).

Social robots: Tools to support the quality of life of older adults

Canada’s older adult population is rapidly growing (1), and tools such as social robots may be able to support the health and quality of life of older adults. Social robots are devices that can provide support to users through interaction. They can be used for health monitoring, reminders, and cognitive training activities (2), and they have been shown to decrease behavioural and psychological symptoms of dementia, improve mood, decrease loneliness, decrease blood pressure, and support pain management (3–6).

End-user driven approaches to improve social robot adoption

 Despite their benefits, social robots are not readily adopted by older adults. Issues raised in the literature include a lack of emotional alignment between end-users and devices (7,8) and the perception of stigma around social robot use (3,9). To address these barriers and improve the design of future devices, it is important to better understand user preferences for social robots, rather than relying only on expert opinion (10).

Older adult perspectives on social robot applications, emotion and stigma

The Neuroscience, Engagement, and Smart Tech (NEST) lab at Neuroethics Canada explored the topics of emotion and stigma in social robots from the perspectives of older adults, people living with dementia, and care partners. This project was co-designed with a Lived Experience Expert Group (LEEG, or “League”). We conducted online workshops where participants had an opportunity to share their thoughts on what a social robot should be able to do, what kind of emotional range it should be capable of displaying, how much emotion it should display, and considerations around using a social robot in various public contexts (Figure 1).

Figure 1 – Results from the SOCRATES workshops.

Participants expressed that they would want a social robot to interact with them and provide companionship. They also suggested that a robot could be a medium to connect with others; for example, a robot could facilitate electronic communication between two people.

Emotional display was something that most participants desired in a social robot, but there were different preferences for how much emotion a robot should display. Participants mentioned that a social robot displaying negative emotions could be stressful for the user, but a social robot displaying only positive emotions could appear artificial. One participant suggested that to get around this issue, a social robot could have a dial to set the level of interactivity that the user desired. Ideas for displays of emotions raised included facial expressions, body movements, or noises. Participants also explained how having a robot that aligned its emotions with the user could facilitate connection between the robot and user.

Participants also discussed considerations around using a robot in front of other people. Some participants voiced concern about attracting negative attention from an audience and feeling judged, while others suggested that a social robot could help to educate and raise awareness about dementia and how technologies can provide support to older adults.

Looking to the future

One key part of the neuroethics field today is co-created research (11), which involves engaging end-users in the creation of interventions meant to support their wellbeing. The results from this study highlight that social robots should have advanced interactive abilities and emotional capabilities to ensure that users can feel connected to these devices. Since older adults have different preferences for emotional range, customizability should be prioritized in the design of future devices. The results for considerations around using a robot in public suggest that social robot marketing may have a significant impact on the way assistive technologies are perceived in the future. Highlighting these devices as support and using them to educate the public about dementia may reduce the stigma around these technologies. These key findings should be incorporated into the design and implementation of future social robots to improve social robot adoption among older adults.

References

1.         Infographic: Canada’s seniors population outlook: Uncharted territory | CIHI [Internet]. [cited 2022 Jun 16]. Available from: https://www.cihi.ca/en/infographic-canadas-seniors-population-outlook-uncharted-territory

2.         Getson C, Nejat G. Socially assistive robots helping older adults through the pandemic and life after COVID-19. Robotics. 2021 Sep;10(3):106.

3.         Hung L, Liu C, Woldum E, Au-Yeung A, Berndt A, Wallsworth C, et al. The benefits of and barriers to using a social robot PARO in care settings: A scoping review. BMC Geriatr. 2019 Aug 23;19(1):232.

4.         Petersen S, Houston S, Qin H, Tague C, Studley J. The utilization of robotic pets in dementia care. Journal of Alzheimer’s Disease. 2017 Jan 1;55(2):569–74.

5.         Robinson H, MacDonald B, Broadbent E. Physiological effects of a companion robot on blood pressure of older people in residential care facility: A pilot study. Australasian Journal on Ageing. 2015;34(1):27–32.

6.         Latikka R, Rubio-Hernández R, Lohan ES, Rantala J, Nieto Fernández F, Laitinen A, et al. Older adults’ loneliness, social isolation, and physical information and communication technology in the era of ambient assisted living: A systematic literature review. J Med Internet Res. 2021 Dec 30;23(12):e28022.

7.         Prescott TJ, Robillard JM. Are friends electric? The benefits and risks of human-robot relationships. iScience. 2021 Jan 22;24(1):101993.

8.         Pu L, Moyle W, Jones C, Todorovic M. The effectiveness of social robots for older adults: A systematic review and meta-analysis of randomized controlled studies. The Gerontologist. 2019 Jan 9;59(1):e37–51.

9.         Koh WQ, Felding SA, Budak KB, Toomey E, Casey D. Barriers and facilitators to the implementation of social robots for older adults and people with dementia: a scoping review. BMC Geriatr. 2021 Jun 9;21:351.

10.       Bradwell HL, Edwards KJ, Winnington R, Thill S, Jones RB. Companion robots for older people: importance of user-centred design demonstrated through observations and focus groups comparing preferences of older people and roboticists in South West England. BMJ Open. 2019 Sep 1;9(9):e032468.

11.       Illes J. Reflecting on the Past and Future of Neuroethics: The Brain on a Pedestal. AJOB Neuroscience. 2023 Mar 31;1–4.

Jaya Kailley is a directed studies student under the supervision of Dr. Julie Robillard in the NEST Lab, and she is pursuing an Integrated Sciences degree in Behavioural Neuroscience and Physiology at the University of British Columbia. She currently supports research projects that aim to include end-users in the process of social robot development. Outside of work, Jaya enjoys playing the piano, drawing, and reading fiction novels.

We are our brains

Dr. Peter B. Reiner speaking at the Peter Wall Institute of Advanced Studies International Roundtable We are our brains, October 2013.

Unintended Consequences of Neuroscientific Explanations

René Magritte, Les Amants (1928)

In Madness and Civilization, French philosopher and activist Michel Foucault wrote that, “from the fifteenth century on, the face of madness has haunted the imagination of Western man.” Foucault argues that the concept of ‘madness’, at least in the 1800s, was characterized as a loss of ‘reason’ – the essential mental faculty which made human beings distinct from other animals. Those who lacked reason (which all rational beings possessed) were perceived as a threat to humanity. Eventually barriers were raised – both literally and figuratively – which separated these ‘irrational’ individuals from the rest of humanity.

Such stigmatizing and social-distancing attitudes towards those living with mental illness are still deeply enmeshed in the social and cultural norms of many industrialized nations. In the 1960s, sociologist Erving Goffman spoke of stigma as a “spoiled identity;” more closer to us Bruce Link and Jo Phelan conceptualize stigma as occurring when “elements of labeling, stereotyping, separation, status loss, and discrimination occur in a power situation that allows them.” A related concept, social distance, is defined as the degree of proximity an individual is comfortable with in relation to an individual who is living with mental illness, particularly as it relates to the perceived dangerousness or unpredictability of that individual. Prevailing views amongst the public see the behaviour of individuals living with mental illness as volitional; at times these individuals might be considered ‘lazy’ (in the case of major depression and the ethical values tied to participation in the workforce), and so, intuitions about mental illness are believed to be justified blaming these individuals, or perhaps even punishing them.

The Depiction of Addiction

Over at The New Republic, Sally Satel, psychiatrist and resident scholar at the American Enterprise Institute, recently reviewed the controversial book Addiction: A Disorder of Choice by psychologist Gene Heyman. Heyman’s thesis is that conventional wisdom about addiction being a ‘disease’, or perhaps a ‘brain disease’, is incorrect. Satel quotes Heyman by saying, “that the idea [of] addiction [as] a disease has been based on a limited view of voluntary behavior.” Accordingly, addiction is not an “irresistible act”, as he claims the term ‘addiction’ implies, and is in fact a “disorder of choice”. [Note: I have not read Heyman’s book, so I will not comment on the book directly]. Indeed, Satel conveys Heyman’s position as one that is in opposition to perspectives from powerful public figures that support the view that “Addiction is a Brain Disease, and it Matters.” Satel agrees with Heyman’s position, namely that if addiction is a disease, it is a disease the person chose for herself.

Satel’s review of Addiction prompted a response from writer Sascha Z. Scolbic and Peter Scolbic (TNR’s executive editor), who argued that Satel was puting up a “straw man” argument. [See Satel’s reply to the Scolbic’s here].

Blogging Canadian Bioethics: Day 3

So many sessions, so little time. It really is a shame. The quality of sessions this year was truly remarkable, so it was difficult to decide on what to see. On Saturday morning I sat down with my breakfast of coffee, muffins and fruit and took a good long look at the program. After Bernard Keating‘s plenary lecture entitled, Squaring the circle: reconciling evidence and equity (which the majority of the audience was plugged into headphones with a direct line to an English translation, as the lecture was presented mainly en français), I made my way over to two fantastic sessions.

The first session by Cynthia Forlini of the Neuroethics Research Unit at the Institut de recherches cliniques de Montreal (IRCM), reported on a qualitative study on stakeholder perspectives on the use of cognitive enhancing drugs, particularly methylphenidate (I do not have the title of the presentation – it was unfortunately absent from the conference program and I lazily didn’t bother to write it down). The presentation was based upon a paper Forlini co-authored with supervisor Eric Racine in press for Neuroethics entitled, Autonomy and Coercion in Academic “Cognitive Enhancement” Using Methylphenidate: Perspectives of Key Stakeholders. Forlini focused on the perspectives of University students using methylphenidate (MPH) to improve alertness and academic performance. She argued that the social context wherein these drugs are taken and obtained is of crucial importance, as many students reported feeling ‘coerced’ to take MPH in these settings to improve their academic performance. Moreover, the impact of social pressures to take cognitive enhancing drugs significantly impact the individual’s autonomy, such that it may cause a “funnel phenomenon” (see both the article and the figure in this post) leading to social acceptance of this type of enhancement despite beliefs in an individuals autonomous choice. During the discussion period, Forlini was hesitant to call for regulation of cognitive enhancers for use in ‘healthy’ populations, in spite of a recent high-profile commentary in Nature by Greely, Sahakian, Harris, et al. who advocate for responsible use and regulation. Forlini stated that she believes it may be too premature to call for regulation on the use of cognitive enhancing drugs in healthy individuals, as more integration of stakeholder perspectives is needed to inform future public health policy discussion.

Up next was Jennifer Bell from UBC. The title of her presentation, Rights, Risks and Smoking: how ‘denormalisation’ mediates patient-provider interactions in primary case settings, was in many ways a response to Bayer (2008) who justified the use of stigma campaigns to “effectively reduce the prevalence of behaivors linked to disease and death” (actually, volume 67 issue 3 of Social Science and Medicine is devoted to the notion of stigma, including a nifty little back-and-forth between Burris and Bayer). Bell and her co-authors argued that the success of smoking denormalisation campaigns can be witnessed in the stigma attached to smoking, and smokers. However, can stigmatising a group ever be justified as Bayer suggests? Bell and her co-authors disagree. In their analysis, they explore some of the ethical and policy implications of denormalisation strategies which construct health care as a privilege that smokers have forgone the ‘right’ to access. Moreover, Bell reminded us that the more severe health consequences of smoking tend to be greater associated with lower socio-economic status (SES) and that smoking denormalisation efforts further marginalise low SES individuals.

During the lunch hour, UBC’s own Michael McDonald was the recipient of the CBS Lifetime Achievement Award. Dr. McDonald has had a distinguished career in the area of research ethics and other areas of applied ethics. He is the founding director of the Centre for Applied Ethics at UBC and continues to significantly contribute to the ethics literature and actively participates in many ongoing projects. A well deserved honour.

Instead of attending more concurrent sessions in the afternoon, I attended a workshop entitled, Qualitative Ethics Research: Evidentiary Needs and Directions. This session was extremely useful, as we outlined various methods and methodological approaches to qualitative ethics research (not to be confused with qualitative research ethics), and discussed amongst the group some of the challenges that people have encountered in their own work. Although worthwhile, I was tempted by another session titled Conceiving the Post-Prozac Self which was occurring at the same time. Oh well. Can’t win ’em all.