Comparison of an AI Chatbot With a Nurse Hotline in Reducing Anxiety and Depression Levels in the General Population: Pilot Randomized Controlled Trial

Chen Chen; Kok Tai Lam; Ka Man Yip; Hung Kwan So; Terry Yat Sang Lum; Ian Chi Kei Wong; Jason C Yam; Celine Sze Ling Chui; Patrick Ip

doi:10.2196/65785

Comparison of an AI Chatbot With a Nurse Hotline in Reducing Anxiety and Depression Levels in the General Population: Pilot Randomized Controlled Trial

Chen Chen, Kok Tai Lam, Ka Man Yip, Hung Kwan So, Terry Yat Sang Lum, Ian Chi Kei Wong, Jason C Yam, Celine Sze Ling Chui, Patrick Ip

Research output: Contribution to journal › Article › peer-review

Abstract

BACKGROUND: Artificial intelligence (AI) chatbots have been customized to deliver on-demand support for people with mental health problems. However, the effectiveness of AI chatbots in tackling mental health problems among the general public in Hong Kong remains unclear.

OBJECTIVE: This study aimed to develop a local AI chatbot and compare the effectiveness of the AI chatbot with a conventional nurse hotline in reducing the level of anxiety and depression among individuals in Hong Kong.

METHODS: This study was a pilot randomized controlled trial conducted from October 2022 to March 2023, involving 124 participants allocated randomly (1:1 ratio) into the AI chatbot and nurse hotline groups. Among these, 62 participants in the AI chatbot group and 41 in the nurse hotline group completed both the pre- and postquestionnaires, including the GAD-7 (Generalized Anxiety Disorder Scale-7), PHQ-9 (Patient Health Questionnaire-9), and satisfaction questionnaire. Comparisons were conducted using independent and paired sample t tests (2-tailed) and the χ2 test to analyze changes in anxiety and depression levels.

RESULTS: Compared to the mean baseline score of 5.13 (SD 4.623), the mean postdepression score in the chatbot group was 3.68 (SD 4.397), which was significantly lower (P=.008). Similarly, a reduced anxiety score was also observed after the chatbot test (pre vs post: mean 4.74, SD 4.742 vs mean 3.4, SD 3.748; P=.005), respectively. No significant differences were found in the pre-post scores for either depression (P=.38) or anxiety (P=.19). No statistically significant difference was observed in service satisfaction between the two platforms (P=.32).

CONCLUSIONS: The AI chatbot was comparable to the traditional nurse hotline in alleviating participants' anxiety and depression after responding to inquiries. Moreover, the AI chatbot has shown potential in alleviating short-term anxiety and depression compared to the nurse hotline. While the AI chatbot presents a promising solution for offering accessible strategies to the public, more extensive randomized controlled studies are necessary to further validate its effectiveness.

Original language	English
Article number	e65785
Number of pages	10
Journal	JMIR Human Factors
Volume	12
Early online date	6 Mar 2025
DOIs	https://doi.org/10.2196/65785
Publication status	Published - 6 Mar 2025

Bibliographical note

Copyright © Chen Chen, Kok Tai Lam, Ka Man Yip, Hung Kwan So, Terry Yat Sang Lum, Ian Chi Kei Wong, Jason C Yam, Celine Sze Ling Chui, Patrick Ip. Originally published in JMIR Human Factors (https://humanfactors.jmir.org), 6.3.2025. This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Human Factors, is properly cited.

Keywords

AI chatbot
anxiety
depression
effectiveness
artificial intelligence

Access to Document

10.2196/65785Licence: CC BY 4.0

Comparison of an AI Chatbot With a Nurse Hotline in Reducing Anxiety and Depression Levels in the General Population
Copyright © Chen Chen, Kok Tai Lam, Ka Man Yip, Hung Kwan So, Terry Yat Sang Lum, Ian Chi Kei Wong, Jason C Yam, Celine Sze Ling Chui, Patrick Ip. Originally published in JMIR Human Factors (https://humanfactors.jmir.org), 6.3.2025. This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Human Factors, is properly cited.
Final published version, 431 KBLicence: CC BY 4.0

Cite this

@article{462ac3018ad44dd8a60d1cf5e9acbded,

title = "Comparison of an AI Chatbot With a Nurse Hotline in Reducing Anxiety and Depression Levels in the General Population: Pilot Randomized Controlled Trial",

abstract = "BACKGROUND: Artificial intelligence (AI) chatbots have been customized to deliver on-demand support for people with mental health problems. However, the effectiveness of AI chatbots in tackling mental health problems among the general public in Hong Kong remains unclear.OBJECTIVE: This study aimed to develop a local AI chatbot and compare the effectiveness of the AI chatbot with a conventional nurse hotline in reducing the level of anxiety and depression among individuals in Hong Kong.METHODS: This study was a pilot randomized controlled trial conducted from October 2022 to March 2023, involving 124 participants allocated randomly (1:1 ratio) into the AI chatbot and nurse hotline groups. Among these, 62 participants in the AI chatbot group and 41 in the nurse hotline group completed both the pre- and postquestionnaires, including the GAD-7 (Generalized Anxiety Disorder Scale-7), PHQ-9 (Patient Health Questionnaire-9), and satisfaction questionnaire. Comparisons were conducted using independent and paired sample t tests (2-tailed) and the χ2 test to analyze changes in anxiety and depression levels.RESULTS: Compared to the mean baseline score of 5.13 (SD 4.623), the mean postdepression score in the chatbot group was 3.68 (SD 4.397), which was significantly lower (P=.008). Similarly, a reduced anxiety score was also observed after the chatbot test (pre vs post: mean 4.74, SD 4.742 vs mean 3.4, SD 3.748; P=.005), respectively. No significant differences were found in the pre-post scores for either depression (P=.38) or anxiety (P=.19). No statistically significant difference was observed in service satisfaction between the two platforms (P=.32).CONCLUSIONS: The AI chatbot was comparable to the traditional nurse hotline in alleviating participants' anxiety and depression after responding to inquiries. Moreover, the AI chatbot has shown potential in alleviating short-term anxiety and depression compared to the nurse hotline. While the AI chatbot presents a promising solution for offering accessible strategies to the public, more extensive randomized controlled studies are necessary to further validate its effectiveness.",

keywords = "AI chatbot, anxiety, depression, effectiveness, artificial intelligence",

author = "Chen Chen and Lam, {Kok Tai} and Yip, {Ka Man} and So, {Hung Kwan} and Lum, {Terry Yat Sang} and Wong, {Ian Chi Kei} and Yam, {Jason C} and Chui, {Celine Sze Ling} and Patrick Ip",

note = "Copyright {\textcopyright} Chen Chen, Kok Tai Lam, Ka Man Yip, Hung Kwan So, Terry Yat Sang Lum, Ian Chi Kei Wong, Jason C Yam, Celine Sze Ling Chui, Patrick Ip. Originally published in JMIR Human Factors (https://humanfactors.jmir.org), 6.3.2025. This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Human Factors, is properly cited.",

year = "2025",

month = mar,

day = "6",

doi = "10.2196/65785",

language = "English",

volume = "12",

journal = "JMIR Human Factors",

issn = "2292-9495",

publisher = "JMIR Publications Inc.",

}

TY - JOUR

T1 - Comparison of an AI Chatbot With a Nurse Hotline in Reducing Anxiety and Depression Levels in the General Population: Pilot Randomized Controlled Trial

AU - Chen, Chen

AU - Lam, Kok Tai

AU - Yip, Ka Man

AU - So, Hung Kwan

AU - Lum, Terry Yat Sang

AU - Wong, Ian Chi Kei

AU - Yam, Jason C

AU - Chui, Celine Sze Ling

AU - Ip, Patrick

N1 - Copyright © Chen Chen, Kok Tai Lam, Ka Man Yip, Hung Kwan So, Terry Yat Sang Lum, Ian Chi Kei Wong, Jason C Yam, Celine Sze Ling Chui, Patrick Ip. Originally published in JMIR Human Factors (https://humanfactors.jmir.org), 6.3.2025. This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Human Factors, is properly cited.

PY - 2025/3/6

Y1 - 2025/3/6

N2 - BACKGROUND: Artificial intelligence (AI) chatbots have been customized to deliver on-demand support for people with mental health problems. However, the effectiveness of AI chatbots in tackling mental health problems among the general public in Hong Kong remains unclear.OBJECTIVE: This study aimed to develop a local AI chatbot and compare the effectiveness of the AI chatbot with a conventional nurse hotline in reducing the level of anxiety and depression among individuals in Hong Kong.METHODS: This study was a pilot randomized controlled trial conducted from October 2022 to March 2023, involving 124 participants allocated randomly (1:1 ratio) into the AI chatbot and nurse hotline groups. Among these, 62 participants in the AI chatbot group and 41 in the nurse hotline group completed both the pre- and postquestionnaires, including the GAD-7 (Generalized Anxiety Disorder Scale-7), PHQ-9 (Patient Health Questionnaire-9), and satisfaction questionnaire. Comparisons were conducted using independent and paired sample t tests (2-tailed) and the χ2 test to analyze changes in anxiety and depression levels.RESULTS: Compared to the mean baseline score of 5.13 (SD 4.623), the mean postdepression score in the chatbot group was 3.68 (SD 4.397), which was significantly lower (P=.008). Similarly, a reduced anxiety score was also observed after the chatbot test (pre vs post: mean 4.74, SD 4.742 vs mean 3.4, SD 3.748; P=.005), respectively. No significant differences were found in the pre-post scores for either depression (P=.38) or anxiety (P=.19). No statistically significant difference was observed in service satisfaction between the two platforms (P=.32).CONCLUSIONS: The AI chatbot was comparable to the traditional nurse hotline in alleviating participants' anxiety and depression after responding to inquiries. Moreover, the AI chatbot has shown potential in alleviating short-term anxiety and depression compared to the nurse hotline. While the AI chatbot presents a promising solution for offering accessible strategies to the public, more extensive randomized controlled studies are necessary to further validate its effectiveness.

AB - BACKGROUND: Artificial intelligence (AI) chatbots have been customized to deliver on-demand support for people with mental health problems. However, the effectiveness of AI chatbots in tackling mental health problems among the general public in Hong Kong remains unclear.OBJECTIVE: This study aimed to develop a local AI chatbot and compare the effectiveness of the AI chatbot with a conventional nurse hotline in reducing the level of anxiety and depression among individuals in Hong Kong.METHODS: This study was a pilot randomized controlled trial conducted from October 2022 to March 2023, involving 124 participants allocated randomly (1:1 ratio) into the AI chatbot and nurse hotline groups. Among these, 62 participants in the AI chatbot group and 41 in the nurse hotline group completed both the pre- and postquestionnaires, including the GAD-7 (Generalized Anxiety Disorder Scale-7), PHQ-9 (Patient Health Questionnaire-9), and satisfaction questionnaire. Comparisons were conducted using independent and paired sample t tests (2-tailed) and the χ2 test to analyze changes in anxiety and depression levels.RESULTS: Compared to the mean baseline score of 5.13 (SD 4.623), the mean postdepression score in the chatbot group was 3.68 (SD 4.397), which was significantly lower (P=.008). Similarly, a reduced anxiety score was also observed after the chatbot test (pre vs post: mean 4.74, SD 4.742 vs mean 3.4, SD 3.748; P=.005), respectively. No significant differences were found in the pre-post scores for either depression (P=.38) or anxiety (P=.19). No statistically significant difference was observed in service satisfaction between the two platforms (P=.32).CONCLUSIONS: The AI chatbot was comparable to the traditional nurse hotline in alleviating participants' anxiety and depression after responding to inquiries. Moreover, the AI chatbot has shown potential in alleviating short-term anxiety and depression compared to the nurse hotline. While the AI chatbot presents a promising solution for offering accessible strategies to the public, more extensive randomized controlled studies are necessary to further validate its effectiveness.

KW - AI chatbot

KW - anxiety

KW - depression

KW - effectiveness

KW - artificial intelligence

UR - https://humanfactors.jmir.org/2025/1/e65785

UR - http://www.scopus.com/inward/record.url?scp=105001046146&partnerID=8YFLogxK

U2 - 10.2196/65785

DO - 10.2196/65785

M3 - Article

C2 - 40048637

SN - 2292-9495

VL - 12

JO - JMIR Human Factors

JF - JMIR Human Factors

M1 - e65785

ER -

Comparison of an AI Chatbot With a Nurse Hotline in Reducing Anxiety and Depression Levels in the General Population: Pilot Randomized Controlled Trial

Abstract

Bibliographical note

Keywords

Access to Document

Other files and links

Fingerprint

Cite this