DebunkBot

[Gryffin]

Professor at Cornell, Gordon Pennycock, conducted a study where participants chatted with a custom ChatGPT bot, which tried to debunk their conspiracy beliefs. You can read about it at the site https://debunkbot.com.
I heard about it in an interview with the Question Everything podcast. Great interview and interesting results. Long story short, within three messages, a bot can sway a conspiracy believer (using facts) 25% of the time.

 

[Mendel]

Pennycook, not -cock.
We have a thread on a preprint at https://www.metabunk.org/threads/can-ai-chatgpt-etc-help-people-out-of-the-rabbit-hole.13433/
The paper is by Costello, Pennycook, and Rand, and they're also listed at https://debunkbot.com/about .

 

[Gryffin]

Pennycook, not -cock.
We have a thread on a preprint at https://www.metabunk.org/threads/can-ai-chatgpt-etc-help-people-out-of-the-rabbit-hole.13433/
The paper is by Costello, Pennycook, and Rand, and they're also listed at https://debunkbot.com/about .

Ah, thank you

 

[Ann K]

Long story short, within three messages, a bot can sway a conspiracy believer (using facts) 25% of the time.

Encouraging, but...
Does that mean that its comparatively good results indicate that the subjects trust that the answers are accurate from the chat bot, more than they would trust the answers from a statistician or an expert that spoke with them? If so, it might only take a short time for them to "relapse" into their previous beliefs once an AI source (or contact with their fellow believers) gives them different responses.

 

[Gryffin]

Encouraging, but...
Does that mean that its comparatively good results indicate that the subjects trust that the answers are accurate from the chat bot, more than they would trust the answers from a statistician or an expert that spoke with them? If so, it might only take a short time for them to "relapse" into their previous beliefs once an AI source (or contact with their fellow believers) gives them different responses.

Surprisingly, the interview actually addresses these. 2 months later, the people who changed their minds remain changed.

 

[Montauk]

Iirc there was another university that did a similar experiment. I can't for the life of me remember it now unfortunately. It might've been Columbia.