How “Adversarial Collaboration” Can Improve Research When Scientists Clash
I hope you’ll watch and share this fresh and fascinating discussion of a project hosted at the University of Pennsylvania aimed at fostering “adversarial collaboration” when researchers - as just one example - clash in the literature over data that could reveal why humans tend to hold fast to certain beliefs and when and how they update them. I can’t imagine a more important question these days. My guests were project co-director Cory J. Clark and Gordon Pennycook, an associate professor of psychology at Cornell who is involved with several such efforts. You can explore some of the resulting papers and other background below.Here’s a description (full story in Penn Today):Led by Cory Clark, a behavioral scientist and visiting scholar in the Department of Psychology, and in partnership with Penn Integrates Knowledge University Professor Philip Tetlock, the Adversarial Collaboration Project encourages scientists with competing perspectives to work together to design research that can adjudicate their dispute and test where the truth lies. Clark’s team is currently running 10 projects with several dozen researchers from some 30 institutions worldwide and recently published on this work in the Journal of Applied Research in Memory and Cognition.You can also watch the conversation and share this Sustain What episode on Facebook, LinkedIn, my X account, Substack Live or on YouTube.We talked about a wide array of insights and issues, including when I asserted that the news media and social media tend to cut against any effort at adversarial collaboration. I used the example of content in The Atlantic on the research camps warring over the impact of mobile phones on children and teens (an issue I’ve tweeted about).The processes used by the project team, in some cases, have included a mediator (Clark has filled that role sometimes). I noted how much time and effort is involved in what is a very bespoke effort and asked whether artificial intelligence could help mediate between clashing teams’ views of relevant date. Can AI mediate research disputes?That hasn’t been explored much, Clark and Pennycook said, but there has been work showing that AI can durably nudge the beliefs of study participants. Here’s the editor’s summary of one such study (Pennycook is a co-author): Durably reducing conspiracy beliefs through dialogues with AI (Science Magazine, 2024):Human participants described a conspiracy theory that they subscribed to, and the AI then engaged in persuasive arguments with them that refuted their beliefs with evidence. The AI chatbot’s ability to sustain tailored counterarguments and personalized in-depth conversations reduced their beliefs in conspiracies for months, challenging research suggesting that such beliefs are impervious to change. This intervention illustrates how deploying AI may mitigate conflicts and serve society. Clark and Pennycook noted that research tends to show this is NOT about the AI bot, but about getting appropriate evidence to people in ways they can ingestHere’s some related research to explore: Is Overconfidence a Trait? An Adversarial Collaboration (pre-print) - Jabin Binnendyk, Sophia Li, Thomas Costello, Randall Hale, Don A. Moore, and Gordon PennycookAn Adversarial Collaboration on the Rigidity-of-the-Right, Rigidity-of-Extremes, or Symmetry: The Answer Depends on the Question (pre-print) - Shauna Marie Bowes, Cory J Clark, Lucian Gideon Conway III, Thomas H. Costello, Danny Osborne, Phil Tetlock, and Jan-Willem van ProoijenOn the Efficacy of Accuracy Prompts Across Partisan Lines: An Adversarial Collaboration - Cameron Martel, Steve Rathje, Cory J. Clark, Gordon Pennycook, Jay J. Van Bavel, David G. Rand, and Sander van der LindenCrisis counseling for scientists clashing over big questionsI noted there’s a lot of synchronicity with a longstanding effort in Earth science - the U.S. Geological Survey’s John Wesley Powell Center for Analysis and Synthesis - essentially a crisis counseling center for intradisciplinary disputes. I held a conversation long ago about a mediation related to earthquake hazard analysis with two seismologists and the project director Jill Baron.The center is in the Rockies and one method for overcoming rancor is taking hikes that make everyone so short of breath it’s harder to argue. Read this EOS commentary to learn more:I’m close to an important threshold for paid supporters and hope those who can afford it can chip in $50 to help me justify the time this takes to do this webcast and newsletter.Thank you Aron Roberts, Kim M., Jeff Jolley, Zvi Leve, Lauren Chua, and many others for tuning into my live video! Join me for my next live video in the app. This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit revkin.substack.com/subscribe
New Zealand