Assessing Chile's Medical Claims Model: An AI Governance Metrics Deep Dive with Mariana Germán
When governments create AI governance policy tools, how are they used in real-world situations? What does the process of assessing a machine learning model used by a government agency look like? In this episode of Privacy on the Ground, you’ll hear all about it from an insider: Mariana Germán, a researcher in the Ethical Algorithms Project at GobLab UAI, the public innovation laboratory at Chile’s Universidad Adolfo Ibáñez’s School of Government. Germán and the team at GobLab helped assess a machine learning model in development for use to help decide medical claims at Chile’s health agency, the Department of Social Security Superintendence or SUSESO. In this full interview recorded in September of 2024, Germán and World Privacy Forum Deputy Director Kate Kaye dig deep into the metrics and measurements used to assess the model and its risks of producing discriminatory decisions, discussing the caveats of the AI governance tools, measures and metrics themselves, and how they were applied. Featured in this episode: Mariana Germán, researcher in the Ethical Algorithms Project at GobLab, the public innovation laboratory at Chile’s Universidad Adolfo Ibáñez’s School of Government WPF’s deputy director and Privacy on the Ground host and producer Kate Kaye This episode of Privacy on the Ground features music by Maciej Sadowski. The Privacy on the Ground intro theme features music by Pangal.
--------
56:03
--------
56:03
Why Rodrigo Moya Changed His Mind about Chile’s AI Governance Tool for Assessing a Medical Insurance Claims AI Model
Inside Chile’s Department of Social Security Superintendence — the country’s social security and medical insurance agency — medical claims processors hold the livelihoods and future health of thousands of people in their hands. They are responsible for deciding whether or not the government should pay wages when workers are on medical leave or cover other expenses such as occupational mental health related costs. Like many government agencies these days, the agency, known by its acronym SUSESO, has begun to use machine learning models to help its limited staff process a high volume of medical claims. The idea is to streamline and in some cases automate certain parts of that claims evaluation process. Use of AI and the tools used to govern and assess these systems have upended traditional government processes. And in Chile, SUSESO project manager Rodrigo Moya is caught in the middle. Moya heads up the Digital Transformation, Innovation and Project Unit in SUSESO’s Technology and Operations Department. He must balance project time and resource constraints with the need to analyze risks and impacts of AI. In this episode of Privacy on the Ground, you’ll hear the story of how Moya and others at SUSESO have used Chile's AI governance tool requiring assessment of AI systems as part of the AI procurement process, and about how Moya has navigated tensions regarding use of automation when it comes to risky government decision making affecting people’s lives. Featured in this episode: Rodrigo Moya, head of the Digital Transformation, Innovation and Project Unit in the Technology and Operations Department at SUSESO Mariana German, researcher in the Ethical Algorithms Project at GobLab, the public innovation laboratory at Chile’s Universidad Adolfo Ibáñez’s School of Government WPF’s deputy director and Privacy on the Ground host and producer Kate Kaye This episode of Privacy on the Ground features music by Maciej Sadowski. The Privacy on the Ground intro theme features music by Pangal.
--------
28:51
--------
28:51
How AI Governance Tools Put Policy into Practice in Canada and Chile
There’s no shortage of principles and policies for governing AI from governments and NGOs around the world. But how do they put those principles and policies into practice? It’s that practical side of AI governance that has been a key focus of our work at World Privacy Forum for more than two years. Rather than look only at government policies, in early 2023 we went layers deeper, looking at the tools that governments and NGOs around the world—from Canada to Chile to Ghana to New Zealand to Singapore—have developed for actually implementing those AI policies. Since then, we have observed actual use of these tools to understand how they govern and measure AI and spot where there’s room for improvement. Key to that work has been talking to people who have actually used those AI governance tools, including people in Canada and Chile. WPF’s forthcoming Privacy on the Ground series—AI Governance Tools on the Ground—features talks with some of those people. In this episode introducing the series, you’ll hear WPF’s founder and executive director, Pam Dixon, along with WPF’s deputy director and Privacy on the Ground host and producer Kate Kaye, discuss what led to this work and how AI Governance Tools have evolved. This episode of Privacy on the Ground features music by Maciej Sadowski. The Privacy on the Ground intro theme features music by Pangal.
--------
21:13
--------
21:13
Emotion Recognition and What Nazanin Andalibi's Research Tells Us about Its Impacts
Emotion recognition is baked into all sorts of software and systems many of us use or experience every day, from video call systems measuring the “mood” at a work meeting, to systems used to gauge distraction at school, or impairment or anger of drivers inside their cars. Despite its increasing proliferation, emotion recognition systems and the data use embedded in them create significant privacy impacts. What is emotion recognition? Would fixing inaccuracy problems in these systems alleviate the potential harms they enable? Should emotion related data be recognized as a sensitive type of information along with health financial and other sensitive data? How might policymakers address potential harms of emotion recognition? Dr. Nazanin Andalibi, Assistant Professor at the University of Michigan School of Information, has a lot to say about all this, and she has the research to back it up. World Privacy Forum Deputy Director Kate Kaye interviewed Dr. Andalibi in June 2024 in Rio de Janeiro, Brazil. This episode of Privacy on the Ground features music by Old Wave. The Privacy on the Ground intro theme features music by Pangal.
--------
39:13
--------
39:13
Te Mihinga Komene on Ensuring Māori Language Data Flourishes in the Generative AI Era
In this episode of World Privacy Forum’s Privacy on the Ground, Māori language expert and educator Te Mihinga Komene shares positive and problematic experiences working with tech companies to help build and correct Māori language translation and learning systems. Komene also discusses extractive data collection practices in AI, and why she hopes her scholarly research will help ensure the Māori language flourishes in the generative AI era. She was interviewed by World Privacy Forum Deputy Director Kate Kaye in June 2024 in Rio de Janeiro at the FAccT conference on fairness, accountability, and transparency in socio-technical systems.
Privacy On The Ground is where privacy meets real life. Discussions about privacy in relation to government policy, legal compliance, or tech can be complicated and inaccessible. But the meaning of privacy and how data use affects us in our real lives is anything but: It is contextual and tangible. That’s what we aim for with Privacy on the Ground. In this podcast, you’ll hear talks and stories that reflect what privacy means for real people and real lives.
Privacy On The Ground is a production of World Privacy Forum, a nonpartisan 501c3 nonprofit public interest research organization. Find us online at www.WorldPrivacyForum.org.
New Zealand