Governing Algorithms and Algorithmic Governance
Course Numbers: INST878D/CMSC839C
This cross-cutting interdisciplinary course, taught jointly between the College of Information Studies and the Department of Computer Science, investigates the role that algorithms and automated decision-making systems play in markets, societies, and policymaking and the technologies we have to address their unintended consequences. The course connects policy and computational conceptualizations of transparency, security, fairness, privacy, manipulation, and accountability through a series of case studies and burning debates. Students will collaborate cross-disciplinary and be encouraged to work through difficult trade-offs to reach consensus. By discussing recent applications of algorithms for social and consumer sorting, and the moderation and generation of content, all in the context of the technologies designed to address algorithmic harms, students will engage with the pressing challenges and opportunities in the governance of and by algorithms.
This course counts as "qualifying" for CS PhD students within the AI area.
Instructors
Ido Sivan-Sevilla
Assistant Professor
College of Information Studies
Email: sevilla (at) umd (dot) edu
Gabriel Kaptchuk
Assistant Professor
Department of Computer Science
Email: kaptchuk (at) umd (dot) edu
Kanav Gupta (Teaching Assistant)
Department of Computer Science
Email: kanav (at) umd (dot) edu
Course Information
Syllabus and Grading: Available here.
Meeting Times: Tuesdays, 11:00am–1:45pm
Course Structure: 1 Hour lecture, 1 hour interative activity or discussion, 1 hour project work time
Course website: ga-ag.com
Course Project: Available Here
Class Slides
[Class 1] [Class 2] [Class 3] [Class 4] [Class 5] [Class 6] [Class 7] [Class 8] [Class 9] [Class 10] [Class 11] [Class 12]
Reading List
Class 1
What does introducing a technology do?
- Neil Postman. Five Things We Need to Know About Technological Change"
The politics of artifacts
- Langdon Winner. Do artifacts have politics? [Start on Page 123 with “in what follows” through stars on page 134].
“Unintended” consequences:
- Lina Dencik, Arne Hintz, and Joanna Redden. Data Justice Book, Introduction (full intro is available within the amazon sample)
- Kimberly Do, Rock Yuren Pang, Jiachen Jiang, and Katharina Reinecke. “That’s important, but...”: How Computer Science Researchers Anticipate Unintended Consequences of Their Research Innovations [Read sections 2,6 and 7]
- Fabio Urbina, Filippa Lentzos, Cédric Invernizzi and Sean Ekins. Dual use of artifcial-intelligence-powered drug discovery
Class 2
How does regulation work?"
- Andrew Sellers. An introduction to United States Law for Technologists
- David Levi-Faur, Yael Kariv-Teitelbaum, and Rotem Medzini. Regulatory Governance: History, Theories, Strategies, and Challenges
Technology and regulation from an STS perspective
- Magnus Eriksson. Theorizing Technology
On Power
- Catherine D'Ignazio and Lauren Klein. Data Feminism, The Power Chapter. [Read “Power and the Matrix of Domination” and “Data Science by Whom” through this paragraph (right after the picture of Joy Buolamwini)]
Optional Additional Readings
- Nick Seaver. Computing taste, Chapter 4: Space is the place (Ask instructors for a copy)
- A readable manifest on power.
- Cary Coglianese. Regulating New Tech: Problems, Pathways, and People
Class 3
What is machine learning (socially)
- Emily M. Bender, Timnit Gebru, Angelina McMillan-Major, and Margaret Mitchell. On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?
- Alexander Campolo Katia Schwerzmann.From Rules to Examples: Machine Learning Type of Authority [Start at "Rules that are impersonal, systematic, explicitly...", Skip section titled “Scaling”]
- Suresh Venkatasubramanian. Repairing the Algorithmic Lens
Case Study
- Julia Angwin, Jeff Larson, Surya Mattu and Lauren Kirchner, ProPublica. Machine Bais
Class 4
What is implicit in the reality machine learning is trying to create?
- Ruha Benjamin. Race After Technology, Chapter 1: Engineering Inequity [Read through "Raising Robots" header]
- Timnit Gebru and Émile P. Torres. The TESCREAL bundle: Eugenics and the promise of utopia through artificial general intelligence [Read Sections 4,5,6]
Hazards in a AI world
- Alan R. Wagner, Jason Borenstein, and Ayanna Howard. Overtrust in the Robotic Age
- Madeleine Clare Elish. Moral Crumple Zones: Cautionary Tales in Human-Robot Interaction [Start with last paragraph of introduction (starting with “Following the discussion of these case,” then jump to Robots on the Road section and read until the end]
- David Gray Widder, Sarah West, and Meredith Whittaker. Open (For Business): Big Tech, Concentrated Power, and the Political Economy of Open AI [Sections 3 and 6]
Secret labor of machine learning
- James Bridle. So, Amazon’s ‘AI-powered’ cashier-free shops use a lot of... humans. Here’s why that shouldn’t surprise you
- Billy Perrigo. Exclusive: OpenAI Used Kenyan Workers on Less Than $2 Per Hour to Make ChatGPT Less Toxic
Class 5
- Sorelle A. Friedler, Carlos Scheidegger, AND Suresh Venkatasubramanian. The (Im)possibility of Fairness: Different Value Systems Require Different Mechanisms for Fair Decision Making
- Ben Green. Escaping the Impossibility of Fairness: From Formal to Substantive Algorithmic Fairness [Skip Section 3]
- Angelina Wang, Sayash Kapoor, Solon Barocas, Arvind Narayanan. Against Predictive Optimization: On the Legitimacy of Decision-Making Algorithms that Optimize Predictive Accuracy (Explore)
- Deborah Hellman. What is discrimination, when is it wrong and why? [Watch until end of her presentation, around 50:00]
- Social Inequalities Explained in a $100 Race
Class 6
Ongoing regulatory efforts
- Blueprint for the AI Bill of Rights [Read Overview at this link. Pick one area you are interested in and read the associated “From Principles to Practice” section]
- SB-1047 Safe and Secure Innovation for Frontier Artificial Intelligence Models Act
- EU AI Act [Read summary at the link. In you are interested, you can poke around the full act]
- The Artificial Intelligence Civil Rights Act (introduced 9/24/24). [You may also be interested in the press announcement associated with the itroduction]
The limits of regulatory approaches
- Mike Annany and Kate Crawford, Seeing Without Knowing: Limitations of the Transparency Ideal and its Application to Algorithmic Accountability
- Cary Coglianese, Regulating Machine Learning: The Challenge of Heterogeneity [Read sections 4,5, and 6].
- Kevin Wei, Carson Ezell, Nick Gabrieli, and Chinmay Deshpande. How Do AI Companies "Fine-Tune" Policy? Examining Regulatory Capture in AI Governance [Read sections 5 and 6]
Optional Additional Readings
- NYC Carceral algorithms Law
- NIST AI Risk Management Framework and the associated Generative AI addendum
- Maranke Wieringa. What to account for when accounting for algorithms: a systematic literature review on algorithmic accountability
- Gabriel Nicholas, Grounding AI Policy: Towards Researcher Access to AI Usage Data
- A. Feder Cooper, Karen Levy, and Christopher De Sa. Regulating Accuracy-Efficiency Trade-Offs in Distributed Machine Learning Systems
Class 7
- Daniel J. Solove, Understanding Privacy [Read chapter 1]
- Helen Nissenbaum, A Contextual Approach to Privacy Online
- Kent Bye, Primer on the Contextual Integrity Theory of Privacy with Philosopher Helen Nissenbaum [Listen]
- Woody Hertzog, What is Privacy? That's the Wrong Question [You may need to be on UMD network or use the Reload Button]
Optional Additional Readings
- Danielle Citron and Daniel J. Solove, Privacy Harms
- Ari Waldman, Privacy as Trust
- Neil M. Richards, Intellectual Privacy
Class 8
Balancing Privacy and Data Use in the Census
- U.S. Census Bureau, Disclosure Avoidance for the 2020 Census: An Introduction [Read section 1, skipping blue page]
- Kobbi Nissim, Thomas Steinke, Alexandra Wood, Mark Bun, Marco Gaboardi, David R. O’Brien, and Salil Vadhan. Differential Privacy: A Primer for a Non-technical Audience [Read Section 2]
The resulting controversy
- danah boyd and Jayshree Sarathy, Differential Perspectives: Epistemic Disconnects Surrounding the US Census Bureau’s Use of Differential Privacy [Read section 3]
- Steven Ruggles and David Van Riper, The Role of Chance in the Census Bureau Database Reconstruction Experiment [Skip "Small Blocks and Swapping"]
- Aloni Cohen, Moon Duchin, JN Matthews, and Bhushan Suwal, Private Numbers in Public Policy: Census, Differential Privacy, and Redistricting [Read sections 2.1 and 2.2]
Class 9
Introduction to PETs
- The United Nations Guide on privacy-enhancing technologies for official statistics [Read sections 1.2, 2.1, 2.6, 2.7]
- Matthew D. Green, Zero Knowledge Proofs: An illustrated primer
Policy-relevant uses of PETs
- Mayank Varia, Cryptographically Secure Data Analysis for Social Good
- David Archer, Amy O’Hara, Rawane Issa, and Stephanie Straus, Sharing Sensitive Department of Education Data Across Organizational Boundaries Using Secure Multiparty Computation. (Tech Report and Policy Report).
- The EU Digital Identity Wallet: A Beginner's Guide
- Carsten Baum, Olivier Blazy, Jaap-Henk Hoepman, Anja Lehmann, Anna Lysyanskaya, René Mayrhofer, Hart Montgomery, Ngoc Khanh Nguyen, abhi shelat, Daniel Slamanig, and Søren Eller Thomsen, Cryptographers’ Feedback on the EU Digital Identity’s ARF (link will cause a download) [Read Sections 1.1, 2, 3]
- Kenneth A. Bamberger, Ran Canetti, Shafi Goldwasser, Rebecca Wexler, and Evan J. Zimmerman. Verification Dilemmas in Law and the Promise of Zero-Knowledge Proofs [Read Sections 2B, 2C, and 2D].
Class 10
GDPR
- https://gdprxiv.org/ [Explore]
- Chen Sun, Evan Jacobs, Daniel Lehmann, Andrew Crouse, and Supreeth Shastri. GDPRxiv: Establishing the State of the Art in GDPR Enforcement
- Ido Sivan-Sevilla. Varieties of enforcement strategies post-GDPR: a fuzzy-set qualitative comparative analysis (fsQCA) across data protection authorities
- Inbar Mizarhi-Borohovich, Abraham Newman, and Ido Sivan-Sevilla. The civic transformation of data privacy implementation in Europe.
Class 11
Assisting & Improving The Regulatory Process:
- Daniel Esty and Reece Rushing. The promise of data-driven policymaking
- Susan Athey. Data-driven policymaking and its limitations
- Helen Margetts and Cosmina Dorobantu. Computational Social Science for Public Policy, Chapter 1 [Read pp. 3-18]
Critique:
- Philip Ball. The Politics of Science-based policymaking
- Helen Margetts & Patrick Dunleavy. The political economy of digital government: How Silicon Valley firms drove conversion to data science and artificial intelligence in public management
Class 12
Note that the response question is different this week:
For each case, we are asking you to think like a reviewer. You don’t have to answer all of the questions below, as different questions may be more relevant to different cases. Instead, we hope that for each case you can holistically respond to the proposal with whatever thoughts you think are most important to apprising the proposal. Some example questions you might want to consider:
- Where is the policy cycle does this proposal live? Does the proposal seem like a reasonable place to intervene in the policy cycle?
- Do you think the proposed approach/intervention makes sense?
- Do you think the technological approach is sound?
- What are the benefits and risks to the approach, and do the benefits outweigh the risks?
- What are the assumptions on which the idea rests? Do these assumptions seem valid?
Cases:
- World Economic Form. RegTech - Rationale & Cases [Read 2 and 3]
- Faiz Surani, Mirac Suzgun, Vyoma Raman, Christopher D. Manning, Peter Henderson, and Daniel E. Ho. AI for Scaling Legal Reform: Mapping and Redacting Racial Covenants in Santa Clara County. [Read 1, 3, 4, skim 6 and 7]
- Neal Jean, Marshall Burke , Michael Xie, W. Matthew Davis, David B. Lobell, and Stefano Ermon. Combining satellite imagery and machine learning to predict poverty
- Paul Ryan, Martin Crane, and Rob Brennan. Design Challenges for GDPR RegTech
A critical vision:
- Ryan Calo and Danielle Keats Citron. The Automated Administrative State: A Crisis of Legitimacy [Read introduction (until the break on page 805) and section IV (starting on page 835)]
Class 13
No reading