Algorithms of Resistance
by Hannah Jacobs
This essay benefitted greatly from feedback provided by scholars in American Studies, African American Studies, and Information Science.
What are algorithms?
According to Merriam-Webster, an algorithm is a procedure for solving a mathematical problem, and the term is commonly used for the set of rules a machine (and especially a computer) follows to achieve a particular goal. Everything we do with a computer is mediated by algorithms. Algorithms determine which search results we receive, which content we see in our social media feeds, which route our GPS follows, which ads are directed at us, and so on. Algorithms accomplish these tasks by following “a set of instructions, rules, and calculations designed to solve problems.” (Benjamin 6) They are not unlike the recipe used to bake a favorite dessert. (Broussard 7)
To deliver internet search results, for example, algorithms use classification (a way of organizing and describing something) and other methods that take search terms, other data knowingly or unknowingly provided through browsers, search terms and data provided by other users, and the mountains of information available online to retrieve a selection of what is determined to be the most relevant search results. (Bowker & Star)
All of this activity, though, is hidden from most people’s view. An algorithm is a “black box” – a “system whose workings are mysterious; we can observe its inputs and outputs, but we cannot tell how one becomes the other.” (Pasquale 3) Think about the description above about how algorithms deliver search results. We might now have some idea about what goes into a search and what comes out of it, but we do not know much about the steps algorithms use to produce search results.
Algorithms of Oppression
According to Safiya Noble and many other scholars and critical thinkers, the ways algorithms are constructed, how they are used, and how they are kept hidden from public view have resulted in the proliferation of what Noble calls “algorithms of oppression.” These are algorithms that are “serving up deleterious information about people, creating and normalizing structural and systemic isolation, or practicing digital redlining, all of which reinforce oppressive social and economic relations.” (Noble 10) Noble’s work focuses on the racialized ways that women in particular are represented in online search results: searches for “black girls” and “black beauty” yield racist, sexist, derogatory material while “white girls” and “beauty” yield stereotypical, no less racist or sexist, ideals of beauty as both white and overtly feminine. What kinds of negative messages do these kinds of search results send to users? How can they negatively impact individuals’ views of themselves and the world around them?
Answers to these questions are just some of the consequences wrought by algorithms of oppression. Cathy O’Neil describes them as “weapons of math destruction.” Algorithms, as Zeynep Tufekci has written, have very real consequences: they are “computational agents who are not alive, but who act with agency in the world.” (Tufekci 207) How can lines of code impact someone in the real world? Here are just a few examples: the difficulty a person of color may experience applying for a loan, which neighborhoods a realtor shows white home buyers and which they show non-white home buyers, whether someone is misidentified as a criminal suspect, or how close someone lives to a grocery store. All of these scenarios have potentially life altering consequences, and they represent the very real oppression that occurs both on and offline every day.
Algorithms of Resistance
Awareness and representation are only the beginning of working against such oppression, however. Recall the definition of algorithms provided by Benjamin: “a set of instructions, rules, and calculations designed to solve problems.” (6) Algorithms and how they are used to construct platforms, perform analyses, and make decisions are created by design. Someone somewhere defined a problem they needed a computer’s help to solve, outlined steps and rules that computers could use to solve the problem, and wrote the code to carry out those steps.
How can we use design theory and ethics to think about how we create and use algorithms? Sasha Costanza-Chock has considered this question from the design perspective in their 2020 book, Design Justice: Community-Led Practices to Build the Worlds We Need. Joy Buolamwini and the Algorithmic Justice League, of which Costanza-Chock is a member, are thinking through and carrying out algorithmic resistance to algorithmic oppression through their focus on exposing and eliminating biased and harmful AI technologies. They work against what Ruha Benjamin has termed “discriminatory design.” We draw from their examples to outline a set of guidelines in use by On the Books for creating and working with algorithms of resistance:
- First, we acknowledge that we live in a society historically built on racist ideologies and practices and while we do not intend to systematically exclude marginalized people, we recognize that any algorithmic platform, analysis, or tool we create or use may still impose harm, whether small or significant. 1
- Despite these challenges, we seek to use algorithms to resist discriminatory policies and to enact positive change by:
- Centering those whose knowledge and experiences directly connect to our work;2 and Enabling community agency and control: 3 We are engaging instructors at the K-12 and college levels with the aim of making them aware of these laws and working with them to identify potential pedagogical uses. Through education and community engagement, we make students, scholars, and the general public aware of our work and our uses of algorithms. In addition to promoting awareness, we seek feedback on how our work with algorithms may be impacting others and how we can address that impact.
- Considering our own identities, assumptions, and privileges: 4 We recognize that the technical part of our team is overwhelmingly white, so we rely on the advice of experts in history and African American studies to inform our work.
- Ensuring transparency in our uses and creation of algorithms: 5 We have posted our documented code on GitHub and are creating step-by-step tutorials and examples.
- Keeping humans in the loop: 6 We used a combination of automated and manual processes to prepare the corpus to the best of our ability. We trained the algorithms we use to identify Jim Crow Laws with training sets identified and refined by scholars in multiple disciplines.
- Prioritizing process over product: 7 Our team spent a year developing a thorough process to reduce as many corpus errors as possible.
The same algorithm can be used for oppression or resistance. How we design and use algorithms, and how they impact those vulnerable to discriminatory policies, determines whether they enact oppression or resistance. The question remains, as Costanza-Chock writes, “What will it take for us to transform the ways that we design technologies (sociotechnical systems) of all kinds, including digital interfaces, applications, platforms, algorithms, hardware, and infrastructure, to help us advance toward liberation?” (“Design Values”)
Sources
Algorithmic Justice League. https://www.ajl.org/.
Benjamin, Ruha. Race After Technology: Abolitionist Tools for the New Jim Code. Cambridge, UK: Polity, 2019.
Bowker, Geoffrey C., and Susan Leigh Star. Sorting things out: classification and its consequences. Cambridge, MA: MIT Press, 1999.
Broussard, Meredith. Artificial Unintelligence: How Computers Misunderstand the World. Cambridge, MA: MIT Press, 2018.
Costanza-Chock, Sasha. Design Justice. Cambridge, MA: MIT Press, 2020. https://design-justice.pubpub.org/
Noble, Safiya Umoja. Algorithms of oppression : How Search Engines Reinforce Racism. New York: NYU Press, 2018.
O’Neil, Cathy. Weapons of Math Destruction: How big data increases inequality and threatens democracy. New York: Crown, 2016.
Pasquale, Frank. The black box society: the secret algorithms that control money and information. Cambridge, MA: Harvard University Press, 2015.
Tufekci, Zeynep. “Algorithmic harms beyond Facebook and Google: Emergent challenges of computational agency symposium essays.” Colorado Technology Law 13, 2015: 203– 18.
1Benjamin, Ruha. “Engineered Inequity.” In Race After Technology: Abolitionist Tools for the New Jim Code. Newark: Polity Press, 2019. Accessed March 5, 2021. ProQuest Ebook Central. 4, 42.; Costanza-Chock, Sasha. “Design Values: Hard-Coding Liberation?” In Design Justice: Community-Led Practices to Build the Worlds We Need. Cambridge, MA: MIT Press, 2020. https://design-justice.pubpub.org/pub/3h2zq86d/release/1.
2Costanza-Chock.
3Costanza-Chock.; Algorithmic Justice League, “Overview.”
4Costanza-Chock.
5Algorithmic Justice League, “Overview.”
6Algorithmic Justice League, “Overview.”; Costanza-Chock.
7Ruha.; Costanza-Chock.