What should count as success for an AI system?
Connect computation, datasets, labour and social consequences. Compare a technical evaluation with the people and purposes it leaves out.
Ways into the question
Begin with Alan Turing
What does a proposed test of machine intelligence actually test?
Begin with Fei-Fei Li
What does a vision benchmark leave outside its score?
Begin with Timnit Gebru
What should a dataset user know before training a model?
Begin with Ruha Benjamin
How can an apparently neutral default distribute unequal consequences?
Begin with Kate Crawford
Which costs disappear when AI is described only as software?
Thinkers to read
Alan Turing
Work on computability, machine intelligence and morphogenesis. Begin by distinguishing a mathematical model, an operational test and a biological mechanism.
Fei-Fei Li
Computer vision and human-centred approaches to AI. Study the relationship among a dataset, a benchmark and the people affected by a system.
Timnit Gebru
Research on dataset documentation, evaluation and harms in machine learning. Begin with how data was collected and which uses its documentation supports.
Ruha Benjamin
Social analysis of race and technological design. Examine how a system’s categories, defaults and institutional uses can reproduce inequality.
Kate Crawford
Research on AI’s material, labour and political conditions. Trace a system beyond its interface to extraction, human work and data infrastructures.