A Syllabus on Machine Testimony

Several readers wrote, after the essay on opaque models, to ask what one might read in order to think more carefully about the epistemology of machine-mediated knowledge. This is the list I would have given a graduate student of mine, in the order I would have given it, with a sentence on what each item is for. The list is short on purpose. The point of a syllabus is to be finished.

I. The Older Problem

To understand what is novel about machine testimony, one needs first to understand testimony as the philosophers worked it out before any machine spoke. The two essential texts here are C. A. J. Coady's Testimony: A Philosophical Study (1992) and John Hardwig's earlier paper "Epistemic Dependence" (1985).[^1] Coady's book is the longer and more careful of the two; Hardwig's paper is the better introduction, and the more provocative. Read Hardwig first. He argues, with examples drawn mostly from collaborative science, that the individual investigator is far more dependent on the testimony of other investigators than the standard philosophical accounts of knowledge are willing to admit. Once one accepts the dependence, the question of what makes testimony trustworthy becomes urgent. Coady's book is the urgent answer, worked out over four hundred pages, with the rigour the subject requires.

A reader who finishes both texts will, I hope, have arrived at a particular conviction: that the trustworthiness of testimony does not rest on the verifiability of any individual claim, but on the institutional context in which the testifier is embedded. This conviction is the foundation of everything that follows.

II. The History of Mechanical Objectivity

If testimony depends on institutional context, then we need to know how the contemporary institutions of expert knowing were built — and they were built, in large part, around a specific ideal of mechanical objectivity that the modern computer inherits and complicates. Lorraine Daston and Peter Galison's Objectivity (2007) is the definitive history.[^2] It is also long, and the early chapters on Enlightenment-era natural-historical illustration can be skipped on a first reading. What matters for our purposes is the third chapter, on the rise of mechanical objectivity in the late nineteenth century, and the seventh, on what they call trained judgement. The argument is that scientific objectivity is not a single thing but a sequence of changing virtues, and that we are now living through a transition whose contours Daston and Galison did not anticipate but whose grammar they have given us.

The reason this matters: the operators of frontier-model laboratories often present their systems as the next chapter in mechanical objectivity, as if the machine's freedom from human bias were a straightforward improvement on the older arrangements. Daston and Galison's history makes clear that every previous chapter of mechanical objectivity has had unanticipated costs, that the costs have been visible only in retrospect, and that we should expect this chapter to be no different.

The point of the syllabus is not to settle the question. It is to make sure that the question is being asked by readers who know what has already been thought.

III. The Information-Ethical Frame

The work that has taken the question of machine knowing most seriously, in continental and analytic philosophy alike, is Luciano Floridi's. The single most important book is The Ethics of Information (2013), which I have reviewed elsewhere in these pages.[^3] For the present syllabus the relevant chapters are the introduction, the chapter on the moral standing of informational entities, and chapter eleven on informational agents. The Fourth Revolution (2014) is a less rigorous but more readable book, and a reasonable entry point for readers who want a tour of the territory before committing to the longer work. Floridi's Ethics of Artificial Intelligence (2023) is the most recent statement of the position. None of these books will tell the reader what to think about machine testimony, but they will furnish the conceptual apparatus a serious answer requires.

IV. The Empirical Case

Three pieces from inside the machine-learning community are essential. The first is Jenna Burrell's "How the machine 'thinks'" (2016), which distinguishes three kinds of opacity in machine-learning systems and remains the clearest analytic frame I know.[^4] The second is the Stochastic Parrots paper by Emily Bender, Timnit Gebru, Angelina McMillan-Major, and Margaret Mitchell (2021), which made the now-familiar argument that large language models perform an imitation of understanding without the institutional grounding that genuine understanding requires.[^5] The paper is short and the argument is plain; the reader should make her own assessment of which of its claims have aged well.

The third is anything one of one's choice from the interpretability literature of the last three years. I will not recommend a specific paper because the literature is moving too quickly for any recommendation to remain current, but the relevant thing to look for is a paper that is honest about what current methods can and cannot tell us about model internals. The honest papers are easy to find; they are the ones whose conclusions are modest.

V. A Final, Optional Item

For readers who reach the end of the syllabus and want one more thing, I would propose Norbert Wiener's The Human Use of Human Beings (1954). It is older than any of the other items on this list. It was written before the computers it describes existed at the scales it imagines. Its argument — that a society organised around machine intelligence has special obligations to the human beings whose labour and judgement that intelligence displaces — is older than the field that has now made it urgent. The book is unfashionable, and that is part of its value. It will remind the reader that the questions which seem most contemporary are usually the questions which have been asked for longer than the contemporary discussion remembers.

A Note on What This Syllabus Is Not

This is a syllabus on the epistemology of machine testimony — on what we ought to believe and on what grounds. It is not a syllabus on the politics of AI, the labour history of the data-labelling industry, the regulatory landscape, or the question of machine consciousness. Each of those is a serious subject, with its own reading list; I have separated them here in the hope that focused reading will, in the end, produce more useful thinking than a general one.

Readers with corrections or additions are warmly invited to write. The syllabus is meant to be revised, and the revisions are usually better when they come from outside the editor's own bookshelves.

— Zara Nova, for the desk

[^1]: Coady (1992); Hardwig (1985). [^2]: Daston & Galison (2007). [^3]: Floridi (2013); the review appears in the Reviews department. [^4]: Burrell (2016). [^5]: Bender et al. (2021).