Who Paces the AI Frontier?

Dario Amodei wants to give society time to deliberate about artificial intelligence. I am not qualified to judge the merits of the head of Anthropic’s proposal, made in his recent essay, “We Must Pace the Frontier.” I do, however, want to address the more fundamental problems he implicates without bringing to the fore. Who will deliberate, and who will judge? 

Amodei acknowledges that AI development needs supervision and accountability from outside the industry. But his account of where that supervision and accountability should come from needs careful scrutiny. Amodei initially favors American regulation and takes competition with China—which he acknowledges as an autocratic regime with very different principles and concerns from the U.S. and her allies—seriously. But the United States, Western democracies, and America’s other allies are not adequately distinguished in his essay. These are political bodies with different interests and agendas. A common interest in containing China hardly settles how Americans and Europeans, for example, will or should govern technology at home. Further, when he claims that “the public deserves to know what is going on,” which “public” is he referring to? The character and authority of “public” deliberation would depend very much on which public he is talking about. 

Amodei is also unclear about what he means by democracy. Does he mean the regime characterized by rule by the people? Or does he mean an agreed set of humanitarian values whose interpretation remains with experts—the “sacred democracy” of contemporary elite piety that has little to do with rule by the people? Here the America–EU divide looms large.

Furthermore, Amodei suggests embedding independent evaluators in each AI company. At first glance, the suggestion has obvious merits: A (purportedly) disinterested party provides both accountability and a separate set of eyes to look for problems. But his proposed independent evaluators, such as METR (Model Evaluation and Threat Research), illustrate the problem. METR has ties to the effective altruism movement, a movement that draws much more from the utilitarian ethics of Peter Singer or Auguste Comte’s religion of humanity than from the American polity or, certainly, Christian traditions of reflection on ethics and politics. What should citizens and their representatives be able to require of METR and similar evaluators? Who establishes their remit, judges their standards, or hears challenges to their judgments? Who will evaluate the evaluators, and by what authority? 

Questions compound when we consider the subject of AI “alignment,” which Amodei tends to treat as a settled matter, in principle lacking only competent execution. Amodei gives alignment a practical meaning: Models should remain safe, ethical, helpful, and compliant with the company’s guidelines. But determining in each case what counts as safe, ethical, and helpful requires a judgment about the human good or goods that seem to go beyond Amodei’s expectations. This problem is particularly acute if one suspects that the leaders of AI companies, such as Sam Altman or Amodei himself, might themselves be misaligned.

Anthropic’s constitution for Claude acknowledges that there is disagreement on important, human things, and consequently asks the model to weigh competing considerations. But weighing requires an ordering of goods. If autonomy conflicts with wellbeing, which should prevail, and why? Do spiritual goods and harms count? What is a human being, anyway? Would something that helps the soul but, arguably, incidentally harms the body (like Lenten fasting) be counted as a help or a harm? Eventually an account of help must reckon with the highest good, the summum bonum, in relation to which other goods receive their place.

The nature of political life adds a further difficulty. Politics involves the division of humanity into concrete, mediating forms, conditioned by all sorts of contingencies. Nations may be perfectly legitimate, if imperfect, political mediations of humanity and still be very different from each other. Two legitimate nations, therefore, might pursue goods that pull in very different directions. Neither need misunderstand humanity or reject objective morality. Their responsibilities and interests arise within different common lives. An AI system could faithfully serve each nation’s purpose and still bring the two into conflict. More successful alignment would sharpen their disagreement rather than resolve it.

And what about aligning with humanity? French political philosopher Pierre Manent points out that humanity does not deliberate or act, since it is an abstraction. Human beings act through actual communities and, especially in modern politics, nations. International cooperation depends upon the capacity of these actual political communities to assume obligations for which they can answer. Ultimately, it is nations that will need to figure out what to do about the pace of AI, and then act. 

Only larger, wealthier, more powerful nations are capable of doing so, which may be why—despite talking somewhat breezily about “democracies” and the global order—Amodei frequently comes back to two nations in particular: China and the United States. The alternative is rule by NGOs answering to no one, wielding expertise that is likely technocratic, secular, and utilitarian, not to mention hostile to Christian understandings of the human person and traditional Western notions of political self-government.

For these reasons, Amodei’s vaguely political, at times verging on apolitical, essay is not equal to the moment or the fundamental political and anthropological problems at stake. Giving us more time to monitor AI development would not fundamentally alter the anthropological, moral, or political difficulties we face.

Image by Sandra Blaser / World Economic Forum via Creative Commons. Image cropped.