What We Oweto Each Other

The First Collision

In February 2026, I opened a Flipboard notification about a confrontation between Anthropic and the Pentagon in an AP article.

The dispute concerned the conditions under which the Department of Defense could use Claude. Anthropic refused to remove safeguards related to mass domestic surveillance and fully autonomous weapons, arguing that current frontier systems remained too unreliable for certain high-stakes uses. Pentagon officials responded that a private company should not determine how the military could lawfully use technology it had purchased. Both institutions possessed something the other could not easily replace: the government held public authority, while Anthropic held technical expertise and control over the system.

At first, I read it as a dispute over one government contract, but it exposed a much larger institutional problem. AI is being built in one world and governed in another.

As a student, I had already watched ChatGPT and Claude develop at a pace that was difficult to reconcile with the slower world of laws, public institutions, and university disciplines. Their rapid improvement of AI frontiers made me wonder whether an equivalent policy world existed around them. And AI policy does exist today, but trying to answer the below questions are much more difficult in today's political landscape:

1. Who has legal or political authority?
2. Who controls the technology or infrastructure?
3. Why is there no reliable institution connecting these groups?

The third question is key. What is expected of us, then, is neither perfect coordination nor a single institution capable of seeing everything. A more realistic starting point is to make the relationships visible: who understands the technology, who can make the decisions, and where responsibility sits.

I now start with named institutions1Map nodes are named institutions. Biosecurity and cybersecurity are overlapping domain tags; defence is a separate use context. Read the node and domain definitions. and documented mechanisms2A documented agreement, evaluation or deployment establishes a specific relationship. It does not by itself measure institutional dependence, authority or effectiveness. Unknown dates and present status remain unverified. Read the evidence rules.. The Institutional Link Map3The institutional map is a provisional historical sample with inspectable source records. It does not claim complete or current coverage. Explore the institutional map. lets us inspect the agreements, evaluations, and deployments through which institutions meet. Each connection has a source and a limit to what it can establish.

In this essay, I try to illustrate that the central governance problem of frontier AI lies in the distance between those who understand it most deeply and those who decide how it should be managed. The greater that distance, the more likely AI policy is to become adversarial, or technically unworkable.

What’s Expected of Us4In Ted Chiang's 2005 short story “What's Expected of Us” (PDF), the story considers what happens when people come to believe that the future is already fixed and that their choices no longer matter. I am using it here less as an argument about free will than as a warning against the passive language that often surrounds technology. We say that models will improve and governments will eventually adapt, as though these were natural events rather than the accumulated result of decisions made by companies, researchers, and institutions.

AI is often described as something that is simply happening to us, passively. Models will become more capable, competition will intensify, and governments will eventually have to adapt. This language captures the speed of change, but it can also make responsibility disappear.

Much of frontier AI is developed inside a small number of private companies. These companies recruit the technical talent, operate the computing infrastructure, evaluate their models, and control how those models are released. Stanford's 2025 AI Index found that industry produced roughly 90 per cent of the notable AI models identified in 2024. At the same time, the U.S. Government Accountability Office has warned of serious shortages of AI expertise across the federal workforce. The people closest to the technology are therefore often separated from the institutions expected to govern its public consequences.

Their timelines rarely match either, as models can change within weeks, while legislation, judicial review, and international effort can take years. It is tempting to reduce this to the claim that policy moves too slowly, but caution is part of what public institutions are for. They are expected to follow procedures and remain accountable to law. Private firms can usually move faster because they are not bound by the same public processes, but speed alone does not give them public legitimacy.

I therefore do not think the central problem is a simple shortage of expertise, as specialization itself is not the problem. In fact, it is probably one of the main reasons humanity has become capable of doing extraordinarily difficult things.5CRISPR is a useful example of how long specialized knowledge can take to accumulate. The unusual DNA sequences were first observed in bacteria in 1987, but their function was not understood at the time. It took roughly twenty-five years of work by researchers across several areas of biology before CRISPR was turned into a practical tool for editing genes. The Broad Institute’s CRISPR Timeline traces that development.

We spend years learning the language, methods, assumptions, and history of one field precisely because no person can know everything. But our institutions tend to preserve those divisions long after the problems themselves have stopped respecting them.

We grow up moving between separate subjects in school, and eventually become very good at speaking to people who were trained to think about problems in roughly the same way we were. Universities certainly create interdisciplinary programs, laboratories, and research groups, but their basic architecture is still largely disciplinary.

This becomes much more consequential as AI begins to increase the amount of intellectual work that specialized fields can produce. One way Dario describes this problem is to ask what becomes the limiting factor once intelligence itself becomes much more abundant. More intelligence does not make every other constraint disappear, as experiments still take time and human beings still have to decide whether something should actually be used.6In Machines of Loving Grace, Dario points to factors such as the speed of the outside world, missing data, and human constraints in a world of abundant intelligence. In Policy on the AI Exponential, he makes a related policy argument for preserving optionality and improving visibility as AI capabilities change quickly. I use the two here to distinguish faster intelligence from the physical and institutional conditions needed to act on it.

And there is another possible bottleneck that I think deserves much more attention: verification.

Mathematics offers an unusually interesting example of what this can look like. In 1946, Paul Erdős posed a problem about something that sounds almost trivial: if you place a large number of points on a plane, how many pairs can be exactly one unit apart? Mathematicians worked on versions of this problem for nearly eighty years. Then, in May 2026, an internal OpenAI model produced a construction that disproved a longstanding conjecture about its answer. External mathematicians subsequently checked the argument and produced human-written papers explaining and verifying the result.7An OpenAI model has disproved a central conjecture in discrete geometry (May 2026). See also Quanta Magazine, Why the Legendary Erdős Problems Are Falling to AI. I use the case here because it separates producing a difficult result from the process by which that result becomes inspectable and credible.

One of those tools is Lean, a formal proof system.8The Lean Prover Community’s Did you prove it? makes an important distinction between verifying a proof and verifying that the formal statement actually corresponds to the theorem someone claims to have proved. In very simple terms, Lean allows mathematicians to express a mathematical statement with extraordinary precision and then check whether a proposed proof actually establishes it. It does not decide which theorem matters, nor does it magically tell us whether we have formalized the question we intended to ask. In fact, the Lean community explicitly warns that this distinction matters: a formally valid proof is not useful if the formal statement does not correspond to the mathematical claim we thought we were proving.

This is becoming particularly interesting alongside AI. DeepMind's Formal Conjectures9Google DeepMind’s Formal Conjectures project is intended in part as a benchmark for automated theorem proving. project is turning large collections of open mathematical problems—including hundreds drawn from Erdős's problem lists—into statements written in Lean. The point is partly to create problems against which automated theorem provers can work, but also to make the questions themselves precise enough that solutions can eventually be formally checked.

I find this incredibly compelling, but also slightly frustrating as someone who works in global affairs, because our problems do not behave like this. We cannot place a statement such as this institution has authority over this AI system into Lean and expect it to tell us whether that statement is true. Even the word authority immediately creates more questions. Authority granted by what? A regulation? A procurement contract? Is the authority legally binding, politically asserted, or merely advisory? Does it apply to this particular institution, jurisdiction, or use? Two perfectly competent researchers can read the same material and still disagree about the answer. The disagreement here is not an error in the system, because sometimes, or most of the time, it is the system.

That means I do not think policy needs, or could realistically have, some universal language that verifies political truth. But the benefit of formal systems points toward something smaller and much more achievable. We may not be able to formalize whether a political judgment is ultimately true, but we can become far better at formalizing what we are claiming, where the claim came from, what evidence supports it, what assumptions were required to reach it, and what we still do not know.

This sounds rather obvious until you look at how much policy research is actually produced. A statute lives in one place, a policy analyst interprets it in a report and another researcher compresses the report into a spreadsheet. Eventually, a conclusion can become separated from the evidence it is based on, making provenance difficult to trace.

People working between policy and technology have already encountered narrower versions of this problem. New Zealand's Better Rules project, for example, found what it called a translation gap between policy analysts and software developers. Each group had its own structured language and professional standards, but each subsequent group had to reinterpret what the previous one produced. There is now an entire Rules as Code movement built around related ideas. Catala, a programming language designed specifically for statutory law, goes further by giving lawyers and programmers a shared medium through which certain computational parts of legislation can be represented and tested. None of these projects makes politics mathematically provable, nor should they. But they demonstrate something important: professional boundaries become less costly when the information crossing them has structure.

What Do We Owe to Each Other?

Then, as people working in policy and technology, we have to ask ourselves: what is expected of us? More importantly, what do we owe one another10I am using this phrase from T. M. Scanlon's What We Owe to Each Other in a institutional sense. When technical and policy institutions jointly shape systems that affect public life, neither can treat the knowledge or responsibilities of the other as someone else's problem. Scanlon's contractualism centres on whether conduct can be justified to others while respecting their separate interests, rather than securing agreement or producing a preferred overall outcome. I borrow that distinction to claim that the worlds of policy and technology do not need to agree on everything, but the decisions they make together should be defensible to the people affected by them. They may not owe one another agreement, but they do owe one another enough openness to make their decisions defensible to the people who will live with them., and what should that require of us? I do not think the answer is that engineers should become policymakers, or that policymakers need to understand every technical detail. These fields exist separately for good reasons. But when different institutions each hold part of the knowledge needed to understand a serious risk, they have some responsibility to make those parts intelligible to one another.

Thus, these relationships form something closer to an institutional system than a simple divide between government and industry. The map below is a preliminary picture of that system:

In regards to the aforementioned concept of Erdős problems and Lean, the lesson I take from mathematics is not that international affairs ought to become mathematics. It is almost the quite opposite. Mathematics can demand a degree of formal verification because it operates under conditions that political life rarely gives us. Global affairs contains ambiguity, incomplete evidence, competing interpretations, and legitimate disagreement. Any system that removed those things for the sake of producing a clean answer would probably make the analysis worse.

But ambiguity does not require disorder: there is a difference between saying we disagree about what this evidence means and not being able to determine which evidence produced the disagreement in the first place. And AI makes this increasingly important because it is not merely producing more text, as it is beginning to act across the same institutional boundaries that we already struggle to describe.

I have not yet figured out how to answer the three questions11These are the three questions posed at the beginning of the essay:
1. Who has legal or political authority?
2. Who controls the technology or infrastructure?
3. Why is there no reliable institution connecting these groups? I proposed at the beginning of this essay. But I do know that the answer is not to create one language in which everyone thinks. A better way forward is to find ways for our different languages to meet, and to make the distance between them easier to cross. Perhaps this is what the worlds of policy and technology owe one another: not to become the same, but to make a serious effort to render their knowledge understandable across the boundary between them.