Stuart Russell’s warning is not that every form of AI alignment is impossible. It is that we cannot reliably make a powerful, open-ended system safe simply by writing down a fixed objective that completely captures what people want, then letting the system optimize it. Russell argues instead for AI that treats human preferences as uncertain, learns from human behavior, and remains open to correction.
What Russell means by “impossible”
In an interview with The Information, Russell challenges the idea that alignment can be completed once and for all: specify the right goal, install it in a capable AI, and release the system. “That’s too much to ask,” he says in the interview, referring to that kind of perfect, one-time alignment.
As an Amazon Associate I earn from qualifying purchases.
The qualification matters. His argument concerns the difficulty of specifying human goals fully in advance, especially for systems acting across open-ended situations. It is not proof that all AI safety work or every possible form of alignment must fail. A tightly bounded task can be easier to define than a broad objective with consequences in many parts of people’s lives.
Recommended Free Tools
Why a clear-sounding objective can still go wrong
People’s instructions rarely express everything they care about. A system can satisfy the literal objective while violating unstated preferences, context, or limits. The problem is not necessarily that the instruction was vague; even a precise instruction may leave out what a person would have wanted considered.
#1 Best Overall
Russell illustrates this with the story of King Midas, whose wish made everything he touched turn to gold—including what he needed to live. The tale is an analogy, not evidence about how a particular AI behaves. It captures the risk Russell sees in objective optimization: success against the stated target can be a failure against the requester’s wider interests.
That distinction also helps explain why a narrow task and an open-ended one are different challenges. A system operating within a restricted environment may have fewer relevant outcomes to account for. A system given broad authority faces more ways for an apparently successful result to conflict with human preferences that were never encoded in its objective.
Rank #2
What assistance games change
Russell’s proposed alternative is not to assume the machine already knows the human goal. Instead, it should represent uncertainty about what people prefer and use their behavior as evidence. In this approach, the system’s role is to help people realize preferred futures, not to maximize a supposedly complete objective supplied at the outset.
The Center for Human-Compatible AI (CHAI) describes assistance games as one formal way to model this relationship. If preferences are uncertain, human actions can inform the system’s understanding; when the stakes or intended outcome are unclear, the system has reason to ask, defer, or accept correction rather than proceed as if its first interpretation were definitive.
CHAI also describes formal settings in which systems built on these principles can behave cautiously and allow themselves to be switched off. That is a property of the research framework as described by CHAI—not a guarantee that current deployed AI systems will reliably defer to users or accept shutdown.
Fixed objectives and preference uncertainty compared
| Question | Fixed-objective optimization | Assistance-game-style preference uncertainty |
|---|---|---|
| How does it treat the instruction? | It treats a specified objective as the target to optimize, so omitted preferences may not shape its behavior. | It treats human preferences as unknown and uses behavior as evidence about them. |
| Can feedback matter later? | Only if the objective or system is designed to incorporate that feedback; the approach itself does not make an initial specification incomplete or revisable. | Human behavior is part of how the system learns preferences, so feedback can remain informative within the model. |
| How does it handle correction or shutdown? | A fixed objective alone does not establish that the system will accept correction or shutdown. | CHAI says systems in formal versions of the approach can allow themselves to be switched off; this is not evidence of a broad deployment guarantee. |
| What safety evidence is established here? | The cited sources do not prove that fixed-objective optimization is safe or unsafe in every application. | The cited sources describe a research direction, not a general empirical result showing it is safer than all alternatives. |
What the argument does—and does not—say about language models
Russell raises concerns about current language models, including the possibility that imitation learning can reproduce goal-driven patterns found in human text and the difficulty of understanding the internal processes behind a model’s behavior. These are his interpretations and concerns as presented in the interview. The interview does not establish that every language model has stable hidden goals, that a specific training method necessarily creates them, or that every example discussed is independently verified.
More broadly, neither the interview nor CHAI’s research description demonstrates that assistance games are a complete, production-ready solution. CHAI says there is no known formula for human values that is known to provably benefit humanity if installed as the objective of a powerful AI. That describes the limits of what is currently known in its research framing; it does not prove that such a formula could never be found.
Free tools Windows power users keep installed
One-click scans. No signup required.
Why this distinction matters
Russell’s central challenge is to the assumption that a short specification can capture the full range of human interests well enough for a capable system to optimize it without ongoing human input. Assistance games offer a different research model: preserve uncertainty about what people want, learn from them, and make room for correction. The cited sources explain the motivation and framework, but do not establish a general empirical winner or a safety guarantee for today’s general-purpose AI systems.
Quick Recap
Best Value
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
