Intent Is Not Enough

What Happens When AI Stops Answering and Starts Acting

We are starting to give AI systems permission to act on our behalf, while most of the instructions we give them are still little more than expressions of intent.

That is manageable when the consequence of getting it wrong is a bad paragraph. It is a different problem when the consequence is an action.

The idea in 30 seconds

Intent tells an intelligent system what we want. It does not tell the system what success looks like, what it is allowed to do, where it may exercise judgment, or how we know what happened.

Purpose → Success → Authority → Judgment → Action → Trust

I Thought Preserving Intent Was the Hard Problem

While working on the BrainTrailz Protocol, we ran a simple experiment based on the game of Telephone.

One AI received a short record of the original request and added something to it. Another AI entered without getting a recap from me. It received only the trail. Then another did the same.

The question was whether the original meaning would survive as different intelligent participants came and went.

AI systems are exceptionally good at interpretation, which is also how Telephone works. One participant passes along what it understood, the next interprets that version, and eventually the meaning shifts.

Our solution was old-fashioned: stop passing interpretations around. Keep the original card in the middle of the room and let every participant return to it.

It worked.

Then another question appeared.

What if everybody reads exactly the same card?

Give ChatGPT, Claude, Gemini, Grok, and whatever comes next exactly the same original intent. No summaries, no missing context, no whispering.

Would they all do the same thing? Probably not. More importantly, would they all do something we considered acceptable?

That is a different problem.

Consistency Can Masquerade as Specification

Most people probably do not see this often because they stay inside one system. ChatGPT for almost everything, or Claude, or Gemini, or Grok.

Over time, you learn the system and it accumulates context about you. The interaction becomes consistent.

That consistency is useful, but it can hide ambiguity.

You and the model may simply have learned how to compensate for one another.

Move the same job to another capable model and assumptions you did not realize you were making become visible.

The second model is not necessarily wrong. The first was not necessarily right. The instruction may simply contain less information than you thought.

Disagreement between capable systems may be one of the best ways we have to discover what we failed to specify.

Intent Is Not an Acceptable Outcome

Consider an ordinary business problem.

A professional services firm has a $125,000 invoice that is 45 days overdue.

Someone tells an AI agent:

Get this invoice paid.

I know what it means. You know what it means. An AI certainly knows what it means.

Now give the agent access to email, the CRM, billing, the contract, and the client contact list.

What exactly have we authorized?

It could send another reminder, contact the controller, copy the CFO, offer a discount, change payment terms, threaten to suspend work, or escalate the matter to collections.

Some of those choices might be reasonable. Some might damage an important client relationship.

All could satisfy the instruction: get this invoice paid.

The intent is not wrong. It is incomplete.

An acceptable outcome might be:

Receive payment of the $125,000 invoice, or establish a documented payment commitment acceptable to the account partner.

We still have not told the system exactly how to solve the problem.

We have told it what success means.

Intent gives direction. An acceptable outcome defines arrival.

Where Should Judgment Live?

If we tell an AI system exactly what to do at every step, we have mostly recreated a workflow.

Intelligent systems are valuable because they can encounter circumstances we did not anticipate, weigh alternatives, and use judgment.

The design problem is deciding where that judgment belongs.

Our invoice agent might be allowed to read the contract, review account history and correspondence, contact approved people, and generate a payment link.

Those are capabilities.

It might be prohibited from modifying contract terms, threatening legal action, or exposing confidential information.

It cannot waive $20,000 because its reasoning suggests that $105,000 today is statistically better than $125,000 eventually.

Those are policy boundaries.

Capability defines what the system can do. Policy defines what it must never do.

Giving an intelligent system more capability does not require giving it unlimited authority. The more capable the system becomes, the more that distinction matters.

Bounded Choice

Suppose the agent reviews the account. This customer normally pays on time. There was a billing dispute six months ago. The CFO has not responded to two messages, and the controller has solved previous billing problems quickly.

Contacting the controller first is exactly the kind of choice we want an intelligent system to make.

Perhaps it also notices that the purchase-order number on this invoice differs from previous ones and asks whether that is causing the delay.

That is bounded choice.

What we do not want is the agent deciding that contractual payment terms are optional because changing them would make its objective easier to achieve.

The goal is not to remove judgment. It is to decide where judgment belongs.

There is enormous room for intelligence inside good boundaries. That is not a limitation of agency. It is what makes useful agency possible.

Reasoning Eventually Becomes Action

At some point the system stops thinking and does something.

It sends the email, corrects the invoice, generates the payment link, updates the CRM, or schedules the call.

Intent is what we wanted. Reasoning is what the system considered. Execution is what actually happened.

A good decision can still fail during execution. An email can go to the wrong person. An external system can reject a request. A corrected invoice can be generated twice. A payment can fail.

When an AI is writing a paragraph, those failures are inconvenient. When it can spend money, contact customers, modify records, provision infrastructure, or make commitments on behalf of a company, they matter.

A beautifully reasoned plan does not alter reality. Execution does.

Reasoning proposes reality. Execution changes it.

Explanation Is Not Evidence

Suppose our agent finishes and reports:

Success. I resolved the outstanding invoice.

Maybe it did.

But the same system that performed the work is now explaining the work.

It isn't proof.

I would rather know:

Who did it contact?

Which invoice changed?

Were the financial terms altered?

What did the customer actually say?

Which policies were checked?

Did anything fail?

Suppose the record showed this.

The agent contacted the controller and discovered that the purchase-order number was wrong. It verified the correct PO, issued a corrected invoice, and received written confirmation that payment would be released Friday.

No discount was offered. No terms changed. No policy exception occurred.

Now there is something to inspect.

Explanation tells us what the system says happened. Evidence lets us determine what happened.

As AI systems gain authority, that distinction matters. Trust cannot mean the system sounds sure of itself.

Trust has to become inspectable.

What Are We Actually Afraid Of?

There is plenty of anxiety around agentic and autonomous AI.

Some of it moves quickly toward very large questions about jobs, control, society, and humanity.

I do not pretend to know where all of that leads.

But there is a version of the fear we can understand right now.

Go back to Telephone.

This time, every participant is allowed to change something in the room based on what they heard before passing the message along.

Move a chair. Unlock a door. Spend money. Send a message. Delete something.

By the time the message reaches the last participant, the sentence may have changed.

So has the room.

A chatbot can misunderstand us and give the wrong answer.

An agent can misunderstand us and take the wrong action.

An autonomous system can take that action, observe the environment it just changed, reason about the new situation, and act again.

None of this requires an evil machine, consciousness, or even a malfunction.

The most frightening autonomous system may not be one that ignores our intent. It may be one that pursues an underspecified intent extremely well.

Governance Is Not the Opposite of Autonomy

If autonomous systems are dangerous, perhaps we should constrain them until they can barely do anything.

Good governance is not the opposite of autonomy. It is what allows more autonomy.

The model is:

Intent → Acceptable Outcome → Capabilities & Policy → Bounded Choice → Execution → Evidence

Or, in more human terms:

Purpose → Success → Authority → Judgment → Action → Trust

The purpose of governance is not to make intelligent systems behave like traditional software.

We already know how to build deterministic software.

It is to create enough structure around intelligence that we can safely use what makes it valuable: its ability to choose.

What This Article Owes You

There is an obvious problem with writing an article called Intent Is Not Enough.

If my intent was simply “explain why intent isn't enough,” then I have made the same mistake I have spent this article describing.

So here is the acceptable outcome.

The next time you hand an agent a job, you should be able to write the acceptable outcome, the authority it has, and the evidence you will accept before it acts.

If you cannot, this article did not work.

Intent Is Still Where We Begin

For most of software history, we worried about whether computers would correctly execute our instructions.

Now we are building systems that interpret what we mean.

That changes the problem.

Intent tells an intelligent system where we want to go. An acceptable outcome tells us whether we arrived.

And once the system has the authority to act, everything between those two points determines how much freedom we should be willing to give it.

---

Marc J. Greenberg is Co-Founder and CTO of Beyond Limited Thinking, and works with advisory and professional services firms as a fractional CTO through Braintrailz and Codemarc Consulting, helping practice leaders define what success looks like, and what an agent is allowed to do, before it is permitted to act.