From Bits to MindsMeasuring What AI Actually Understands

Try this. Give a frontier AI agent a task that's boring in exactly the way real work is boring: open three tabs, compare prices, pick the cheapest option, then fill a checkout form right up to the final pay button. Watch it fly. It'll read pages fluently, do arithmetic instantly, and narrate its plan with unnerving confidence. Then it will click Submit, the page will quietly refuse because a required dropdown is still set to "Select…", and the agent will report: done.

Most benchmarks log that as a plain failure. True, but not very diagnostic. The interesting part is the shape of the failure. The agent didn't collapse on language, or arithmetic, or even decision-making. It collapsed on the tiny, unglamorous act of noticing whether the world actually changed.

That's a metacognitive failure: not a lack of capability, but a lack of reliable self-assessment. This essay proposes a way to measure that gap using tools that are almost comically old-school: Shannon's information theory (how uncertainty moves) and Wiener's cybernetics (how feedback stabilises behaviour). We'll use them to decompose agent behaviour into measurable cognitive operations, identify where errors concentrate, and design hybrid systems that route work to the right kind of mind—human, model, or both—based on quantitative cognitive profiles rather than vibes.

In 1948, two people working on two different problems accidentally gave us a toolkit for thinking about minds.

Claude Shannon, at Bell Labs, asked what it takes to transmit messages through a noisy channel. His answer was entropy: the expected "surprise" in a signal, measured in bits. If a system sees a world that could be in many states, its entropy is high; if the world is predictable, entropy is low. His deeper point was the channel capacity theorem: every channel has a hard limit on how much information can be carried reliably.

Norbert Wiener, at MIT, asked a different question: how does a system use information to control itself? His answer was the cybernetic feedback loop: act, observe the result, compare against a goal, and feed the error back into the next action.

Shannon tells you about information content. Wiener tells you about information control. Cognition needs both. A brain that can compute but can't detect its own error signal is not "less intelligent"; it's just unsafe.

The bridge between them is mutual information: how much knowing one variable reduces uncertainty about another. It's a way of asking: how much of the signal survived the channel?

Now swap the nouns. The "transmitter" is a user's intent, the "channel" is an agent's perception-and-reasoning stack, and the "receiver" is the outcome in the external world. A modern GUI agent is a walking demo of partial observability: it sees pixels, not state; it issues clicks, not guarantees. Crucially, its feedback channel is often the weakest link—subtle highlights, silent validation errors, delayed updates, and modal dialogs that fail off-screen.

So when we ask whether an agent "understands", we're really asking two separate questions: (1) how much task-relevant information can it extract and propagate through its internal processing, and (2) how much information does it have about its own success and failure? The second question is the metacognitive one, and it's where today's systems most visibly wobble.

That leads to a useful (and testable) notion: an agent has a kind of cognitive channel capacity. As tasks get longer, noisier, or more visually ambiguous, information degrades. At some point the agent is no longer steering by feedback; it's steering by story. The metric we introduce next—OSKR—measures exactly how much of that steering is real.

Getting the right answer and knowing you got the right answer are different cognitive operations. A student can land on the correct result by accident; a good student can tell you which step was fragile, and how they'd check it. Psychologists call that capacity metacognition: monitoring and evaluating your own thinking.

For AI agents, metacognition is not a philosophical flourish. It's the difference between "helpful assistant" and "confident saboteur". If the agent can't tell whether a click succeeded, it will fill the gap with plausible narration—and your system will treat fiction as telemetry.

We can put a number on this by defining a variable for outcome (did the step succeed?) and a variable for the agent's self-assessment (its confidence, error flag, predicted probability of success, or any internal proxy). Then we ask: how many bits of the outcome does the self-assessment actually contain?

I call this the Outcome Self-Knowledge Ratio (OSKR): a normalised mutual information that measures the fraction of outcome uncertainty removed by the agent's self-assessment. It uses the I(S;T)/H(T) normalisation—what fraction of total outcome uncertainty does the self-assessment remove?—which is directly actionable for agent deployment decisions.

The idea of entropy-normalising mutual information between confidence and accuracy as a metacognitive-efficiency measure has roots in Dayan's (2023) information-theoretic treatment of metacognition. A related but distinct quantity, Relative Metainformation (RMI), was introduced by Meyen et al. (2025) using bound-normalisation that accounts for accuracy-imposed information limits. OSKR uses the simpler absolute normalisation; the two measures answer complementary questions.

If the agent's self-assessment is statistically independent of reality—it feels confident regardless of success—then the mutual information is 0 and OSKR is 0. If its self-assessment perfectly predicts success and failure, the conditional entropy drops to 0 and OSKR is 1.

A neat feature is the unit. For a balanced success/failure outcome, H(T)=1 bit. An OSKR of 0.8 means the agent's self-assessment carries 0.8 bits of the 1 bit you'd need to know whether it worked. That's calibration expressed in the native currency of information theory.

Notice what OSKR is not. It's not raw task accuracy. Two systems can have the same success rate and radically different OSKR. One might succeed often but be clueless about when it failed; the other might succeed less often but reliably detect the failures and escalate. In deployment, the second system is frequently the safer choice.

High-OSKR agents pause when something feels off, request another observation, run a verification step, or hand control back. Low-OSKR agents barrel on with unwarranted confidence. A lot of what we call "hallucination" in agentic settings is this dynamic: a missing or ambiguous feedback signal gets replaced by a plausible narrative, and the narrative is mistaken for evidence.

To apply metacognitive measurement to real agents doing real work, you need a decomposition that's faithful to how tasks actually unfold. Otherwise everything collapses into a single pass/fail number, and you learn nothing about why the system failed.

Cognitive Chain Modelling (CCM) is a taxonomy for GUI-based tasks that draws on a long lineage in HCI: Norman's seven-stage action cycle, the GOMS family of task analyses, and most recently the TaskSense framework (Li et al. 2025), which defines a "cognitive chain" of ordered cognitive steps preceding GUI actions using the same eight operation types we adopt here. What we add is not the taxonomy itself, but its combination with per-operation OSKR measurement and deployment routing—turning the chain into an instrument for deciding where to trust the agent, not just whether it passed.

The eight operations, in some form, appear in almost every workflow. Think of them as the primitive moves of applied cognition:

Each operation maps onto well-studied constructs in cognitive science. Orient is belief-state and working-memory maintenance: knowing where you are in a workflow and what matters next. Find is visual search and attention. Extract is perception plus parsing. Recall is memory retrieval. Decide is action selection under uncertainty. Compute is transformation and inference. Create is generation. Verify is the metacognitive checkpoint that closes the loop.

Now the information-theoretic part. At each link you can estimate: (a) local uncertainty (entropy), (b) how much task-relevant information survives to the next step (mutual information), and (c) local metacognitive reliability (OSKR). When the chain breaks—and it always breaks somewhere—you can locate the failure mode instead of merely recording it.

This turns benchmarking from "did it pass?" into a cognitive profile: a map of strengths and blind spots you can use to design prompts, build guardrails, choose tools, and decide when a human should take the wheel.

If human and AI cognitive strengths are complementary rather than overlapping, the implication is immediate: stop delegating tasks wholesale and start routing by cognitive dimension. In practice, you don't "hand over a job" to an agent—you hand over the parts of the job that sit above its reliability threshold.

The decision logic maps directly from the cognitive profile: where the agent's OSKR is high enough for the local task load, delegate. Where it isn't, keep a human in the loop, or improve the feedback channel until the agent can genuinely verify what it did.

The key variable is always agent OSKR relative to task difficulty in the relevant dimension. If the agent's metacognitive reliability for that operation exceeds the task's cognitive load, delegate. If it doesn't, treat the agent as an optimiser, not an autonomous actor.

The Deployment Matrix

The simplest way to see this: plot accuracy against self-knowledge. The four quadrants yield four distinct deployment postures.

Same Accuracy, Different Self-Knowledge

This is the central insight, sharpened. Two systems can have identical success rates and radically different value in deployment. Meyen et al.'s ambiguity result formalises this: accuracy alone does not determine the information a system's confidence carries about its correctness.

Actor vs Verifier: Where the Field Is Going

The sharpest practical implication of this framework: the same task can produce radically different reliability depending on how verification is structured. Below is a schematic comparison.

This isn't about AI replacing humans or humans "supervising" AI. It's about composing hybrid cognitive systems where each type of mind operates in its zone of competence—measured, not assumed.

What makes this framework usable is that three historically siloed fields are converging on the same questions using the same mathematical language.

Information theory supplies the units: entropy, mutual information, channel capacity. It lets you replace fuzzy words like "complexity", "signal", and "uncertainty" with quantities you can estimate from data.

Cognitive science supplies the ontology: working memory, executive control, metacognition. It tells you which quantities matter if you care about reliability, not just raw performance.

AI systems supply the laboratory. You can instrument every internal variable you can access, run the same task thousands of times, ablate one capability at a time, and watch the cognitive profile change. That makes agents an unusually sharp testbed for theories of mind.

The traffic is two-way. AI gives cognitive scientists controllable "minds" that run at scale. Cognitive science gives AI engineers a vocabulary for escaping the flatland of accuracy and asking the questions that actually matter in deployment: where does this system's thinking break down, and does it know when it's breaking?

If OSKR becomes a standard deployment metric—reported alongside accuracy the way calibration is reported alongside Brier score—the implications reach beyond any single agent. It would change what benchmarks measure, what procurement contracts require, and what "ready for production" means.

The Channel Changed

Shannon's original channel ran from transmitter through noise to receiver. Update it for 2026: the transmitter is user intent, the channel is a perception-and-reasoning stack interacting with a messy interface, and the receiver is a real-world outcome that often provides only ambiguous feedback.

The maths hasn't changed. Entropy still accumulates. Capacity still constrains. Feedback still corrects—but only if the system can see its own error signal.

The agents you'll trust with real tasks, real data, and real consequences won't merely be the ones with the best benchmark scores. They'll be the ones with the best metacognitive telemetry: the ones that can answer a single, brutally practical question.