Agency and control are not the same
Control is the real ability to act. Agency is the sense that your action caused the outcome.
In the Post Office scandal, the Horizon IT system showed what happens when the two diverge. Sub-postmasters were told the accounts didn’t balance, yet they had no way to see, correct, or contest the system’s numbers.
The software had control, but they had no felt agency. Some paid thousands to cover phantom shortfalls. Others were prosecuted, even imprisoned, on the basis of errors they could not influence.
Everyday examples illustrate the same tension. A thermostat offers many settings, but if the room takes hours to warm, the user feels powerless. A cyclist on black ice feels in control until the slip reveals the illusion.
This distinction matters for design. Strong agency keeps people engaged and accountable; weak agency reduces them to spectators. Interfaces that give clear, immediate, reversible feedback sustain agency. Opaque prompts or delayed responses erode it.
Psychologists call this locus of control. A strong locus builds confidence in the link between effort and outcome. A weak one undermines trust.1
Why people give agency to machines
Humans often attribute agency to systems. The “media equation” shows we treat computers like social actors, responding with politeness or trust.
Design choices such as display size or conversational tone shape this perception. Get it right and systems feel supportive. Get it wrong and you enter the uncanny valley, where near-human systems feel untrustworthy.1
The tipping point of assistance
Assistance is when a system quietly helps a user achieve their goal. Correcting small errors, smoothing input, or suggesting next steps. At low levels this support feels invisible and useful.
But assistance is not linear. In a study of computer-assisted pointing, people felt fully in charge with no help or light correction. Once the system began to intervene more strongly, their sense of agency collapsed.
Agency shapes how we experience time. With it, even delays feel tolerable because progress is under our control.
How tight or loose the reins are matters
Agency depends not only on what systems do, but on how much freedom they leave to people.
A loose rein gives humans space to adjust and correct. A tight rein makes the system decisive, but risks eroding trust if there’s no way back.
Government: citizen communications
- Loose rein: AI drafts guidance; a caseworker reviews tone and content, then sends.
- Tight rein: The system issues standard letters automatically. Human override and audit trail remain visible.
Lost agency, higher risk
When control slips, the consequences are magnified in government and finance.
Efficiency gains are fragile if they come at the expense of ownership. Without agency, errors spread faster, reputations are harder to defend, and public confidence is slow to rebuild.
AI should reduce friction without stripping out ownership. Assistance should be dosed carefully, reins adjusted by level of risk, and correction paths left open.
The Post Office Horizon scandal showed what happens when agency is ignored. Sub-postmasters were prosecuted based on faulty system data they could not contest. With no override, no transparency, and no path to appeal, the system’s efficiency came at catastrophic human cost. Trust in the institution has still not recovered.
When people feel their actions matter, they remain engaged. With agency preserved, trust and outcomes improve together.
Sources and further reading
- Reeves, B., & Nass, C. (1996). The Media Equation. Media as social actors and why form factor matters.
- Coyle, D., Moore, J., Kristensson, P. O., Fletcher, P., Blackwell, A., & Lindley, S. (2012). I did that! CHI. Measuring agency and the assistance tipping point.