Third and final article in a three-part series on artificial intelligence: definitions, professional applications, and governance.
The second article in this series ended on a chain of reasoning: a model’s performance measures whether it hits the objective it was given, but says nothing about whether that objective was legitimate in the first place — and it’s precisely this shift, from performance to objective, and from objective to the values it’s meant to reflect, that makes AI governance indispensable. This third article picks up from there to tackle the underlying question head-on: how do we make sure an AI system stays aligned with what we actually want from it?
Alignment and specification: related, but different problems
In AI research, the value alignment problem refers to a broad question: how do we ensure a system pursues objectives compatible with human intentions and values? One of the main difficulties that makes this problem concrete is called the specification problem: how do you translate a human intention — inevitably nuanced — into an objective precise enough for a machine to optimise?
In practical terms, a “precise enough” objective for a programmer is a number to push up or down: “minimise the gap between predicted and actual price,” “maximise the click-through rate on a recommendation,” “maximise time spent on a page.” The trouble is that the human intention behind these objectives is almost always broader than the number chosen to represent it. Nobody actually wants to “maximise time spent on a page” for its own sake — what’s wanted is content that’s genuinely useful or interesting, and “time spent” gets chosen as the objective because it’s easy to measure and assumed to reflect user interest. That gap between the real intention and the number chosen to approximate it is what the specification problem is really about.
This distinction isn’t just a terminology detail. It lets us state something far more interesting than “AI should be ethical”: the problem isn’t always that a system fails to follow the objective it was given. Sometimes it follows that objective perfectly — but the objective itself was a poor stand-in for what was actually wanted.
This same trap shows up outside AI too, any time an objective gets replaced by an indicator meant to measure it: ask a sales team to maximise the number of calls made rather than the number of sales closed, and they’ll end up making more short, low-value calls — the indicator climbs while the real objective drifts further away. This is sometimes called Goodhart’s Law: once an indicator becomes the target you optimise directly, it stops being a good measure of whatever it was meant to represent.
The same thing can happen with AI, just more systematically and at greater scale. Take a hypothetical example: a recommendation algorithm is told to maximise the time users spend on a platform, chosen as a rough stand-in for their “satisfaction.” If it turns out that the most polarising or addictive content holds attention the longest, the algorithm will learn to recommend more of it — not because it malfunctioned, but because it optimised the given indicator very well, an indicator that, once turned into a target, no longer really represented the satisfaction it was meant to measure.
This is the exact same mechanism we saw in the previous article with biased credit scoring: the model isn’t “wrong” in a statistical sense — it faithfully optimises an objective that simply didn’t capture what a fair decision actually meant.
When there’s no universally right answer
The self-driving car dilemma illustrates a different facet of the problem: what should an autonomous vehicle do if a crash becomes unavoidable — protect its passengers first, or minimise total casualties, including pedestrians?
In 2018, a team at MIT explored this question at scale with the Moral Machine platform, presenting millions of such scenarios to participants around the world. The results, published in Nature, showed that moral preferences vary significantly across cultures and regions — there’s no universal consensus on the “right” answer.
That raises a question that’s too often sidestepped: who actually decides which values get encoded into a system? It’s almost never a single actor — not the data scientist who builds the model, not the engineer who deploys it, not the technology vendor, not the auditor who reviews it, not even the regulator. It’s usually some mix of all these roles, and often, in practice, nobody explicitly at all — values stay implicit in a string of technical choices, never formulated or validated as values in the first place.
This is exactly where governance becomes necessary: the values that govern a system can’t be left implicit in the code. They need to be made explicit, documented, and open to revision.
The auditor’s role: evaluating the control system, not the model
This is what explains why functions like internal audit are starting to take a closer look at AI systems deployed inside organisations — even though, today, this practice remains young and applied unevenly across organisations and sectors.
The auditor’s job isn’t to answer “is this model mathematically optimal?” — that’s a technical question for data scientists. The auditor’s role is instead to verify:
- who defined the system’s objective, and why that choice was made;
- what data was used, and what assumptions were made along the way;
- what risks were identified, and by whom;
- who is accountable for the system once it’s in production;
- what controls exist, and how the system is monitored over time;
- what happens when it gets something wrong, and who can suspend, correct, or shut it down.
In other words: the auditor doesn’t validate the model — they evaluate the control system that allows the organisation to trust it. That’s an important nuance, and it’s exactly what allows someone without machine-learning expertise to play a legitimate, useful role in governing highly technical systems.
One item on that list deserves to stand on its own: human control and intervention. Is there a mechanism to suspend, correct, or disable the system when its behaviour drifts from what was expected? Without that capacity to intervene, everything else — traceability, documented risk assessments, bias testing — remains largely theoretical.
Governance doesn’t stop at deployment
There’s a piece missing from this picture if we stop at the design and deployment phase: what happens afterwards. A model validated at the point of deployment can drift over time — the data changes, user behaviour evolves, the regulatory context shifts, or the system ends up being used in a setting different from the one it was designed and validated for.
A properly governed model, then, isn’t just one that was properly validated before deployment. It’s a system whose behaviour continues to be monitored, reassessed, and, where necessary, corrected after it goes into production. This is, in fact, exactly the logic behind the NIST AI Risk Management Framework in the United States — a voluntary framework built around four functions, Govern, Map, Measure, Manage — which explicitly frames AI risk management as an ongoing process across the system’s full lifecycle, not a one-off milestone.
On the regulatory side, the EU AI Act takes a risk-based approach, with obligations that scale up according to a system’s risk classification; some of the transparency obligations under Article 50 have in fact applied since 2 August 2026. The NIST AI RMF, unlike the AI Act, carries no binding legal weight — it’s a voluntary framework meant to help organisations structure their own approach to managing AI-related risk.
Where I stand
I’ve sat on both sides of this problem: building and analysing models, and later having to assess the risks they introduce once deployed. That dual perspective has convinced me that the future of AI governance doesn’t lie in increasingly rigid rules imposed on technical teams after the fact. The strongest cases I’ve come across are the ones where the alignment question gets asked at the design stage, not bolted on afterwards as a compliance layer.
That requires closer dialogue between data scientists, auditors, and business decision-makers — three roles that, even today, often only start talking to each other too late in an AI project’s lifecycle.
What this series leaves us with
This series has followed a single thread: first understanding what AI actually is — imitation or rationality — then seeing how that rationality plays out in high-stakes professional decisions, and finally recognising that a rational decision is never automatically a good one, nor a decision that stays governed over time. The question that remains, for any organisation deploying AI today, isn’t “is our model performing well?” but “do we still know why it decides what it decides, and are we prepared to answer for that — today, and a year from now?”
What about your organisation — who’s asking these governance questions about deployed AI systems today: the technical team, compliance, or no one yet? Let me know in the comments.