Second article in a three-part series on artificial intelligence: definitions, professional applications, and governance.
In the first article of this series, we saw that the real question to ask about AI is no longer “does it seem intelligent?” but “are its decisions reliable, justifiable, and aligned with our goals?”. That question stays fairly abstract when we’re talking about chatbots or games of Go. It becomes far more concrete the moment AI — or, more broadly, advanced data analysis — starts making decisions that affect real people, real budgets, or real risks.
When AI supports a high-stakes decision
One of the most well-documented examples is AI-assisted medical screening. In 2020, a team of researchers at Google Health published a study in Nature showing that a deep learning model could reduce both false positives and false negatives compared to radiologists, on this specific task and on the datasets studied — trained on mammograms from several hundred thousand women. This isn’t a system that’s “better than radiologists” in some absolute sense, but a solid result on a well-defined detection task.
This result nicely illustrates a point from the first article: the system doesn’t imitate a radiologist’s reasoning — it acts rationally based on statistical patterns found in its training images. But it also illustrates an important limitation: a model that performs well on one dataset won’t necessarily perform as well in a new hospital, with different imaging equipment, or on a population different from the one it was trained on. Performance measured in a lab is never a guarantee of reliability in the real world.
That tension — measurable performance versus real-world reliability — is exactly what defines AI at work today.
AI in control functions: audit, compliance, fraud detection
This is ground I know directly. Advanced data analysis and machine learning are starting to transform functions like internal audit, compliance, and fraud prevention — historically dominated by manual review and sampling — even though, in most organisations, these methods still coexist heavily with deterministic rules, classic statistical tests, and expert systems.
Banks have for several years used anomaly-detection systems to monitor transactions and flag money-laundering patterns that fixed rules wouldn’t catch. Large audit firms are increasingly incorporating data-analysis tools to examine an entire general ledger rather than a sample, surfacing anomalies that traditional testing would have essentially no statistical chance of catching.
My own example fits this same logic — an application of unsupervised machine learning rather than “AI” in the strict sense, though the line between the two is often blurry anyway.
Specifically, I built a collusion-detection model based on clustering: rather than looking for explicit rules (“this supplier submitted a suspicious bid”), the algorithm groups bidding behaviour by statistical similarity — bid rotation patterns, price gaps, the structure of tenders won. That work identified confirmed cases of supplier collusion that a standard manual review would not have surfaced.
This case captures well what these tools promise when applied to control functions: they don’t replace the auditor’s judgment — they change its starting point. Instead of starting from a sample and a hypothesis, you start from the full dataset and let the model flag what deserves closer human review. It’s a shift away from the “AI replaces humans” narrative toward something more accurate: AI widens the field of observation, which reshapes the decision process — and creates a new need for human oversight, not less of it, just different.
A sector-by-sector snapshot
| Domain | Application | Potential gain | Main risk |
|---|---|---|---|
| Healthcare | Diagnostic support (medical imaging) | Faster, more consistent detection | Errors when generalising to a new context |
| Finance | Anti-money-laundering (AML) detection | Continuous monitoring at scale | False positives, reproduced historical bias |
| Audit & compliance | Anomaly and collusion detection | Full coverage instead of sampling | Misinterpreting a statistical signal |
| Transport | Assisted or autonomous driving | Real-time decision-making | Errors with potentially irreversible consequences |
This table points to something all these cases share: the higher the stakes of a decision, the more central the question of the system’s reliability becomes — and the riskier it is to deploy it without an accompanying control mechanism.
Error and bias are two different problems
It’s tempting to reduce the risks of these systems to a single word — “bias.” But there are really two distinct problems to separate out.
A model can simply be wrong: a fraud-detection system can correctly flag the statistically most unusual transactions without those transactions actually being fraudulent. That’s an error of bounded rationality — exactly the kind described in the first article. The model reasons well from what it observes, but what it observes isn’t enough to guarantee the truth of its conclusion.
A model can also be biased: a statistically strong credit-scoring system can reproduce historical inequalities present in its training data, systematically disadvantaging certain groups — not because it’s statistically wrong, but because the objective it was given (“maximise predictive accuracy”) doesn’t capture what we actually mean by a “fair” decision.
A model, then, can be technically strong yet socially problematic — and, conversely, fairer by some given criterion yet statistically weaker. That raises a question no single metric can answer on its own: what makes a decision “good”?
And this is where the reasoning needs to go one step further: a good decision isn’t necessarily the one that maximises a metric. A model can maximise accuracy, cut costs, or raise a detection rate — but none of those metrics, on their own, say whether the resulting decision is acceptable to the organisation, to the people affected, or to society more broadly. Performance tells you whether the model hits the objective it was given; it says nothing about whether that objective was the right one to begin with. It’s precisely this shift — from performance to objective, and from objective to the values it’s meant to reflect — that makes AI governance indispensable.
Toward an unavoidable question
This is exactly where the distinction between rationality and truth, introduced in the first article, becomes meaningful in a professional context: a rational agent is neither omniscient nor infallible. It can make the best decision available given the information it has, and still get it wrong — or unintentionally reproduce choices that don’t actually reflect our real objectives.
Once that’s clear, a question follows naturally: who verifies that these systems, once deployed inside an organisation, keep doing what we expect of them — over time, and fairly?
That is precisely the role that AI governance and audit are increasingly being asked to play — a field still young today, but on track to become as central as financial audit became in the 20th century.
In the final article of this series, we’ll tackle this question head-on: the problem of value alignment, and the growing role of internal audit in governing AI systems within organisations.
Have you ever encountered an automated decision whose reliability you questioned? Let me know in the comments.