Most organisations do not decide to let AI make consequential decisions. It happens by drift. A tool is introduced to save time on a routine task: summarising, sorting, calculating. Its output is good, so people start to rely on it. The output starts appearing in the next step of the process without anyone re-checking it. A year later, nobody can say exactly where the tool stopped helping and started deciding.
That drift is the subject of this part. A system can help a person decide without being authorised to decide. The line between those two things is rarely written down, and it is often not where the task's name suggests. A calculation sounds harmless until it becomes the figure in an affidavit.
This series is general information about governing AI in organisations, not legal advice. Where it describes court judgments or legislation, it summarises them for a non-legal audience; read the primary sources and take advice from a qualified legal practitioner before relying on any interpretation for your organisation.
Spekboom Finance is fictional, and its processes and figures are illustrative. Spekboom, the South African consumer lender we follow through this series, uses an AI tool to reconcile customers' claimed payments against its ledger when a customer disputes arrears. In Part 1, a near miss with a non-existent legal authority prompted Spekboom to look at all its AI uses. The reconciliation tool turned out to be the one that worried its Head of Legal most.
Assistance and authority are different things
Some tasks help a person reach a decision. Others decide something about a person. The difference is easiest to see side by side:
| Lower-authority assistance | Consequential authority requiring stronger control |
|---|---|
| Transcribe or classify documents | Determine credibility |
| Build a chronology for review | Decide which evidence is true |
| Reconcile figures | Approve or deny a benefit |
| Find possible sources | Treat generated sources as verified |
| Draft a neutral summary | Decide guilt, liability or sanction |
| Flag anomalies | Accuse a person of misconduct |
The left-hand column is where AI is often genuinely useful and relatively safe, provided a competent person checks the result. The right-hand column is where the decision affects someone's rights, money, reputation or access to a service. Those decisions need stronger control: a named decision-maker, evidence they have actually examined, and a way for the affected person to challenge the outcome.
But the table is a starting point, not a rule. The boundary depends on context. "Reconcile figures" sits in the left column, and as a task it is arithmetic. If its output is the basis for a debt collection step, an insurance payment or a court order, it has real influence over a consequential outcome. Risk follows the consequence and the system's actual influence on it, not the apparently simple name of the task.
How a reconciliation reaches a courtroom
Spekboom's reconciliation tool shows how that happens. When a customer says "I paid, and you have not credited me", a collections agent loads the customer's claimed payments and the account ledger, and the tool matches them and produces a reconciled statement of arrears. Most of the time the agent uses that to explain the position to the customer. In a handful of cases a month, the account ends up in a repossession matter, and the arrears figure goes into an affidavit.
At that point, a figure produced by an AI tool and glanced at by an agent is being placed before a court under oath. Nobody decided that the tool should have that authority. The process simply carried its output forward.
A recent reported judgment illustrates why this matters. In ABSA Bank Limited v Masina (2025/049787) [2026] ZAGPJHC 986, decided in the Gauteng Division in Johannesburg in August 2026, the defendant in a vehicle repossession matter said, according to ITWeb's report, that his payments had not been allocated. As reported by ITWeb, the judgment discloses that a reconciliation of the payments was produced at the judge's request by a tool the reported quotation calls "Chat GBT", and states "I have interrogated this reconciliation and am satisfied that it is accurate" (ITWeb, 7 October 2026). According to the same report, the court found that the payments had been allocated and ordered the return of the vehicle. The judgment itself is on SAFLII.
The disclosure is useful, and the statement that the reconciliation was interrogated is exactly the kind of human check this series argues for. Our point is not about that case's outcome, which we have no basis to question. It is about what an organisation should build into its own processes when a calculation like this can influence a legal outcome. For Spekboom's own reconciliations, a production governance model should also be able to answer:
- What input was supplied to the tool: which payments, from which source, for which period?
- What output did it produce, exactly as produced?
- Which independent records were checked: bank statements, the ledger, payment processor reports?
- Who checked them, and what did they find?
- What was retained, and where?
Those questions are not a criticism of using AI to reconcile figures. Reconciliation is a reasonable use. They are what lets the organisation, a court or the customer test the result later.
An authority matrix for every AI use
Spekboom's response was to write down, for every AI use, what the tool is and is not allowed to do. We recommend the same exercise for any organisation, using a short list of verbs:
- Recommend: suggest an option or a decision.
- Draft: produce text or a document for a person to review.
- Retrieve: find information.
- Calculate: produce a figure.
- Initiate: start a process that affects someone, such as sending a letter or opening a collections step.
- Approve: make the decision.
- Execute: carry out the decision.
For each AI use, record which verbs the tool may perform, which require a person, which human role that person holds, and the point at which reversal becomes difficult.
Spekboom's matrix, simplified, looks like this:
| AI use | The tool may | A person must | Reversal gets hard at |
|---|---|---|---|
| Call summaries | Draft | Approve (the agent edits and saves) | Saving to the customer record |
| Payment reconciliation | Retrieve, calculate, draft | Initiate, approve, execute | Any collections step or affidavit |
| Legal research | Retrieve, draft | Initiate, approve, execute | Sending a letter or filing papers |
| Hardship triage | Recommend, retrieve, calculate, draft | Initiate, approve, execute | Confirming a payment arrangement |
| Staff assistant (enterprise tool) | Retrieve, draft | Everything else | Not used for decisions |
The pattern is deliberate. The tools may help with the work that comes before a decision. From the point where an action affects a customer and becomes hard to undo, a named person decides, using evidence they have examined.
The reconciliation tool got one additional rule, because of where its output can end up: any reconciliation used in an affidavit or court document is independently re-performed from bank statements and the ledger by a person who signs for it. The tool's version can be a starting point. It cannot be the evidence.
Authority should be enforced, not just written
An authority matrix in a policy document is better than nothing. An authority matrix built into the systems is much better. Where the AI tool is connected to other systems, the limits should be enforced outside the model: the reconciliation tool has no permission to open a collections case; the hardship tool can propose an arrangement but cannot confirm one; the research assistant cannot send a letter.
This is the same principle we describe for AI agents in Agents change the risk boundary: the model may propose, and deterministic controls decide what actually happens. A limit that exists only in the instructions given to a model is a suggestion. A limit enforced by permissions is a control.
Watch for drift
Even a well-drawn matrix erodes if nobody looks at it. The signs that assistance is becoming authority are usually visible in ordinary operational data:
- Override rates near zero. If people never change the tool's output, either it is perfect or nobody is really reviewing it.
- Review times falling steadily. Faster can mean familiarity. It can also mean the check has stopped happening.
- Output appearing further downstream. When a tool's output turns up in documents or systems it was never designed for, its influence has grown.
- "The system says" in conversations with customers. When staff explain decisions by pointing at the tool, they are no longer making them.
Spekboom's Head of Risk now reviews these indicators for each AI use every quarter. Part 6 sets out how that fits into a wider set of controls.
Common mistakes
- Judging risk by the task's name. "Reconcile" and "summarise" sound harmless; what matters is where the output ends up.
- Letting output flow downstream unchecked. A figure that is fine for a phone conversation is not automatically fine for an affidavit.
- Writing the authority matrix but not enforcing it. Limits held only in prompts or policy can be bypassed by accident.
- Treating disclosure as the whole answer. Saying that AI was used is useful. Recording what went in, what came out and what was checked is what makes the result testable.
- Ignoring drift. Override rates and review times tell you whether assistance is quietly becoming authority.
Questions to ask this week
- For each AI use in our organisation, which of the seven verbs can it perform today?
- Where does each tool's output end up, including places it was never designed for?
- At what point does each process become hard to reverse, and who decides at that point?
- Are the tool's limits enforced by permissions, or only described in a policy?
- What are the override rates and review times for each use, and when did someone last look?
- If a customer disputed a figure the AI produced, could we show the input, the output and the independent check?
What to take from this part
Assistance is not authority. A tool can help a person decide without being allowed to decide.
Risk follows consequence, not the task's name. A calculation that reaches a court order needs the controls of a consequential decision.
Write an authority matrix, then enforce it. Seven verbs, a named human role, and the point where reversal gets hard.
Watch for drift. Near-zero overrides and falling review times are early signs that assistance has become authority.
Next in the series: The prompt is part of the evidence.
How CloudNala can help
We help organisations map where AI output travels, draw the line between assistance and authority for each use, and build that line into systems through permissions and workflow rather than leaving it in policy. We also set up the operational indicators that show when assistance is drifting into authority. Where a question turns on legal interpretation, we work alongside your legal advisers.
Work with CloudNala
CloudNala helps organisations move from technology ambition to practical execution across cloud, AI, data, platform engineering and digital services.
Whether you are exploring AI, modernising your cloud environment, building a public-sector digital service, or turning an idea into a working MVP, we can help you shape the roadmap and deliver the next step.
Book an AI Readiness Workshop or write to us at consult@cloudnala.co.za