Why APRA Says AI Is Not Just Another Technology Risk

A business approves an AI system, completes a security review, signs off the vendor and goes live.

Six months later, the underlying model has changed. The workflow has gained new tool access. The AI is handling more customer interactions than it did at launch. A vendor dependency has shifted. The original controls are still on file, but nobody has asked the more important question:

What changed after approval?

That is the deeper warning in the Australian Prudential Regulation Authority's 30 April 2026 Letter to Industry on Artificial Intelligence.

APRA's message is aimed at regulated banks, insurers and superannuation trustees, but the lesson reaches much further.

AI cannot always be governed like ordinary software.

Its outputs can be probabilistic. Its behaviour can change as models, prompts, tools and data change. Important dependencies may sit several layers below the vendor a business has contracted with. Autonomous or agentic systems can create new attack paths. And when AI becomes part of a critical operation, organisations need to know how they will keep operating if it fails.

The mistake is not using AI.

The mistake is treating an adaptive, supplier-dependent system as a set-and-forget technology rollout.

What APRA found

In late 2025, APRA conducted a targeted review of selected large banks, insurers and superannuation trustees.

The regulator found that AI adoption was accelerating, but maturity in governance, risk management, operational resilience and assurance varied significantly.

One of APRA's clearest findings was that assurance was not keeping pace with the scale, speed and complexity of AI adoption.

APRA also found that many boards were still developing the technical literacy needed to challenge AI-related risks effectively. Some were relying too heavily on vendor presentations and summaries without enough independent examination of unpredictable model behaviour or the effect on critical operations.

The review also identified a tendency to treat AI risk as just another technology issue.

That approach misses the characteristics that make AI different:

• predictive and probabilistic behaviour
• models that can adapt or degrade
• bias and ethical considerations
• privacy and data risks
• opaque upstream dependencies
• new cyber attack paths
• operational reliance on systems the business does not fully control

The result is a gap between how quickly AI is being deployed and how quickly organisations are adapting their controls.

Why the usual software assumptions start to break

Traditional technology governance often relies on a familiar sequence:

design → test → approve → release → monitor

That model still matters, but it is not enough for AI.

The difference is not that every AI system is unpredictable all the time.

The difference is that the risk surface can keep changing after launch.

A prompt can change.

A model can be replaced.

A new tool can be connected.

The system can receive access to more data.

The use case can expand from internal support to customer-facing decisions.

The surrounding threat environment can change.

A control that was appropriate at launch may no longer be enough six months later.

Lesson 1: boards need enough AI literacy to challenge, not just approve

APRA's letter places clear weight on board understanding.

That does not mean every director needs to become a machine-learning engineer.

It does mean a board should be able to ask better questions than:

Has the vendor told us this is safe?

Useful challenge looks more like:

• What can the system do?
• What can it access?
• Which decisions can it influence?
• What happens when it is uncertain?
• What changed since approval?
• Which upstream providers does it depend on?
• How would we know if its behaviour degraded?
• Who can stop it?
• What happens if the provider becomes unavailable?

This matters because vendor presentations naturally focus on capability.

Boards and executives also need an independent view of failure modes, dependency risk and operational consequences.

Interest in AI's productivity benefits is not the same as readiness to govern it.

Lesson 2: point-in-time assurance is not enough

This may be APRA's most important practical lesson.

Many organisations are built around point-in-time reviews:

• security assessment before launch
• model validation before release
• a risk sign-off
• periodic sample audits
• annual control testing

Those controls still have value.

The problem is that AI systems can change between reviews.

The risk surface may shift because of:

• model updates
• prompt changes
• workflow changes
• new tools or integrations
• expanded permissions
• new knowledge sources
• changing customer behaviour
• provider-side changes
• model drift or degraded outputs

The operating question therefore changes from:

Did we approve this system?

to:

What has changed since we approved it, and do the original controls still hold?

That is why continuous assurance matters.

A good deployment should make important changes visible and give teams a way to monitor:

• behaviour
• exceptions
• review rates
• output quality
• failed handoffs
• unexpected tool use
• policy breaches
• provider changes
• incidents and near misses

For voice AI specifically, this means auditable call workflows where the team can see what was said, what was captured, what changed, and why.

The purpose is not to create endless compliance work.

It is to detect when the system no longer behaves like the system that was originally approved.

Lesson 3: your AI supply chain is bigger than your vendor contract

A business may think:

We use Vendor X.

The real dependency chain may look more like:

Business → SaaS product → AI platform → foundation model → cloud and data dependencies

That matters because the organisation may not control the most important layer.

The SaaS vendor may switch models.

The model provider may change terms, capabilities or availability.

A critical feature may depend on a single offshore provider.

A fourth-party dependency may create security, resilience or concentration risk.

APRA's April letter highlights supplier concentration and opacity as real challenges. Its later remarks at the 2026 AFIA Risk Summit go further, framing frontier AI dependency as a third-party, concentration and sovereign access issue.

For businesses deploying AI, the practical questions are:

• Which model actually powers this use case?
• Can that model change without our approval?
• Which other providers sit underneath the service?
• Where does customer data move?
• Can we substitute the provider?
• How long would substitution take?
• Can the business keep operating while the dependency is unavailable?

Supplier mapping should not stop at the company whose logo appears on the invoice.

Lesson 4: agentic systems create new security paths

APRA explicitly calls out AI-specific attack pathways including prompt injection, data leakage, insecure integrations, exploit injection and misuse of autonomous AI agents.

This is important because many AI systems are no longer limited to generating text.

They can:

• retrieve data
• update records
• call tools
• trigger workflows
• book appointments
• create tasks
• route customers
• write code
• make decisions that affect downstream systems

The more a system can do, the more important its permissions become.

A useful security review should ask:

• What can the agent access?
• What can it change?
• Which actions require approval?
• Can external input manipulate the agent?
• Can the agent expose sensitive data?
• Can it trigger a downstream action without review?
• Are privileges broader than the use case requires?
• Can the system be stopped quickly?

The security model for an AI agent should not be:

It follows the prompt.

Prompts are part of behaviour design.

They are not access control.

Lesson 5: critical AI needs a credible fallback

APRA is especially clear when AI supports critical operations.

A business should know how it will continue operating if the AI:

• becomes unavailable
• starts behaving unpredictably
• loses access to a provider
• produces unacceptable outputs
• is affected by a security incident

A fallback should be more than a sentence in a risk document.

It should answer:

• Who decides the AI should stop?
• How is it disabled?
• What happens to work already in progress?
• Can a human team take over?
• Is the manual process still usable?
• Has the fallback been tested?
• How long can the business operate without the AI?

A kill switch that nobody has tested is not much of a fallback.

This matters even more as AI moves from internal experimentation into customer-facing and decision-critical workflows.

Human oversight has to be designed, not claimed

"Keep a human in the loop" sounds reassuring.

It is also too vague.

A real human-control design should answer:

What does the human review?

• every output
• only high-risk decisions
• low-confidence cases
• complaints
• policy exceptions
• unusual tool use
• failed outcomes

When do they step in?

• before an action
• after an action
• only when a trigger fires
• sampled periodically

What evidence do they receive?

• the original input
• the conversation
• the model output
• captured fields
• the reason for escalation
• the proposed next action

Can they override the system?

And if they do:

• is the override recorded
• can it change future behaviour
• does the workflow continue automatically
• does anyone review repeated overrides

Human oversight should be an operating mechanism, not a slogan.

A practical AI deployment checklist

APRA's letter is written for regulated financial institutions.

But many of the underlying questions are useful for any business deploying AI into real operations.

Visibility

• Can we identify every meaningful AI use case?
• Do we know which ones are customer-facing?
• Do we know which ones support critical operations?

Ownership

• Who owns each AI system?
• Who owns the risk?
• Who can stop the system?

Dependencies

• Which vendor provides the service?
• Which foundation model sits underneath it?
• Are there important fourth-party dependencies?
• Can we substitute the provider?

Behaviour

• What is the system allowed to do?
• What is it prohibited from doing?
• How does it behave when uncertain?
• What happens outside the expected workflow?

Monitoring

• What do we monitor after deployment?
• Can we detect behaviour changes?
• Can we see exceptions, failures and overrides?
• Do we know when the underlying model changes?

Human control

• When must a person review?
• What evidence do they receive?
• Can they override the outcome?
• What happens if nobody reviews?

Resilience

• Can we stop the system?
• Can we roll back a change?
• Can humans take over?
• Has the fallback been tested?

These questions are not a substitute for regulation, legal advice or industry-specific risk management.

They are a way to make AI deployment more deliberate.

What this means beyond financial services

APRA regulates financial institutions, so its expectations should not be casually presented as universal legal requirements.

But the operating lessons travel well.

Any business using AI to:

• speak with customers
• qualify leads
• route work
• access sensitive data
• trigger actions
• support important decisions

should be asking how the system is governed after launch.

This is especially relevant as AI becomes more agentic.

A system that can act needs stronger controls than a system that can only suggest.

A customer-facing AI system needs stronger reviewability than an internal experiment.

A workflow that supports critical operations needs a stronger fallback than a convenience tool.

The level of control should match the consequence of failure.

The real lesson from APRA

The important part of APRA's warning is not that AI is uniquely dangerous.

It is that AI changes the assumptions behind familiar technology controls.

A one-time approval may not be enough.

A vendor summary may not be enough.

A policy may not be enough.

A human-in-the-loop statement may not be enough.

The stronger model is continuous and operational:

know what is deployed → know what it can do → monitor what changes → keep human accountability clear → maintain a credible fallback

That is the shift.

AI should not be governed as a black box.

And it should not be treated as just another technology rollout when the system itself, its dependencies and its behaviour can keep changing after launch.

For the capability, testing and agentic-safety perspective that complements APRA's governance warning, see our guide to what Australia's AI Safety Institute tells businesses about safe AI deployment.

---

Primary sources

• APRA Letter to Industry on Artificial Intelligence (AI), 30 April 2026
• APRA Member Therese McCarthy Hockey's remarks to the 2026 AFIA Risk Summit, 24 June 2026
• APRA's System Risk Outlook, May 2026