When an AI issue becomes an incident
A customer asks a support assistant whether a cancelled service will still be billed. The assistant confidently says no. The approved policy says the opposite, and several customers may have received the same answer overnight.
This constructed scenario starts with uncertainty. The team doesn't yet know whether one answer was wrong or a wider group was affected. It knows customers could act on inaccurate information and the system may repeat it. That's enough to activate the AI incident response plan.
An AI incident is a suspected or confirmed event in which an AI-enabled system causes, or could cause, material harm, loss of control or material business impact. The trigger may be inaccurate customer information, unfair treatment, exposure of data, an unsafe action, an unannounced provider change, an outage or an agent acting beyond its authority. This is an Off Piste operating definition, designed to help a team act while the facts are still forming.
Cybersecurity, privacy and reporting are parallel assessment routes for the right owners and qualified advisers. The playbook coordinates the first business decisions without treating every AI incident as a cyber incident, privacy breach or legally reportable event.
The Australian Government's Guidance for AI adoption implementation guidance asks organisations to include response, recovery and communications in deployment planning for residual risks. It also calls for serious incidents, corrective measures and relevant reporting obligations to be tracked and assessed. The guidance is a governance resource, not a universal legal duty.
The NIST Generative AI Profile connects AI risk management with incident response, recovery, deactivation, stakeholder communication and continual improvement. Together, these sources support a practical rule. Activate the response when potential consequence or loss of control requires coordinated decisions, even before the team knows the root cause.
Prepare the response before anything goes wrong
The first hour works better when ownership and stop controls already exist. Build the playbook into the business's current systems and incident arrangements.
At minimum, keep a current record of:
- live AI uses, owners, providers, connected systems and data boundaries
- the incident lead and alternates who can coordinate the response
- who can pause a workflow, restrict access, disable an action or disconnect a system
- available prompts, outputs, logs, versions, permissions and business records
- supplier contacts, contractual notification routes and evidence duties
- manual service routes and customer continuity arrangements
- privacy, cyber, legal and sector advisers who may need to assess the facts
- communication ownership and the person authorised to approve recovery
Record the owners and activation thresholds in your AI governance policy. For third-party systems, test the supplier escalation and incident route before approval or renewal. A contact name is useful only if the route produces timely help, relevant evidence and a workable suspension or exit option.
The ASD guidance for deploying AI systems securely advises organisations to extend existing incident response and business continuity arrangements to AI systems, define responsibilities and retain ways to block access or disconnect systems during a major incident. These controls support security and continuity. The broader playbook also has to cover harmful output, unfair impact and loss of human control when no attack has occurred.
NIST SP 800-61 Rev. 3 places incident response within wider cybersecurity risk management through preparation, detection, response, recovery and improvement. Off Piste extends that familiar structure to the model, data, workflow, supplier, human oversight and affected-person dependencies found in AI-enabled systems.
Activate a proportionate response
The incident lead needs a fast way to decide whether the support answer can be corrected locally or whether the whole playbook should activate. Two dimensions make that judgement more useful than a vague high, medium or low label.
Consequence considers effects on people, rights, privacy, safety, money, customer trust and essential operations. Reversibility or spread considers whether the team can identify affected cases, stop repetition and repair the outcome. Each business should set its own thresholds according to its services, obligations and risk tolerance.
Consequence and reversibility determine the response
Greater consequence
Activate the playbook
High consequence with bounded spread requires incident leadership, containment and impact assessment
Stop and escalate
High consequence with difficult reversal requires an immediate stop and qualified specialist assessment
Correct and record
Lower consequence with easy reversal can follow a documented local correction and monitoring route
Contain and investigate
Lower immediate consequence with uncertain spread requires containment before the impact grows
Harder to reverse or contain
This Off Piste model is an operating aid, not a legal standard. Each business still needs context-specific response times and reporting categories. In the support scenario, an isolated draft caught before sending may be corrected and recorded. Repeated advice already delivered to customers has wider spread and needs incident leadership, containment and impact assessment. A response involving safety, significant rights, sensitive information or continuing unauthorised action may require an immediate stop and specialist escalation.
The person with stop authority can act without waiting for consensus when the agreed threshold is met. They should record what was stopped, why, who approved the action and which continuity route now applies.
Contain harm without destroying evidence
Containment protects the next customer. Evidence explains what happened and supports later decisions. The team needs both.
Choose the narrowest control that reliably limits further harm. Pause the affected workflow, disable a customer-facing answer, remove tool access, restrict a data source, revert a version or move new cases to a named manual queue. If the system cannot be bounded confidently, disconnect it under the organisation's stop authority.
Before a retry, prompt edit or record correction changes the trail, preserve the original input, output, time, system and model version, instructions, retrieved material, tool calls, permissions, affected records and human decisions. Keep supplier messages and screenshots where they add evidence. Apply normal access, privacy and retention controls to the incident record.
Avoid turning the response meeting into a detailed technical investigation. Once immediate harm is contained, use the companion guide to trace the failed workflow boundary. That guide owns root-cause tracing, repair assignment and outcome retesting. This playbook owns the coordinated business response and recovery decision.
Assess impact through parallel routes
Impact assessment should proceed while technical investigation continues. Start with the people and business outcomes, then route specialist questions to accountable owners.
Identify who received, relied on or was affected by the output. Determine which period, version, channel and customer journey may be involved. Check whether staff overrode the system, whether an unfair pattern affected a group, whether an unsafe action completed and whether a provider or connected system changed. For customer-facing events, audit the affected customer journey to find where information, choice or escalation failed.
Run privacy and security assessments as separate workstreams when the facts justify them. The OAIC's data breach preparation and response guidance explains that a written data-breach response plan should cover containment, assessment and management. It also explains the assessment and notification route under the applicable Australian privacy framework.
An AI incident involving personal information is not automatically an eligible or notifiable data breach. Coverage and obligations depend on the organisation and event. Preserve evidence, involve the responsible privacy owner and seek qualified advice where the assessment or notification duties are uncertain. Apply the same discipline to cyber, consumer, employment and sector-specific questions.
Keep one coordinated incident record while allowing each specialist route to reach its own conclusion. Record affected people and systems, known facts, uncertainty, containment, advice sought, possible obligations and the next decision owner.
Communicate what is known and what happens next
Good incident communication reduces further harm and gives people a usable next step. It also avoids filling gaps with confident guesses.
Internal updates should name what happened, what has been contained, which services are operating, what remains uncertain and who owns the next decision. Supplier escalation should identify the affected configuration and evidence required, then request a clear response time and contact. Keep contractual duties and specialist advice visible to the incident lead.
When customers may be affected, separate four things in plain language. State what's confirmed, what remains uncertain, what the business has done to protect them and when they'll hear more. Provide a human contact and practical remediation route. Give people immediate protective action before root-cause certainty, while keeping speculation out of the update.
The Australian Government implementation guidance links incident reporting with trust, corrective action and relevant obligations. The NIST Generative AI Profile also supports defined stakeholder communication and value-chain contacts. Neither source turns one template into the right answer for every incident. Audience, consequence, confidentiality, accessibility and legal advice shape the message.
Approve recovery with evidence
Recovery is a controlled business decision. A patched prompt or successful test run is only part of the evidence.
Before reopening, confirm that the system has been repaired, narrowed or safely reverted. Restore the required safeguards and data or permission boundaries. Test the original failure, normal cases, relevant edge cases and the manual fallback. Confirm that customer remediation and required communications have owners. Record the residual risk and the monitoring signal that would trigger another stop.
The recovery approver should be named before an incident and independent enough to challenge an optimistic technical update. Their decision should identify the approved version and scope, evidence reviewed, open limits, monitoring owner and next review point. If confidence remains weak, keep the system narrowed or paused.
One live record helps the team connect the first report to that approval. It also prevents critical decisions from disappearing across chat messages, supplier tickets and meeting notes.
Turn the incident into stronger controls
Closure means the response work has accountable follow-through. Record the confirmed causes, affected people, remediation and recovery decision. Then identify which governance threshold, supplier condition, test, monitor, training step or customer safeguard must change.
Turn confirmed failure modes into repeatable release tests with the AI workflow pre-launch testing guide. Update the supplier record when missing notice, access or evidence delayed the response. Record durable owners and revised stop thresholds in the AI governance policy. Assign each action a due date and a way to verify completion.
Return to the constructed support incident. The immediate repair may correct a knowledge source and contact affected customers. The lasting controls may also require a policy-source owner, a test for cancellation questions, monitoring for unsupported billing claims, a clearer human handoff and a supplier escalation route that returns version evidence. Those changes make recurrence easier to detect and contain.
A growing business is ready when it can identify live AI uses, activate a named lead, stop the right function, preserve useful evidence, assess obligations through qualified routes and approve recovery from recorded evidence. If those decisions remain unclear, an AI incident-readiness and workflow review can map the systems, owners, data boundaries, stop controls, evidence access, communication routes and recovery tests before a real customer becomes the test case.