An AI system can be technically impressive and still be the wrong system to deploy, the wrong system to use in a particular context, or the wrong system to allow to take a particular action. AI impact assessment, risk prioritization, and risk acceptance are the governance disciplines that turn that observation into a documented decision: what could happen, to whom, how serious it could be, how likely it is, what safeguards are required, who has authority to approve the remaining exposure, and when that decision must be revisited.
This matters because AI risk is not confined to the model. A deployed AI system may combine a model provider, enterprise data, retrieval, application logic, user interfaces, tools, identity permissions, agents, human operators, and downstream business processes. A model that performs well in isolation can create a very different risk profile when its output is used to update a record, recommend an action, communicate with another person, or trigger an external process.
Consider this fictional scenario: Northstar, an internal document-answering assistant, is initially read-only. Later, the organization proposes adding two narrowly scoped capabilities: updating a service record and sending an internal message. The model has not suddenly become a different model. The system has become more consequential. The governance question therefore changes.
- Why impact assessment matters
- Ethics, governance, risk, security, safety, assurance, and compliance
- Define the system boundary and accountability
- How to perform an AI impact assessment
- How to prioritize AI risks
- How risk acceptance should work
- Residual risk, monitoring, and reassessment
- Implementation in practice
- Common mistakes
- FAQ
- References & Further Reading
- Summary
| Activity | Main question | Typical output | Decision consequence |
|---|---|---|---|
| Impact assessment | Who or what could benefit or be harmed, in this context? | Impact map, assumptions, affected groups, scenarios, alternatives | Shapes system scope, safeguards, evaluation, and deployment conditions |
| Risk assessment | What uncertain events could produce those harms or benefits? | Risk statements, causes, likelihood, impact, evidence, controls | Creates a basis for treatment and monitoring |
| Risk prioritization | Which risks need attention first and why? | Priority, treatment order, escalation level, resource allocation | Determines where limited governance and engineering capacity goes |
| Risk acceptance | Who is authorized to knowingly retain the remaining exposure? | Explicit acceptance record, conditions, expiry or review trigger | Creates accountable permission to proceed within defined limits |
01
Why AI Impact Assessment Comes Before Risk Scoring
A common governance mistake is to start with a spreadsheet: probability, severity, score, red or green status, approval. That looks objective, but it can create false confidence when the underlying question has not been defined.
Imagine deciding whether a child should cross a road. Before calculating how likely an accident is, you need to know which road, where the child will cross, how fast the traffic moves, whether an adult is present, and what happens if something goes wrong. A number without the situation is not useful safety information.
An AI impact assessment asks about the consequences of using the system in a particular context. It examines intended benefits, foreseeable negative impacts, affected people, organizational consequences, operating assumptions, alternative approaches, and the conditions under which the system will be used.
NIST's AI Risk Management Framework explicitly treats context, intended purpose, potential positive and negative impacts, users, affected communities, and organizational risk tolerance as part of AI risk management. It also emphasizes that impacts can change as systems, contexts, and capabilities evolve. NIST AI RMF — Framing Risk
Do not ask only, “How risky is the model?” Ask, “What is this complete AI system allowed to do here, to whom, under what conditions, and with what consequences?”
This is especially important for agents. A read-only assistant can create misinformation or privacy problems, but an agent with permission to modify records, approve transactions, send messages, or invoke external tools adds another layer of consequence. The assessment therefore needs to follow the action boundary, not stop at the model boundary.
Version A: Northstar retrieves approved internal documents and produces answers. Users still perform all business actions themselves.
Version B: Northstar can update a service record and send an internal message after interpreting a user's request.
The model may be unchanged. The system context, permissions, possible failure paths, affected people, reversibility, and control requirements are not. Version B therefore deserves a fresh impact and risk assessment rather than a simple extension of Version A's approval.
A proposed AI feature changes who is affected, what data is used, what the AI can influence, what actions can happen automatically, or how difficult a failure would be to reverse.
02
Do Not Collapse Ethics, Governance, Risk, Security, Safety, Assurance, and Compliance
These terms frequently appear together in AI discussions, but they answer different questions. Confusing them creates weak ownership. A security test may demonstrate resistance to a particular attack without answering whether the business should use the system in that context. A policy may state that human review is required without proving that the human review actually happens. An impact assessment may identify harm without deciding whether the residual risk is acceptable.
| Concept | Core question | Typical evidence |
|---|---|---|
| AI ethics | What values, interests, rights, or consequences should guide the decision? | Principles, stakeholder input, impact findings, value trade-offs |
| AI governance | Who has authority, what rules apply, and what evidence is required? | Policies, role assignments, approvals, decision records |
| Risk management | What uncertainty could cause harm or loss, and how should it be treated? | Risk register, assessment, treatment plan, residual risk |
| Security | How could unauthorized or adversarial behavior compromise the system or its data? | Threat models, tests, access controls, telemetry, incidents |
| Safety | How could unintended system behavior cause harmful outcomes? | Safety analyses, boundary tests, fail-safe controls, incident evidence |
| Assurance | What evidence supports the claim that a control or system behaves as intended? | Evaluation results, audit evidence, test records, monitoring results |
| Legal compliance | What binding requirements apply to this system, organization, role, sector, and jurisdiction? | Applicable law, regulatory guidance, legal analysis, mandated records |
NIST AI RMF is a voluntary framework rather than universal law, and NIST explicitly says risk tolerance is contextual rather than prescribing a single universal threshold. ISO/IEC 42001 is an international AI management-system standard, while ISO 31000 provides general risk-management guidance. These sources can help structure a governance program, but applicability and legal effect depend on the organization's situation. NIST AI RMF · ISO/IEC 42001:2023 · ISO 31000:2018
Risk → A team treats successful penetration testing as proof that deployment is acceptable.
Control → Require a separate business and impact decision covering purpose, affected people, consequences, alternatives, and residual risk.
Evidence → Approved impact assessment, security evaluation, risk register, and decision record.
Remaining risk → Some societal, operational, or contextual harms may remain even when security controls work as designed.
03
Define the AI System Boundary and Assign Accountability
An impact assessment becomes weak when the assessor cannot say what exactly is being assessed. “Our chatbot” is usually too vague. The relevant system may include the model, retrieval layer, enterprise records, prompts or instructions, tools, identity controls, monitoring, human reviewers, operational processes, and downstream decisions.
Think of a car engine versus a complete vehicle on a public road. The engine matters, but the real-world outcome also depends on steering, brakes, road conditions, driver behavior, traffic rules, maintenance, and the environment. The same distinction applies to AI governance: model-level evidence is important, but deployment-level decisions require system-level evidence.
NIST notes that different AI actors can hold different perspectives and responsibilities across the lifecycle. The developer of a general model may not know how a deployer will integrate it, while the deployer may create risks that do not exist in the model's original context. The assessment should therefore identify the roles that actually control the relevant risks. NIST — Framing Risk
| Role | Governance question | Example evidence |
|---|---|---|
| Provider | What does the upstream component claim and what evidence supports those claims? | Technical documentation, service terms, model information, provider test evidence |
| Developer | How was the application designed and what controls are built into it? | Architecture, evaluation, access rules, test results, change history |
| Deployer / system owner | Is this system appropriate for this purpose and organizational context? | Impact assessment, risk decision, deployment conditions |
| Operator / control owner | Are controls actually operating after deployment? | Logs, review records, monitoring results, incident records |
| Risk owner | Who has authority to accept the defined residual risk? | Risk acceptance record and delegated authority |
| Affected person or group | What consequences can occur to people who are not the person operating the system? | Impact findings, complaints, feedback, outcome analysis, remedy records |
The boundary should also identify what is deliberately outside the decision. For example, an organization may assess an internal knowledge assistant but separately govern the underlying HR database, enterprise identity service, third-party model provider, and corporate messaging platform. Those dependencies are not irrelevant simply because another team owns them.
The fictional Northstar assessment includes: the model service; approved internal documents used for retrieval; the retrieval and ranking layer; application instructions; the user interface; identity and role permissions; two business tools; human review for selected actions; audit logging; incident handling; and the downstream service-management process.
The assessment does not treat the model provider's general documentation as proof that Northstar itself is safe. Provider evidence is an input. The deployment decision belongs to the organization operating the complete system in its own context.
Different teams own the model, data, application, identity, business process, or human review. Write the boundary before assigning risk scores so every risk has an identifiable owner.
04
How to Perform an AI Impact Assessment
An impact assessment is not simply a checklist of famous AI principles. It is a structured investigation of consequences in a particular context. The goal is to expose assumptions before those assumptions become expensive incidents.
NIST's AI RMF includes context establishment, potential beneficial and negative impacts, organizational risk tolerance, stakeholder perspectives, and consideration of non-AI alternatives. It also emphasizes that impact measurement may be difficult and can change across lifecycle stages. NIST AI RMF Core
A practical sequence is:
- State the purpose. What business or user problem is the AI intended to solve?
- Test whether AI is actually needed. Could a rule, workflow, search system, human process, or simpler automation achieve the objective with less risk?
- Identify affected people and stakeholders. Include people who do not directly use the AI but may experience its outputs.
- Map intended uses and foreseeable misuse. Consider what users are expected to do and what reasonable users may attempt anyway.
- Describe possible benefits and harms. Cover direct and indirect effects, not only technical accuracy.
- Assess severity and reversibility. A small but irreversible harm can deserve more attention than a frequent but easily corrected inconvenience.
- Record uncertainty. Identify where evidence is weak, measurement is incomplete, or real-world behavior is not yet understood.
- Connect impacts to controls. Every material concern should lead to a treatment decision, an evidence plan, or an explicit reason for accepting the exposure.
- Decide what must be true before deployment. Convert abstract concerns into testable conditions and named owners.
Imagine a new bridge. You do not ask only whether the bridge can carry weight. You ask who will use it, where they will enter, what happens during rain, what happens if a lane closes, who is nearby, how rescuers will reach it, and what happens if the bridge becomes unavailable. An AI impact assessment similarly follows the system into its real environment.
One useful way to make an impact assessment concrete is to classify impacts by the characteristics that change their governance significance.
| Impact lens | Question | Why it changes the decision |
|---|---|---|
| Who benefits? | Who gains time, access, accuracy, convenience, or capability? | Benefits help explain value but do not automatically justify risk. |
| Who bears harm? | Who could lose money, opportunity, privacy, safety, time, reputation, or control? | The risk owner may not be the person who experiences the consequence. |
| Direct or indirect? | Does the AI directly affect someone or influence another process that later affects them? | Indirect effects are easy to miss during a narrow technical review. |
| Reversible? | Can the harm be corrected quickly and completely? | Hard-to-reverse consequences deserve stricter controls. |
| Known or uncertain? | How strong is the evidence behind the assessment? | Uncertainty may itself justify staged deployment or stronger monitoring. |
Northstar's read-only version primarily creates informational risks: incorrect answers, misleading summaries, inappropriate disclosure, or users relying on unsupported conclusions.
The proposed record-update capability introduces a new impact pathway: an incorrect interpretation can change operational data. The proposed messaging capability adds another: the system could communicate an incorrect or inappropriate statement to another employee.
The assessment therefore recommends different controls for the two tools rather than one universal “human in the loop” statement.
Risk → The assessment considers only direct users and ignores people affected downstream.
Control → Require an explicit affected-person and downstream-process review for each material use case.
Evidence → Affected-stakeholder map, scenario analysis, complaints or feedback path, and documented assumptions.
Remaining risk → Some indirect consequences may remain difficult to predict, especially when the system is reused outside its original purpose.
Stakeholders disagree about whether a use case is “low risk.” Move the discussion from labels to consequences, affected people, reversibility, uncertainty, alternatives, and operating conditions.
05
How to Prioritize AI Risks Without Pretending the Score Is the Truth
Risk prioritization is about attention and resource allocation. It does not mean every risk can be reduced to one mathematically precise number.
NIST describes AI risk in terms of the magnitude of negative impact and likelihood, while also emphasizing that measurement can be difficult, context-dependent, incomplete, or misleading when reduced to oversimplified metrics. It also states that risk prioritization should reflect context and organizational risk tolerance. NIST — Risk Tolerance and Risk Prioritization
Imagine ten problems at an airport. A broken vending machine, a long security queue, a lost passport, and a fire alarm are all “problems,” but they should not receive the same response. Risk prioritization is deciding which problem deserves immediate attention, which can wait, and which requires stopping the normal process entirely.
A useful professional risk statement is more specific than “hallucination risk.” Instead, write:
If Northstar misinterprets the user's request and invokes the record-update tool, an incorrect service record may be written, creating operational disruption or requiring manual correction.
That structure identifies the cause, event, and consequence. It can then be evaluated against evidence rather than vague categories.
An illustrative 1–5 scoring model
Organizations often use simple matrices because decision-makers need a common language. The following model is intentionally fictional and should not be treated as a NIST, ISO, regulatory, or universal scoring formula.
| Impact level | Illustrative meaning | Typical governance response |
|---|---|---|
| 1 — Minimal | Small, recoverable inconvenience | Routine controls and monitoring |
| 2 — Limited | Noticeable disruption or localized harm | Documented treatment and owner |
| 3 — Material | Meaningful operational, financial, privacy, safety, or human impact | Formal treatment, stronger evidence, review gate |
| 4 — Severe | Serious harm or difficult recovery | Senior risk authority, independent review, strong controls |
| 5 — Critical | Catastrophic, prohibited, or otherwise unacceptable consequence in context | Stop, redesign, or obtain the applicable formal authority before proceeding |
Likelihood can be scored similarly, but evidence quality matters. “We think this happens rarely” is not equivalent to production telemetry, controlled testing, incident history, domain evidence, or a strong empirical estimate. Where evidence is weak, the uncertainty should be documented rather than hidden inside a reassuring score.
A practical prioritization decision can therefore consider several dimensions:
- Potential magnitude of harm
- Likelihood or credible scenario frequency
- Reversibility and recovery time
- Number and vulnerability of affected people
- Degree of uncertainty in the evidence
- Ability to detect the failure before harm occurs
- Whether a legal, regulatory, contractual, safety, or policy boundary changes the decision
- Whether the risk is concentrated in a small group that has little ability to protect itself
| Risk | Illustrative score | Priority decision |
|---|---|---|
| Incorrect answer in read-only document Q&A | Impact 2 × likelihood 3 | Strong answer-quality evaluation, source visibility, user reporting |
| Incorrect service-record update | Impact 4 × likelihood 2 | Higher priority because the action changes durable business data |
| Incorrect internal message sent to another employee | Impact 3 × likelihood 2 | Require approval for specified message types and audit the decision path |
The scores are invented solely to demonstrate prioritization. A real organization should define its own criteria and authority model.
Notice an important governance point: the highest numeric score is not automatically the only risk worth discussing. A low-frequency risk with a severe and irreversible consequence may deserve stronger treatment than a frequent but trivial error. Likewise, a risk with poor evidence may deserve staged deployment until evidence improves.
Risk → A single numerical score hides uncertainty, reversibility, and affected-group differences.
Control → Require narrative justification for material risks and identify evidence quality separately from the score.
Evidence → Scoring criteria, rationale, source evidence, control effectiveness results, and reviewer comments.
Remaining risk → Any scoring model remains a simplification. Governance judgment is still required.
06
Risk Acceptance Is a Decision, Not a Checkbox
Organizations cannot usually eliminate every uncertainty. The governance challenge is to distinguish unavoidable residual exposure from unexamined exposure.
NIST's AI RMF describes risk-response options that include mitigating, transferring, avoiding, and accepting, and its Playbook says residual risks should be documented. It also emphasizes that risk tolerance is contextual and should align with applicable requirements and organizational governance. NIST AI RMF Playbook — Manage
Imagine a building manager knows that a small maintenance problem cannot be repaired until tomorrow. “Accepting the risk” does not mean pretending the problem does not exist. It means someone with appropriate authority knows about it, understands the consequence, confirms that the situation is within the permitted limit, defines what happens next, and knows when the decision expires.
A professional risk-acceptance process should pass several gates.
- Applicability gate: Is the proposed activity allowed under applicable law, regulation, contract, policy, safety requirements, or other mandatory constraints? A risk cannot simply be “accepted away” when a binding rule prohibits the conduct.
- Context gate: Is the residual risk understood in the actual deployment context rather than in a generic model description?
- Treatment gate: Have reasonable mitigation, avoidance, transfer, redesign, human-review, or non-AI alternatives been considered?
- Authority gate: Does the person accepting the risk have the delegated authority to do so?
- Evidence gate: Is there sufficient evidence to explain why the remaining risk is within the defined tolerance?
- Monitoring gate: Are measurable triggers defined for escalation, suspension, redesign, or withdrawal of the acceptance?
System: Northstar
Risk ID: NS-RA-004
Residual risk: An approved user may still cause a low-frequency incorrect record update after all defined controls operate as intended.
Treatment completed: permission scoping; constrained tool parameters; confirmation step; audit logging; targeted evaluation.
Risk owner: Service Operations Owner
Acceptance authority: Assigned business risk authority under internal delegation
Decision: Accepted for the defined use case only.
Conditions: Tool scope remains unchanged; monitoring remains active; monthly review of tool-related incidents and near-misses.
Reconsideration trigger: Material tool change, new affected population, control failure, repeated near-miss, or material incident.
Expiry / review date: 2027-01-05
Evidence references: impact assessment, evaluation report, control-test results, monitoring plan.
The record above deliberately includes both a decision and a scope. “Risk accepted for Northstar” is too broad. “Risk accepted for this specific version, use case, permission set, and operating condition” is much more defensible.
Acceptance should also be distinguished from approval. A project manager may approve a release while a designated risk owner accepts a defined residual risk. Those may be the same person in a small organization, but they do not have to be.
Risk → The application owner clicks “approved” and the record is treated as formal risk acceptance.
Control → Separate release approval from residual-risk acceptance and enforce delegated authority thresholds.
Evidence → Signed or attributable decision record, authority mapping, residual-risk statement, conditions, review date.
Remaining risk → Even a valid acceptance decision does not make an undesired event impossible and does not replace incident response.
A team says, “There is always some risk.” Ask the next four questions: What remains? Why does it remain? Who is authorized to accept it? What would cause us to stop accepting it?
07
Residual Risk Must Survive Deployment, Change, and Reality
Risk acceptance is not permanent ownership of a risk for all future versions. AI systems evolve. Models change. Retrieval sources change. user populations change. Tool permissions expand. Business processes change. A control that was effective at deployment may become ineffective after a seemingly unrelated change.
NIST's AI RMF describes risk management as continuous across the AI lifecycle and its Playbook calls for post-deployment monitoring, feedback, incident handling, change management, and reassessment. NIST AI RMF Core · NIST AI RMF Playbook — Manage
A practical reassessment model has two parts: scheduled review and event-driven review.
| Trigger | Example | Governance response |
|---|---|---|
| Model change | Provider model or version changes | Re-evaluate relevant performance and risk assumptions |
| Data change | New source type, retrieval corpus, retention rule, or user data | Revisit impact, privacy, quality, access, and downstream effects |
| Permission change | New tool or broader action scope | Re-open the impact assessment and action-risk analysis |
| Incident / near-miss | Unexpected tool invocation or harmful output | Investigate, contain, update risk treatment, and reconsider acceptance |
| Context change | New geography, population, business process, or use case | Repeat context and impact analysis before treating the old decision as valid |
This is where residual risk becomes operational rather than theoretical. NIST's Playbook describes residual risks as those remaining after treatment and recommends documenting risks that are accepted, transferred, or minimally mitigated. NIST AI RMF Playbook — Manage 1.4
Northstar's risk owner accepts the remaining risk of occasional incorrect record updates only while the tool remains limited to two fields, confirmation remains mandatory for specified actions, monitoring remains active, and the affected service group remains unchanged. A future release that adds automatic multi-record updates is not covered by the old acceptance. The old record is evidence of a previous decision, not permission for the new capability.
A good reassessment trigger is therefore often more useful than a vague instruction to “review annually.” Calendar reviews are valuable, but major changes should trigger review immediately.
Risk acceptance has a scope, owner, evidence base, operating conditions, and review trigger. Without those elements, it is closer to an informal opinion than a governance decision.
08
Implementation in Practice: From Idea to Defensible Decision
The following workflow is an original practical implementation pattern rather than a mandated sequence from any one framework. It can sit inside an organization's existing enterprise-risk, product, compliance, technology-risk, or AI-management process.
Step 1 — Create a minimum system record
Purpose: Internal service-information assistant
Users: Authorized service employees
AI capability: Retrieval-augmented question answering
External actions: Record update, internal message
Model provider: [PROVIDER_NAME]
Data sources: Approved internal knowledge repositories
Human involvement: User confirmation for defined actions
System owner: Service Operations
Risk owner: [ROLE / DELEGATED AUTHORITY]
Assessment status: Draft / Reviewed / Approved / Suspended
Step 2 — Write an impact statement before scoring risk
Reduce time required for employees to locate approved service information.
Potential negative impact:
Incorrect or incomplete information may be treated as authoritative by a user.
Additional impact from enabled action:
An interpretation error may produce an incorrect change to an operational record or an inappropriate internal message.
Key affected parties:
Direct users; employees whose records or communications may be affected; operations teams responsible for recovery.
Step 3 — Build the risk register
| Risk ID | Risk | Control | Evidence |
|---|---|---|---|
| NS-R01 | Incorrect factual answer | Approved source retrieval, evaluation set, user feedback | Evaluation report and monitoring results |
| NS-R02 | Incorrect record modification | Least-privilege tool, validation, confirmation, logging | Control tests, tool audit, action logs |
| NS-R03 | Inappropriate internal message | Message scope restrictions, confirmation trigger, audit record | Scenario tests and sampled review |
A risk register is more useful when each row can answer five questions without a meeting: what can happen, why can it happen, who may be affected, what control reduces it, and what evidence tells us the control is working?
Step 4 — Evaluate the controls, not the policy sentence
“Human review required” is not a sufficient control description. A real control must specify the trigger, the person or system responsible, the information available to the reviewer, the decision they must make, what happens when they reject the action, and how operation of the control is verified.
Trigger: Any Northstar request that would modify a service record.
Required reviewer: Authorized service user.
Review evidence: Proposed field changes + source context + requested action.
Allowed outcomes: Confirm / Reject / Escalate.
Enforcement point: Application blocks tool execution until confirmation is recorded.
Test: Sample action requests monthly and verify no write occurs without the required confirmation event.
Escalation: Repeated control bypass or unexpected tool use suspends the capability pending review.
Step 5 — Record residual risk after treatment
Controls reduce risk; they do not necessarily eliminate it. The residual-risk statement should be written after the treatment is considered, not copied from the original assessment.
Step 6 — Decide: avoid, reduce, transfer, accept, stage, or stop
Staged deployment is often valuable when evidence is incomplete. A read-only pilot with restricted users may provide information that a broad launch cannot. Likewise, removing an unnecessary autonomous action can be a stronger risk response than adding another monitoring dashboard.
Step 7 — Attach review triggers to the decision
Decision: Deploy the read-only assistant broadly within the approved employee population.
Decision with conditions: Enable record-update actions only for a smaller authorized population after the defined control tests pass.
Decision not yet accepted: Automatic outbound messaging remains disabled until message categories, approval requirements, and evaluation evidence are sufficient.
This illustrates an important governance pattern: one AI system can contain multiple risk tiers and therefore multiple deployment decisions.
For security-specific inputs, an organization may use relevant threat knowledge alongside its broader risk process. For example, OWASP's 2025 guidance addresses risks in LLM applications and its Agentic AI material addresses threats associated with systems that can plan and act; MITRE ATLAS provides a living knowledge base of adversary tactics and techniques involving AI systems. These resources can enrich risk identification, but they do not by themselves determine whether a particular deployment should proceed. OWASP Top 10 for LLM Applications · OWASP Top 10 for Agentic Applications · MITRE ATLAS
Risk → The organization approves the AI system as one indivisible unit.
Control → Segment decisions by use case, data sensitivity, action scope, affected population, and consequence level.
Evidence → Capability inventory, separate risk records, control tests, approval conditions, and deployment scope.
Remaining risk → A future capability can cross a decision boundary unexpectedly unless change management detects it.
09
Common Mistakes and Their Corrections
Mistake 1 — Starting with the score instead of the context
Cause: Risk matrices are familiar and easy to complete.
Consequence: Teams debate whether something is “3 or 4” before agreeing on what the system actually does.
Correction: Complete system boundary and impact analysis first; score only after the scenario is clear.
Mistake 2 — Treating a vendor claim as system-level assurance
Cause: Provider documentation is often easier to obtain than deployment-specific evidence.
Consequence: Evidence for a model is mistaken for evidence that the integrated application is appropriate for a particular context.
Correction: Treat provider evidence as one input, then test the integrated system with its own data, permissions, tools, users, and operational conditions.
Mistake 3 — Calling a human click “meaningful oversight”
Cause: The workflow records a confirmation event but does not verify whether the person could understand or challenge the proposed action.
Consequence: The human becomes a rubber stamp rather than an actual control.
Correction: Give the reviewer relevant context, a clear decision boundary, a reject or escalate path, appropriate time, and monitoring that tests whether the control is genuinely functioning.
Mistake 4 — Ignoring people who do not use the AI
Cause: The assessment is written entirely from the operator's perspective.
Consequence: Downstream subjects, employees, customers, or communities may bear the harm without having a voice in the decision.
Correction: Map affected people separately from direct users and identify feedback, contestability, remedy, and escalation mechanisms where relevant.
Mistake 5 — Accepting residual risk without defining the residue
Cause: “Low risk” becomes a conclusion rather than a statement about what remains after controls.
Consequence: No one can tell what was actually accepted.
Correction: Write the specific remaining event, consequence, affected party, evidence, owner, conditions, and review trigger.
Mistake 6 — Treating the assessment as a one-time document
Cause: Governance is attached to the original launch date rather than to system change.
Consequence: Old evidence continues to justify a materially changed system.
Correction: Tie reassessment to model changes, data changes, new tools, new populations, incidents, near-misses, business changes, and other material triggers.
Risk → Governance artifacts exist, but nobody can prove that the documented controls operate in production.
Control → Pair every material control with a verification method and owner.
Evidence → Test results, monitoring data, sampled transactions, review records, incident data, or other objective evidence appropriate to the control.
Remaining risk → Evidence can show control operation over a defined period; it cannot guarantee future performance under every condition.
10
❓ FAQ
An impact assessment focuses on who or what may benefit or be harmed and how those consequences arise in a particular context. A risk assessment then examines uncertain events that could produce those outcomes and supports treatment and monitoring decisions. They overlap, but the impact assessment should not be reduced to a simple risk score.
Record the uncertainty instead of hiding it inside a precise-looking number. Use credible scenarios, domain evidence, testing, expert judgment, and conservative treatment where the potential impact is severe or difficult to reverse. A lack of measurement does not prove that the risk is low.
The person or governing body with appropriate delegated authority for that class of risk and decision should accept it. The application developer, project manager, technical reviewer, or business owner may participate, but participation is not automatically the same as risk-acceptance authority.
Sometimes, depending on the context, applicable requirements, consequence, control effectiveness, and organizational tolerance. However, external actions increase the importance of permission boundaries, action-specific testing, monitoring, reversibility, and explicit approval conditions. Risk acceptance should never substitute for a required safety, security, legal, or policy control.
Use both scheduled reviews and event-driven triggers. Reassessment is especially important after material model, data, permission, tool, user-population, business-process, or regulatory changes, and after incidents or meaningful near-misses. An old acceptance should not automatically cover a materially changed system.
11
🔗 References & Further Reading
- NIST AI Risk Management Framework — NIST's official AI RMF resource page. As of October 5, 2026, NIST states that AI RMF 1.0 is being revised and that the Playbook will be updated after the revision.
- NIST AI RMF — Framing Risk — used for risk, impact, risk tolerance, prioritization, residual-risk, lifecycle, and context concepts.
- NIST AI RMF Core — used for the Govern, Map, Measure, and Manage functions and the continuous lifecycle approach.
- NIST AI 600-1 — Generative Artificial Intelligence Profile — published July 26, 2024 and updated April 8, 2026; used as relevant background for generative-AI-specific risk management.
- ISO 31000:2018 — Risk management — Guidelines — used for general risk-management concepts. ISO currently lists the 2018 edition as the published version while a third edition is under development.
- ISO/IEC 42001:2023 — Artificial Intelligence — Management System — used to distinguish an AI management-system standard from a universal risk-acceptance rule. Clause-level requirements are not reproduced here.
- OECD AI Principles — adopted in 2019 and updated in 2024; used for accountability and ongoing risk-management context.
- OWASP Top 10 for LLM Applications 2025 — used only as an example of security-risk input relevant to AI applications.
- OWASP Top 10 for Agentic Applications — used as a current security-risk input for agentic systems.
- MITRE ATLAS — used as a current example of an AI-specific adversarial threat knowledge base.
Framework, standard, project, and organization names belong to their respective owners. This article does not establish legal compliance, certification, or a universal definition of acceptable AI risk.
12
📝 Summary
Impact assessment: Understand the purpose, affected people, benefits, harms, alternatives, assumptions, uncertainty, and real-world context before relying on a risk score.
Risk assessment: Turn concrete impact pathways into explicit risk statements with causes, consequences, evidence, and controls.
Risk prioritization: Direct scarce governance and engineering attention toward the risks that matter most in the specific context.
Risk treatment: Prefer avoiding, redesigning, reducing, transferring, staging, or otherwise treating risk before accepting what remains.
Risk acceptance: Name the residual risk, owner, authority, evidence, operating conditions, monitoring, and review trigger.
Lifecycle governance: A past approval is not permanent permission for a materially changed AI system.
The mature AI governance question is not “Did we complete the assessment?” It is “Can we explain, with evidence, why this AI system is appropriate for this use, what remains uncertain, who owns the decision, what could make us stop, and how we will know when that decision is no longer valid?”
Comments
Post a Comment