AI risk management for engineers, not auditors

AI risk management is the practice of identifying what an AI system can break, deciding which of those failures you will prevent, detect, or accept, and putting a name next to each decision. That is the whole discipline. The reason it fills frameworks and conference tracks is that most organizations skip one of the three parts: they identify risks without deciding anything, or they decide without an owner, or they assign owners to risks nobody actually enumerated.

I spend a good part of my week in this territory as an enterprise architect in an AI enablement group, and the pattern I see most often is not recklessness. It is paperwork. A forty-row spreadsheet gets produced for the steering committee, everyone attends the review, and then the system ships and evolves while the spreadsheet stays exactly as it was. The assessment was an event. Risk is not an event.

The fix is not more process. It is treating AI risk the way engineering already treats every other unreliable dependency: name the failure modes, build controls proportional to blast radius, watch the system in production, and keep a decision trail. What follows is the version of that I can defend in front of both a delivery team and an audit committee.

A risk taxonomy an engineer can use

Start with a taxonomy, because “AI risk” as a single category produces conversations that go nowhere. Five domains cover essentially everything I encounter in enterprise systems. The value of the split is that each domain has a different owner, a different detection mechanism, and a different control set, which is exactly what a taxonomy is for.

Domain	What fails	Typical owner	Primary detection
Model behavior	Wrong, fabricated, biased, or degraded outputs	Product engineering	Eval suites, production sampling
Data	Leakage, poisoning, lineage gaps, retention violations	Data platform / engineering	Data flow mapping, access audits
Security	Prompt injection, tool abuse, artifact tampering, exfiltration	Security engineering	Adversarial testing, runtime detection
Operational	Provider outages, cost runaway, latency, silent dependency changes	Platform / SRE	Telemetry, budgets, SLOs
Legal and compliance	Regulatory exposure, IP contamination, contractual breach, records obligations	Legal with engineering input	Inventory review, counsel checkpoints

Three observations from using this in practice.

Model behavior risk is the one everyone discusses and few can measure. Hallucination, drift, and bias are real, but a risk you cannot measure is a risk you cannot manage, only worry about. The prerequisite control is an eval suite tied to defined expectations, which is a solved engineering problem with an unsolved adoption problem. The mechanics are covered in evaluating AI systems in production; for risk purposes the point is simpler: until evals exist, every entry in the model behavior column is a guess.

Data risk is where the actual incidents live. Most AI failures I have seen discussed candidly were not model failures. The model did what it was built to do, with data it should never have had: a retrieval corpus indexed without permission checks, prompts logged into a store half the company could read, customer data sent to a provider under the wrong contract terms. Data risk is unfashionable because it is plumbing. It is also, in my experience, the domain most likely to produce the incident that gets executives involved.

Operational risk is the one nobody assesses because it feels mundane. A frontier API provider having a bad day is not a hypothetical; it is a weather condition. So is a model version being deprecated under you, a retry storm turning a per-token price into a budget incident, or latency quietly doubling after a provider-side change. These are classic availability and cost problems wearing a new jacket, and the SRE toolkit applies almost unchanged. They belong in the register precisely because they are boring, boring risks are the ones that skip review.

Security risk gets its own article in securing LLM applications, and the OWASP LLM Top 10 and MITRE ATLAS are the right external references for enumerating it. The one point worth repeating here: a system that can be steered by anyone able to write text into its context window has a security problem, not a quality problem, and it needs to be assessed by people who think in attack paths.

Risk assessment that isn’t paperwork

An assessment is useful exactly to the degree that it changes a decision. That gives you a simple test for every artifact the process produces: which decision does this row change? If the answer is none, delete the row.

The version that works fits inside design review rather than existing as a parallel process. For each AI system, before it ships and at every significant change, the team answers five questions in writing:

1. What is the worst plausible output or action, and who does it reach? Not the worst imaginable, the worst plausible. An internal drafting assistant whose output a human edits has a different answer than an agent that emails customers.
2. What data can the system see, and what can it emit? Draw the flow. One page. If nobody can draw it, that is the finding.
3. Who can influence the system’s inputs? Users, document authors, upstream systems, the internet. Everyone on that list is inside your threat model whether you invited them or not.
4. What happens when the model is unavailable or wrong? Degraded mode, human fallback, or outage. “We had not considered unavailability” is a common and honest answer, and it is why the question is on the list.
5. Which failures will we prevent, which will we detect, and which will we accept? This is the row that changes decisions. The other four exist to make this one answerable.

Score severity by blast radius, reversibility, and reach, and resist the urge to invent a 25-cell heat map with false precision. Three severity tiers are enough: failures that end up in front of a regulator or a customer, failures that cost real money or time internally, and failures that are annoying. Tier one gets prevention controls and human gates. Tier two gets detection and alerting. Tier three gets accepted, in writing, which is not negligence, it is the entire point of doing the analysis.

The output is a risk register per system: a short list of named failure modes, each with a decision (prevent, detect, accept), a control or monitor implementing that decision, and an owner. If the register for a consequential system fits on one page, that is not a sign of shallow analysis. It is a sign the taxonomy did its job.

Mapping to the NIST AI RMF without drowning in it

The NIST AI Risk Management Framework organizes the discipline into four functions: Govern, Map, Measure, Manage. It is voluntary, sensible, and written at a level of abstraction that makes engineers’ eyes glaze. The useful move is to treat it as a completeness check over artifacts you were going to build anyway, not as a program to be implemented.

RMF function	The engineering artifact that satisfies it
Govern	Named owners per system, a written risk acceptance policy, decision records for accept/mitigate calls
Map	The AI system inventory, the one-page data flow per system, the five-question assessment
Measure	Eval suites, production sampling, drift and cost telemetry
Manage	The risk register with decisions, incident thresholds, the reassessment triggers

Two things make this mapping worth doing even if no one is forcing you.

First, the RMF is becoming the shared vocabulary. When a customer security review, a regulator, or an internal audit function asks how you manage AI risk, answering in RMF terms with concrete artifacts behind each function ends the conversation quickly. I have watched teams with genuinely solid engineering practices fail these reviews on translation alone.

Second, the Generative AI Profile (NIST AI 600-1) is a decent checklist of failure modes specific to generative systems, confabulation, prompt injection, data leakage into training, harmful content, that teams coming from classical ML routinely miss. Read it once, steal the categories for your taxonomy, and move on.

What the RMF will not do is make decisions for you. It tells you to “prioritize risk based on impact and likelihood.” It does not tell you whether your customer-facing summarizer needs a human gate. That judgment stays with your engineers and your risk owners, which is where it belongs. How to institutionalize those judgments without building a bureaucracy is the subject of AI governance for engineers.

Risk acceptance and ownership

Every risk register ends in one of three verbs: prevent, detect, accept. Organizations are comfortable with the first two and allergic to the third, so the third happens implicitly, which is the worst possible way.

Unwritten risk acceptance is still risk acceptance. Shipping a RAG system without permission-aware retrieval is accepting data leakage risk. Skipping injection testing is accepting manipulation risk. The only question is whether the acceptance was a decision made by someone with authority or a default made by whoever was busiest that sprint.

The workable model has three rules.

One named owner per system, with shutdown authority. Not a committee, not a RACI matrix with four letters spread across nine people. One person who can say “turn it off” and be obeyed. If nobody in the org can shut the system down without a change advisory board meeting, then nobody owns its risk, whatever the register says. I made the same argument about accountability in the AI integrity framework, because it is the same property viewed from a different angle.

Acceptance is written, scoped, and dated. A risk acceptance record is three sentences: what we are accepting, why the mitigation is not worth it right now, and what trigger reopens the decision. It takes five minutes and it converts a future incident postmortem from “how did nobody know” into “we knew, we decided, here is the record, and here is what we got wrong.” Those are very different meetings. Only one of them ends careers.

Acceptance authority scales with blast radius. A team lead can accept tier-three risks on an internal tool. Nobody below the executive who owns the P&L should accept a risk that ends in front of a customer or a regulator. Writing this gradient down once, in the governance policy, prevents both failure modes: engineers accepting risks above their pay grade, and executives being asked to review risks below theirs.

Monitoring as continuous risk management

An assessment describes a system at a point in time. AI systems do not hold still: providers update models under stable API names, retrieval corpora grow, users find inputs no one tested, prompts get “improved” on a Friday. A register that is not connected to production telemetry is a photograph of a moving object.

Monitoring closes the loop by turning each register entry into something watchable. The mapping is direct:

• Model behavior risks become sampled production scoring: a fraction of real traffic evaluated continuously against the same expectations that gate releases, with alert thresholds tied to the severity tier. Drift stops being a philosophical concern and becomes a paging event.
• Data risks become access audits and egress checks: who read the prompt logs this month, what left the trust boundary, did a new data source appear in the flow diagram’s real-world counterpart without appearing in the diagram.
• Security risks become runtime detection: injection pattern hits, tool calls outside expected scopes, anomalous retrieval volumes.
• Operational risks become ordinary SLOs and budgets: latency, error rate, token spend per feature per day. Cost alerts deserve particular respect; a runaway agent loop is a denial-of-wallet incident, and it will not announce itself.

The organizational half matters as much as the telemetry. Alerts route to the risk owner, not to a governance mailbox. Incident thresholds are defined in advance, “more than X tier-two failures per thousand interactions is an incident”, so that the decision to invoke response process is mechanical rather than political. And every incident, anywhere in the portfolio, is a reassessment trigger for every system sharing the failure mode. This is the same discipline as incident response for infrastructure teams, applied to a new class of component.

When monitoring is wired this way, the risk register stops being a compliance artifact and becomes what it should have been from the start: the specification for what production watches.

Where this goes wrong

Honesty about failure modes applies to the risk program itself.

The program can outgrow the portfolio. Ten AI systems do not justify a risk platform, a committee, and a quarterly attestation cycle. The overhead of the program should be visibly smaller than the exposure it manages, or engineering will route around it, quietly and correctly.

Registers rot. Any register without reassessment triggers is a snapshot decaying toward fiction. If you cannot fund the monitoring loop, a smaller register that stays true beats a comprehensive one that drifts.

Scoring theater is worse than no scoring. Multiplying a guessed likelihood by a guessed impact to two decimal places manufactures confidence that nobody should have. Blast radius and reversibility are estimable; probability of a novel manipulation technique is not. Rank, don’t calculate.

And risk aversion is itself a risk decision. Blocking every AI use case until the framework is perfect accepts a different exposure: the organization learns nothing while its competitors and its own shadow-IT users learn plenty. The systems you refuse to sanction do not disappear. They just stop appearing in your inventory. How to sequence adoption so that governance enables rather than obstructs is the subject of the enterprise AI adoption framework.

What to write down

If you take one action from this article, build the inventory: every AI system, its model, its data paths, its owner. If you take three, add the five-question assessment for the system with the largest blast radius, and wire its top risks into production monitoring.

Then keep four artifacts alive: the inventory, the per-system register with explicit prevent/detect/accept decisions, the acceptance records with their reopening triggers, and the monitoring that watches what the register names. Everything else in AI risk management is elaboration on those four.

None of this is new discipline. Engineering has always managed components it depends on but cannot fully trust, disks that lie, networks that partition, vendors that deprecate. Models join that list with new failure modes and an old obligation: know what can break, decide what you will do about it, and be able to show your work. Frameworks will keep multiplying. That obligation will not change.