Your AI Models Are in Production. Who Is Watching Them?

Eighteen months ago, a bank I advise had 11 AI models in production. When I asked who owned monitoring for each one, the room went quiet. Not uncomfortable quiet. Confused quiet. Nobody had ever been asked that question. Three months later, one of those models started approving credit applications it should have flagged. It ran that way for six weeks before anyone noticed. The exposure was north of $4M.

The Gap Between Deployment and Ownership

Every enterprise I've worked with has a process for getting AI into production. Intake forms, architecture reviews, security sign-offs, sometimes even ethics panels. That process ends the moment the model goes live. After that? Silence.

Here is the pattern I see repeated across industries. The data science team builds the model. Engineering deploys it. The business unit that requested it moves on to the next initiative. Nobody is assigned ongoing responsibility for the model's behavior in the real world.

A 2025 Gartner survey found that 62% of enterprises with production AI had no formal model monitoring cadence. Not 'inadequate monitoring.' No monitoring at all. The models were deployed and forgotten.

This is not a technology problem. Monitoring tools exist. DataRobot, Fiddler, Arthur AI, even open-source options like Evidently and WhyLabs all do this well. The problem is organizational. Nobody has been told this is their job.

The Accountability Map: A One-Page Fix

At a Top 10 bank where I spent six years, we solved this with what we called the Model Accountability Map. One page per model. Five fields. No committee required to create it.

Field one: Model Owner. This is a named person, not a team. Not 'the data science group.' A human with a phone number who is responsible for this model's behavior. At our bank, this was always someone on the business side, not engineering. The business owns the outcome, so the business owns the model.

Field two: Technical Steward. The engineer or ML ops person who can actually pull logs, check drift metrics, and retrain if needed. Again, a named person. When the Model Owner gets an alert, this is who they call.

Field three: Monitoring Cadence. Weekly, monthly, or quarterly, depending on the model's risk tier. A credit decisioning model gets weekly reviews. An internal document classifier might get quarterly. The cadence is written down and calendar-blocked.

Field four: Drift Thresholds. Specific numbers. 'If accuracy drops below 91%, alert the Model Owner within 24 hours.' 'If false positive rate exceeds 8%, pause the model and escalate.' These are not vague guidelines. They are trigger points with defined responses.

Field five: Escalation Path. If the model needs to be paused or retrained, who approves that? At what dollar exposure does this become a risk committee conversation? We used three tiers. Under $500K potential exposure, the Model Owner and Technical Steward can act independently. Between $500K and $5M, the line-of-business VP must approve. Over $5M, the CRO's office gets involved.

What Model Drift Actually Looks Like in Enterprise

Model drift sounds academic until it costs you money. Here is what it looks like in practice.

At one financial institution, a fraud detection model was trained on transaction data from 2022 and 2023. By mid-2025, the pattern of legitimate transactions had shifted. Remote work changed spending patterns. New payment platforms changed transaction sizes and frequencies. The model started flagging 23% of legitimate transactions as suspicious, up from a baseline of 6%. Customer complaints spiked. The call center absorbed an extra 1,200 hours of work per month handling false flags.

Nobody was watching. The model was 'working' in the sense that it was running. It just was not working well. The business impact was roughly $180K per month in added call center costs alone, not counting the customer attrition nobody measured.

The fix took two weeks. Retrain on fresh data, validate, redeploy. The drift had been happening for five months. That is $900K in avoidable cost because nobody had a monitoring cadence written down.

This is not a rare story. I have seen versions of it at four different organizations in the last two years. The models are different. The failure mode is always the same: nobody was assigned to watch.

How to Build This in 30 Days Without a New Team

You do not need a new department. You do not need to hire a Head of ML Ops (though you might want one eventually). You need 30 days and a spreadsheet.

Week one: Inventory. List every AI model in production. At most companies, this takes longer than expected because models are scattered across teams. One VP told me he thought they had 8 models. The actual count was 19. Include anything making automated decisions or recommendations that humans act on.

Week two: Assign owners. For each model, name a Model Owner and Technical Steward. This will surface uncomfortable conversations. Some models were built by people who left the company. Some were deployed by vendors who handle 'everything' but have no SLA for monitoring. Good. Surface those gaps now.

Week three: Set thresholds. Work with your data science team to define drift thresholds for each model. If your team cannot tell you what 'good' looks like for a model's performance metrics, that is a red flag. It means the model was deployed without a performance baseline. Fix that first.

Week four: Activate the cadence. Put the reviews on the calendar. The first review for each model should happen in week four. Use it to validate that you are measuring the right things and that alerts actually reach the right people. I have seen 'alerting systems' that sent emails to distribution lists nobody read.

Total cost: zero dollars if you use existing tools. Maybe $30-50K annually if you add a monitoring platform for the models that do not have one. Compare that to the $900K example above and the math is obvious.

One more thing. Document each Accountability Map in a shared location that your risk and compliance teams can access. When regulators ask how you govern production AI (and they will ask, not if but when), you want to hand them a clean answer in under five minutes. The bank I mentioned earlier used this exact documentation during an OCC exam. The examiner called it 'the clearest AI governance artifact we have seen.' That is not a marketing quote. That is what happens when you do the boring work before you are forced to.

Actionable Takeaway