Why Your AI Center of Excellence Is Failing (And the Lean Model That Works)

I have watched four different AI Centers of Excellence get stood up at major banks. Three of them were dead or irrelevant within a year. The fourth one worked, and it looked nothing like what McKinsey told the board it should look like. The difference was not budget, talent, or executive sponsorship. It was the operating model itself.

The Classic CoE Trap: How Good Intentions Become Bureaucracy

Here is how it usually goes. The CEO reads an article, attends a conference, or gets pressured by the board. Someone gets tapped to 'stand up an AI Center of Excellence.' That person hires five to eight data scientists, maybe a product manager, writes a charter document nobody reads, and starts accepting intake requests from business units.

Within 90 days, the CoE has a backlog of 40 requests. They are working on three. The other 37 business owners are frustrated. The data scientists are spending 60% of their time on data access requests and compliance reviews instead of building models. The CoE leader is in back-to-back meetings explaining why things are taking so long.

By month six, business units start going around the CoE. They hire contractors. They buy point solutions from vendors who promise 'no code, no IT involvement.' Shadow AI spreads. The CoE becomes the thing everyone complains about in steering committee meetings.

I saw this exact pattern at two banks between 2019 and 2023. One had a CoE of 12 people with an $8M annual budget. They delivered four production models in 18 months. That is $2M per model, not counting the opportunity cost of the 30+ projects stuck in the queue. The CIO eventually dissolved the team and redistributed the headcount.

Why the Traditional Model Breaks: Three Structural Flaws

The traditional CoE model has three fatal flaws that no amount of talent or funding can fix.

Flaw one: it centralizes execution. When every AI project must flow through a single team, you create a bottleneck by design. A centralized execution team cannot scale linearly with demand. Every new business unit request adds queue time. At one bank, the average time from intake request to project kickoff was 14 weeks. By the time the CoE started working, the business problem had changed or the sponsor had moved on.

Flaw two: it separates AI knowledge from domain knowledge. The best AI applications come from people who deeply understand the business problem. When you put data scientists in a central team, they spend their first three weeks on every project just learning the domain. A fraud analyst who has been doing the job for eight years will spot patterns that a centralized data scientist will miss entirely. The CoE model assumes AI expertise is the bottleneck. In most enterprises, domain expertise combined with basic AI literacy is far more valuable.

Flaw three: it creates a permission culture. When you have a formal CoE with an intake process, you are implicitly telling the organization that AI is special, dangerous, and requires central approval. That might have been appropriate in 2020. In 2026, when AI capabilities are embedded in tools your teams already use, treating every AI application like a custom model build is like requiring IT approval to use a spreadsheet formula.

The Lean Model: Centralize Standards, Distribute Execution

The CoE that worked, the fourth one I mentioned, flipped the model. Instead of centralizing execution and distributing demand, they centralized standards and distributed execution. The team was five people. Not fifty. Five.

Person one: the AI governance lead. Owned the policy framework, the risk classification tiers, and the approval workflows. This person did not review every project. They built the rubric so business units could self-classify. Tier 1 projects (low risk, internal facing, no PII) needed no central approval. Tier 2 needed a lightweight review. Only Tier 3 (customer-facing, regulated, or using sensitive data) required full governance review. Result: 70% of AI projects moved without waiting for central approval.

Person two: the platform engineer. Owned the approved tool stack, the API integrations, the sandboxed environments. Their job was to make it easy to build within guardrails. They maintained three pre-approved environments: one for prototyping, one for internal deployment, one for production with full monitoring. Business units did not need to negotiate infrastructure for every project.

Person three: the AI enablement lead. This was the force multiplier. Their entire job was training and coaching. Not classroom training. Embedded coaching. They spent two days per week rotating through business units, sitting with teams, helping them frame problems as AI-solvable, showing them how to use the approved tools, and reviewing early prototypes. They ran a monthly 90-minute clinic where teams presented work-in-progress and got feedback.

Persons four and five: two senior ML engineers who handled only Tier 3 projects. Custom model builds, complex integrations, anything that touched production customer systems. They also served as the technical review board for Tier 2 projects, doing asynchronous code reviews that took 48 hours, not 14 weeks.

This five-person team supported an organization of 11,000 employees. In the first year, 23 AI applications made it to production. The previous 12-person CoE had delivered four.

How to Transition: The 90-Day Shift From CoE to Lean Model

If you already have a CoE that is struggling, you do not need to blow it up overnight. Here is the 90-day transition I recommend based on what I have seen work.

Days 1 through 30: Classify your backlog. Take every project in your intake queue and assign it a risk tier. Use a simple 2x2 matrix. One axis is data sensitivity (public or internal versus PII or regulated). The other axis is deployment scope (internal tool versus customer-facing). Anything in the low-sensitivity, internal-tool quadrant gets immediately released to the requesting business unit with a one-page guide on approved tools and a link to your AI policy. At one bank, this single step cleared 60% of the backlog in three weeks.

Days 30 through 60: Reassign your team. Stop treating every data scientist as a project executor. Identify your two strongest ML engineers and dedicate them to Tier 3 work only. Take your most business-savvy team member and make them the enablement lead. Start the rotation schedule. If you have people who are primarily doing data prep and access requests, that is a platform engineering problem, not an AI problem. Move that work to your data engineering team or automate it.

Days 60 through 90: Publish your governance tiers and self-service criteria. Make it public internally. Run three lunch-and-learn sessions where you walk business units through the new model. The message is simple: 'We are here to help you move faster, not to approve your requests.' Track two metrics from this point forward. First, time from idea to production deployment. Second, number of AI applications running in production. Stop tracking intake requests. That metric rewards the wrong behavior.

One thing I want to be direct about: this transition will make some people on your current CoE team uncomfortable. Data scientists who joined to build models will resist becoming coaches or reviewers. Some will leave. That is okay. The goal is not to keep a team happy. The goal is to get AI into production across the organization.

The Numbers That Matter: Measuring a Lean AI Operating Model

Traditional CoEs measure the wrong things. They track projects in flight, models in development, team utilization rates. These are activity metrics. They tell you nothing about impact.

The lean model tracks four metrics. First: time from business request to production deployment. The target should be under 60 days for Tier 1, under 120 days for Tier 2, and under 180 days for Tier 3. If your Tier 1 projects are taking longer than 60 days, your self-service tooling is broken.

Second: number of distinct business units with at least one AI application in production. This measures adoption breadth. A CoE that has delivered 10 models all in the same department has not proven organizational capability. At the bank where the lean model worked, they went from AI production use in 2 departments to 9 departments within 14 months.

Third: percentage of AI applications that are still in active use after six months. This is your quality check. If teams are building things that get abandoned, your enablement process is not qualifying problems well enough. The target is 75% or higher. The bank I referenced hit 78% in year one.

Fourth: ratio of self-service deployments to centrally-built deployments. This tells you whether you are actually distributing capability or just relabeling the old model. Target is 3:1 or higher. For every project your central team builds, three should be shipping from business units using your standards and tools.

Actionable Takeaway