The AI Center of Excellence Was a Detour — Build a Platform Team Instead
AI Operating ModelCenter of ExcellencePlatform EngineeringOrg DesignEnterprise AI

The AI Center of Excellence Was a Detour — Build a Platform Team Instead

T. Krause

Three years ago, every Fortune 1000 stood up an AI Center of Excellence. Most are now quietly being restructured or wound down. The pattern of what's replacing them tells a clearer story about how AI gets done at scale than any analyst report — and it has more to do with platform engineering than with centers.

The AI Center of Excellence had a good run. From late 2023 to mid-2025, almost every large organization stood one up. The mandate was familiar: gather AI talent in one place, set the standards, evangelize the practice, and incubate the early use cases. For a while it worked — the CoE built the first wins, ran the literacy programs, and gave the rest of the company someone to call.

By 2026, most of these centers have either been quietly absorbed, restructured into something else, or are in the awkward phase before that happens. The pattern is not a failure of execution. It is the predictable end state of an organizational pattern that was always temporary — a scaffold that gets removed once the building underneath it can stand on its own.

What is replacing the CoE in the organizations doing this best looks a lot more like a platform team than a center, and the implications for how AI gets done at scale are significant.

Why the CoE Pattern Has a Half-Life

The Center of Excellence was the right answer for an early-stage problem: a few experts, a few use cases, a large organization that needed someone to consult. The pattern stops working once the scale changes, for reasons built into what a CoE is.

Centralizes scarcity and bottlenecks delivery. When the AI talent is concentrated in one team, every use case in the company has to go through that team. The CoE becomes a queue. Use cases sit waiting for capacity, and the slowest part of the process is always the same team. The bottleneck is the structure.

Lives outside the business and loses credibility. CoE teams sit organizationally apart from the business units they serve. The work they produce is suspected of not understanding the business context. The business units, in turn, are suspected of not understanding the technology. Both suspicions are usually partly true, and the friction reduces both speed and quality.

Standards by evangelism do not scale. The CoE's role of "setting standards" works in a small organization where the CoE can know every team. It does not work in a large one where dozens of teams are doing AI in parallel. Standards have to be enforced by something stronger than persuasion, and the CoE has no enforcement mechanism.

Use cases pile up faster than the center can handle. Early on, the CoE shipped the few use cases the company was attempting. As AI adoption broadened, every team had ten ideas, and the CoE's throughput was an order of magnitude too small. The backlog became the dominant feature of the relationship.

What the Platform Team Pattern Looks Like

The companies replacing the CoE successfully are converging on a platform engineering model — the same pattern that internal developer platforms have followed in software organizations over the past decade.

A standing platform with shared infrastructure. The platform team builds and runs the foundational services every AI use case needs: model access, evaluation, prompt management, observability, security guardrails, data integration, deployment infrastructure. The platform is the product, and business teams are the customers.

Self-service for business unit teams. Business teams don't queue for the platform team's attention. They use the platform directly. The platform team's job is to make the platform good enough that business teams can build their own use cases on top of it without needing a project request.

Engineering culture, not consulting culture. Platform teams ship code, run production systems, and own uptime. CoE teams produce decks, write standards, and hand off projects. The cultural difference shows up in everything from hiring to operating cadence to how problems get diagnosed.

Standards enforced by the platform, not by review. Security, governance, evaluation, and cost discipline are encoded into the platform itself. If a use case is built on the platform, it is automatically compliant with the standards. If it isn't on the platform, it doesn't get to production. Enforcement is structural, not procedural.

Where Platform Teams Already Outperform CoEs

The comparison is now visible in companies that have made the transition far enough to measure. The differences are not marginal.

Time to deployed use case. Platform-based organizations are shipping new AI use cases in weeks. CoE-based organizations are shipping them in quarters. The difference is mostly the elimination of the queue at the central team and the reuse of platform components that no longer need to be built each time.

Cost per AI system. Platform-based systems amortize shared infrastructure across all use cases, so each new one is cheaper than the last. CoE-based systems tend to rebuild the foundations each time, because each project is scoped separately. The cost curves diverge over twelve months and the gap keeps widening.

Consistency of governance. When governance is enforced by the platform, every system on it is consistent. When governance is enforced by review at the CoE, the systems are inconsistent in proportion to the variability of the reviewers. The platform model produces a more defensible posture for compliance and audit.

Resilience to leadership change. A platform is durable infrastructure. A CoE is often the personal initiative of an executive sponsor. When the sponsor moves on, the CoE is fragile in a way the platform is not. Many of the CoEs being wound down are casualties of leadership transitions as much as of structural limits.

How to Make the Transition

The transition from CoE to platform team is a deliberate organizational decision, and it goes better when it is approached as such. The companies that have done it well share a sequence.

Audit what the CoE actually does today. Separate the work into platform-shaped tasks (infrastructure, tooling, standards, monitoring) and consulting-shaped tasks (project work, advising business teams, evangelism). The first group becomes the seed of the platform team. The second group either federates out to the business or shrinks.

Separate platform work from project work explicitly. As long as the same team is doing platform engineering and shipping individual use cases, the project work will always crowd out the platform work because projects have deadlines and platforms have priorities. The two functions need different teams or, at minimum, different time allocations.

Stand up platform engineering with real scope. The platform team needs engineers with platform experience, not just AI experience. Treat it like an internal developer platform team and staff it accordingly. The skill profile is different from the typical CoE staff profile, and that difference matters.

Federate use-case development to the business. Business units own their AI use cases. They build on the platform, they own the production system, they are accountable for the outcomes. The platform team supports them but does not deliver for them. This is the structural shift that breaks the queue.

Sunset the CoE when the platform is real. Once the platform is good enough that business teams can ship on it without the CoE's involvement, the CoE has done its job. The graceful path is to absorb the remaining CoE functions into either the platform team or the business teams. The ungraceful path is to keep the CoE running as a vestige until the next reorganization eliminates it.

The Stakes

Organizational structures harden faster than they unfreeze. The CoE that was the right pattern in 2024 has become an obstacle in 2026, and the companies still running it are spending effort maintaining a structure that is no longer producing value. The opportunity cost is invisible — every use case that was slow because of the CoE bottleneck, every initiative that was abandoned because it was stuck in the queue, every business team that gave up and built shadow AI to get around the wait.

The companies that made the transition early are now operating at a different speed. Their business teams are shipping AI directly, on a platform that handles the governance and the foundations automatically, with the platform team supporting at scale rather than gatekeeping. The CoE-based companies are still arguing about what the CoE's mandate should be.

The decision is not whether to dissolve the CoE — it is whether to recognize that the pattern has a half-life and to plan the transition deliberately rather than letting it happen as the byproduct of an eventual reorganization. The companies that plan the transition land it cleanly. The ones that don't end up doing it anyway, three years later, with more disruption and less to show for it.

Related Articles

We use cookies

We use cookies to ensure you get the best experience on our website. For more information on how we use cookies, please see our cookie policy.

By clicking "Accept", you agree to our use of cookies.
Learn more.