AI-assisted software development can make a team look dramatically faster. That does not automatically mean the enterprise is producing better returns.
At the CIO 100 Leadership Live Chicago gathering on October 13, senior technology leaders weighed the difference between quick wins in code and proof in the numbers. Several speakers argued that AI can speed up prototyping and proposal writing, yet the heavy work of architecture, integration, security and quality still determines whether an application is ready for production use.
The return is not the same as the demo
Laura Walsh, the technology chief at Culligan, outlined the squeeze that comes with executive mandates. She said leaders are asked to speed adoption, defend short-term financial results, keep standards intact and pay for new work from existing budgets. Her answer leaned on long-standing disciplines, including data stewardship, disciplined delivery and technology business management.
Other executives pushed back on the idea that AI transformations succeed by being dropped on top of the existing landscape. Scott Behrens of PwC cautioned that disconnecting AI tools from underlying workflows can raise consumption without producing the business benefits executives were promised.
This split is the part worth noticing. A proposal drafted in minutes is a delivery gain. The proposal still has to win a customer, support a decision or move an operating metric. The benefit lives downstream of the speed boost.
Different measures for different investments
Andy Xydakis, a global CIO at Aon, told the room his group has lifted its cloud footprint from roughly 35 percent to about 85 percent of the application portfolio. Around 60,000 employees now have access to AI features. Proposal work that once consumed 40 to 50 hours is reported to take closer to four or five.
Chris Campbell, the technology chief at DeVry University, made the case that technology projects should be reviewed against institutional goals, including enrollment, persistence, retention and graduation, rather than only against local savings. That is a fair test for any leader: a project can look efficient at the team level while quietly undermining the wider mission.
Prashant Shanti Kumar of Marsh suggested starting from clear business outcomes and slicing complex programs into manageable phases. The advice sounds obvious. It also has a strong track record.
Our analysis: measure the system, not the shortcut
The harder problem for any project lead is timing. AI tends to deliver visible intermediate savings long before the durable business result appears. That gap can attract the wrong metric, or the wrong celebration.
A useful discipline in agile project management is to treat the promised outcome as a constraint on backlog decisions. Ask the team which workflow actually moves the institutional target before chasing the headline number. Prototyping speed matters, but shipping it is what pays.
The opposite risk is equally pressing. Setting too aggressive a financial return at the start can pull projects toward demos that look profitable and leave the integration and operating costs uncovered. Alignment between sponsor and team about what counts as a win is itself part of the deliverable.
A worked example for an executive sponsor
Say a senior leader wants to fund an AI assistant that drafts sales proposals. Ask the project team to compare cycle time at the proposal stage, but also ask what happens next: approval rates, cycle time after the draft, customer feedback, the time sales staff spend adjusting the output. Decisions about scaling can wait until those downstream signals are collected.
Run the pilot on a single team for a defined period. Agree on the success criteria in advance. Pause for a review before expanding scope. This is the same logic Prashant Shanti Kumar outlined in 90-day increments, and it protects the next budget from being spent on enthusiasm.
What to watch next
Look for whether boards and CIOs start tying AI budgets to institutional outcomes such as revenue per process, customer retention, employee impact and technology cost. The companies that win will redesign the workflow around the technology, not the other way around.
Bottom line: AI productivity gains are real only when they survive the journey from output to business result. Measure that journey.
Key takeaways
- Faster development work does not automatically create measurable enterprise returns.
- AI transformations work best when business processes are redesigned, not only extended with new tools.
- Connect each investment to an institutional outcome and split complex programs into manageable phases.
- Track downstream signals such as adoption, quality, customer impact and operating cost.
Frequently asked questions
Why is AI productivity difficult for executives to measure?
AI can compress intermediate tasks, yet the financial benefit often appears only after integration, governance and real-world use.
What should leaders measure alongside time saved?
Adoption, quality, customer impact, error rates and institutional outcomes all belong in the scorecard, alongside local savings.
Why is bolting AI onto an existing process risky?
Speakers from PwC warned that disconnected AI tools can raise consumption without producing the corresponding business benefits leaders were promised.
How can an executive sponsor test an AI investment?
Pick a bounded workflow, set the baseline, capture the downstream signals and expand only after a review.