From Chat to Change: How Executives Close the AI ROI Gap
The difference between AI that generates words and AI that generates results
Walk into most organizations today and you'll find the same scene: employees enthusiastically using AI chat tools, generating summaries, drafting emails, asking questions. Leadership sees the activity and concludes the company is embracing AI. Budgets are approved. Announcements are made.
Then someone asks a harder question: what has actually changed? What takes less time? What costs less? What do customers experience differently? The answers are often surprisingly thin.
The problem isn't the technology. The problem is that most organizations are stuck at the lowest tier of AI capability — and don't realize there are two more tiers above it that deliver the actual returns.
Tier One: The Chat Window (Where Most Companies Are Stuck)
When most people think of AI today, they think of a chat window. You type a question. The AI responds. You follow up. It's impressive the first time you use it. It feels like having a very smart, very fast assistant available at any hour.
And it is useful; for individuals. A marketing manager drafting a campaign brief. A lawyer doing initial research. An executive preparing talking points. For personal productivity, AI chat is genuinely valuable.
But here is the critical limitation: every conversation starts from scratch. The AI doesn't know your business. It doesn't know your customers, your standards, your processes, or your history. It's responding to whatever you typed just now, with no memory of anything before it. The quality of what you get depends entirely on how well you asked, and that varies enormously from person to person, day to day.
This means that at Tier One, AI is a personal productivity tool. It is not a business capability. There is no consistency, no scalability, no repeatability. Two employees asking the same question will get meaningfully different answers. There is no way to ensure quality, measure output, or improve over time.
If this is where your AI investment lives, you are spending money on a sophisticated search engine with better sentence structure. The ROI will always be marginal.
"If this is where your AI investment lives, you are spending money on a sophisticated search engine with better sentence structure."
Tier Two: Orchestrated Workflows (Where ROI Starts)
The second tier is where AI stops being a tool that individuals use and starts being a capability that the organization runs.
An orchestrated workflow is an AI system designed to do a specific job reliably, consistently every time, without requiring a skilled human to direct it in the moment. The AI has been given detailed instructions in advance. It knows what it's supposed to do, how it's supposed to do it, what format the output should be in, and what constraints it operates within. A user triggers it; the system executes.
Think of the difference this way. Tier One is like hiring a brilliant freelancer and briefing them from scratch every time you need something done. Tier Two is like having a trained specialist on staff who knows the job, follows your standards, and produces consistent work without constant supervision.
Real-world example: A financial services firm uses an orchestrated workflow to analyze incoming client documents, extract key data points, flag compliance issues, and generate a structured summary — in under two minutes, with consistent formatting, every time. Previously this took a junior analyst 45 minutes and the quality varied. The analyst now reviews the AI's output rather than producing the first draft.
The outputs of orchestrated workflows are tangible and measurable: a completed document, a structured report, a flagged risk, a populated record, a drafted communication. Not a conversation but a deliverable. This is what makes ROI calculable. You can count how many documents were processed, how long it took, what it used to cost, and what it costs now.
The secret ingredient that makes this work, the thing most organizations overlook, is something called prompt engineering. This is the craft of writing precise, structured instructions that tell the AI exactly what you need it to do, in what context, with what constraints. It's the difference between telling a new employee "handle customer complaints" and giving them a detailed playbook with examples, escalation criteria, tone guidelines, and output formats.
Done well, prompt engineering is what transforms a generic AI model into a specialized tool that behaves as if it were trained specifically for your business. It requires expertise and iteration. But once it's done, it runs the same way every time — and that consistency is the foundation of real returns.
"Prompt engineering is the craft of writing precise instructions that tell the AI exactly what to do. It's what transforms a generic model into a tool that behaves as if it were built for your business."
I’ll dive deeper into prompt engineering in a subsequent article.
Tier Three: Agentic Applications (Where Transformation Happens)
The third tier is the most powerful and the most misunderstood. Agentic AI doesn't just execute a predefined workflow. It pursues a goal.
You give an agentic system an objective, and it figures out the steps and makes decisions along the way based on what it finds. When one approach doesn't work, it tries another. It keeps going until the goal is achieved or it determines it needs human input.
It does this using Tools and Skills. It uses Tools ( searching databases, calling APIs, reading documents, sending communications ) to take Actions. Think of tools as the raw capabilities or "hands" of the agent. Skills are procedural knowledge; packaged instructions that teach the agent how to perform specific tasks well. Each Skill packages instructions, metadata, and optional resources (scripts, templates) that the agent uses automatically when relevant.
A simple analogy: a Tier Two workflow is a skilled employee following a detailed checklist. A Tier Three agent is a project manager who reads the brief, builds the plan, delegates to the right tools, checks the results, and adjusts when something unexpected happens.
Real-world example: A professional services firm deploys an agentic system to handle new client onboarding. Given a client name and engagement type, the agent retrieves the signed contract, extracts key terms, creates the project folder structure, drafts the kickoff agenda, populates the CRM record, and sends a welcome email — all without human intervention. What used to take an administrator two hours now takes four minutes.
Agentic applications are where AI stops augmenting individual employees and starts restructuring how work gets done. The returns are correspondingly larger, but so are the risks if the system isn't built carefully. Agentic systems need guardrails: clear boundaries on what they can and cannot do autonomously, human review checkpoints for high-stakes decisions, and monitoring to catch errors before they propagate.
The organizations building agentic capabilities today are not just improving efficiency. They are redesigning processes that haven't fundamentally changed in decades. That is a different kind of competitive advantage than productivity gains.
The Real Question: Are You Generating Words or Generating Value?
Here is the diagnostic that matters. Look at what your AI investment is producing. Is it generating words — summaries, drafts, responses, or content that a human then has to evaluate, edit, and act on? Or is it generating outcomes where decisions are made, processes completed, records updated, actions taken?
Words are not worthless. Saving a knowledge worker two hours of drafting time has value. But if your entire AI strategy is "help people write things faster," you are at Tier One, and the returns will plateau quickly.
Outcomes are where the compounding returns live. An orchestrated workflow that processes 500 documents a day with consistent quality and zero marginal cost after buildout is a fundamentally different economic proposition than 50 employees each saving 20 minutes a day on email.
The right questions to ask your team: What specific business process does this AI system own end-to-end? What is the measurable output? What did that output cost before, and what does it cost now? What happens when the system makes an error, and who catches it?
If your team can't answer those questions, your AI investment is probably at Tier One — and the value is being measured in employee satisfaction surveys rather than business outcomes.
What Getting This Right Actually Looks Like
Organizations that are generating genuine returns from AI share a few common characteristics.
They started with a specific problem, not a technology. They didn't ask "how do we use AI?" They asked "what takes too long, costs too much, or produces inconsistent results?" Once they’ve gone through that thought process they then could focus on the appropriate AI solution.
They invested in the architecture above the model. The AI model itself, whether it's Claude, GPT-4, or something else, is a commodity. What isn't a commodity is the system built on top of it: the carefully engineered instructions, the quality controls, the integrations with business data, the human review checkpoints. That's where the proprietary value lives.
They measure outcomes, not activity. Not "how many employees are using AI" but "how many contracts were reviewed, how many hours were saved, what is the error rate, what did compliance cost last quarter versus this quarter."
They treat AI like infrastructure, not a pilot. A chat window subscription is a pilot. An orchestrated workflow that runs a core business process is infrastructure. It gets maintained, monitored, improved, and governed — the same way you'd treat any other critical system.
"The AI model itself is a commodity. What isn't a commodity is the system built on top of it. That's where the proprietary value lives."
The Bottom Line
AI is not magic, and it is not hype. It is a genuinely transformative technology that is delivering real returns for organizations that have moved beyond the chat window.
The path from chat to orchestrated workflow to agentic application is not a technology journey. It is a strategy and architecture journey. It requires clarity about which business problems are worth solving, discipline in how the systems are built, and rigor in how returns are measured.
The gap between organizations that use AI and organizations that are transformed by it will widen significantly over the next three years. Which side of that gap you land on is a decision being made right now; often without executives realizing it's a decision at all.
More words is not the goal. The goal is better outcomes.
Tom Ackerman is the founder of Ackerman Strategic Advisory, specializing in enterprise AI architecture and technology transformation for mid-market and enterprise organizations.
