If you can’t show me the “before” number, you can’t measure AI impact. That is my position, and I hold it firmly.
You need one workflow, one number before and the same number after. That’s the whole method. It’s the I in FLIP, and it is where most AI plans stall.
Key points
I’m ZAK, founder of ORKA AI. This is part four of my FLIP series, after the FLIP framework overview, AI fluency and AI literacy for leaders. Fluency and Literacy are about people. Impact is where the business gets its answer.
What does it mean to measure AI impact?
To measure AI impact is to compare a real workflow before and after AI using a number your leadership already cares about: hours, dollars, errors or turnaround time. If you cannot name one process that runs better with a number attached, you do not have impact yet. You have potential.
Potential is fine. It just doesn’t go in the board deck.
Which number should you use to measure AI impact?
Use the number the workflow already lives by. For a support inbox, it’s response time. For invoicing, it’s errors per hundred. For proposals, it’s hours per draft. Pick one, measure it for a normal week before AI, then measure the same thing the same way after.
Cost matters too, and it’s trickier than the sticker price. Our look at pricing AI work by the finished task makes the case for counting cost per completed job. Tokens are an input. I agree. Finished work is the unit that matters.
How do you run a thirty-day AI impact test?
Choose one repetitive workflow and record its baseline this week. Run the AI version for 30 days, measuring the exact same thing. Then report the result in your regular leadership meeting, whether it’s good, flat or bad. Thirty days is long enough to get past novelty and short enough to keep focus.
- Week 0: baseline. Same metric, a normal week.
- Weeks 1 to 4: AI version, one owner, everyone on the workflow uses it.
- Day 30: compare, then decide. Keep, change or stop.
Be honest about who uses it. In our piece on the leader gap in workplace AI adoption, the warning was that a leader’s own heavy use can hide a team that never touches the tool. Measure the whole team. Champions skew the average.
What makes a fair baseline when you measure AI impact?
A fair baseline uses a normal week, the same volume of work and the same quality bar you will use after AI. Skip holiday weeks and month-end crunches. Count the full job, including the time people spend checking and fixing AI output, or the comparison will flatter the tool.
Three traps I watch for:
- Moving the goalposts. If the AI version answers emails faster but you stop checking tone, you changed two things. Keep the quality check identical.
- Counting only the fast part. A draft in two minutes means little if review takes twenty. Time the whole job, start to finish.
- Measuring the keen people. Early adopters always look good. Include everyone who does the workflow.
Write the baseline down somewhere leadership can see it, with the date and who measured it. A number nobody can find in 30 days is a number nobody will trust.
Why does Canadian AI use rarely show measurable impact?
Because use is growing faster than measurement. In its June 2025 release, Statistics Canada put business AI use at 12.2%, measured over the prior 12 months. Few of those teams set baselines first, so wins cannot be proven and losses never get cut.
The figure comes from Statistics Canada’s analysis of business AI use. Adoption is the easy headline. What I want every Canadian leader to ask is the follow-up: of the AI we use, which piece moved a number?
When should you stop an AI project?
Stop or change it when 30 days of honest measurement show no gain on the metric that matters. Killing a weak use frees budget and attention for one that works. In FLIP, cutting what doesn’t pay off scores as Impact, because it shows you are managing AI by results.
Your move: score the Impact pillar on the free FLIP scorecard, pick one workflow and write down its baseline before Friday. One real number will change the AI conversation in your next meeting.
Who should own the job to measure AI impact?
The person who owns the workflow. They know the real metric, they see the edge cases and they can tell if the AI version is faster or just different. Give them the baseline, the 30 days and a seat at the leadership meeting to report what happened.
Avoid handing the job to IT or to the vendor. IT can help set up the tool. The vendor will always have a case study. Neither knows your Tuesday afternoon. When the workflow owner reports the number, leadership trusts it, and you build a habit to measure AI impact on every new tool before it spreads.
Over a year, that habit gives you something rare: a short list of AI uses you can defend with real numbers, and a longer list of ideas you tested and dropped for good reason.
This won’t prove value for every AI use. Some gains, like better first drafts or fewer late nights, are real but hard to count. Note them, but don’t let them replace one hard number.
Frequently asked questions
How do you measure AI impact?
Pick one workflow, record a baseline in time, cost, errors or speed, run the AI version for 30 days and measure the same thing the same way. Then compare and decide.
What is a good metric for AI impact?
The one the workflow already lives by, such as response time for a support inbox, errors per hundred invoices or hours per proposal draft.
How long should an AI pilot run before judging it?
Thirty days works for most workflows. It gets past the novelty phase while keeping the test small and focused.
Where does Impact fit in the FLIP framework?
Impact is the third FLIP pillar. Score it with the five Impact statements in the free FLIP scorecard.
Disclosure: ZAK is the founder of ORKA AI; aiwithzak.com is his website. This column describes his own framework. It is analysis and opinion.
