What does “AI fluency” actually mean?
At the start of 2025, I made it mandatory for every employee at Concord to use AI in their daily workflow. The question then became: How do we measure whether that employee is AI-fluent?
Using ChatGPT every day does not – in itself – make anyone better at their job. In fact, forced AI usage slows many people down at first.
We realized that AI fluency meant thinking differently about solving problems.
Here’s how we measured that.
First, what we realized *doesn’t* work:
– Tracking tool usage (frequency ≠ fluency)
– Traditional skills tests
– Requiring documentation of AI usage
All the above actually slow people down. They waste time trying to “prove” they’re using AI instead of *actually using it* to overhaul their workflows. It’s pointless.
Here’s what we found that works:
1) The “before/after” method
Teams write out how they solve problems pre-AI. Then they plan out how to use AI to solve those same problems, faster. Then they implement it. When we tried this, one sales ops person went from 3-hour contract analytics to 15-minute AI queries.
2) Peer teaching Fridays
Every Friday, several team members explain how they used AI to solve a specific problem. This celebrates AI achievements that have real impact on the company, and encourages others to ask questions. The best sign is when they go, “Oh cool!”
3) Innovation tracking
We don’t count “usage,” we count new use cases that measurably save time. Last quarter we tracked 5 new applications just from the marketing team.
4) Problem-first thinking
Teams need to identify problems that need AI, vs. ones that need human judgment. Correct identification of an AI-suited problem predicts successful adoption more than any other factor.
The biggest surprise for us:
Our most AI-fluent employees aren’t the youngest, and they’re not the most technical. They’re experts who exercise lateral thinking about how to apply AI in new domains.
For example, we have a BizOps person who connected AI to Notion, and used it to automate the creation of project planning boards. Days of work reduced to an hour.
Results: 60% efficiency gains – but that’s just a number. I could be pulling that out of you-know-where.
But I’ll tell you something else: We’re solving problems we previously thought were impossible, as we continue to work to build the best contract management software.
Stop counting usage. Start measuring transformation.
What’s working at your company?
Matt Lhoumeau is the CEO and co-founder of Concord, a leading provider of Agreement Intelligence solutions. Concord empowers growing businesses to make smarter operational decisions by unlocking actionable insights from contracts, and is trusted by over 1,500 companies worldwide.
