In 2019 we deployed a document classification system for a large enterprise. It ingested tens of thousands of scanned files a month, figured out what each one was, pulled the fields that mattered, and routed them to the right queue.
There was no ChatGPT. There was no prompt. There were no agents, no copilots, no one on LinkedIn explaining that everything had changed. There was a model you trained yourself on data you labeled yourself, and it took months, and when it went wrong you did not get an apology in polite prose — you got a silent misfile that somebody found six weeks later.
The technology from that era is obsolete. Almost every lesson from it is not.
Lesson one: the model was never the hard part
We spent maybe fifteen percent of that project on the model. The other eighty-five was spent on questions like: where do these documents actually come from, who touches them before we do, what happens when one arrives that doesn't fit any category, who gets told, and what does that person do next.
That ratio has not moved. It has arguably gotten worse, because the models are now so good that they make the surrounding work look optional. It isn't. Today you can stand up something that reads a lease and answers questions about it in an afternoon. What you cannot stand up in an afternoon is the answer to "what happens when it's wrong, and how will we know."
Every failed AI project we've been called in to clean up failed there. Not at the model. At the seams.
Lesson two: nobody trusts a black box, and they're right not to
The 2019 system was accurate. It was also, for about three months, ignored — because the team downstream had no way to tell a confident right answer from a confident wrong one, so they checked everything by hand anyway. We had automated the work and doubled it.
What fixed it wasn't better accuracy. It was surfacing the machine's own uncertainty. When the system said "I'm 96% sure this is a purchase agreement," people trusted it and moved on. When it said "I'm 61% sure, please look," they looked — and crucially, they stopped looking at everything else.
Confidence, exposed honestly, is what makes automation adoptable. A system that is right 94% of the time and tells you which 6% to check is more useful than a system that is right 98% of the time and never tells you anything. This is deeply counterintuitive to people buying software, and it is the single most reliable predictor of whether your team will actually use the thing.
Lesson three: the process you automate is not the process you have
Every organization has a documented process and a real process, and they are never the same document.
In 2019 we mapped a workflow with the operations lead, built to the map, and watched it fail on contact with reality — because the map didn't include the fact that three of the regional offices had, for perfectly good reasons, been doing it differently for years, and nobody had told corporate, because why would you.
That's not dysfunction. That's what a living business looks like. But it means that if you build to the org chart's version of the process, you will build something correct and useless.
The fix is unglamorous: go sit with the person who does the work. Watch them do it. Not a workshop, not a survey — watch. You will learn more in ninety minutes at their desk than in six hours in a conference room, and you will discover the two exceptions that would have quietly destroyed the project.
What's actually different now
Two things, and they're big.
The first is capability. Work that required a labeled dataset and a data science team now requires a well-written instruction and a good evaluation harness. The unlock isn't that the machines got smarter — it's that the on-ramp collapsed.
The second is who it's for. In 2019, this technology was available to companies with a seven-figure innovation budget and a team of specialists. That's it. That was the whole market. A twenty-person brokerage or a forty-person hospitality group could not have touched it, and would have been badly served by anyone who tried to sell them a version of it.
That's the change worth caring about. The same class of system that used to require an enterprise transformation office can now be built for an owner-led company in weeks, at a price that clears a 2x return in the first year. Not a watered-down version. The same thing.
The thing that didn't change
Discipline.
The reason so many small companies got burned in the last eighteen months is that the on-ramp collapsed but the diligence didn't come with it. It became so easy to build something that people stopped asking whether they should. Pilots multiplied. Nothing shipped. Everyone got tired.
The old constraints — cost, scarcity, difficulty — used to enforce a kind of rigor by accident. You couldn't afford to build the wrong thing. Now you can, easily, and many people have.
So we carry the old discipline into the new tools. Price the problem before you scope the solution. Watch the real process, not the documented one. Show the machine's uncertainty. Decide up front how you'll know if it worked.
Six years of doing this before it was fashionable taught us mostly that the fashion is irrelevant. The businesses that win with this stuff are not the ones with the newest model. They're the ones who were honest with themselves about what was actually broken, and who gave their people something worth doing with the time they got back.