Pumpkin
    The AI Mistakes Firms Keep Making
    AI

    The AI Mistakes Firms Keep Making

    By Aaron WatersNovember 19, 2024Updated August 15, 20267 min read

    Most AI mistakes firms make are versions of one mistake, which is buying the technology before naming the problem. Everything downstream of that, the shelfware, the frustrated staff, the vague sense that this whole thing was oversold, follows from the same original error.

    The technology is good enough now to deliver real value. Whether it does depends almost entirely on decisions that have nothing to do with the software.

    Here are the ones that keep showing up.

    The tool that arrived before the problem

    A partner sees a demo, gets excited, buys. Six months on it's a line item nobody can defend, because it was never connected to a workflow anyone actually runs.

    The fix costs nothing. Write down the specific task that's consuming hours, name the person consuming them, and only then go shopping. If you can't complete that sentence before the purchase, you're buying a feeling. There's a sequencing argument for exactly this in what firm owners should automate first with AI.

    Expecting it to read your mind

    These tools can't infer context they were never given. They don't know your client, your preferences, your firm's history with a particular treatment. Given a one-line prompt they produce one-line-prompt quality, and then the firm concludes the technology doesn't work for accountants.

    Output quality tracks input quality almost linearly. The firms getting good results learned to write requests the way they'd brief a capable new hire: here's the situation, here's what I need, here's an example of what good looks like, here's what to avoid. It takes ninety seconds and it changes everything about what comes back.

    Handing client data to a vendor nobody checked

    This is the one that should worry you most, and it gets the least attention in the buying process.

    Your firm holds tax returns, financial records, Social Security numbers, business strategy, and personal details people would be mortified to see published. Some tools take what you feed them and use it to train their models. Some store it in jurisdictions with rules you haven't read. Some have security practices that wouldn't survive ten minutes of questioning.

    Before any tool touches client data, get answers in writing to six things. Where is the data stored. Is it encrypted at rest and in transit. Is our data used to train models. Who at the vendor can access it. How long is it retained after we cancel. What certifications do they hold and can they show the report rather than the logo.

    A vendor who answers these crisply has been asked before, which is itself the signal. A vendor who gets vague has given you your answer.

    Six workflows at once

    Enthusiasm is fine. Attempting a firm-wide transformation in one quarter is not. It overwhelms the team, creates integration problems that nobody can isolate, and typically ends with nothing running properly.

    Sequential works. One workflow, running well, understood, then the next. The second one goes faster because most of what you learned in the first was about your own firm rather than about software. A workable pace for a small firm is a first automation live in two to four weeks, the next one a month or so later, and a third after that. Inside six months you'd have three things running and the experience to keep going. The structured version is in AI workflow automation for small firms.

    Announcing it instead of asking

    Some owners pick the tools, then inform the team at a Monday meeting. Then they wonder about the adoption numbers.

    People resist changes they had no part in, especially changes they suspect are aimed at their job. They find workarounds. They use the old process quietly and let the new tool rot.

    Involve them in identifying the pain first, since they're the ones in the pain. Let them test the candidates. Take their objections seriously, because staff objections to software are usually accurate and usually about integration.

    And address the job question directly. If the goal is to eliminate tedious work rather than positions, say so plainly. If the goal actually is headcount, say that too, because they'll know by the second quarter regardless and the alternative is spending your credibility to buy a few months of quiet.

    The review step that quietly disappears

    Every implementation starts with a review step. Somewhere around month four, when the outputs have been right for weeks, the review starts happening faster, then in a glance, then not at all. Nobody decided this. It just drifted.

    Then one output is wrong in a way that looks completely normal, and it goes to a client.

    Match the review depth to the stakes. A routine internal email needs a scan. Anything with numbers going to a client needs a person who actually checks the numbers. That's especially true for anything generating narrative on top of financial data, where a plausible-sounding sentence about why margin moved can be confidently, entirely incorrect. It's why financial statement analysis should be built as a first draft for a professional rather than a deliverable, no matter how good the drafts get.

    You can loosen review over time, per task, on evidence. Start tight.

    Counting tools instead of outcomes

    Some firms measure AI progress by how many tools they've adopted. That's like measuring fitness by counting gym memberships.

    Pick the outcomes before you implement: hours recovered on a named task, error rate on a specific process, time from client inquiry to first response, capacity per staff member during your worst month. Then measure them, and be willing to conclude at ninety days that a tool isn't working.

    Related and just as common: the reporting dashboard nobody opens. If a report has never changed a decision anyone made, it isn't reporting. It's decoration, and it's costing you a subscription.

    Skipping the unglamorous foundation

    Firms rush toward the visible applications, client-facing chat, automated reporting, anything that demos well, while ignoring the fact that nobody has written down how their processes work.

    That's backwards, and it shows up as failed implementations six weeks in, when the vendor asks how the current process runs and three people give three answers. Using AI for SOPs is the lowest-risk application there is, it captures the knowledge that's currently walking out at 5pm, and it produces the documentation every later automation depends on.

    You can't automate a process you can't describe.

    Assuming the risk only points one direction

    One more, because it doesn't fit the pattern of the others. The same technology that's making your document workflow faster has made phishing dramatically better at scale, and firms holding financial data are a favorite target during the exact weeks they're least alert.

    The partner meeting that approves an AI budget should spend ten minutes on that too. Not as a reason to slow down. As a reason to include security questions in the same conversation rather than a different one nine months later.

    Waiting, which costs the most

    The irony of an article like this is that reading a list of mistakes makes a certain kind of careful person decide to wait. Wait for the tools to settle. Wait for a better budget year. Wait until after busy season, then after the next one.

    The firms that started experimenting a couple of years ago aren't ahead because they chose better software. They're ahead because they've made all these mistakes already, in small, cheap, recoverable ways, and now they know things you can only learn by doing.

    Pick one problem. One tool. One workflow. Get it wrong in a way you can afford, and go from there. Our guide to AI for accounting firms lays out the order that tends to work.