A custom AI tool is software built around one specific workflow in your business, grounded in your own documents and rules, instead of a general-purpose platform you adapt to, and it does one job end to end and hands the decision back to a person. That definition matters, because "custom AI" gets used for everything from a system prompt in a chat window to a full production application, and the real difference between them shows up the first time an answer is wrong.
We build these tools for a living, so we have every reason to tell you to build one... That being said, the honest first answer is usually buy, and the rest of this post is how to tell which side of the line you're on.
So, it helps to know how often AI projects go nowhere. In 2025, MIT's Project NANDA reported that about 95% of the generative-AI pilots it looked at, drawn from roughly 300 public deployments, produced no measurable profit-and-loss impact. Treat that figure as directional, because it comes from a preliminary report that spread through press coverage and was never peer reviewed, and an August 2026 arXiv study that cites it treats it the same way. The long and short of it is that the study traces those failures to sponsorship, skills and changes in how people work, and every item on that list is about people and process.
When buying is the right answer
Buy when the workflow is common, the stakes of a wrong answer are low, and a product already fits how your team works. Scheduling, email, bookkeeping, CRM, transcription and document storage are solved problems, and buying is faster, cheaper, and someone else maintains it.
The trap is buying a platform and then bending your business to fit it. For instance, if the product needs three people to keep it fed with data it should be reading itself, you bought a second job.
When building is the right answer
Look for three signals, and you usually need at least two:
- The work is specific to you. Your process is the product of years of decisions no vendor modeled, and a tool that doesn't know those rules produces answers your team has to check, which is the work you were trying to remove.
- The answers have to be exactly right. Where a wrong answer costs real money or a real relationship, a general model that guesses confidently is worse than no tool at all.
- It has to reach systems you already run. A lot of the value sits in the connection, and that means pulling from the database, writing to the CRM and reading the documents nobody has time to read.
Why prompt-stuffed agents break
The common failure is asking one large model to do everything, so it reads the inputs, applies the rules, does the math and writes the answer. It works in a demo and breaks in production, because a model that is asked to reason about rules will eventually reason its way to a wrong one, and it will do it in a fluent, confident paragraph nobody catches.
That failure is documented. The GSM-Symbolic study (October 2024) found that changing only the numbers in a math question lowered the accuracy of every model tested, and adding one irrelevant clause cut performance by up to 65%. OpenAI's own researchers explain why models produce plausible but wrong answers instead of admitting uncertainty, and the short version is that training and grading reward a confident guess over "I don't know."
"AI Agent" projects, where a model is handed a whole job, carry the most risk. In June 2025 Gartner predicted that more than 40% of agentic AI projects will be cancelled by the end of 2027, pointing to rising costs, unclear business value and weak risk controls. That number is an analyst forecast, and the same arXiv study labels it as one.
The architecture that survives contact with real work separates the two jobs. A deterministic rules engine makes every routing and recommendation decision; the model only writes the prose. Same inputs, same answer, every time, and that is something a model can't promise on its own. In September 2025, Thinking Machines Lab sampled one prompt about 1,000 times at temperature zero, the setting meant to remove randomness... and got about 80 different completions. With the rules engine deciding, every answer can be traced to the rule that produced it, and the model does what models are good at, turning a structured result into language a person can act on, and nothing else.
That is the difference between a tool your team trusts after a month and a tool your team quietly stops opening.
What this looks like in practice
The flagship system we built runs daily in a regulated industry where a wrong answer costs a real person a real outcome. It answers from 170+ dense source documents, was validated against 1,000+ real cases, and turned a research job that took 90–120 minutes into about 45 seconds. The person still makes every recommendation, and they just stopped spending the afternoon assembling the inputs.
The same architecture runs the Bottleneck Finder™, our free diagnostic: 7 routing layers, 17 routing rules, 19 mapping rules and 59 framing moves, tested across 42,000 answer combinations and 57 industries. None of that is AI cleverness. It is the boring work of writing down what the rules actually are, which is also the work that makes the output auditable.
That is what a build that fits gets you: less time assembling inputs, fewer answers anyone has to double-check, the same answer on a Friday afternoon as on a Monday morning, people spending their day on the part that needs a person... all of it.
Where this doesn't work
Plenty of AI work gets abandoned. In a survey of more than 1,000 respondents in North America and Europe, S&P Global Market Intelligence found that the share of companies abandoning most of their AI initiatives rose to about 42% in 2025, up from about 17% the year before, and the average organization scrapped about 46% of its proofs of concept before they reached production (CIO Dive, March 2025).
Custom builds are the wrong answer when the process itself is undefined. For instance, if five people do the same job five different ways and nobody can say which is correct, a tool will just automate the disagreement. Define the process first, on paper, with the people who do it, and sometimes that exercise is the fix and no software gets built.
They're also wrong when the volume doesn't justify them. If a task happens twice a month and takes about twenty minutes, the honest answer is to leave it alone.
So, if either of those describes your business, don't hire us yet. Write the process down first, and call us if it's still costing you hours once it's on paper.
How to decide in one sitting
Write down the workflow that costs your team the most hours this month, and then answer three questions. Does a product already do this the way we work? Would a wrong answer cost us real money? Does the tool need to read or write something in a system we already run?
Three nos means buy something. Two yeses means the build case is real, and the next step is scoping it before anyone goes shopping.
If you'd rather have the answer in numbers, the Bottleneck Finder asks seven questions and gives you a written brief with what to fix, what it costs and what to skip, and it can and does tell people to spend nothing. If you'd rather read first, here's how we build custom AI tools and what we fix. Is that worth three minutes to find out? No worries either way.