Thanks AndersonInc.
What We Fix Custom AI Tools Sales Training AI for Agencies AI Visibility Governance Better Prompt Blog About Contact →
← All posts

What safeguards should I expect before an AI vendor touches my website?

Before an AI vendor changes your website, ask them to show you six things: where your prices and facts live, what checks every change, a preview you approve, who makes the decisions, a test after launch, and the plan when something breaks. A vendor who has these can walk you through all six on one short call. A vendor who doesn't will tell you not to worry.

A safeguard is a check that catches a mistake before your customer does. Some run on their own, like a script that refuses any change with a broken link in it, and some are a person, like you opening a preview on your phone before it goes live. An AI vendor is anyone who builds or runs something for you that uses AI, short for artificial intelligence, such as a chatbot that answers your customers at night or a tool that writes and updates your web pages.

What does it cost me if nobody checks?

Usually it's small and embarrassing, and you hear about it from a customer. A price on one page that doesn't match another, a booking button that goes nowhere, a chatbot that promises a refund you don't offer... each one costs you a sale or an awkward phone call, and you never find out how many people saw it and just left.

AI makes this more likely. Tools like ChatGPT write their answers fresh every time, so they rarely say the same thing the same way twice, and an AI tool can be right on Monday and wrong on Thursday. That's why the checks have to run on every change and not just once when the vendor sets things up.

What should I ask an AI vendor to show me?

These six, and ask them to show you rather than tell you:

  • Where your facts live. Prices, hours, names and numbers should sit in one place that every page reads from. Change a price once and it changes everywhere. If they type it into each page by hand, sooner or later one page will be wrong.
  • What checks every change. Something should run automatically before anything goes live, and it should stop the change on its own if it finds a broken link, a wrong number or a word that shouldn't be there.
  • A preview you approve. You should see every change before your customers do, on a link you can open on your phone.
  • Who makes the decisions. If the AI quotes a price or recommends something, that answer should come from your rules or your own data, and the AI only writes the sentence around it.
  • A test after launch. Someone should click through the live site the way a customer would, forms and booking included.
  • The plan when something breaks. Undo it first, then a plain-English note to you, then a fix so the same thing can't happen again.

A good answer is specific: a file, a check or a step they can pull up while you watch. "Don't worry, we're really careful" is an answer too, just not the one you want. Here's what each kind sounds like.

Six questions to ask an AI vendor, with what a good answer and a red flag sound like
What a good answer and a red flag sound like for each of the six.

How do I know these six actually matter?

Because we run them on our own site, and they've caught us. On October 21 we wrote up eight small mistakes that had reached thanksanderson.com and our own tools, like a booking link that pointed at a page we'd deleted and a page asking visitors to type their email into a box that wasn't there. None of them were big, and every one was the kind of thing a customer finds before you do. Each one traced back to one of the six being missing or only half done.

So, every change to our site now runs a check for broken links, banned wording and anything that looks like a password before it can go live. Our How we build page lists the rest, including the parts we're still building, and it's the same baseline we bring to an AI governance and evaluation engagement.

Do the people who build with AI trust it?

Mostly not, and that's the best argument for these checks. Google's 2025 DORA report, published September 23, 2025 from a survey of nearly 5,000 technology professionals, found 90% of software professionals use AI at work, while only 24% trust it "a great deal" or "a lot." The people closest to these tools use them every day and still check the work, and your vendor should be doing the same on your site.

None of the six are our invention, either. They're how careful software teams already work, and Google's engineers have written about the habit of fixing the process after every failure, instead of blaming a person, in their Site Reliability Engineering book. Done right, you get fewer surprises, fewer refunds, fewer awkward calls with customers and a vendor you can actually hold to something... all of it.

Where do safeguards fall short?

A check only catches what someone thought to check, which is how most of our eight got through. Safeguards also cost time, and a small change can take a day longer with a preview and a test. On a site that takes bookings, that's worth it. On a brochure page you touch twice a year, ask for the preview and the undo and skip the rest.

They don't replace you, either. A preview only works if someone who knows the business reads it, and no script knows a price is wrong if it's the price you gave the vendor. I sell this kind of work, so weigh all of this accordingly.

Can your current vendor show you those six? If you're not sure, send me your website through the contact page and I'll tell you which of the six I can see from the outside. No worries either way.

Free AI Audit — the Bottleneck Finder™ No email · No signup · Or book a call now

Common questions

What safeguards should an AI vendor have before changing my website?

Six, and they should be able to show you each one. One place where your prices and facts live, a check that runs on every change, a preview you approve before anything goes live, your rules deciding anything the AI quotes, a test of the live site after launch, and a plan for when something breaks. If a vendor answers with "don't worry, we're careful," you're being asked to trust them instead of check them.

Why does AI need more checking than regular software?

Regular software breaks the same way every time, so once you find a problem it stays found. AI tools such as ChatGPT write their answers fresh each time and rarely word things the same way twice, so an answer that was right on Monday can be wrong on Thursday. That's why the checks have to run on every change, not just once at setup.

What is a blameless postmortem?

A postmortem is the write-up a software team does after something breaks. Blameless means the write-up is about how the process let the mistake through, never about who made it, so people report problems early instead of hiding them. Google's engineers describe the practice in their Site Reliability Engineering book.

Is it normal for a small AI vendor not to have SOC 2?

Yes. SOC 2 is a formal outside audit of a company's security controls, and it costs more than most small firms can justify. What a small vendor can show you is a written standard and a record of following it. If your contracts require SOC 2, ask on the first call, before anyone builds anything.