Your Software Vendor Is Pitching an AI Add-On. How Do You Evaluate It?

If you pay for business software, you’ve probably gotten this email: “Introducing [Product] AI.” Your CRM, your practice management system, your phone provider, your accounting software, your marketing platform. Everyone has an AI add-on now, and your account rep would love to schedule a demo.

Some of these add-ons are genuinely useful. Because they live inside software your team already uses all day, with your data already in it, they can be the easiest AI win available. Others are thin features with a new price tag, or a reason to push you onto a more expensive plan.

Here’s how to tell the difference.

Why Vendor Add-Ons Deserve a Fair Look

It’s easy to be cynical, but built-in AI has real advantages:

It’s where the work already happens. No copying information between tools.
Your data is already there. A CRM’s AI can see the account history. A standalone chatbot can’t, unless someone pastes it in.
It’s one less vendor. Your existing contract, security review, and admin controls often already apply.
Adoption is easier. A button in a screen people already use gets used more than a new app.

So the question isn’t “is vendor AI good or bad?” It’s “is this feature worth this price for our team?”

Step 1: Pin Down Exactly What It Does

Demos are designed to impress. Before you evaluate anything, get a specific answer to: what task does this do, for whom, and what does the output look like?

“AI-powered insights” isn’t an answer. “Drafts a follow-up email after each logged call, based on the call notes, for the rep to review and send” is.

Then ask your team: how often do we do that task, and how long does it take now? If the answer is “rarely” or “a few minutes,” the feature can be excellent and still not be worth paying for.

Step 2: Understand the Pricing Model

AI add-on pricing comes in a few forms, and some are easier to predict than others:

Per user per month. Simple, but check whether you must buy it for every user or can choose who gets it.
Plan upgrade. The feature is only available on a higher tier. Calculate the full cost of the upgrade, not just the AI part.
Credits or usage-based. You buy a pool of AI “credits” or pay per action. Ask what a typical user consumes, what happens when credits run out, and whether unused credits expire.
“Free for now.” Included at no cost during a launch period. Ask what the pricing will be later and whether you’ll be automatically enrolled.

Get the price in writing for your actual number of users, for a full year.

Step 3: Ask About Your Data

This is where you need clear answers, especially if you handle customer, patient, or financial information.

Is our data used to train AI models, either yours or a third party’s? Is that on or off by default, and can we opt out?
Which AI provider powers the feature? Many vendors use models from OpenAI, Anthropic, Google, or others behind the scenes. That’s normal, but you should know who is processing your data and under what terms.
Where is the data processed and stored, and for how long are prompts and outputs kept?
Does the AI respect our existing permissions? If a user can’t see a record, can the AI show it to them?
Do our existing contract terms cover it? For healthcare, does the Business Associate Agreement extend to the AI feature?
Can we turn it off, for specific users or entirely?

A good vendor will have a written answer to each of these. If the rep has to “check with the team” on every question, ask for the documentation.

Step 4: Test It on Your Own Work

Never decide based on the demo alone. Demo data is clean, and demo scenarios are chosen because they work.

Ask for a trial with a few real users on real work. Two to four weeks is usually enough.
Pick specific tasks to test and note how long they take with and without the feature.
Check accuracy. Have someone who knows the work review the output. Look for errors that would be embarrassing or costly if they reached a customer.
Ask the users. Did they keep using it after the first few days, or did they drift back to the old way?

If a vendor won’t offer any kind of trial for a feature you’d be paying for all year, that’s worth noting.

Step 5: Compare It to What You Already Have

This is the step vendors hope you skip. Before paying for an AI add-on, ask whether you already have something that does the same thing:

Microsoft 365 Copilot Chat, Microsoft 365 Copilot, or Gemini in Workspace may already handle the drafting and summarizing the add-on offers.
ChatGPT, Claude, or another assistant your team already pays for may do it well enough, even if it’s less convenient.
Another vendor in your stack may be adding the same feature. Your phone system and your CRM might both offer call summaries. You don’t need both.

Paying twice for the same capability is one of the most common ways AI spending gets out of hand. Looking across your whole stack, not one vendor at a time, is how you avoid it.

Red Flags

Vague outcomes. “Transform your business,” “unlock insights,” with no specific task attached.
Numbers you can’t check. “Customers save 10 hours a week” without saying which customers, doing what, measured how.
Mandatory bundling. You can only get the AI feature by upgrading every seat to a higher plan.
Unpredictable usage pricing. Credits with no clear estimate of what you’ll use, or overages with no cap.
Evasive data answers. No clear statement on training, retention, or which providers process your data.
No trial, long commitment. A one-year or multi-year contract for a feature you haven’t used.
Pressure tactics. “This price is only good until the end of the month” for a brand-new product.
It automates decisions you’re accountable for (approvals, eligibility, billing codes, hiring) without clear human review.

When It’s Not Worth It

The task is rare or already fast. Saving two minutes on something done twice a week won’t justify a per-user fee.
You’d only use a fraction of the upgrade. If the AI feature is the only reason to move to a higher tier, the math often doesn’t work.
You already have an assistant that covers it. A little less convenience may not be worth a new monthly bill.
The feature is brand new. First versions of vendor AI features often improve a lot within six months to a year. Waiting is a legitimate choice.
Your data in that system is a mess. AI built on incomplete or outdated records produces confident, incomplete, outdated answers. Clean up first.
Nobody will change how they work. If the team won’t use it, the price doesn’t matter.

A Simple Decision Checklist

Before saying yes, you should be able to answer:

What specific task does it do, and how often do we do that task?
What will it cost per year for the users who actually need it?
Do we have clear, written answers about our data?
Did it work on our real work during a trial?
Do we already pay for something that does this?
Who on our team will own getting people to use it?

If you can answer all six and the numbers make sense, it’s probably a good buy. If you can’t, you’re buying the demo.

The Bottom Line

Vendor AI add-ons can be the easiest, most practical AI improvements you make, precisely because they live where your team already works. But they’re sold one vendor at a time, and nobody at those vendors is looking at your whole stack. Evaluate each one on a specific task, a real trial, clear data terms, and whether you already own something that does the same job.

If you’re getting AI pitches from several vendors and want an objective read on which ones are worth it, book a strategy session. I’m not paid by any software company, so I’ll tell you plainly which ones to take, which ones to skip, and which ones to revisit later.