5 Questions That Filter AI Vendor Hype

Three demo requests in your inbox. Your board asking why you haven't "done something with AI" yet. A competitor slapping "AI-powered" on something that worked fine without it. Sound familiar? Yeah.

You don't need more demos. You need a filter.

Twenty years. That's how long I've been sitting across from vendors who promise the world. I've developed five questions that kill 80% of AI pitches in the first meeting. Most vendors can't survive question three. Run every AI product through these five questions. If it can't pass all five, it's not ready for your money.

Question 1: What Specific Task Does This Replace or Improve?

Not "what category does it serve." Not "what problem space does it address." I mean the actual task. Performed by a specific person. On a specific day. What does this product improve?

If the vendor can't name the task, walk. They're selling a solution looking for a problem. Here's what the difference sounds like:

Both describe the same product. One tells you exactly what you're buying. The other tells you nothing.

Question 2: What Does It Get Wrong, and What Happens When It Does?

Every AI system gets things wrong. Every single one. The question isn't IF it fails. It's whether you can survive the failure.

A 95% accuracy rate sounds great. Until you do the math. On 10,000 customer interactions per day, that's 500 wrong answers. Every. Single. Day. A wrong shoe recommendation costs you a return. A wrong medical dosage costs you a lawsuit. Same accuracy rate. Very different consequences.

Ask the vendor for their error rate on data similar to yours. If they don't have one, they haven't tested it on data similar to yours. Walk away until they have.

Question 3: What Does It Need From Us to Work?

AI products don't run on enthusiasm. They need data, infrastructure, and people. Here's the thing. Nobody wants to talk about this part. Ask anyway:

• •Data: What format? How much history? How clean does it need to be?
• •Infrastructure: Cloud, on-premise, or theirs? What are the security implications?
• •People: Who configures it? Who maintains it? Who monitors the output?

The vendor will minimize these requirements. Always. Your implementation team will not. Get the real numbers from the people who actually build it, not the people who sell it.

Question 4: What's the Total Cost for the First Year?

Not the license fee. The total cost. Let me be blunt. In most enterprise AI deployments, the license fee is 20-30% of the first-year total. The rest is everything else. Implementation. Infrastructure. People. Opportunity cost.

Vendor quotes you $100K for a license? Plan for $300-500K all-in for year one. That gap is where budgets die and projects stall. Every time.

Question 5: How Do We Measure Success in 90 Days?

Not "what's the long-term vision." I want a specific, measurable outcome you can see in 90 days that tells you this was worth the money.

Good metric: "Invoice processing time drops from 4 hours/week to 30 minutes/week." Bad metric: "Increased operational efficiency." One is a number. The other is a press release.

If the vendor can't define success with you before you sign, they don't understand your use case well enough to deliver on it. Period.

The Red Flags to Watch For

Beyond these five questions, here are the phrases that should make you reach for your wallet. To put it back in your pocket:

• •"Our AI is proprietary." Unless they're training foundation models, they're wrapping someone else's. That's fine. But don't pay a premium for something they're renting.
• •"Up and running in two weeks." For a demo, maybe. For production with your data and security requirements? No.
• •"This will pay for itself." Discount vendor ROI projections by 50%. If the business case still holds, you might have something.

Putting It All Together

Here's what this looks like in practice. Your CTO brings you a proposal for an AI-powered customer support system. You run the filter. Task is specific: Tier 1 tickets. Error rate is honest: 87% on your data. Requirements are clear. Total cost is $205K against $180K in savings. The 90-day metric is measurable: 60% automation rate.

That's a real business case. Not exciting. Not revolutionary. Just math that works. That's what good AI adoption looks like. Not a TED talk. A spreadsheet.