How to not blow up your pilot

Title: How to not blow up your pilot


Read Time: 2.5 min

Pilot purgatory is real and looks a lot different in AI than it did a year ago, but the cause is exactly the same, nobody ran a real scoping call with specific success criteria.

I can't tell you how many companies I know that are running pilots with zero defined success outcomes. Just hoping the customer will be so impressed they'll come tripping over themselves to write a check.

Because there's such a big push to adopt AI right now, the customer wants to look like they're evaluating it, but no one on their side has actually agreed on what success means, who owns the decision, or what happens if you deliver precisely what they asked for. Without that, you're not running a pilot. You're providing free R&D to someone who was never going to buy.

Once success is actually defined, the pilot's only job is to prove you can hit it. If you don't know what "it" is, you can't hit anything. That blind spot is where AI pilots specifically get smoked, because the tech is new enough that customers themselves often can't articulate what they're validating until you force the conversation.

We see the same pattern across founders. The deal looks somewhat alive on usage. A few people are logged in, poking around the tool. But nobody's actually committed to anything. There's no defined problem, no metric, no date by which the customer has to say yes or no. It feels like momentum.

And when nothing's defined, nobody owns it. The customer says they're, "still evaluating." You say you're "in a pilot." Neither of those means anything without an agreed definition of what a win looks like, and that's how a pilot rots.

The $40K pilot that sat for six months

We had a client come on in late May with a pile of deals stuck at the bottom of the funnel. One in particular had been sitting in a $40K pilot since early April, six months, no movement.

People were logging in. Nobody was actually engaged. Shallow usage, no depth, no real signal either way. The deal had come in through an investor intro, which meant it skipped the step that mattered most, there was never a scoping call, no agreed success criteria, no owner on either side.

So we reset the whole thing. We got the team back on a call and ran the scoping conversation that should've happened on day one. We forced specifics on what success actually looked like, and then asked the question directly: if every box gets checked, are you ready to move forward? Once we had a real yes, we built a plan around it, milestones, owners, review dates, and a defined next step for every box.

We re ran the pilot against that criteria over 45 days. We're now close to the finish line, and we're almost certain it closes, not because the product got better, but because the pilot finally had a target.

The whole point of a scoping call is to get painfully specific about what the customer wants, what "validated" means to them, and then build the pilot backward from that outcome. Once you know the target, you design the pilot to walk them straight to it instead of hoping they stumble into the value on their own.

Here's how to actually run that conversation.

3 tactical steps for running a real scoping cal

1. Set the agenda and replay their pain back to them.

Open with a short frame: the goal today is to define what success looks like for this pilot, map how we get there, and lock in what happens next if it hits.

Then play back what they've already told you, the problem they described, what they were hoping this would solve,in their own words. Ask if that's still accurate, and whether anything's shifted since you last talked. It shows you were listening, and it gives them room to correct the picture before the real conversation starts.

2. Nail down success, ownership, and risk.

Work through this in buckets, and don't leave any of them soft:

• Success metric: what specifically has to change or improve for this to count as a win?l

• What they're actually validating: performance, reliability, workflow fit, cost, something else entirely?

• Who owns it: who's accountable for the result on their side?

• How it'll run: who's using it, how many people, what does real usage look like versus a login?

• Check-ins and friction: when do you check in, and what could slow this down, security review, data access, approvals, integration work?

Walk away from this part with a real scoreboard, a real owner, and a short list of known blockers. If you don't have all three, the scoping isn't done.

3. Turn it into a plan with dates and a decision point.

Once success is defined, make it tangible. Restate the commitment on both sides, what you're doing, what they're doing. Lock the pilot window: start date, end date, one or two milestones in between.

Book two things before you hang up: the kickoff and the decision review at the end. Follow up with a short mutual action plan, success metrics, owners, dates, and where you'll communicate, whether that's Slack, email, or wherever they actually live.

Nobody should hang up that call wondering what happens next, or what happens if the pilot works. The path from "trying it" to "signed" needs to already exist on paper before the pilot even starts.

Key takeaways

A scoping call exists for one reason: to define what a win looks like before anyone touches the product. If you skip that, you have no idea what actually moves the needle for them, you're guessing.

Every pilot needs:

• Clear metric

• Clear owner

• Clear timeline before it kicks off.

• Start date

• End date

• Decision meeting booked before day one.

If you can't name the boxes that need checking before you start, the pilot isn't ready to start.

In AI specifically, watch for pilots that exist because of a mandate, not a problem, those are the ones most likely to sit in purgatory for six months and go nowhere.

The pilots that turn into revenue are the ones where both sides agreed, up front, what winning looks like, who owns it, and what happens the moment you hit it.

That’s it for today!

See you all next week.


Darren

P.S. If finding PMF and scaling to $1M in ARR through founder-led sales is on your radar, book a call with me here.

💡 How We Can Help