Back to all posts

When Stripe Radar Isn't Enough: Closing the Free-Trial Fraud Gap With an AI Agent

Stripe Radar is excellent at what it does: scoring a transaction the moment money moves. But the moment money doesn't move is exactly where our problem lived.

We run a 7-day free trial. To start it, a user saves a card but is never charged — under the hood, that's a Stripe SetupIntent, not a payment. No charge means no charge-time event, which means Radar's most powerful rules have nothing to evaluate at the moment that matters most: when a fraudulent account is being created and starting to burn trial credits.

This is the story of how we went from firefighting fraud in one-off chats to running an automated, minute-by-minute fraud layer — built on top of our own product, Zero.

Radar is great. "Great" isn't "perfect."

Stripe Radar is genuinely good at what it does. Trained on more than $1 trillion in annual payment volume, its models recognize a card they've seen before 92% of the time, and Stripe reports Radar reduces fraud by 32% on average for the businesses that use it.

Read that number again, though: reduces, not eliminates. A 32% average dent is a big one — but it means a large share of fraud pressure is still arriving at your door, and Stripe is refreshingly honest about why. In their own words, "reducing the number of false positives often increases the likelihood of more true fraud slipping through the cracks." Leakage isn't a bug in the model; it's the deliberate cost of not blocking your real customers. Every fraud team is choosing, on purpose, how much to let through.

For a trial-led product, that leak is wider than the headline number suggests — because the most powerful part of Radar never even runs.

Why a gate can only ever reduce

There's a deeper reason a charge-time engine can reduce fraud but never end it, and it isn't about Stripe's models — it's about timing and information. At the instant a card is added, the customer has done nothing yet. No sign-in history, no product behavior, no usage pattern to learn from — just a card and a handful of network signals. Radar is making its best call on the thinnest possible slice of evidence, at the one moment it has the least to go on.

That's the real ceiling. You can tune a gate, but you can't give it data that doesn't exist yet. The information that actually separates an abuser from a customer is mostly created after the gate has already had to decide.

Stage 1: Firefighting in the chat window

Our first response was entirely manual. When a wave of abuse hit, someone opened a conversation with our agent and worked the incident live: pull the recent signups, look at the cards, find the pattern, shut the accounts down by hand.

It worked, but it didn't scale. Every incident started from zero. The knowledge — what does this ring look like, which signals are reliable — lived in one person's head and one chat thread, and evaporated the moment the window closed.

We also learned an expensive lesson on the good side of the funnel: when we wanted to win back genuine users who abandoned checkout, a careless outreach email to a fraudulent address does real harm — it confirms the address is live. Any automation we built would have to tell the difference between a real prospect and a planted one.

Stage 2: Hardening the front door with Radar

Our next move was to push everything we could upstream into Radar — block lists and velocity rules at the card-add step, tuned to the pressure we were seeing. It's the right first move, and it stopped the crudest attacks.

But two ceilings showed up fast. The first is the one Stripe names itself: a charge-time engine is a tradeoff dial, not a wall — turn it up and you block real customers, turn it down and more fraud slips through. The second is structural and specific to trials: that 32% is earned at charge time, and a SetupIntent trial has no charge to score. Unless you explicitly enable Radar on saved payment methods, the engine doesn't run at all — and even when it does, only Block, Allow, and 3DS rules apply on a SetupIntent. The "Review" action, the one that buys a human a second look, never fires.

So the single best-performing layer in the stack is, by default, sitting out the exact moment we needed it most. Radar is a gate at the front door. We needed a patrol behind it.

Stage 3: A second layer, built on Zero

After signup, the information picture flips. The account starts leaving a trail — and the richest part of that trail lives in a system the payment processor never sees: our identity provider, Clerk.

So we gave Zero access to it. Clerk knows things Stripe can't: how the account was created, the signup method and email, the session and device behind it, and the precise timing relative to every other signup that day. Stripe knows the card. Neither system, on its own, tells the whole story — but the agent can read both and correlate across them. The abuse that's invisible at the gate becomes obvious side by side: a brand-new account whose sign-in identity doesn't line up with its billing identity, created moments apart from a cluster of lookalikes. That cross-system join is exactly what a charge-time gate can't make — and exactly what an agent sitting on top of both systems can.

On that richer evidence base, we run a set of scheduled Zero tasks every few minutes against live signup and billing data. Three principles shape them:

1. Patrol, don't just gate. Instead of evaluating one moment, the agent sweeps every recent signup on a short loop, correlating account data and payment metadata to surface accounts that slipped past the front door.

2. Tier the response by confidence. Not every signal deserves the same action. High-confidence patterns are handled automatically — the account is suspended and its trial canceled on the spot, because the action is reversible and the cost of waiting is real. Lower-confidence signals are never auto-actioned; they're collected into a report for a human to review. Decisive where it's safe, conservative where it isn't.

3. Keep a human audit trail. Every automated action records the exact signal that triggered it, so it can be reviewed — and reversed — in seconds. Automation you can't audit is automation you can't trust.

We applied the same rigor to the friendly side of the funnel. A separate task finds genuine abandoned-checkout users and drafts a win-back email for a human to approve — behind a deliberately heavy fraud filter, so we never validate a bad address. Same engine, opposite goal.

What it did to the economics

Once the patrol was live, the numbers moved immediately. Look at the 90th-percentile signup — the heavy accounts that did the real damage. The trial compute one of them burned in its first hours fell from about $4 to about $0.25 — a 94% drop. Averaged across every new signup over the same window, per-account loss fell by roughly 85%.

The shape of the change is the tell. Most signups never cost us anything; the loss was always concentrated in a long, heavy tail of accounts that existed only to drain trial credits. The patrol didn't shrink the funnel — it cut that tail off. Not fewer signups. Cleaner ones.

Why an agent, and not a script

We could have written a cron job. We didn't, for one reason: the threat changes faster than a release cycle. When an attacker shifts tactics, we update a task's instructions in plain language and the new logic is live on the next run — no deploy, no schema migration, no ticket. The "rules" are a prompt, and the prompt is as fast to change as the attacker is.

That's the real lesson. Radar is the right tool for charge-time risk, and we lean on it hard. But a trial-led business has a structural blind spot that no charge-time tool can cover — and the fix isn't a bigger rules engine. It's a fast, auditable, always-on second layer that you can reprogram at the speed of the threat.

Takeaways for trial-led teams

Curious what an always-on agent could automate in your own stack? Explore Zero.


Notes: Radar figures are Stripe's own published numbers (stripe.com/radar; "A primer on machine learning for fraud detection"). Loss figures reflect compute cost per new signup in the first hours after registration, measured over matched windows before and after launch, with internal accounts excluded; the post-launch sample is still early and will firm up over time.

Stay in the loop

// Get the latest insights on AI teammates and collaboration.

SubscribeJoin Discord