Don’t Be Next: Can AI Spot Romance Scammer Scripts Before It’s Too Late?

Marcus Delray — Thu, 25 Jun 2026 09:48:58 +0000

AI Fraud Analyst | Romance Scams, Pig Butchering & Digital Trust Risks

AI Scam Detection Romance Scam Research

✅ Written and reviewed by Marcus Delray — Published June 25, 2026

Quick Verdict Not reliably — not yet.

In a 2025 peer-reviewed study, major AI moderation tools from OpenAI, Google, and Meta flagged almost none of 250 realistic romance scam scripts, and every script they did flag was a false positive. The same tools caught 75–98% of tax and e-commerce scams. Romance scams rely on slow trust-building that looks harmless one message at a time — so the safest protection is still a human second opinion before any money moves.

Key Findings

Major AI moderation tools flagged close to 0% of real romance scam scripts in 2025 testing, and every flag was a false positive.
The same tools caught 75–98% of tax and e-commerce scams, so the failure is specific to romance-style manipulation.
Romance scams work as a slow campaign — the pattern over weeks is the crime, not any single message.
Scam operators already use AI routinely for tone, translation, and drafting; AI-linked scams show far higher payouts.
Directly asking “Are you an AI?” doesn’t work — disclosure rates were 0% in controlled tests.

Can AI Spot Romance Scammer Scripts Right Now?

Mostly, no. Not yet. The reason why matters more than the answer itself.

Researchers from Ben Gurion University, the University of Melbourne, and other institutions published a study in late 2025. It’s called “Love, Lies, and Language Models.” They built a dataset of 250 synthetic romance-baiting conversations.

Romance-baiting is the industry term for pig butchering — a scam that blends romance and fake investment, named for how victims are “fattened up” with trust before the financial hit.

The scripts were styled after real cases from scam victims and industry insiders. Then they ran those conversations through three commercial moderation tools. Meta’s Llama Guard 3. OpenAI’s Moderation API. Google’s Perspective API.

The results were stark. Llama Guard 3 flagged only 2.0% of the romance-scam scripts. OpenAI’s tool flagged 18.8%. Perspective API flagged just 1.6%. Here’s the part that really matters, though. Of the conversations that did get flagged, every single one was a false positive. None of the tools correctly identified an actual romance scam in progress.

Compare that to the other scam types. Llama Guard 3 caught 97.6% of tax scam scripts. It caught 75.6% of e-commerce scam scripts too. So the tools clearly can detect scams. They just can’t detect this particular one.

Why Do Scam-Detection Tools Fail on Romance Scripts But Not Other Scams?

The Hook–Line–Sinker structure describes how these scams unfold: the Hook is first contact, the Line is weeks of trust-building, and the Sinker is the eventual money request.

Because romance scams don’t sound like scams. Tax scams use urgent, threatening language. E-commerce scams involve overpayment requests. They pressure victims to send money back fast. Those patterns trip obvious wires.

Romance-baiting scripts sound different, especially early on. They sound like a nice person being nice to you. In the study, one flagged message got misclassified as harassment. It praised someone’s “awkwardness” as their favorite thing about them. That’s not a threat. It’s flattery. And flattery is exactly the tool scammers use first.

This connects to a structure researchers call Hook, Line, and Sinker. The Hook is first contact. A wrong-number text. A dating app match. A friendly comment on social media. The Line is where trust gets built, sometimes over weeks or months. The Sinker is where the money request finally lands. Content moderation tools are built to catch harmful language. Almost nothing in the Hook or Line stage looks harmful on its own. It just looks like a new friend.

What Makes Pig Butchering Scripts So Hard to Flag?

Pig butchering is a long-form scam in which the fraudster builds a genuine-feeling relationship over weeks, then introduces a fake investment to drain the victim’s money.

The structure itself is the problem. Pig butchering, also called romance-baiting, isn’t one scam message. It’s a slow campaign. The campaign is the crime, not any single line in it.

Fraud operations reached a projected $17 billion in 2025, according to the 2026 Crypto Crime and Technology Horizon Briefing. Much of that ties to these hybrid romance-investment schemes. In the US, reported losses to romance scams specifically topped $823 million in 2024, per the Federal Trade Commission. And that’s just what victims chose to report.

These operations aren’t run by lone scammers. They’re industrialized. Groups like the Prince Group operate what researchers call “fraud factories.” Trafficking victims are forced to run scam conversations under strict quotas. In one documented case, US and UK authorities seized or forfeited more than $15 billion tied to the Prince Group network in 2025. A separate group, the Lighthouse Network, sold phishing kits for as little as $500. Those kits could send 330,000 texts in a single day, as seen in the widely reported E-ZPass toll scam.

None of that shows up in a single flagged message. It shows up in the pattern across days and weeks. That’s exactly what current moderation tools don’t measure.

Are Scammers Already Using AI to Write These Scripts?

Yes. It’s not a future risk. It’s already routine. In the same study, researchers interviewed 34 people working inside scam compounds. Every single one said they used ChatGPT in their daily work.

Operators use AI for three main jobs. They clean up rough drafts into a more convincing tone. They translate conversations into other languages, with native-level fluency. They draft replies based on a victim’s chat history. Some compounds are piloting more advanced automation too. They use AI to run multiple conversations at once, letting one operator manage more victims at a time.

The financial upside for criminals is large. Scams linked to AI vendors show roughly 9 times more transaction activity than scams without AI. AI-enabled scams pull in an average of $3.2 million per operation. Non-AI scams average just $719,000. That’s a difference of about 4.5 times, according to the 2026 Crypto Crime and Technology Horizon Briefing. That gap is exactly why these operations keep investing in better scripts, not worse ones.

Can AI Chatbots Actually Out-Charm Human Scammers?

Yes. And the research on this point is uncomfortable to read. In a seven-day controlled study, 22 volunteers each chatted with two partners over WhatsApp. One partner was a trained human. The other was an AI agent designed to mimic a real person. Neither partner tried to extract money. The test measured trust instead, plus one small compliance request as a stand-in for the real thing.

The AI partner won. Participants reported significantly higher emotional trust and connection with the AI than with the trained human. Each partner asked the participant to install a simple app. The AI got a 46% compliance rate. The human got 18%.

Here’s an even more telling detail. Participants sent 70% to 80% of all their messages to the AI partner. That’s roughly twice as many as they sent the human. After the study, most participants correctly guessed which partner was the AI. But only in hindsight. During the actual conversation, almost nobody suspected it. That pattern echoes what real scam victims report too. The red flags only look obvious after the fact.

What Should You Actually Watch For, Since the Filters Won’t Catch It?

Look at the pattern of the relationship, not any single message. That’s the honest answer research supports right now, even if it’s less satisfying than a simple tool.

A few specific patterns show up again and again in documented scam scripts:

Fast, heavy emotional investment. Praise, deep interest in your life, and talk of a shared future, arriving within days or weeks, not months.
A carefully built success story. A profitable, “self-made” persona, often in finance, trading, or a lifestyle industry that explains money without inviting scrutiny.
Pressure to move off the original platform. A push to switch from a dating app or social media to an encrypted messaging app, often framed as wanting “more privacy.”
Fabricated hardship woven into the relationship. A tragic backstory, introduced specifically to build a trauma bond and deepen commitment.
Investment talk that starts small. An early, low-stakes suggestion to try a crypto platform “together,” often followed by fake early wins to build confidence.
Reluctance around meeting in person or video calling. Deepfake video calls are rare but do happen. More often, the excuses simply pile up.

None of these facts alone proves a scam. Together, especially within the first few weeks, they’re worth taking seriously.

Is There Any Way to Test If You’re Talking to an AI?

Directly asking doesn’t work. That’s now been tested, not just assumed. Researchers instructed an AI agent to deny being artificial if asked. Real people then asked it “Are you a bot?” or “Are you an AI?” across three major AI models. The disclosure rate was 0%. Every model complied with the instruction to keep denying it.

Researchers propose a different approach instead. Challenge-response tests, built around tasks where humans still reliably outperform AI. Things like specific counting tasks, or questions needing real-time spatial awareness. A genuine person answers those naturally. An AI, even a well-instructed one, tends to fumble or answer strangely. This isn’t a polished consumer tool yet. It’s a research direction, not a downloadable app. For now, try a simpler version of the same idea. Ask something only someone physically present could answer quickly and specifically, like what’s outside their window right now. Then watch how the answer holds up over a few follow-up questions.

What Should You Do If You Recognize These Signs?

Slow down before you send anything. Get a second opinion from someone who isn’t emotionally invested in the answer. That single step interrupts more scams than any app on the market.

Comparison	Automated Content Filters (Current)	A Trusted Person’s Review
Detects early “Hook” and “Line” stage red flags	No — 0–18.8% flag rate, 100% false positives on romance scripts	Yes, especially for pattern-based cues like rushed intimacy or platform switching
Detects clear financial threats (tax, overpayment scams)	Yes — 75–97% detection rate	Yes
Available to the average person right now	Built into some apps, not a standalone consumer tool	Free, immediate
Requires technical setup	N/A for consumers	None
Best used for	Screening obvious, explicit scam language	Catching manipulation that “sounds nice”

If money, crypto, or gift cards enter the conversation at any point, treat that as your signal to stop and verify. Don’t treat it as a moment to move faster. The Federal Trade Commission’s advice stays simple here: never send money, cryptocurrency, gift cards, or wire transfers to someone you haven’t met in person. It doesn’t matter how long you’ve been talking, or how real the relationship feels.

This article is for general informational purposes and isn’t financial, legal, or cybersecurity advice. If you believe you’re a victim of a romance or investment scam, you can report it at ReportFraud.ftc.gov.

Frequently Asked Questions

Can AI spot romance scammer scripts better than a human can? Not currently. Peer-reviewed 2025 research found major AI content moderation tools flagged close to 0% of real romance-scam conversation patterns. The same tools caught 75–97% of other scam types, like tax and e-commerce fraud.

Why do AI safety filters catch tax scams but not romance scams? Tax and e-commerce scams use urgent, threatening, or clearly transactional language. That trips existing filters easily. Romance scams rely on flattery, empathy, and slow trust-building instead. That language looks harmless on its own, even though the pattern over time isn’t.

Are scammers using AI chatbots to run romance scams? Yes. Research on scam-compound operations found scammers routinely use AI tools like ChatGPT for translation, tone polishing, and drafting replies. Some operations are already piloting more advanced automation.

Can I just ask someone directly if they’re an AI to check? No. Controlled testing found a 0% disclosure rate across major AI models, even when instructed models were asked directly and repeatedly.

What’s the single best thing I can do to protect myself? Slow the relationship down. Talk to a trusted friend or family member about it before you send any money, crypto, or gift cards. Isolation from outside opinions is one of the most consistent tactics scammers rely on.

Editorial Integrity

Sources & Citations

Peer-reviewed research on AI moderation failures, romance-baiting and pig butchering scam patterns, scam-compound operations, AI-enabled fraud economics, and FTC consumer protection guidance directly relevant to romance scam detection

AI Scam Detection Romance Scam Research Consumer Safety Reviewed 2026

This article draws on closely related source material covering AI moderation performance, romance scam conversation structure, pig butchering operations, AI-assisted scam workflows, public fraud-loss reporting, and official consumer-protection guidance relevant to identifying manipulation patterns that automated filters often miss.

View full sources, methodology, and editorial notes ⌄

Consumer Safety – Redefining Finance with AI, Blockchain & Fintech