ScamLingua ScamLingua

Academic research project · Old Dominion University & Christopher Newport University

Academic Research on Scam Detection

A university research project studying the language of scams to build AI that detects and prevents fraud against older adults. We do not send, facilitate, or endorse scams.

Data disclosure All datasets are fully synthetic and AI-generated — no real people, no real data.

Request dataset access →

Researchers
Publications
AI safety evaluations
Contact (Attn: Carl Lochstampfor)
Status notice COVA-X is a synthetic research dataset — every conversation is machine-generated, and it contains no real victim data. It is released for non-commercial defensive research use under the access terms stated below. It is not a certified benchmark, a safety or detection product, or legal and compliance advice, and it carries no regulatory standing. It is published as independent academic research by Carl Lochstampfor (Old Dominion University) and Dr. Ayan Roy (Christopher Newport University).

What is COVA-X?

COVA-X is a fully synthetic dataset of approximately 11,000 AI-generated, English-language, multi-turn scam conversations modeled on fraud techniques used against older adults. It exists for one purpose: to help researchers build and evaluate systems that detect social-engineering fraud. Every conversation is machine-generated — no real victims, no real communications, and no personal data of any kind.

Generation runs entirely on local, offline research hardware using an open-weight language model (Qwen 2.5 14B via Ollama) — no cloud services and no third-party data exposure at any stage. The full methodology, generation pipeline, and quality-assurance process are documented in the papers linked above (arXiv preprints, both in the peer-review process).

COVA-X generation pipeline: prompt templates feed a locally hosted Qwen 2.5 14B language model that drives a multi-agent attacker and victim dialogue loop; a quality pipeline then applies contamination scanning, stage-direction stripping, and automated relabeling to produce the labeled synthetic dataset.
Generation methodology: prompt templates → local LLM (Qwen 2.5 14B) → multi-agent dialogue → quality pipeline → labeled synthetic dataset.

Each record carries structured metadata alongside the dialogue — scam type, outcome label, turn count, victim parameters, and data-quality flags:

{ "conversation_id": "romance_20260214_...", "scam_type": "romance", "outcome_label": "verification_attempt", "total_turns": 13, "victim_age_range": "70-74", "model": "qwen2.5:14b (local)", "legacy_contamination": false, "turns": [ ...synthetic dialogue... ] }

Illustrative structure only — field names and format match the dataset; values shown are placeholders. Full schema documentation accompanies each data distribution.

Evaluations of AI safety systems

Separate from the COVA-X dataset: independent tests of commercial AI safety systems, designed and run by Carl Lochstampfor, with a second evaluator checking labels.

Independent evaluation · Lead evaluator: Carl Lochstampfor

Tuteliq text-safety detection: Phase 1 (October 2026)

  • 1,344 test cases
  • 5 harm categories
  • Blind second-evaluator check

In September and October 2026, Carl Lochstampfor ran an independent black-box evaluation of Tuteliq's text-safety detection API, with Dr. Ayan Roy (Christopher Newport University) as second evaluator. The test set had 1,294 synthetic multi-turn conversations and 50 single messages across five harm categories: romance scams, social engineering, coercive control, bullying, and self-harm and emotional distress. The conversations were generated locally with an open-weight language model (Gemma 3 12B).

Romance scams, coercive control and self-harm were detected in 96% to 100% of harmful conversations, with false alarms at or below 6.9%. Social engineering was the weakest category (83.7% detected, 13.2% false alarms; corrected October 28, 2026). Both reports are published with Tuteliq's permission, weaknesses included.

Social engineering figures were corrected on October 28, 2026; see the erratum. Results are from synthetic, English-language text conversations (detector tuteliq-detect-v5.20261015, tested October 19, 2026). This is an independent evaluation, not a certification, accreditation or audit. The evaluation was unpaid. Tuteliq had no role in the test design, scoring or conclusions, and reviewed the reports only for factual accuracy. Quote figures only as set out in the claims register in each report.

Coming before the end of 2026: the five-category evaluation dataset behind these reports will be added to ScamLingua, with its own access terms. Like COVA-X, every conversation is synthetic.

Requesting dataset access

The dataset is shared by request with researchers and organizations working on fraud detection and prevention. All conversations are in English — the dataset does not currently include other languages.

  1. Email requests@scamlingua.org (Attn: Carl Lochstampfor) with your name, institutional affiliation, and a brief description of your intended use — the link opens a pre-filled request template for you.
  2. Access is granted for non-commercial defensive research use — detection, prevention, and evaluation (see Responsible use). Redistribution is not permitted.
  3. Any publication or product of research using the dataset must cite the reference below.

By submitting a request, you confirm that you have read and agree to these terms. Approved requests receive an access code invitation by reply email — please confirm receipt within 72 hours so it doesn't get lost in spam.

Tip for faster review: requests sent from an institutional email address (university or organization domain) are typically processed fastest. Students — CC'ing your supervising professor or thesis advisor helps us confirm research alignment quickly. Requests from personal email accounts are welcome, but may involve a brief follow-up to confirm affiliation. Email link not opening? Write to requests@scamlingua.org from your own mail client — everything to include is listed below.

All requests are reviewed for research alignment.
Please allow at least 3 - 7 calendar days for a response.

Include in your email:

  • Your full name and role or title
  • Institution or organization — please write from your institutional email address, which helps us verify your affiliation
  • A brief description of your intended use (project, paper, or system)
  • A confirmation that you agree to the terms — copy the statement below into your email
I confirm that I will use the COVA-X dataset for non-commercial defensive research use only, that I will not redistribute it, and that any resulting work will cite: Lochstampfor, C., & Roy, A. (2026). COVA-X. arXiv:2606.06879.
Lochstampfor, C., & Roy, A. (2026). COVA-X. arXiv:2606.06879.
https://arxiv.org/abs/2606.06879