Scam Check benchmark results
This public benchmark reruns labeled synthetic evidence through the same deterministic functions used by Scam Check. Dangerous cases must not receive the lowest concern band.
Current test result
- Method
- hide-verify-evidence-v5
- Labeled cases
- 59
- Exact matches
- 59 of 59
- False reassurance
- 0
- Missed warnings
- 0
- False alarms
- 0
Checks included
The set covers sensitive requests, phone regions, reported sender authentication, encoded and mixed-script domains, claimed-brand lookalikes, payment checksums, redirect completion, HTTPS downgrades, cross-domain handoffs, redirect loops, every threat-list state and the final aggregation of completed and unavailable checks.
| Check kind | Cases |
|---|---|
| message | 11 |
| email auth | 7 |
| domain presentation | 3 |
| brand domain | 5 |
| iban | 3 |
| crypto address | 9 |
| phone | 3 |
| redirect trace | 6 |
| threat intel | 5 |
| verdict aggregation | 3 |
| engine coverage | 4 |
Metric definitions
False reassurance means a case labeled harmful received Scam Check's lowest concern band. A missed warning means a harmful case received neither a high nor medium concern. A false alarm means a case labeled legitimate received a high concern. Unknown examples remain unknown where the available evidence cannot support a conclusion. The final result can use the lowest concern band only when every check completed; one unavailable check keeps an otherwise low-concern result incomplete.
Benchmark limitations
Live provider accuracy is not published because the deterministic set does not make live provider requests or establish real-world scam prevalence. The labels are maintained regression expectations. No independent certification has been performed, and real-world outcomes still depend on user reports.