ProofAgent releases 98M on-device privacy detection model Nano
Understand this story
ProofAgent releases ProofAgent Nano, a 98M-parameter on-device model that can run inference locally on CPU or GPU, with text never leaving the device. The company says it achieves an average F2 of 0.935 on the full English PIIMB test set, higher than the 36 published models compared; on a 2,000-sentence sample it scored 0.933 against five frontier LLMs, above Claude Fable 5.1's 0.901. The model is proprietary, and the release did not disclose architecture, training methods, data composition or calibration details. Only English has been evaluated so far; the mobile and quantized versions of this release have not yet been benchmarked.
AI overview based on source reports · Updated 31 min ago
Coverage timeline
Follow the coverage to explore different perspectives.
- Hacker News AITiny AI Security Model Beats Five Frontier LLMs on Privacy Detection
ProofAgent Nano, a 98M-parameter on-device model, reports 0.935 average F2 on the full English PIIMB test set, ahead of 36 published models including OpenMed's 1.4B filter at 0.918. On a separate 2,000-sentence sample scored with the same PIIMB tool, Nano scored 0.933 versus 0.901 for Claude Fable 5.1 and 0.828 for Claude Haiku 5.5, though the full-set result was computed locally and not submitted to the public leaderboard. A multilingual version covers English, French and Arabic, and the model flags PII, prompt injection, jailbreaks and data exfiltration with mappings to 38 regulatory frameworks.
Activity for this story
Not enough continuous observations to show a trend yet.