EU-Sovereign AI Platform · Applied Research · Vertical Products
Privacy-preserving NLP for
European languages
The open research hub of KlusAI — benchmarks, datasets, models and papers for PII/PHI detection, anonymization, and re-identification-risk evaluation across European languages (8 published datasets: ro, en, pl, de, fr, es, it, nl — scaling toward EU-24), spanning the legal and clinical domains.
Flagship benchmark
EuroPriv-Bench
EuroPriv-Bench is the first unified pan-European de-identification benchmark. Unlike prior work that reports only detection-F1 on English, it measures re-identification risk alongside detection on a single GDPR-aligned taxonomy. Eight languages are published — ro, en, pl, de, fr, es, it, nl. Seven have general-text tracks (de, en, es, fr, it, nl, ro), and three carry contamination-controlled, decode-bearing real-skeleton tracks: Romanian (CNP), Polish (PESEL), and Italian (codice fiscale). The roadmap scales toward the full EU-24.
The headline finding: detection-F1 does not track re-identification protection —
demonstrated on decode-bearing national identifiers (RO CNP, PL PESEL, IT codice fiscale).
The deeper mechanism is general but the proof is not yet: an aggregate detection-F1 can stay
high while a model misses the rare, high-stakes tokens that actually carry the
re-identification. National IDs are the clearest, provable case of that —
each un-redacted national ID deterministically discloses several quasi-identifiers at once (a
Romanian CNP, for instance, decodes date of birth, sex, and county) — not the whole of it.
On contamination-free, realistic-structure documents the dissociation holds across
three decode-bearing identifiers in three languages (RO CNP, PL PESEL, IT
codice fiscale), across two independent Romanian template families, and is
reproduced by independent third-party submissions on the public board: spaCy, with no
structured-ID recognizer, leaks 89.0% of Romanian CNPs at a detection-F1 of
just 0.14, while GLiNER — the strongest detector on the track (F1 0.85) — still leaks 30.2%.
The contrast is KlusAI's reference de-identifier kp-deid, the strongest protector
that still detects — 0% CNP leakage at detection-F1 0.74. These are measured,
contamination-controlled signals on development-track gold (config_status = dev),
pending native-speaker and inter-annotator-agreement validation — a finding, not yet a
validated or citable claim. Extending the measure to quasi-identifier-combination
re-identification is in progress, so the broad reading remains a hypothesis under test.
Artifacts
Open from day one
Benchmark
EuroPriv-Bench — versioned, provenance-tracked, openly redistributable. The live leaderboard is open for external submissions.
Models
The KlusAI Privacy (kp-*) family — kp-deid-mdeberta-280m is the strongest protector that still detects (0% CNP leakage at detection-F1 0.74).
Papers
The EuroPriv-Bench preprint — reproducible numbers that trace back to a commit. In-progress working paper; arXiv pending.
What we publish here
A citable home for the program
- Leaderboards — versioned, provenance-tracked results you can cite.
- Release notes & methodology — the technical story behind each artifact.
- Paper companions — reproducible numbers tracing back to a commit.
For the company, see klusai.com. For the platform, see kOS, our EU-sovereign AI platform.
Updates
Latest posts
-
Privacy on gateway traffic: first results
Our first gateway-trained models leave fewer protected identifiers exposed than the two Presidio baselines on an eight-language chat-style test set. The 280m encoder repeats that result across three training seeds and removes less harmless text. The mixture of experts (MoE) also reduces leakage, but still fails the wider evaluation contract because of over-redaction. These are development results: independent review of the synthetic test set is pending, and no gateway-wide contract win is established. The gateway-trained models remain private and are not yet released.
-
October update
The programme restarted on October 11. The next horizon is a deployment-grade privacy layer for EU AI gateway traffic. We have useful evidence to build on, and gaps that need to stay visible. A good legal-text result is a starting point; it does not establish protection on gateway conversations.
-
Every detector leaked half the IDs on real court text — except one
A few weeks ago we argued that a high-F1 detector can still leak every ID it finds — but the evidence was on our own synthetic national-ID tracks. The obvious objection: show it on real data. So we did. On real ECHR court judgments — the peer-reviewed Text Anonymization Benchmark (TAB; Pilán et al. 2022), manually annotated, that no model on our board trained on — the dissociation holds, and it is starker than on synthetic text.
-
Making the benchmark honest: a harder external eval, and our first real data
Two changes landed this week that make EuroPriv-Bench harder to game — including by us. We added an external, third-party detection eval that pulls our own synthetic scores apart, and we wired in the first real-data gold config in the suite. Both make our numbers look worse, on purpose. A benchmark you can’t fail isn’t measuring anything.