Ploni finds what the leading tools miss
We tested 16 tools on Hebrew PII removal and anonymization: 4 commercial and 12 open source. Ploni finds up to 6× more of the descriptions that give a person away, with a false-alarm rate up to 3× lower. In our tests, no tool beat it on privacy, and it does about half the damage of the leading privacy tools.
The scoreboard
Green is Ploni, red is clearly worse than Ploni. Higher is better in the two "found" columns, lower in the others. Each range runs across our two Hebrew test sets. Ploni appears on Strict, our strictest mode.
Why no names?
Most leading providers restrict publishing comparisons with their tools, so we don't name them. Don't take our word for it: test it free on your own text, against any tool you like. Found a tool that works better? We train Ploni free until it wins, for up to 12 months, and give you $2,000 in credit as a gift. Full terms
The tools that protect the most get there by hiding more. Ploni understands which words point to a person, so it finds more, hides less for nothing, and leaves what requests usually need. Finding more descriptions is what separates Ploni from the big cloud tools, which leave 9 to 32 of every 100 people identifiable. Measure by measure:
- BetterDescriptions that give a person away: Ploni finds 89 of every 100, up to 6× more. Commercial tools: 15 to 58.
- BetterFalse alarms, words hidden for nothing: 12 to 15 of every 100 things Ploni marks to hide, up to 3× fewer. Commercial tools: 21 to 39.
- Better than the leading privacy toolsTexts that can no longer be used: on Strict, on our general test, 3 of every 100, against 7 to 13 with the leading privacy tools.
- BetterTexts where the answer gets noticeably worse: on Strict, about half as many as with the leading privacy tools.
- Better than the big cloud tools, on par with the leading privacy toolsPeople a strong AI could still identify in cleaned text: far ahead of the big cloud tools (one 9 to 23, the other 32 of every 100). On par with the leading privacy tools on both test sets (general: 2 of 205, against 2 of 205 and 3 of 210; rare types: 0* of 178, against 0 of 181 and 0 of 190).
- Better than the big cloud tools, on par with the leading privacy toolsNames and numbers missed: 0.1 to 0.3 of every 100. Far ahead of the big cloud tools (one 7 to 13, the other 32 to 40). On par with the closest tool.
Ploni on Strict, our strictest mode. The leading providers are the commercial tools we tested; most leading providers restrict publishing comparisons, so they appear by number. In the text above, the leading privacy tools are the two that leave the fewest people identifiable, and the big cloud tools are the two large cloud services.
| Tools | People a strong AI could still identify (measured results) | Texts ruined or damaged by the cleaning | Names and numbers found | Descriptions that give a person away: found | False alarms: share of what it marks to hide |
|---|---|---|---|---|---|
| Ploni | General: 2 of 205 Rare types: 0* of 178 AI requests: 0* of 180 | 20–24% | 99.7–99.9% | 89% | 12–15% |
| Leading provider 1 | 9–23% | 19–36% | 87–93% | 27–28% | 22–31% |
| Leading provider 2 | General: 2 of 205 Rare types: 0 of 181 | 40–55% | 98–99.9% | 54–58% | 21–27% |
| Leading provider 3 | 32%† | 28%† | 60–68% | 15–16% | 38–39% |
| Leading provider 4 | General: 3 of 210 Rare types: 0 of 190 | 39–49% | 99.8–99.9% | 45–52% | 31–36% |
| HebSafeHarbor (open source) | 8%‡ | 28%‡ | 93% | 24–31% | 13–16% |
People a strong AI could still identify (measured results)
- PloniGeneral: 2 of 205
Rare types: 0* of 178
AI requests: 0* of 180 - Leading provider 19–23%
- Leading provider 2General: 2 of 205
Rare types: 0 of 181 - Leading provider 332%†
- Leading provider 4General: 3 of 210
Rare types: 0 of 190 - HebSafeHarbor (open source)8%‡
Texts ruined or damaged by the cleaning
- Ploni20–24%
- Leading provider 119–36%
- Leading provider 240–55%
- Leading provider 328%†
- Leading provider 439–49%
- HebSafeHarbor (open source)28%‡
Names and numbers found
- Ploni99.7–99.9%
- Leading provider 187–93%
- Leading provider 298–99.9%
- Leading provider 360–68%
- Leading provider 499.8–99.9%
- HebSafeHarbor (open source)93%
Descriptions that give a person away: found
- Ploni89%
- Leading provider 127–28%
- Leading provider 254–58%
- Leading provider 315–16%
- Leading provider 445–52%
- HebSafeHarbor (open source)24–31%
False alarms: share of what it marks to hide
- Ploni12–15%
- Leading provider 122–31%
- Leading provider 221–27%
- Leading provider 338–39%
- Leading provider 431–36%
- HebSafeHarbor (open source)13–16%
Any tool can reach zero by hiding everything. The real question is how much privacy you get for how much damage. The leading privacy tools get there by hiding more: 7 to 13 of every 100 texts can no longer be used, and about twice as many are damaged as with Ploni.
*0 was measured in our test, on the same texts for all tools. 0 can't be guaranteed in every test. Ploni Strict: latest measurement, two test sets of 100 documents each. Other tools: measured 25 October, not re-run; on the general test, the counts of people for the leading privacy tools cover 99 of the 100 documents; texts that can no longer be used, in the lines above, are from the general test. †General test only. ‡Measured on one 100-document test sample. Identification test: a strong AI reads each cleaned text and tries to work out who the people are. Descriptions found is what Ploni detects; by default it hides places, workplaces, jobs and ages and keeps health, finances, events, habits and family, among others, and you choose by type. False alarms are counted against the detail types our benchmark scores. Some tools also mark dates, numbers and amounts it doesn't score, which raises their figure. The big cloud tools ruin fewer texts than Ploni Strict, and leave far more people identifiable.
Balanced: for teams that choose less hiding
Ploni starts on Strict, and that is the setting compared with other tools above. Balanced hides a little less, so fewer texts are affected and slightly more people can be identified. A strong AI identified 3 of 209 people on our general test and 0 of 172 on rare types. 1 of every 100 texts could no longer be used on the general test, and 83 to 84 of every 100 lost nothing or almost nothing. Measured on 25 October.
Results are for the tools' default behaviour: each hides everything it detects.
More open-source tools we tested: none came close
| Tools | Names and numbers found | Descriptions that give a person away: found |
|---|---|---|
| Ploni | 99.7–99.9% | 89% |
| Stanford Stanza | 55–67% | 21–27% |
| NVIDIA GLiNER-PII | 64–67% | 14–16% |
| DICTA BERT-large NER | 64% | 28% |
| DICTA BERT NER | 49–65% | 26–32% |
| GLiNER multilingual PII | 49–54% | 19–20% |
| OpenAI Privacy Filter | 35–53% | 8–12% |
| OpenMed multilingual | 41% | 9% |
| Another open-source model | 33% | 11% |
| ai4privacy | 29% | 6% |
| Microsoft Presidio | 22% | 0% |
| Knowledgator GLiNER-PII | 4% | 1% |
Microsoft Presidio has no Hebrew model, so it ran with its pattern recognisers alone. Some of the others were not built for Hebrew either; all ran on the same Hebrew texts.
What the numbers mean
People identified, and what it costs
On our general test, a strong AI identified 2 of 205 people with Ploni Strict, against 2 of 205 and 3 of 210 with the leading privacy tools. On rare types, 0* of 178, against 0 of 181 and 0 of 190. But on the general test, with those tools 7 to 13 of every 100 texts could no longer be used. With Ploni, 3. In AI requests alone: 0* of 180 people identified, 4 of 92 requests impossible to carry out.
Finds what gives a person away
A rare job, a place, an age: these identify people as surely as a name. Ploni finds 89 of every 100. The leading privacy tools find 45 to 58, the big cloud tools 15 to 28, open-source tools 0 to 32.
Hides less for nothing
Of every 100 things Ploni marks to hide, 12 to 15 are false alarms. The leading privacy tools: 21 to 36. The big cloud tools: 22 to 39. With names, Ploni is right 99 to 100 times out of 100. One leading privacy tool: 70 to 74, and it also cuts Hebrew prefix letters off words.
Cheap privacy is not privacy
The big cloud tools ruin only 1 of every 100 texts, on our general test. But a strong AI identifies 9 to 32 of every 100 people with them.
What happens to a text after cleaning
Mild
The answer is slightly less specific, and the job still gets done. On Strict, 76 to 80 of every 100 texts land here or lose nothing at all. On Balanced, 83 to 84.
Damaged
The job still gets done, with a noticeably weaker or partly wrong answer in places. On Strict, about half as common as with the leading privacy tools.
Ruined
The text can no longer be used. That's the number to compare: on Strict, 3 of every 100, against 7 to 13 with the leading privacy tools, on our general test.
Run it on your own texts
All tools ran on the same internal set of Hebrew texts across many everyday genres, and were scored the same way. A separate AI reviewer read each cleaned text to judge damage and identifiability. The benchmark itself is not published. Your own texts are the better test, and our guarantee stands behind the result.