zh-pii-tagger-35m

A 35M-parameter token tagger for personal data in Chinese and mixed Chinese–English text. It is a fine-tune of cjhwong/seatbelt-pii-tagger with the same architecture, tokenizer and 18 types / 73 BIOES labels.

The fine-tune adds:

• real Chinese contexts (FineWeb-2) with inserted synthetic private names and addresses;
• self-distillation replay, which aims to reduce forgetting of the base model's other types and its English behaviour.

In short:

• It misses fewer Chinese person names. On test sets never used during development, person recall rises by about 2 to 10 points; precision moves between −3.5 and +3.5 points.
• Person F1 gains are modest:
  • +0.6 to +2.1 on two freshly generated test sets;
  • +4.2 to +6.7 on an MSRA NER train sample (never used during development) and +3.3 to +5.7 on the MSRA NER test split (used during development);
  • on Weibo and CLUENER, not significant under the base model's own decoder (the intervals include 0).
• Addresses: no consistent improvement over the base model. It finds slightly more addresses written without a street or city (a room, door or building number), and it also predicts slightly more addresses in messages that contain none. On the synthetic full-address sets the base model is slightly better.
• An optional filter (cue_filter.py, off by default) removes predicted addresses that are a bare number with no address word nearby. That cuts most address predictions in messages without an address, but it also drops about 33 points of recall on addresses written as digits alone (such as 我住502). With the filter, the base model scores highest on the synthetic full-address sets.
• Under the base model's own decoder, it is slightly below the base model on PII Bench ZH person (−0.4 to −0.65; −1.0 with the released int8 on an x86 CPU).
• English was evaluated only for person, email and phone.

Use it as a local, per-request filter before text leaves the machine. It is not a legal anonymisation tool, and it is not a state-of-the-art or across-the-board improvement; see Results and Limits.

Usage

The labels are BIOES, so the generic pipeline("token-classification") output splits entities into pieces. This short decoder joins them and strips leading and trailing punctuation (tested on CPU with transformers 5.9.0):

import torch
from transformers import AutoTokenizer, AutoModelForTokenClassification

repo = "ickma2311/zh-pii-tagger-35m"
tok = AutoTokenizer.from_pretrained(repo, revision="v1.0")
model = AutoModelForTokenClassification.from_pretrained(repo, revision="v1.0").eval()
PUNCT = " \t，。、；：！？,.;:!?()（）「」\"'"

def tag(text):
    enc = tok(text, return_offsets_mapping=True, return_tensors="pt", truncation=True, max_length=512)
    offsets = enc.pop("offset_mapping")[0].tolist()
    with torch.no_grad():
        ids = model(**enc).logits[0].argmax(-1).tolist()
    spans, cur = [], None
    for (s, e), i in zip(offsets, ids):
        if s == e:
            continue
        lab = model.config.id2label[i]
        if lab == "O":
            cur = None; continue
        pre, typ = lab.split("-", 1)
        if pre in ("B", "S") or cur is None or cur[2] != typ:
            cur = [s, e, typ]; spans.append(cur)
        else:
            cur[1] = e
        if pre in ("E", "S"):
            cur = None
    merged = []
    for sp in spans:  # join same-type spans that touch (only spaces between)
        if merged and merged[-1][2] == sp[2] and not text[merged[-1][1]:sp[0]].strip():
            merged[-1][1] = sp[1]
        else:
            merged.append(sp)
    out = []
    for s, e, typ in merged:
        while s < e and text[s] in PUNCT: s += 1
        while e > s and text[e - 1] in PUNCT: e -= 1
        if s < e: out.append((typ, text[s:e]))
    return out

print(tag("我叫欧阳晓燕，住在杭州市西湖区文三路 88 号 502 室，电话 13812345678。"))
# [('person', '欧阳晓燕'), ('address', '杭州市西湖区文三路 88 号 502 室'), ('phone', '13812345678')]

This is simple argmax decoding. The numbers in Results use the decoders in the evaluation repository. For the optional number filter, see cue_filter.py.

Demo

A 3.5-minute comparison of this model and the base model, both running locally (int8 ONNX on a Mac CPU): five fictional messages, the held-out set, the test sets never used during development, and the address trade-off. English narration, with English and Chinese captions burned in. Caption files: English, Chinese.

Compatibility

• Same architecture as the base model; the ONNX file uses the same file layout (fused int8 graph).

• Spans need one post-processing step: strip leading and trailing punctuation from each span.

• Optional second step, off by default: cue_filter.py (in this repository) drops address spans that are only a number unless an address word is next to them. See Addresses for what it gains and loses.

• This release contains no Core ML package, and Core ML was not exported or tested. The upstream agent-seatbelt hook prefers Core ML on Apple Silicon. If you swap files in a directory that still holds the base model's model.mlpackage, remove it, or the hook may silently keep running the base model.

• int8 scores depend on the CPU. We ran the same script, the same files and onnxruntime 1.31.0 on zh-pii-ctx-heldout (person F1, project decoder):

  weights / hardware	this model	base model
  int8, Apple Silicon	0.732	0.697
  int8, x86 cloud CPU	0.722	0.704
  fp32	0.731	0.698

  If accuracy matters, use model.safetensors, or check int8 on your own hardware.

• Other runtime paths were not verified.

Results

All numbers are measured by us:

• metric: person strict character-span F1, from the fp32 weights on CPU (the English Nemotron-PII row: GPU, fp16 autocast);
• base model: cjhwong/seatbelt-pii-tagger at revision c2ca0cd;
• 3 training seeds (7 / 77 / 777). The released checkpoint is seed 7, chosen by validation loss;
• Δ is this model minus the base model, in points;
• the bracket spans the paired-bootstrap 95% intervals of all 3 seeds (lowest lower bound to highest upper bound), resampling documents.

Both models are scored with the same decoder. Two decoders are reported:

• project: argmax per token; touching same-type pieces are merged; leading and trailing punctuation is stripped;
• seatbelt: our reimplementation of the base model's runtime decoder (BIOES-constrained Viterbi, as in agent-seatbelt hooks-pii/server/detectors/tagger.py), followed by the same punctuation strip. Given the same logits, it gave exactly the same spans before the strip as that file's decoder at commit 326ebf8 on 2,400 sampled documents (both models, 4 test sets). The hook's public-figure / place rule chain is not applied.

Test sets never used during development

These sets were built or chosen after the model was trained. None shares a passage or a near-duplicate context (character 8-gram Jaccard above 0.5) with the 40,000 FineWeb-2 fine-tuning passages; overlap with the 57,062 replay texts (OpenPII, nym-pii and Nemotron-PII training data, labelled by the base model) was not checked.

• zh-pii-fresh and zh-pii-fresh-2: 2,000 passages each, built from LCCC valid dialogues and from Wikipedia articles not used before. Faker names (zh_CN, plus en_US for about 8% of them) and zh_CN addresses were inserted with templates written independently of ours. The two sets share no Wikipedia article; 13 of their 1,300 LCCC dialogue contexts each are short contexts that appear in both.
• Weibo NER train: real social media, 1,350 sentences, PER.NAM labels.
• MSRA NER train sample: real news, 5,000 sentences.
• CLUENER2020 dev: real news text, 1,343 sentences, name labels.

set	gold names	decoder	base	this model	Δ	recall: base → this model
zh-pii-fresh-2	3,512	project	0.923	0.933–0.938	+1.0 to +1.5	0.926 → 0.954–0.959
		seatbelt	0.933	0.939–0.943	+0.6 to +1.0	0.932 → 0.955–0.958
zh-pii-fresh	3,452	project	0.918	0.933–0.939	+1.6 to +2.1 [+1.0, +2.7]	0.918 → 0.954–0.959
		seatbelt	0.928	0.938–0.942	+1.0 to +1.4 [+0.5, +1.9]	0.925 → 0.953–0.957
Weibo NER train	574	project	0.338	0.357–0.367	+2.0 to +2.9 [−0.1, +4.9]	0.345 → 0.408–0.420
		seatbelt	0.384	0.394–0.403	+1.0 to +2.0 [−0.9, +3.9]	0.368 → 0.423–0.436
MSRA NER train sample	1,883	project	0.554	0.611–0.620	+5.7 to +6.7 [+4.2, +8.1]	0.547 → 0.630–0.648
		seatbelt	0.605	0.647–0.651	+4.2 to +4.5 [+2.9, +5.8]	0.574 → 0.638–0.651
CLUENER2020 dev	465	project	0.461	0.492–0.494	+3.1 to +3.3 [+0.7, +5.7]	0.546 → 0.617–0.622
		seatbelt	0.509	0.515–0.523	+0.6 to +1.4 [−1.2, +3.3]	0.576 → 0.619–0.626

• zh-pii-fresh and zh-pii-fresh-2 use Faker names. The base model's training data also contains Faker-generated text, which may favour the base model on these sets.
• Brackets for zh-pii-fresh-2 were not computed.
• These sets were not used to develop this model. Some of them were used in the address error analysis behind the optional filter (see Limits); the person scores above are unaffected.

Sets used during development

We looked at these sets while choosing the training recipe and the punctuation strip, so they are not independent estimates.

set	what it is	decoder	base	this model	Δ
zh-pii-ctx-heldout, person	ours: 1,875 real LCCC test / zh-Wikipedia passages. Built with the same generator code as the training data, but with disjoint name pools and different contexts and templates	project	0.698	0.731–0.736	+3.4 to +3.9 [+2.1, +5.1]
		seatbelt	0.756	0.779–0.786	+2.3 to +3.0 [+1.4, +4.0]
PII Bench ZH, person	public (wan9yu/pii-bench-zh), 8,000 template docs	project	0.971	0.972–0.973	+0.1 to +0.2 [−0.2, +0.4]
		seatbelt	0.975	0.969–0.972	−0.4 to −0.65 [−0.9, −0.2]
Weibo NER test, person	public, real social media, 111 names	project	0.351	0.398–0.417	+4.8 to +6.6 [−0.7, +13.8]
		seatbelt	0.404	0.410–0.436	+0.6 to +3.2 [−5.5, +10.3]
MSRA NER test, person	public, real news	project	0.487	0.538–0.543	+5.1 to +5.7 [+3.7, +7.3]
		seatbelt	0.521	0.554–0.559	+3.3 to +3.8 [+2.0, +5.0]
Nemotron-PII test (1,000 en docs), typed overlap F1 person / email / phone; GPU, fp16 autocast	public, English; same distribution as part of the replay data	project	0.952 / 0.994 / 0.824	0.950–0.953 / 0.993–0.994 / 0.824–0.830	about 0

Addresses

Overlap scores (any overlapping prediction of the right type counts as a hit), fp32, seatbelt decoder, 3 seeds. "+ filter" applies cue_filter.py to the predictions; the filter is the same for both models. int8 was not evaluated together with the filter.

set	base	base + filter	this model	this model + filter
zh-pii-numaddr, recall on the 313 addresses written as digits alone (我住502, 门牌66号)	0.732	0.431	0.760–0.773	0.438
zh-pii-numaddr, recall on the 255 short addresses with a unit word (18号楼, 3单元401, 5-302)	0.604	0.604	0.624–0.722	0.624–0.722
zh-pii-numaddr, F1 on all 668 gold addresses (568 short + 100 full addresses)	0.748	0.692	0.758–0.770	0.697–0.722
zh-pii-fresh-2, F1	0.957	0.976	0.946–0.947	0.967–0.970
zh-pii-ctx-heldout, F1 (the filter's number pattern was set after looking at errors on this set)	0.969	0.984	0.945–0.953	0.976–0.978
PII Bench ZH, F1	0.999	0.999	0.999	0.999
zh-pii-numaddr, the 550 messages with no address: address predictions (all false positives)	99	24	101–114	23–28
number-only address predictions that match no gold address: zh-pii-fresh-2 + MSRA NER test + zh-pii-numaddr (count)	193	45	195–213	43–48

• zh-pii-numaddr (data/numaddr.jsonl on GitHub): 1,200 short messages: 550 whose address has no street or city, only a number that may carry a unit word (栋, 号楼, 单元, 室; 568 gold spans from 58 templates), 550 with numbers that are not addresses (prices, dates, order numbers, jersey numbers, bus routes, ...), and 100 with a full address plus a confusable number.
• How it was made: a separate generator that could not see the filter, but whose templates were based partly on the same example sentences as the filter's word list. The share of messages with no address word next to the number (about 30%) is a design choice, not a measured real-world rate. A few positives are debatable (a market stall, a hotel room, an office).
• The messages with no address have complete gold, so every address predicted there is a false positive. The last row is not a clean false-positive count: MSRA NER has no address labels and zh-pii-fresh-2 labels only the inserted addresses, so some of those predictions may be real addresses (for example 住在８层).
• So the filter trades about 33 points of recall on addresses written as digits alone for far fewer false positives (99 → 24 for the base model, 101–114 → 23–28 for this model, on the messages with no address). It only removes predicted spans that are a bare number: on the tested subset no prediction of an address with a unit word was removed, because those predictions include the unit word; a prediction that stops before the unit (such as 5号 out of 5号院) would be removed. It is off by default. Which is better depends on whether a missed room number or a masked price costs you more.
• This model marks a whole address, including building, unit and room suffixes, as one span. Strict-span scores are therefore much higher than the base model's (zh-pii-ctx-heldout 0.848–0.857 vs 0.279, project decoder); that gap reflects span conventions, not better detection.
• With the released int8 on an x86 CPU, zh-pii-ctx-heldout address overlap F1 is 4.0 to 5.1 points below the base model (fp32: 1.6 to 2.4).
• The address test sets are synthetic. Accuracy on real addresses was not measured.

Speed and size

• Setup: int8 ONNX, model already loaded, 4 CPU threads, Apple M-series, held-out passages (median 78 characters).
• Statistic: median over 3 interleaved rounds of the per-round mean time per passage.
• Result: 1.969 ms for this model vs 1.980 ms for the base model; file size 35.6 MB.
• Measured with the project decoder and without the filter. This is processing time only; start-up and model loading are excluded. The filter adds a regular-expression check per address span.

Reference points (not comparable in cost)

• Qwen3-8B: single zero-shot prompt, greedy decoding, fp16 on a T4, about 2.3 s per request.
  • zh-pii-ctx-heldout: person 0.680, address 0.878.
  • 1,000-doc PII Bench sample: person 0.978, address 0.918.
• Rule baselines on zh-pii-ctx-heldout person: jieba nr + regex 0.522, argus-redact fast 0.197.

Limits

• Gains are mostly in recall on person names. Precision is flat or slightly lower on most sets, so expect a few more false positives.
• No consistent address improvement over the base model, with or without the filter.
• The synthetic test sets use real contexts but synthetic entities, so they do not measure accuracy on real private data.
• Real-text person scores are still low in absolute terms: Weibo 0.36–0.44, MSRA 0.54–0.65. Their gold labels include public figures, which the base model's guideline does not target.
• The training-address format and the punctuation strip were chosen after error analysis on PII Bench ZH, so PII Bench address scores are not independent estimates.
• The filter's number pattern and word list were written after looking at address errors on zh-pii-fresh, zh-pii-ctx-heldout, Weibo NER train, the MSRA NER train sample and CLUENER2020 dev, and at a few example sentences; some of the same example sentences also shaped zh-pii-numaddr's templates.
• The labelling guideline is the base model's: private persons only. Public-figure handling was not measured separately.

Training

• One epoch, 3,008 steps, batch 32, lr 3e-5, fp16, on one Kaggle T4.
• Data:
  • 39,200 FineWeb-2 passages with entities inserted by our own name and address generator. Names that jieba flagged but the base model did not tag were masked out of the loss.
  • 57,062 replay texts labelled by the base model.
• Training and evaluation code and the test sets: https://github.com/ickma2311/zh-pii-tagger-eval

Versions

• v1.0 (main, released 2026-10-13): the model described above.
• A v1.1 published on 2026-10-14 was withdrawn the same day because it missed most house and room numbers written as digits. If you downloaded it then, please re-download from main.

Licence

Apache 2.0. See NOTICE for attributions (base model, data). cue_filter.py is Apache 2.0.