BCCard/MoAI-Privacy-Filter-INT8

1. Overview

This repository provides the INT8 weight-only ONNX Runtime serving artifact for a Korean/English PII detection model for the finance domain. The model was built by full fine-tuning openai/privacy-filter (1.4B MoE, 50M active) on synthetic finance-domain PII data. It tags 18 PII entity types (73 BIOES classes) at token level.

The model is designed as the NER layer of a multi-layer PII-masking gateway in front of LLM services and as a PII detection component for offline privacy review and audit workflows. High-sensitivity deployments require in-domain evaluation and ongoing monitoring.

This repository does not contain PyTorch weights. AutoModelForTokenClassification.from_pretrained() and the standard Transformers token-classification pipeline therefore do not load this artifact. Use ONNX Runtime as shown in Usage.

The serving chain is:

text
-> tokenizer
-> INT8 weight-only ONNX graph
-> FP32 logits
-> constrained BIOES Viterbi
-> character-offset spans
-> whitespace boundary refinement

1.1. TL;DR

• Base model: openai/privacy-filter — 1.4B-parameter MoE (128 experts, 50M active), 8 layers, hidden 640, bidirectional banded attention (±128), o200k tokenizer
• Domain / Language: Finance (BC Card — cards, accounts, national IDs, customer service text) / Korean + English
• Task: Token classification (BIOES) → character-offset PII spans → masking
• Labels (18): PERSON, RRN, FRN, CARD_NUMBER, ACCOUNT_NUMBER, SECRET, USER_ID, EMAIL, PHONE, PASSPORT, DRIVER_LICENSE, GENERIC_ID, ADDRESS, ZIPCODE, DATE, CARD_EXPIRY, CVC, IPIN
• Method: Full fine-tuning (all parameters incl. experts & router) with a re-initialized 73-class head (rows copied from the base head by taxonomy mapping)
• Decoding: constrained BIOES Viterbi (not per-token argmax) + whitespace span refinement — the bundled viterbi_calibration.json exposes precision↔recall operating-point biases without retraining
• Format: INT8 weight-only ONNX (MatMulNBits, QMoE, GatherBlockQuantized) with FP32 activations and logits; graph + external tensor data + tokenizer + label taxonomy + Viterbi calibration sidecar
• Sequence length: trained on sequences ≤768 tokens — chunk longer inputs
• Intended use:
  1. In-house PII masking gateway (detect → mask before text reaches an LLM)
  2. Offline privacy review and audit support (PII discovery in stored text, logs and documents)

1.2. Label Taxonomy (N=18)

The 18 labels re-map the upstream ai4privacy source labels to the granularity a Korean financial masking policy needs - merging fragments into single spans (GIVENNAME/SURNAME → PERSON, CITY/STREET/BUILDINGNUM → ADDRESS) and adding Korea-specific classes absent upstream (RRN, FRN, IPIN, CARD_EXPIRY, CVC, SECRET). data source records the row-source buckets in which each label occurs: ko means openpii-1.5m-ko, en means openpii-1.5m-en, and domain means locally synthesized rows.

label	description	data source
PERSON	full name (surname + given, single span)	ko, en, domain
RRN	resident registration number (Korea)	ko, domain
FRN	foreign registration number (Korea)	domain
CARD_NUMBER	credit/debit card PAN	ko, en, domain
ACCOUNT_NUMBER	bank account number	ko, domain
SECRET	auth secret (password / API key / token)	ko, domain
USER_ID	online member ID	ko, en, domain
EMAIL	email address	ko, en, domain
PHONE	phone number (mobile / landline)	ko, en, domain
PASSPORT	passport number	ko, en, domain
DRIVER_LICENSE	driver's license number	ko, en, domain
GENERIC_ID	generic identifier without a more specific taxonomy class	ko, en, domain
ADDRESS	address (city / street / building, single span)	ko, en, domain
ZIPCODE	postal code	ko, en, domain
DATE	date / time	ko, en, domain
CARD_EXPIRY	card expiry date	domain
CVC	card verification code	domain
IPIN	I-PIN number (Korea only)	domain

Each entity type has B-, I-, E- and S- boundary classes, plus the background class O.

1.3. Usage

Install the CPU runtime and client dependencies:

pip install "onnxruntime>=1.28,<1.29" "huggingface-hub>=1.5" "transformers>=5.6" numpy

The following example downloads the repository and runs the INT8 ONNX graph. To reproduce the validated output, use the functions in Viterbi Reference Implementation to apply constrained BIOES decoding and recover refined character-offset spans. Independent token argmax is not equivalent to the validated decoding chain.

import json
from pathlib import Path

import numpy as np
import onnxruntime as ort

from huggingface_hub import snapshot_download
from transformers import AutoTokenizer

model_id = "BCCard/MoAI-Privacy-Filter-INT8"
model_dir = Path(snapshot_download(repo_id=model_id))
tokenizer = AutoTokenizer.from_pretrained(model_dir)
config = json.loads((model_dir / "config.json").read_text(encoding="utf-8"))
labels = tuple(
    config["id2label"][str(index)]
    for index in range(len(config["id2label"]))
)
calibration = json.loads(
    (model_dir / "viterbi_calibration.json").read_text(encoding="utf-8")
)
biases = calibration["operating_points"]["default"]["biases"]

session = ort.InferenceSession(
    str(model_dir / "model_quantized.onnx"),
    providers=["CPUExecutionProvider"],
)
text = "고객 모아이님(000000-0000000)께서 010-0000-0000로 연락 요청하셨습니다."
encoding = tokenizer(
    text,
    return_offsets_mapping=True,
    add_special_tokens=False,
    return_tensors="np",
)
offsets = encoding.pop("offset_mapping")[0].tolist()
feeds = {
    model_input.name: np.asarray(encoding[model_input.name], dtype=np.int64)
    for model_input in session.get_inputs()
}
logits = session.run(["logits"], feeds)[0]
length = int(feeds["attention_mask"][0].sum())

# Copy constrained_viterbi() and decode_spans() from Section 4.3.
path = constrained_viterbi(logits[0, :length], labels, biases)
tags = [labels[class_id] for class_id in path]
spans = decode_spans(tags, offsets[:length], text)

print(logits.shape)
print(spans)

Output from the INT8 graph:

(1, 30, 73)
[
  {'start': 3, 'end': 6, 'label': 'PERSON'},
  {'start': 8, 'end': 22, 'label': 'RRN'},
  {'start': 26, 'end': 39, 'label': 'PHONE'}
]

The offsets use Python's half-open character interval [start, end). Masking is downstream policy logic. For example, the spans above can produce:

고객 [PERSON]님([RRN])께서 [PHONE]로 연락 요청하셨습니다.

For batches, enable right padding and pass only input_ids and attention_mask to the graph. offset_mapping stays outside ONNX and is used only to map decoded token tags back to the original text. Inputs are INT64 and the graph always returns FP32 logits.

1.4. Training Data

Dataset	Role	Size
(Public) BCCard/pii-masking-openpii-finance (v2)	Training / validation	~58.5k train rows · ~14.5k validation rows
(Private) BCCard/pii-masking-openpii-finance-test (v2)	Golden Set (release evaluation; not used for training/tuning)	2,000 rows (ko 1,460 / en 540)

• Sources: curated Korean subset of ai4privacy/pii-masking-openpii-1.5m (label taxonomy remapped, name spans merged & naturalized) + finance-domain synthetic templates + ~30% English replay (catastrophic forgetting guard)
• Hard-example design baked into v2: surface-similar non-PII decoys (FP suppression), label-confusion pairs in one sentence (RRN↔FRN, DRIVER_LICENSE↔GENERIC_ID), weak-context true PII (FN suppression), long-span address boundary variants
• All values are synthetic; validity-pattern collisions with real identifiers are removed at generation time (e.g. card numbers are forced to fail Luhn)

1.5. Training Procedure

Item	Value
Method	Full fine-tuning (1.4B params — experts and router included)
Head	33-class base head → 73-class head, initialized by copying base rows via taxonomy mapping
Loss	Token-level cross-entropy
Batch	effective 16 (per-device × world × accum), fixed across hardware layouts
LR / scheduler	1e-4 / linear decay, warmup 3%
Optimizer	AdamW (fused), weight decay 0.0, max_grad_norm 1.0
Epochs	5 — best checkpoint by validation span micro-F1, decoded with the same constrained Viterbi as deployment
Precision	FP32 master weights + BF16 autocast; MoE router/experts explicitly kept FP32 during compute
Hardware	1× NVIDIA H100 (~5h)

2. Evaluation

2.1. INT8 Validation Results

The INT8 artifact was evaluated on all 14,543 v2 validation rows with the same tokenizer, actual character offsets, constrained Viterbi decoder and whitespace refinement used in the serving example.

Metric	INT8 ONNX
strict micro F1	0.9599
macro F1	0.9607
ko strict micro F1	0.9560
ko macro F1	0.9571
en strict micro F1	0.9689
en macro F1	0.9641
masking coverage	0.9980
ko masking coverage	0.9987
en masking coverage	0.9967

Observed masking coverage is 99.8% overall and at least 99.6% in both the Korean and English validation slices. This leakage-oriented metric is reported as a diagnostic rather than a release gate.

2.2. INT8 Parity Against the BF16 Parent Model

The BF16 parent and INT8 artifacts were evaluated on the same 14,543-row v2 validation dataset with an equivalent character-span decoding chain. Delta values are calculated from the unrounded scores as INT8 minus BF16.

Metric	BF16 PyTorch	INT8 ONNX	Δ
strict micro F1	0.959921	0.959886	-0.000035
macro F1	0.960316	0.960662	+0.000346
ko strict micro F1	0.956171	0.956047	-0.000124
ko macro F1	0.956826	0.957084	+0.000258
en strict micro F1	0.968736	0.968910	+0.000174
en macro F1	0.963133	0.964131	+0.000997
masking coverage	0.997930	0.997980	+0.000049
ko masking coverage	0.998695	0.998662	-0.000033
en masking coverage	0.996540	0.996739	+0.000199

The INT8 artifact remained effectively equivalent to the BF16 parent: strict micro F1 changed by -0.000035 and masking coverage by +0.000049. The small positive and negative deltas reflect a few changed boundary decisions and do not show that either precision format is intrinsically more accurate.

2.3. Limitations and Deployment Guidance

• The graph uses Microsoft contrib operators. ONNX Runtime 1.28.0 with CPUExecutionProvider was validated. Confirm operator placement and fallback behavior before selecting another provider.
• The M4 validation run was a functional observation, not a serving benchmark. Measure warmup, latency percentiles, throughput, RSS and peak memory on target hardware.
• Independent token argmax can create invalid BIOES paths and does not reproduce the reported metrics. Use constrained Viterbi decoding.
• Weak-context Korean person names remain the main known miss channel. Names without honorifics, particles or nearby role cues require particular monitoring.
• Alphanumeric identifiers can swap among USER_ID, SECRET, GENERIC_ID and ACCOUNT_NUMBER. The value may still be masked even when the semantic label is wrong.
• Training and evaluation use synthetic data. Robustness to real customer text, slang, OCR noise and previously unseen credential formats has not been established.
• The label policy is fixed to the 18 types above. Changing taxonomy or boundary policy requires fine-tuning.
• Inputs beyond 768 tokens were not part of this release validation. Chunk longer inputs with overlap and reconcile spans at chunk boundaries.
• Use deterministic pattern rules as a backstop for structured identifiers and retain review paths for high-sensitivity workflows.

3. Future Work

• Benchmark the INT8 artifact on the actual serving hardware and execution provider.
• Improve weak-context PERSON coverage and collect privacy-safe shadow-mode failure patterns for a future data revision.

4. Meta Info

4.1. Citation

@misc{bccard2026moaiprivacyfilter,
  title        = {MoAI Privacy Filter INT8: A Korean Finance-Domain PII Detection Model},
  author       = {BC Card AX Team},
  year         = {2026},
  howpublished = {https://huggingface.co/BCCard/MoAI-Privacy-Filter-INT8},
  note         = {INT8 weight-only ONNX artifact of a full fine-tune of openai/privacy-filter}
}

4.2. See Also

• Base model - openai/privacy-filter
• Parent BF16 model - BCCard/MoAI-Privacy-Filter
• Training dataset - BCCard/pii-masking-openpii-finance
• Source data attribution - ai4privacy/pii-masking-openpii-1.5m under CC-BY-4.0

4.3. Viterbi Reference Implementation

These functions apply the BIOES transition constraints and calibration biases used for validation, then recover character-offset spans with whitespace boundary refinement. config.json defines the class-ID order for logits, while viterbi_calibration.json supplies the additive transition biases used by the decoder.

Show the constrained Viterbi and span-decoding reference code

import numpy as np


def transition_bias(
    from_prefix: str,
    to_prefix: str,
    biases: dict[str, float],
) -> float:
    if from_prefix == "O" and to_prefix == "O":
        return biases["transition_bias_background_stay"]
    if from_prefix == "O" and to_prefix in {"B", "S"}:
        return biases["transition_bias_background_to_start"]
    if from_prefix in {"E", "S"} and to_prefix == "O":
        return biases["transition_bias_end_to_background"]
    if from_prefix in {"E", "S"} and to_prefix in {"B", "S"}:
        return biases["transition_bias_end_to_start"]
    if from_prefix in {"B", "I"} and to_prefix == "I":
        return biases["transition_bias_inside_to_continue"]
    if from_prefix in {"B", "I"} and to_prefix == "E":
        return biases["transition_bias_inside_to_end"]
    raise ValueError(f"Unsupported transition family: {from_prefix} -> {to_prefix}")


def constrained_viterbi(
    logits: np.ndarray,
    labels: tuple[str, ...],
    biases: dict[str, float],
) -> list[int]:
    scores = np.asarray(logits, dtype=np.float32)
    num_classes = len(labels)
    parsed = [
        ("O", None) if tag == "O" else tuple(tag.split("-", 1))
        for tag in labels
    ]
    start_scores = np.full(num_classes, -np.inf, dtype=np.float32)
    end_scores = np.full(num_classes, -np.inf, dtype=np.float32)
    transition_scores = np.full(
        (num_classes, num_classes),
        -np.inf,
        dtype=np.float32,
    )

    for class_id, (tag_prefix, _) in enumerate(parsed):
        if tag_prefix in {"O", "B", "S"}:
            start_scores[class_id] = 0.0
        if tag_prefix in {"O", "E", "S"}:
            end_scores[class_id] = 0.0

    for from_id, (from_prefix, from_label) in enumerate(parsed):
        for to_id, (to_prefix, to_label) in enumerate(parsed):
            allowed = (
                from_prefix in {"O", "E", "S"}
                and to_prefix in {"O", "B", "S"}
            ) or (
                from_prefix in {"B", "I"}
                and to_prefix in {"I", "E"}
                and from_label == to_label
            )
            if allowed:
                transition_scores[from_id, to_id] = transition_bias(
                    from_prefix,
                    to_prefix,
                    biases,
                )

    delta = start_scores + scores[0]
    backpointers = np.zeros((scores.shape[0], num_classes), dtype=np.int64)
    for token_id in range(1, scores.shape[0]):
        candidates = delta[:, None] + transition_scores
        backpointers[token_id] = np.argmax(candidates, axis=0)
        delta = np.max(candidates, axis=0) + scores[token_id]

    last_class = int(np.argmax(delta + end_scores))
    path = [last_class]
    for token_id in range(scores.shape[0] - 1, 0, -1):
        path.append(int(backpointers[token_id, path[-1]]))
    return list(reversed(path))


def decode_spans(
    tags: list[str],
    offsets: list[list[int]],
    text: str,
) -> list[dict[str, int | str]]:
    spans = []
    current = None

    def flush() -> None:
        nonlocal current
        if current is None:
            return
        start = current["start"]
        end = current["end"]
        while start < end and text[start].isspace():
            start += 1
        while end > start and text[end - 1].isspace():
            end -= 1
        if start < end:
            spans.append(
                {
                    "start": start,
                    "end": end,
                    "label": current["label"],
                }
            )
        current = None

    for tag, (start, end) in zip(tags, offsets, strict=True):
        if tag == "O":
            flush()
            continue
        tag_prefix, label = tag.split("-", 1)
        if tag_prefix in {"B", "S"} or current is None or current["label"] != label:
            flush()
            current = {
                "start": start,
                "end": end,
                "label": label,
            }
        else:
            current["end"] = end
        if tag_prefix in {"E", "S"}:
            flush()
    flush()
    return spans