temsa commited on
Commit
641ff0e
·
verified ·
1 Parent(s): 562db84

Refresh portfolio comparison benchmark tables

Browse files
Files changed (1) hide show
  1. PORTFOLIO_COMPARISON.md +3 -1
PORTFOLIO_COMPARISON.md CHANGED
@@ -8,12 +8,14 @@ Use this section for the fastest public comparison across the `temsa` PII maskin
8
  - The first PPSN table only includes public artifacts that ship comparable PPSN accuracy and CPU throughput.
9
  - Missing cells in the archive tables mean the older release did not ship that metric in its public bundle.
10
  - DiffMask rows use the reconciled `clean_single_pass` harness that matches the deployed runtime.
 
11
  - The same content is shipped as `PORTFOLIO_COMPARISON.md` inside each public model repo.
12
 
13
  ### Irish Core PII: Comparable Public Checkpoints
14
 
15
  | Repo | Stack | Full Core F1 | Q8 Core F1 | Q8 Multilingual PPSN F1 | Q8 Core ex/s |
16
  |---|---|---:|---:|---:|---:|
 
17
  | [`temsa/OpenMed-mLiteClinical-IrishCorePII-135M-v2-rc8`](https://huggingface.co/temsa/OpenMed-mLiteClinical-IrishCorePII-135M-v2-rc8) | Raw-only token-span | 0.9737 | 0.9737 | 0.9176 | 46.1 |
18
  | [`temsa/OpenMed-mLiteClinical-IrishCorePII-135M-v2-rc7`](https://huggingface.co/temsa/OpenMed-mLiteClinical-IrishCorePII-135M-v2-rc7) | Hybrid classifier + generated scanner spec | 1.0000 | 0.9934 | 1.0000 | 30.0 |
19
  | [`temsa/OpenMed-mLiteClinical-IrishCorePII-135M-v2-rc6`](https://huggingface.co/temsa/OpenMed-mLiteClinical-IrishCorePII-135M-v2-rc6) | Hybrid classifier + repair decoders | 1.0000 | 0.9934 | 1.0000 | 29.5 |
@@ -55,4 +57,4 @@ Finance-boundary q8 F1 is `1.0000` for `OpenMed-mLiteClinical-IrishCorePII-135M-
55
  | [`temsa/OpenMed-PPSN-v5`](https://huggingface.co/temsa/OpenMed-PPSN-v5) | irish_reg_v5 raw 0.8235; irish_reg_v5 hybrid strict 1.0000 | Hybrid PPSN-only checkpoint; predates the canonical multilingual suite packaging. |
56
  | [`temsa/OpenMed-PPSN-v4`](https://huggingface.co/temsa/OpenMed-PPSN-v4) | synthetic non-PPSN drift check only | Predates the current PPSN eval suite; no packaged apples-to-apples multilingual CPU row. |
57
 
58
- If you need one CPU-first Irish core model today, compare `OpenMed-mLiteClinical-IrishCorePII-135M-v2-rc8` against `IrishCore-DiffMask-135M-v1-rc6`. If you need a PPSN-only artifact, compare the canonical `fp32`, `fp16`, and `q8` variants of `OpenMed-mLiteClinical-IrishPPSN-135M-v1` directly in the table above.
 
8
  - The first PPSN table only includes public artifacts that ship comparable PPSN accuracy and CPU throughput.
9
  - Missing cells in the archive tables mean the older release did not ship that metric in its public bundle.
10
  - DiffMask rows use the reconciled `clean_single_pass` harness that matches the deployed runtime.
11
+ - GlobalPointer rows use the public raw-only span-matrix release bundle and its packaged q8 ONNX artifact.
12
  - The same content is shipped as `PORTFOLIO_COMPARISON.md` inside each public model repo.
13
 
14
  ### Irish Core PII: Comparable Public Checkpoints
15
 
16
  | Repo | Stack | Full Core F1 | Q8 Core F1 | Q8 Multilingual PPSN F1 | Q8 Core ex/s |
17
  |---|---|---:|---:|---:|---:|
18
+ | [`temsa/IrishCore-GlobalPointer-135M-v1-rc1`](https://huggingface.co/temsa/IrishCore-GlobalPointer-135M-v1-rc1) | GlobalPointer raw-only span-matrix | 0.9934 | 0.9934 | 0.9371 | 36.6 |
19
  | [`temsa/OpenMed-mLiteClinical-IrishCorePII-135M-v2-rc8`](https://huggingface.co/temsa/OpenMed-mLiteClinical-IrishCorePII-135M-v2-rc8) | Raw-only token-span | 0.9737 | 0.9737 | 0.9176 | 46.1 |
20
  | [`temsa/OpenMed-mLiteClinical-IrishCorePII-135M-v2-rc7`](https://huggingface.co/temsa/OpenMed-mLiteClinical-IrishCorePII-135M-v2-rc7) | Hybrid classifier + generated scanner spec | 1.0000 | 0.9934 | 1.0000 | 30.0 |
21
  | [`temsa/OpenMed-mLiteClinical-IrishCorePII-135M-v2-rc6`](https://huggingface.co/temsa/OpenMed-mLiteClinical-IrishCorePII-135M-v2-rc6) | Hybrid classifier + repair decoders | 1.0000 | 0.9934 | 1.0000 | 29.5 |
 
57
  | [`temsa/OpenMed-PPSN-v5`](https://huggingface.co/temsa/OpenMed-PPSN-v5) | irish_reg_v5 raw 0.8235; irish_reg_v5 hybrid strict 1.0000 | Hybrid PPSN-only checkpoint; predates the canonical multilingual suite packaging. |
58
  | [`temsa/OpenMed-PPSN-v4`](https://huggingface.co/temsa/OpenMed-PPSN-v4) | synthetic non-PPSN drift check only | Predates the current PPSN eval suite; no packaged apples-to-apples multilingual CPU row. |
59
 
60
+ If you need the strongest current raw-only Irish core model, start with `IrishCore-GlobalPointer-135M-v1-rc1`. If you need the fastest CPU-first raw-only line, compare it against `IrishCore-DiffMask-135M-v1-rc6`. If you need a PPSN-only artifact, compare the canonical `fp32`, `fp16`, and `q8` variants of `OpenMed-mLiteClinical-IrishPPSN-135M-v1` directly in the table above.