NajahUniv/AraUni-Granite-97M-Intent-Classifier

AraUni Granite 97M Intent Classifier

Release: v0.3.0 · Lifecycle: preview

This is a 20-label Arabic university intent classifier fine-tuned from ibm-granite/granite-embedding-97m-multilingual-r2. It routes one question to zero, one, or several university-information labels. The model uses first_token pooling and independent sigmoid outputs; it is not a generative model.

Model details

• Architecture: ibm-granite/granite-embedding-97m-multilingual-r2 encoder + dropout + linear multi-label head
• Pooling: first_token
• Maximum input length: 256 tokens
• Model release: v0.3.0
• Release lifecycle: preview
• Weights: SafeTensors
• Training data: NajahUniv/arabic-univeristy-chatbot-qa-cleaned
• Dataset release: v0.1.0
• Pinned dataset revision: 9136ae156c49e1cc014020cef75c20a8b8d9aae0
• Decision thresholds: selected on validation data and stored in config.json and thresholds.json

The repository includes custom Transformers modeling code so it loads through the standard AutoClass API. Review the code in this repository before enabling trust_remote_code; in a production deployment, pin revision to a reviewed model commit SHA.

Release notes

v0.3.0 adds an optional, validated dynamic-INT8 ONNX package for CPU server inference under onnx/, together with backend- and precision-aware CLI and FastAPI examples. The canonical SafeTensors checkpoint remains the portable source of truth, and the verified Q8-linear/Q4- embedding browser package remains available for private WebGPU or WASM CPU inference. Trained weights, label taxonomy, saved thresholds, and dataset release are unchanged from the earlier releases.

Evaluation

Split	Macro F1	Micro F1	Subset accuracy	LRAP
Validation	0.9825	0.9839	0.9630	0.9965
Test	0.9848	0.9863	0.9700	0.9965

Threshold selection used only the validation split. The test split remained held out until the final comparison. Full aggregate and per-label results are available in validation_metrics.json and test_metrics.json.

Local inference speed benchmark

MPS

Model	Parameters	Batch 1 p50 (ms)	Batch 1 q/s	Batch 8 q/s	Batch 32 q/s	Peak RSS (MiB)
AraUni Granite 311M Intent Classifier	311.7M	10.9	92.1	387.4	629.9	3363
AraUni MARBERTv2 Intent Classifier	162.3M	10.1	113.3	460.1	807.6	1774
AraUni Granite 97M Intent Classifier	97.4M	3.7	170.3	889.9	1640.6	1552

CPU

Model	Parameters	Batch 1 p50 (ms)	Batch 1 q/s	Batch 8 q/s	Batch 32 q/s	Peak RSS (MiB)
AraUni Granite 311M Intent Classifier	311.7M	18.3	57.1	112.9	115.9	3261
AraUni MARBERTv2 Intent Classifier	162.3M	11.0	80.6	139.2	116.4	1771
AraUni Granite 97M Intent Classifier	97.4M	5.8	162.4	338.7	369.9	1551

Measured on Apple M5 Max (PyTorch 2.13.0, Transformers 5.14.1). Each batch size used 5 warm-up and 30 measured iterations over the same deterministically shuffled questions derived from the test split of NajahUniv/arabic-univeristy-chatbot-qa-cleaned@v0.1.0, pinned to commit 9136ae156c49e1cc014020cef75c20a8b8d9aae0, after applying the repository's standard dataset-preparation pipeline.

Numbers include tokenization, padding, device transfer, model forward pass, sigmoid, thresholding, and result construction. They exclude model loading and HTTP overhead. Latency and throughput depend on hardware, software versions, input lengths, batch size, and thermal state; compare only rows from this same run. Peak RSS is whole-process memory and MPS uses unified memory.

The full machine-readable benchmark report is included as benchmark_results.json. Use the direct benchmark for model-to-model speed comparisons; use the FastAPI load test below to size a specific deployment.

Optional CPU INT8 server inference

The onnx/ directory contains a dynamic-shape, dynamic-INT8 ONNX model for CPUExecutionProvider. It is a deployment alternative, not a replacement for the canonical SafeTensors checkpoint. onnx/onnx_config.json pins the graph checksum, label order, thresholds, runtime contract, numerical-verification report, and full validation-set quantization report. Its decision threshold is recalibrated on the validation split for the quantized logits; the SafeTensors checkpoint continues to use the canonical threshold in config.json. Use MODEL_BACKEND=onnx MODEL_PRECISION=int8 in the included FastAPI example. The conservative auto default retains PyTorch/SafeTensors on every device because INT8 speedups depend on the server CPU's instruction set and workload; benchmark before enabling it in production.

Validation result	FP32 ONNX	INT8 ONNX
Graph size	373.4 MiB	292.2 MiB
Macro F1	0.9825	0.9805
Micro F1	0.9839	0.9814
Decision threshold	canonical	0.20

The measured macro-F1 change after INT8 threshold recalibration is +0.0020 (positive means a drop). See onnx/quantization_report.json for source-threshold results, logit differences, and local latency.

Private browser inference (WebGPU and WASM CPU)

The webgpu/ directory contains a verified, fixed-shape ONNX Runtime Web model for private, client-side inference. It uses Q8 linear weights and Q4 embedding weights, batch size 1, and the same saved per-label thresholds as the Transformers model. Load webgpu/webgpu_config.json first; it records the graph checksum, tokenizer artifacts, input contract, and validation report. Applications can execute the same graph with ONNX Runtime Web's WebGPU provider or its WASM CPU provider. WASM is more broadly compatible but can be substantially slower. A privacy-preserving fallback should try WASM locally and require explicit consent before switching to server inference.

Install the browser libraries with:

npm install @huggingface/transformers onnxruntime-web

Run inference inside a Web Worker so model loading does not block the page. This WebGPU example loads the pinned release and applies the saved per-label thresholds from webgpu_config.json:

import { AutoTokenizer } from "@huggingface/transformers";
import * as ort from "onnxruntime-web/webgpu";

ort.env.wasm.wasmPaths = "/ort/"; // Serve matching ORT Web helper files here.
const executionProviders = ["webgpu"];

const modelId = "NajahUniv/AraUni-Granite-97M-Intent-Classifier";
const revision = "v0.3.0";
const artifactRoot =
  `https://huggingface.co/${modelId}/resolve/${revision}/webgpu/`;
const config = await fetch(`${artifactRoot}webgpu_config.json`).then((response) => {
  if (!response.ok) throw new Error(`Could not load browser config: ${response.status}`);
  return response.json();
});
const tokenizer = await AutoTokenizer.from_pretrained(modelId, { revision });
const session = await ort.InferenceSession.create(
  `${artifactRoot}${config.model_file}`,
  { executionProviders, graphOptimizationLevel: "all" },
);

const text = "ما هي شروط القبول في الجامعة؟";
const encoded = await tokenizer(text, {
  padding: "max_length",
  truncation: true,
  max_length: config.max_length,
});
const feeds = Object.fromEntries(
  config.input_names.map((name) => [
    name,
    new ort.Tensor("int64", encoded[name].data, encoded[name].dims),
  ]),
);
const output = await session.run(feeds);
const logits = output[config.output_name].data;
const predictions = config.labels.map((label, index) => {
  const score = 1 / (1 + Math.exp(-Number(logits[index])));
  const threshold = Number(config.thresholds[label]);
  return { label, score, threshold, selected: score >= threshold };
});
console.table(predictions);

For private CPU execution, put the same shared inference code in a separate worker and change the runtime setup to:

import * as ort from "onnxruntime-web/wasm";

ort.env.wasm.wasmPaths = "/ort/";
ort.env.wasm.numThreads = 1; // Works without cross-origin isolation.
const executionProviders = ["wasm"];

Pass executionProviders to InferenceSession.create. Copy the matching ONNX Runtime Web .mjs and .wasm files from node_modules/onnxruntime-web/dist/ to the public /ort/ directory. Keep WebGPU and WASM in separate workers/bundles; do not assume that adding "wasm" after "webgpu" in one provider list provides a compatible fallback for every quantized operator. If WebGPU fails, start the WASM worker explicitly. Only switch to a server runtime after obtaining user consent.

This is multi-label classification: apply sigmoid independently to every logit, compare each score with that label's saved threshold, and allow zero, one, or several labels to be selected. The sigmoid scores are ranking/confidence signals and are not guaranteed to be calibrated probabilities.

Basic usage

import torch
from transformers import AutoModelForSequenceClassification, AutoTokenizer

model_id = "NajahUniv/AraUni-Granite-97M-Intent-Classifier"
model_revision = "v0.3.0"
tokenizer = AutoTokenizer.from_pretrained(
    model_id,
    revision=model_revision,
    trust_remote_code=True,
)
model = AutoModelForSequenceClassification.from_pretrained(
    model_id,
    revision=model_revision,
    trust_remote_code=True,
).eval()

text = "ما هي شروط القبول في الجامعة؟"
inputs = tokenizer(text, return_tensors="pt", truncation=True, max_length=model.config.max_length)
with torch.inference_mode():
    probabilities = torch.sigmoid(model(**inputs).logits[0])

result = {}
for index, probability in enumerate(probabilities.tolist()):
    label = model.config.id2label[index]
    threshold = model.config.thresholds[label]
    result[label] = {"probability": probability, "selected": probability >= threshold}

selected_labels = [label for label, value in result.items() if value["selected"]]
print(selected_labels)

A complete command-line example is included at examples/basic_inference.py:

python examples/basic_inference.py \
  --model-id NajahUniv/AraUni-Granite-97M-Intent-Classifier \
  --revision v0.3.0 \
  --backend auto \
  --precision auto \
  --text "ما هي شروط التسجيل؟"

Complete basic inference example

"""Run multi-label inference with a published PyTorch or CPU INT8 ONNX model."""

from __future__ import annotations

import argparse
import json
from pathlib import Path

import numpy as np
import torch
from huggingface_hub import hf_hub_download
from transformers import AutoConfig, AutoModelForSequenceClassification, AutoTokenizer

DEFAULT_MODEL_ID = "NajahUniv/AraUni-Granite-97M-Intent-Classifier"


def choose_device(requested: str) -> str:
    if requested != "auto":
        return requested
    if torch.cuda.is_available():
        return "cuda"
    if torch.backends.mps.is_available():
        return "mps"
    return "cpu"


def resolve_runtime(backend: str, precision: str, device: str) -> tuple[str, str]:
    resolved_backend = "pytorch" if backend == "auto" else backend
    resolved_precision = precision
    if precision == "auto":
        if resolved_backend == "onnx":
            resolved_precision = "int8"
        elif device == "cuda":
            resolved_precision = "bf16" if torch.cuda.is_bf16_supported() else "fp16"
        else:
            resolved_precision = "fp32"
    if resolved_backend == "onnx" and (device != "cpu" or resolved_precision != "int8"):
        raise ValueError("the published ONNX artifact supports CPU INT8 only")
    if resolved_backend == "pytorch" and resolved_precision == "int8":
        raise ValueError("precision=int8 requires backend=onnx")
    if resolved_backend == "pytorch" and resolved_precision in {"bf16", "fp16"} and device != "cuda":
        raise ValueError("the example enables bf16/fp16 only on CUDA")
    return resolved_backend, resolved_precision


def hub_or_local_file(model_id: str, filename: str, revision: str | None) -> str:
    local = Path(model_id) / filename
    if local.is_file():
        return str(local)
    return hf_hub_download(model_id, filename, revision=revision)


def main() -> None:
    parser = argparse.ArgumentParser(description=__doc__)
    parser.add_argument("--model-id", default=DEFAULT_MODEL_ID)
    parser.add_argument("--revision", help="Pin a release tag or reviewed commit SHA")
    parser.add_argument("--text", required=True)
    parser.add_argument("--device", default="auto", choices=("auto", "cpu", "mps", "cuda"))
    parser.add_argument("--backend", default="auto", choices=("auto", "pytorch", "onnx"))
    parser.add_argument(
        "--precision",
        default="auto",
        choices=("auto", "fp32", "bf16", "fp16", "int8"),
    )
    parser.add_argument("--top-k", type=int, default=5)
    args = parser.parse_args()

    device = "cpu" if args.backend == "onnx" and args.device == "auto" else choose_device(args.device)
    backend, precision = resolve_runtime(args.backend, args.precision, device)
    load_kwargs = {"revision": args.revision} if args.revision else {}
    tokenizer = AutoTokenizer.from_pretrained(
        args.model_id,
        trust_remote_code=True,
        **load_kwargs,
    )
    config = AutoConfig.from_pretrained(
        args.model_id,
        trust_remote_code=True,
        **load_kwargs,
    )
    thresholds = config.thresholds
    if backend == "onnx":
        import onnxruntime as ort

        onnx_config_path = hub_or_local_file(
            args.model_id,
            "onnx/onnx_config.json",
            args.revision,
        )
        onnx_config = json.loads(Path(onnx_config_path).read_text(encoding="utf-8"))
        thresholds = onnx_config["thresholds"]
        model_path = hub_or_local_file(
            args.model_id,
            "onnx/model_int8.onnx",
            args.revision,
        )
        session = ort.InferenceSession(model_path, providers=["CPUExecutionProvider"])
        encoded = tokenizer(
            args.text,
            return_tensors="np",
            truncation=True,
            max_length=config.max_length,
        )
        logits = session.run(
            ["logits"],
            {
                "input_ids": encoded["input_ids"].astype(np.int64),
                "attention_mask": encoded["attention_mask"].astype(np.int64),
            },
        )[0][0]
        probabilities = 1.0 / (1.0 + np.exp(-logits))
    else:
        dtype = {
            "fp32": torch.float32,
            "bf16": torch.bfloat16,
            "fp16": torch.float16,
        }[precision]
        model = AutoModelForSequenceClassification.from_pretrained(
            args.model_id,
            trust_remote_code=True,
            torch_dtype=dtype,
            **load_kwargs,
        ).to(device).eval()
        encoded = tokenizer(
            args.text,
            return_tensors="pt",
            truncation=True,
            max_length=model.config.max_length,
        ).to(device)
        with torch.inference_mode():
            probabilities = torch.sigmoid(model(**encoded).logits[0]).cpu().float().numpy()

    labels = [config.id2label[index] for index in range(config.num_labels)]
    ranked = sorted(
        (
            {
                "label": label,
                "probability": float(probabilities[index]),
                "threshold": float(thresholds[label]),
                "selected": float(probabilities[index]) >= float(thresholds[label]),
            }
            for index, label in enumerate(labels)
        ),
        key=lambda item: item["probability"],
        reverse=True,
    )
    print(
        json.dumps(
            {
                "text": args.text,
                "backend": backend,
                "precision": precision,
                "device": device,
                "selected_labels": [item["label"] for item in ranked if item["selected"]],
                "scores": ranked[: max(1, args.top_k)],
            },
            ensure_ascii=False,
            indent=2,
        )
    )


if __name__ == "__main__":
    main()

FastAPI hosting

The included service loads the model once, supports batching, uses MPS automatically on Apple Silicon, and optionally requires a bearer token:

pip install -r requirements.txt
MODEL_ID=NajahUniv/AraUni-Granite-97M-Intent-Classifier MODEL_REVISION=v0.3.0 MODEL_BACKEND=auto \
  MODEL_PRECISION=auto MODEL_API_KEY=change-me \
  uvicorn examples.fastapi_app:app --host 0.0.0.0 --port 8000

curl http://localhost:8000/classify \
  -H 'Content-Type: application/json' \
  -H 'Authorization: Bearer change-me' \
  -d '{"texts":["كيف يمكنني دفع الرسوم؟"],"top_k":5}'

In Swagger UI at http://localhost:8000/docs, click Authorize and enter only the MODEL_API_KEY value (change-me in the example). Swagger adds the Bearer prefix.

Complete FastAPI server example

"""FastAPI service with automatic PyTorch or CPU INT8 ONNX inference."""

from __future__ import annotations

import hmac
import json
import os
import threading
from contextlib import asynccontextmanager
from pathlib import Path
from typing import Annotated

import numpy as np
import torch
from fastapi import Depends, FastAPI, HTTPException
from fastapi.security import HTTPAuthorizationCredentials, HTTPBearer
from huggingface_hub import hf_hub_download
from pydantic import BaseModel, Field
from transformers import AutoConfig, AutoModelForSequenceClassification, AutoTokenizer

DEFAULT_MODEL_ID = "NajahUniv/AraUni-Granite-97M-Intent-Classifier"
MODEL_ID = os.getenv("MODEL_ID", DEFAULT_MODEL_ID)
MODEL_REVISION = os.getenv("MODEL_REVISION")
MODEL_DEVICE = os.getenv("MODEL_DEVICE", "auto")
MODEL_BACKEND = os.getenv("MODEL_BACKEND", "auto")
MODEL_PRECISION = os.getenv("MODEL_PRECISION", "auto")
MODEL_API_KEY = os.getenv("MODEL_API_KEY")
MAX_BATCH_SIZE = int(os.getenv("MAX_BATCH_SIZE", "64"))


def choose_device() -> str:
    if MODEL_BACKEND == "onnx" and MODEL_DEVICE == "auto":
        return "cpu"
    if MODEL_DEVICE != "auto":
        return MODEL_DEVICE
    if torch.cuda.is_available():
        return "cuda"
    if torch.backends.mps.is_available():
        return "mps"
    return "cpu"


def resolve_runtime(device: str) -> tuple[str, str]:
    if MODEL_BACKEND not in {"auto", "pytorch", "onnx"}:
        raise ValueError("MODEL_BACKEND must be auto, pytorch, or onnx")
    if MODEL_PRECISION not in {"auto", "fp32", "bf16", "fp16", "int8"}:
        raise ValueError("MODEL_PRECISION must be auto, fp32, bf16, fp16, or int8")
    backend = "pytorch" if MODEL_BACKEND == "auto" else MODEL_BACKEND
    precision = MODEL_PRECISION
    if precision == "auto":
        if backend == "onnx":
            precision = "int8"
        elif device == "cuda":
            precision = "bf16" if torch.cuda.is_bf16_supported() else "fp16"
        else:
            precision = "fp32"
    if backend == "onnx" and (device != "cpu" or precision != "int8"):
        raise ValueError("the published ONNX artifact supports CPU INT8 only")
    if backend == "pytorch" and precision == "int8":
        raise ValueError("MODEL_PRECISION=int8 requires MODEL_BACKEND=onnx")
    if backend == "pytorch" and precision in {"bf16", "fp16"} and device != "cuda":
        raise ValueError("this example enables bf16/fp16 only on CUDA")
    return backend, precision


def hub_or_local_file(filename: str) -> str:
    local = Path(MODEL_ID) / filename
    if local.is_file():
        return str(local)
    return hf_hub_download(MODEL_ID, filename, revision=MODEL_REVISION)


class ClassifyRequest(BaseModel):
    texts: list[str] = Field(min_length=1)
    top_k: int = Field(default=5, ge=1)
    threshold: float | None = Field(default=None, gt=0, lt=1)


class ModelRuntime:
    def __init__(self) -> None:
        load_kwargs = {"revision": MODEL_REVISION} if MODEL_REVISION else {}
        self.tokenizer = AutoTokenizer.from_pretrained(
            MODEL_ID,
            trust_remote_code=True,
            **load_kwargs,
        )
        self.config = AutoConfig.from_pretrained(
            MODEL_ID,
            trust_remote_code=True,
            **load_kwargs,
        )
        self.device = choose_device()
        self.backend, self.precision = resolve_runtime(self.device)
        self.lock = threading.Lock()
        self.thresholds = self.config.thresholds
        self.model = None
        self.session = None
        if self.backend == "onnx":
            import onnxruntime as ort

            onnx_config = json.loads(
                Path(hub_or_local_file("onnx/onnx_config.json")).read_text(encoding="utf-8")
            )
            self.thresholds = onnx_config["thresholds"]
            self.session = ort.InferenceSession(
                hub_or_local_file("onnx/model_int8.onnx"),
                providers=["CPUExecutionProvider"],
            )
        else:
            dtype = {
                "fp32": torch.float32,
                "bf16": torch.bfloat16,
                "fp16": torch.float16,
            }[self.precision]
            self.model = AutoModelForSequenceClassification.from_pretrained(
                MODEL_ID,
                trust_remote_code=True,
                torch_dtype=dtype,
                **load_kwargs,
            ).to(self.device).eval()

    def probabilities(self, texts: list[str]) -> np.ndarray:
        if self.backend == "onnx":
            encoded = self.tokenizer(
                texts,
                return_tensors="np",
                padding=True,
                truncation=True,
                max_length=self.config.max_length,
            )
            with self.lock:
                logits = self.session.run(
                    ["logits"],
                    {
                        "input_ids": encoded["input_ids"].astype(np.int64),
                        "attention_mask": encoded["attention_mask"].astype(np.int64),
                    },
                )[0]
            return 1.0 / (1.0 + np.exp(-logits))
        encoded = self.tokenizer(
            texts,
            return_tensors="pt",
            padding=True,
            truncation=True,
            max_length=self.config.max_length,
        ).to(self.device)
        with self.lock, torch.inference_mode():
            return torch.sigmoid(self.model(**encoded).logits).cpu().float().numpy()

    def classify(self, request: ClassifyRequest) -> list[dict[str, object]]:
        if len(request.texts) > MAX_BATCH_SIZE:
            raise HTTPException(413, f"at most {MAX_BATCH_SIZE} texts are allowed per request")
        probabilities = self.probabilities(request.texts)
        labels = [self.config.id2label[index] for index in range(self.config.num_labels)]
        results = []
        for text, row in zip(request.texts, probabilities, strict=True):
            scores = []
            for index, label in enumerate(labels):
                threshold = (
                    request.threshold
                    if request.threshold is not None
                    else float(self.thresholds[label])
                )
                scores.append(
                    {
                        "label": label,
                        "probability": float(row[index]),
                        "threshold": threshold,
                        "selected": float(row[index]) >= threshold,
                    }
                )
            scores.sort(key=lambda item: item["probability"], reverse=True)
            results.append(
                {
                    "text": text,
                    "selected_labels": [item["label"] for item in scores if item["selected"]],
                    "scores": scores[: min(request.top_k, len(scores))],
                }
            )
        return results


runtime: ModelRuntime | None = None


@asynccontextmanager
async def lifespan(_: FastAPI):
    global runtime
    runtime = ModelRuntime()
    yield
    runtime = None


app = FastAPI(title="AraUni Multi-label Intent Classifier", lifespan=lifespan)
bearer_scheme = HTTPBearer(
    auto_error=False,
    scheme_name="BearerAuth",
    description="Enter the MODEL_API_KEY value. Swagger adds the 'Bearer' prefix.",
)


def authorize(
    credentials: Annotated[
        HTTPAuthorizationCredentials | None,
        Depends(bearer_scheme),
    ],
) -> None:
    if MODEL_API_KEY is None:
        return
    if (
        credentials is None
        or credentials.scheme.lower() != "bearer"
        or not hmac.compare_digest(credentials.credentials, MODEL_API_KEY)
    ):
        raise HTTPException(
            401,
            "invalid bearer token",
            headers={"WWW-Authenticate": "Bearer"},
        )


@app.get("/health")
def health() -> dict[str, object]:
    return {
        "status": "ok",
        "model_id": MODEL_ID,
        "device": runtime.device if runtime else None,
        "backend": runtime.backend if runtime else None,
        "precision": runtime.precision if runtime else None,
    }


@app.get("/labels", dependencies=[Depends(authorize)])
def labels() -> dict[int, str]:
    if runtime is None:
        raise HTTPException(503, "model is not ready")
    return dict(runtime.config.id2label)


@app.post("/classify", dependencies=[Depends(authorize)])
def classify(request: ClassifyRequest) -> list[dict[str, object]]:
    if runtime is None:
        raise HTTPException(503, "model is not ready")
    return runtime.classify(request)

For public production hosting, also add TLS, request-size/rate limits, monitoring, and a pinned MODEL_REVISION commit SHA. On macOS, use one Uvicorn worker so multiple processes do not each load a separate copy of the model into memory.

Labels

• 0: academic_calendar
• 1: academic_programs
• 2: admissions
• 3: campus_services
• 4: contact_and_location
• 5: courses_and_study_plans
• 6: exams_and_grades
• 7: general_university_information
• 8: graduation
• 9: library
• 10: news_and_events
• 11: out_of_scope
• 12: registration
• 13: research_and_postgraduate
• 14: scholarships_and_aid
• 15: staff_and_departments
• 16: student_services
• 17: technical_support
• 18: transfer_and_equivalency
• 19: tuition_and_payments

Intended use and limitations

The model is intended for routing Arabic university-chatbot questions within the label taxonomy above. It should not be treated as an authoritative source of admissions, academic, payment, or policy advice. The training data is task-specific; consult the pinned dataset release for its synthetic and real-user composition. The strong held-out scores may not transfer to other universities, taxonomies, dialect distributions, spelling patterns, or production traffic. Inputs outside the training distribution can still receive confident scores. Evaluate on real, independently collected traffic and add human fallback/escalation before deployment. The sigmoid values are classification scores, not guaranteed calibrated probabilities.

Reproducibility and repository contents

• model.safetensors: complete encoder and classifier weights (the only weight copy)

• config.json: architecture, label mappings, pooling, maximum length, and thresholds

• tokenizer.json and tokenizer_config.json: tokenizer artifacts

• configuration_arauni.py and modeling_arauni.py: AutoClass code

• dataset_provenance.json, training_args.json, and metadata.json: provenance

• validation_metrics.json, test_metrics.json, and thresholds.json: evaluation artifacts

• benchmark_results.json: shared MPS/CPU speed comparison and environment metadata

• onnx/: validated dynamic-INT8 ONNX CPU graph and verification reports

• webgpu/: verified ONNX Runtime Web graph, tokenizer, thresholds, and reports

This model card reports the saved checkpoint artifacts; consult the linked base-model card for its pretraining details, license, intended uses, and limitations.