sevclassifier-jev-style-1.5b

Local System One model for incident severity classification. JEV-compatible typed decisions, no API key required.

Drop-in local alternative to TypeSafe Jev for incident management pipelines. Returns SEV1-4, escalation flag, and blast radius โ€” swap the endpoint, keep the agent.


Overview

sevclassifier-jev-style-1.5b is a fine-tuned Qwen2.5-1.5B-Instruct that produces typed incident severity classifications โ€” matching the output contract of TypeSafe System One / Jev for incident triage pipelines.

It outputs a single JSON object per incident:

{
  "severity": "SEV1 | SEV2 | SEV3 | SEV4",
  "rationale": "<one sentence>",
  "escalate": true,
  "page_oncall": false,
  "affected_blast_radius": "<scope description>"
}

Feed it SIEM alerts, PagerDuty payloads, or Datadog monitor events. Get a typed decision the agent acts on immediately โ€” page oncall for SEV1/2, create ticket for SEV3, suppress SEV4 noise.


Why local?

Jev (TypeSafe API) sevclassifier-jev-style-1.5b
Typed output yes yes
Latency ~115ms (network) ~76ms (local, MPS)
Cost per-request billing free after download
Alert data leaves org yes no
Fine-tune on your runbook no yes (LoRA)
Accuracy (SevBench) 93.6% F1 92.9% F1
SEV1 recall (no missed majors) 99.1% 98.8%

Incident alerts contain hostnames, IPs, service names, and error messages โ€” the exact fingerprint of your infrastructure. Run the classifier on-prem.


Quickstart

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch, json, re

tok = AutoTokenizer.from_pretrained("enterprise-ai-lab/sevclassifier-jev-style-1.5b")
model = AutoModelForCausalLM.from_pretrained(
    "enterprise-ai-lab/sevclassifier-jev-style-1.5b", dtype=torch.float32
).eval()

SYS = (
    "You are an incident severity classification assistant. For each incident report output ONE "
    "JSON object with keys: severity (SEV1|SEV2|SEV3|SEV4), rationale (one sentence), "
    "escalate (boolean), page_oncall (boolean), affected_blast_radius (string). "
    "SEV1=critical outage/breach, SEV2=major impact, SEV3=minor/degraded, SEV4=informational/noise. "
    "Output only the JSON."
)

def classify_incident(title, details):
    content = f"Incident: {title}\n\nDetails: {details}"
    msgs = [{"role": "system", "content": SYS}, {"role": "user", "content": content}]
    prompt = tok.apply_chat_template(msgs, tokenize=False, add_generation_prompt=True)
    ids = tok(prompt, return_tensors="pt")
    with torch.no_grad():
        out = model.generate(**ids, max_new_tokens=150, do_sample=False,
                             pad_token_id=tok.eos_token_id)
    text = tok.decode(out[0][ids["input_ids"].shape[1]:], skip_special_tokens=True)
    m = re.search(r"\{.*\}", text, re.DOTALL)
    return json.loads(m.group(0)) if m else {}

result = classify_incident(
    "Production database unreachable",
    "Primary RDS instance not responding. All pods returning 500. 12k users affected."
)
# {"severity": "SEV1", "rationale": "Full production outage.", "escalate": true, "page_oncall": true, "affected_blast_radius": "All production traffic"}

PagerDuty / OpsGenie integration

Wire it as an OpenAI-compatible endpoint and drop it into your existing alert routing agent:

# Before (Jev):
llm = ChatOpenAI(base_url="https://api.typesafe.ai/v1", api_key=JEV_KEY, model="jev-latest")
# After (SevClassifier local):
llm = ChatOpenAI(base_url="http://localhost:8000/v1", api_key="not-needed", model="sevclassifier-jev-style-1.5b")

Routing pattern:

verdict = classify_incident(title, details)
if verdict["severity"] in ("SEV1", "SEV2") and verdict["page_oncall"]:
    pagerduty.trigger(title, urgency="high")
elif verdict["severity"] == "SEV3":
    jira.create_ticket(title, priority="medium")
# SEV4: suppress, log only

Training

Fine-tuned with LoRA (r=16, alpha=32) on a curated incident corpus:

  • 78,500 incidents from SRE runbooks, post-mortems, and SIEM alert exports (2019-2024)
  • Ground truth: retrospective human severity labels from SRE teams
  • Covers: infrastructure outages, security incidents, data pipeline failures, user-impacting bugs
  • Held-out validation: 4,200 incidents stratified by severity
  • Training: 12 epochs, AdamW lr=2e-4, MPS/CUDA

Benchmarks

SevBench (holdout, n=4,200)

Model Precision Recall F1
sevclassifier-jev-style-1.5b 93.3% 92.5% 92.9%
Jev (TypeSafe API) 94.0% 93.3% 93.6%
GPT-4o-mini (zero-shot) 87.4% 86.8% 87.1%
Rule-based (keyword) 74.2% 68.9% 71.5%

SEV1 Recall (no missed critical incidents)

Model SEV1 Recall False Escalation Rate
sevclassifier-jev-style-1.5b 98.8% 6.2%
Jev (TypeSafe API) 99.1% 5.9%
GPT-4o-mini (zero-shot) 96.3% 11.4%

High SEV1 recall is the critical metric: a missed SEV1 is an undetected outage. False escalations are expensive but recoverable.

Latency (Apple M2, 16GB, batch=1)

Model p50 p95 p99
sevclassifier-jev-style-1.5b (local MPS) 76ms 108ms 129ms
Jev (TypeSafe API, US-West) 115ms 198ms 372ms

Intended use

  • Alert fatigue reduction (auto-suppress SEV4 noise)
  • Automatic PagerDuty escalation for SEV1/2
  • Incident ticket creation and prioritization
  • SOC triage assist for security incidents

Limitations

  • Trained on English incident descriptions
  • Severity scales vary by org โ€” fine-tune on your own runbook for best results
  • Not a replacement for domain-specific monitoring thresholds

License

Apache 2.0. Base model (Qwen2.5-1.5B-Instruct) is subject to its own Qwen license.


Downloads last month
20,426
Safetensors
Model size
2B params
Tensor type
F32
ยท
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for JackKozmo29/sevclassifier-jev-style-1.5b

Finetuned
(1951)
this model

Space using JackKozmo29/sevclassifier-jev-style-1.5b 1