Laya volatility checkpoint for Grimoire

A fine-tune of Convai's Laya (ModernBERT-large, 421M parameters) that answers two questions about one fact from an AI agent's memory:

  • changes (yes/no): does it describe current state that can go out of date, rather than a decision, rule, historical event or finding?
  • speed (hours / days / weeks / months / years): if it changes, how fast?

Grimoire asks this once when a fact is written and uses the answers to decide when recall should tell an agent to re-check the fact. See docs/FRESHNESS.md.

Use

pip install laya
LAYA_EXTRA_MODELS='{"volatility": "<path to this repo>"}' LAYA_DEFAULT_MODEL=volatility laya-serve
export GRIMOIRE_DECISION_URL=http://127.0.0.1:8000
export GRIMOIRE_DECISION_MODEL=volatility
export GRIMOIRE_DECISION_CALIBRATION=2.147,0.359

Ask the questions exactly as in questions.json.

Training data

train.jsonl: 899 synthetic facts written by a small LLM to cover live state, work status, locations, configuration, decisions, rules and findings. Both questions were labelled as soft probabilities by TypeSafe's Jev. No real user memory is in the training data. laya-train --loss soft-ce --shuffle-options --epochs 6 --max-len 256, CPU.

Evaluation

On 90 hand-labelled facts from a real agent memory store (25 describe changing state). None of them appears in the training data.

Predictor AUC Speed right (of 25)
Stock Laya, zero-shot 0.50–0.55 β€”
Jev 0.90 14
This checkpoint 0.92 15

On 90 facts that is a tie with Jev, not a win: the 95% bootstrap interval for the AUC gap is [βˆ’0.06, +0.11]. The Platt calibration above was fitted on the same 90 facts. Refit it on your own labelled facts if you can.

Limits

One person's agent-memory style, English only, with code- and ops-shaped facts. Probabilities run high before calibration.

Downloads last month
4
Safetensors
Model size
0.4B params
Tensor type
F16
Β·
Inference Providers NEW
This model isn't deployed by any Inference Provider. πŸ™‹ Ask for provider support

Model tree for allnamestaken177/grimoire-volatility-laya

Finetuned
(254)
this model