mrvictoru/energydecision-dt-v2-sdp

AEMO SDP-Teacher Decision Transformer (planner-distilled)

The shipped / preferred model of the energydecision benchmark. A standalone Decision Transformer distilled from an honest (non-clairvoyant) SDP-planning teacher — no solver at inference — that beats the PPO reference on all four identity surfaces and passes the market-impact gate, with bootstrap 95% CIs excluding zero on every comparison.

Files in this repo

File	Model	Status
aemo_dt_sdp_jtsoc_fullcorpus.pt	Stage C — J_t(soc) RTG, combined corpus (640 eps)	✅ Shipped — use this one
aemo_dt_fcas_model.pt	Stage B — conservative teacher (320 eps), discounted RTG	Historical (superseded; kept for the ablation)
aemo_dt_fcas_best_checkpoint.pt	Stage B — best checkpoint of the same model	Historical

Shipped model results (aemo_dt_sdp_jtsoc_fullcorpus.pt, rtg_mode="auto")

Surface	This model	PPO	Ratio
Standard Oct	$11,573/ep	$2,353	4.9×
Dispatch-matched	$35,320/ep	$22,530	1.57×
Expanded broad-2024	$34,761/ep	$19,504	1.78×
2025 OOD	$25,862/ep	$6,498	3.98×
Impact gate (merit-order)	+$63,064/cell vs PPO, 95% CI [+$33k, +$101k], win 9/9	—	2.5–3.1×

All six paired-difference bootstrap 95% CIs exclude zero; expanded broad-2024 additionally significant under paired Wilcoxon (p = 0.0002, n = 27). Methodology and full experiment ladder: report.md §8.2.10.

What is new vs the Stage B files in this repo

1. State-dependent RTG prompting (J_t(soc)). Stage B used a hand-tuned constant return-to-go prompt; Stage C prompts with the teacher's cost-to-go table (-J_t(s_t) — "how much value remains from this battery level"), recovering the energy-arbitrage gap on the broad surface ($12.0k → $34.8k on expanded broad-2024).
2. Combined conservative + aggressive teachers. Trained on the concatenated corpus (640 eps; deg=$50/MWh + deg=$20/MWh) instead of conservative-only, diversifying energy-arbitrage behaviour.
3. Wins all four identity surfaces (Stage B lost expanded broad-2024 to PPO), and ships with surface-aware rtg_mode="auto": j_t_soc prompting on price-taking surfaces, conservative constant-RTG fallback under market impact. Do not force j_t_soc under merit-order impact — it self-suppresses on 194–250 MWh batteries.

Architecture (both models share the modern v2 backbone)

Parameter	Value
Blocks / hidden dim	8 / 768
Attention	GQA (12 Q / 6 KV), QK-Norm
FFN	SwiGLU, dropout 0.15
Weight tying	Enabled
State / action dim	18 / 9 (full_fcas)
Action head	Mixed (Tanh energy, Sigmoid FCAS)

Action space: dim 0 energy dispatch in [-1, 1]; dims 1–8 FCAS bids in [0, 1] (RAISEREG, LOWERREG, RAISE6SEC, LOWER6SEC, RAISE60SEC, LOWER60SEC, RAISE5MIN, LOWER5MIN). State space: 18-dim normalized market observations (time features, RRP, demand, 8 FCAS prices, generation mix, SOC).

Training Data

• Stage C (shipped): dt_trajectories_jtsoc_combined.parquet — 640 episodes, 6.2M rows, conservative + aggressive SDP teachers concatenated, per-episode J_t(soc) cost-to-go RTG.
• Stage B (historical): dt_trajectories_full.parquet — 320 episodes, conservative teacher only.
• Both generated by scripts/generate_sdp_dt_trajectories.py in the source repo; method in report.md §4.3/§5.4.

Usage

Load with the DecisionTransformer class from the energydecision repo and run through AEMOAgent with rtg_mode="auto" (recommended):

import torch
from huggingface_hub import hf_hub_download
from decision_transformer import DecisionTransformer
from decision import AEMOAgent

import json
repo = "mrvictoru/energydecision-dt-v2-sdp"
ckpt = hf_hub_download(repo, "aemo_dt_sdp_jtsoc_fullcorpus.pt")
meta = json.loads(hf_hub_download(repo, "aemo_dt_sdp_jtsoc_fullcorpus.pt.meta.json"))

model_kwargs = dict(
    state_dim=meta["model"]["state_dim"], act_dim=meta["model"]["act_dim"],
    h_dim=meta["model"]["h_dim"], context_len=meta["model"]["context_len"],
    n_block=8, n_heads=12, n_kv_heads=6, drop_p=0.15,
    max_timestep=2016, rope_enabled=True, qk_norm=True, tie_weights=True,
    action_head_mode="mixed",
)
model = DecisionTransformer(**model_kwargs)
model.load_from_checkpoint(ckpt)           # raw state_dict
model.return_scale = meta["return_scale"]  # J_t(soc) prompt scaling (~25,988)
model.eval()

agent = AEMOAgent(env, algorithm="dt", model=model, rtg_mode="auto")

Intended Use

Research into offline RL / sequence modeling for energy markets; simulation of multi-market BESS dispatch in Australia's NEM; baseline for planner-distilled offline RL. Not intended for live trading without further validation (sim-to-real is open), risk management, and regulatory compliance.

Citation

@misc{energydecision-dt-v2-sdp,
  title = {AEMO SDP-Teacher Decision Transformer (planner-distilled)},
  author = {mrvictoru},
  year = {2026},
  url = {https://huggingface.co/mrvictoru/energydecision-dt-v2-sdp}
}

Related

• Dataset: mrvictoru/AEMO_simulated_trade
• Modern v2 (behaviour-cloning era, superseded): energydecision-dt-v2
• Impact-aware variant: energydecision-dt-v2-impact
• Source repo: mrvictoru/energydecision