EnergyDecision-DT: Decision Transformer for AEMO FCAS Battery Trading

Model Description

EnergyDecision-DT is a Decision Transformer model trained on simulated battery dispatch data from the Australian Energy Market Operator (AEMO) Frequency Control Ancillary Services (FCAS) market. It models optimal battery dispatch as a sequence prediction problem, conditioning on returns-to-go, observed states, and past actions to predict the next action.

The model learns to dispatch battery energy storage (charge/discharge) and bid into 8 FCAS contingency markets simultaneously, making it a 9-dimensional action space controller.

See https://github.com/mrvictoru/energydecision

Key Features

• Action Space (9-dim):
  • Dim 0: Energy dispatch in $$[-1, 1]$$ (charge/discharge)
  • Dims 1-8: FCAS contingency bids in $$[0, 1]$$
• State Space (18-dim): Normalized market observations including prices, demand, renewables penetration, and battery state-of-charge
• Context Length: 180 timesteps (looks back ~15 hours of history)
• Training Approach: Offline reinforcement learning via return-conditioned sequence modeling

Intended Use

This model is intended for:

• Research into offline RL for energy markets
• Simulation of battery trading strategies in the AEMO FCAS market
• Baseline for comparing decision transformer approaches against traditional RL

It is not intended for live trading without further validation, risk management, and regulatory compliance.

Training Data

• Source: AEMO simulated trade dataset
• Size: 75,945,600 rows across 2,405 episodes
• Format: Parquet file with 5 columns

Dataset Schema

Column	Type	Description
episode_id	str	Unique episode identifier (e.g. nsw1_2021_2023_a2c_long_large_ep000 — encodes region, date range, RL algorithm, and episode number)
step	int64	Timestep within the episode (0-indexed, 5-minute intervals)
norm_observation	list[f64] (18-dim)	Normalized market observations including prices, demand, renewables penetration, and battery state-of-charge
action	list[f64] (9-dim)	Battery dispatch action: energy charge/discharge (dim 0) + 8 FCAS contingency bids (dims 1-8)
reward	float64	Scalar reward from the simulated market interaction

Observation Space (18-dim)

The normalized observation vector includes:

• Market prices and demand metrics
• Renewables penetration indicators
• Battery state-of-charge (SoC)
• Time-of-day and seasonal features
• Historical bid-ask spreads and clearance rates

Episode Structure

Each episode represents a simulated battery trading trajectory over a continuous period, with actions taken at 5-minute dispatch intervals. Episodes were generated using an A2C RL agent interacting with a market simulator calibrated to AEMO NEM data (NSW region, 2021-2023).

Preprocessing

• Observations normalized to zero mean, unit variance per feature
• Returns-to-go computed with discount factor $$\gamma = 0.95$$
• Rewards used as-is from the simulator
• No additional augmentation or filtering applied

Model Architecture

DecisionTransformer(
  (embed_return): Linear(1 -> 384)
  (embed_state): Linear(18 -> 384)
  (embed_action): Linear(9 -> 384)
  (embed_timestep): Embedding(100000 -> 384)
  (blocks): 8x TransformerBlock(
      (ln1): RMSNorm(384)
      (attn): MultiheadAttention(384, 8 heads)
      (ln2): RMSNorm(384)
      (ffn): SwiGLU(384 -> 1536 -> 384)
      (dropout): Dropout(p=0.15)
    )
  (ln_f): RMSNorm(384)
  (predict_action): Linear(384 -> 384) -> GELU -> Linear(384 -> 9) -> Tanh
  (predict_state): Linear(384 -> 18)
  (predict_return): Linear(384 -> 1)
)

Hyperparameters

Parameter	Value
Blocks	8
Hidden dim	384
Attention heads	8
Context length	180
Dropout	0.15
State dim	18
Action dim	9
Discount factor	0.95
Return scale	2.0
Action loss weight	0.999
State loss weight	0.002
Return loss weight	0.0001

Training Procedure

• Hardware: NVIDIA RTX PRO 6000 Blackwell (102 GB VRAM)
• Optimizer: AdamW (lr=3e-5, weight_decay=1e-4)
• Batch size: 128
• Epochs: 3
• Mixed precision: AMP (Automatic Mixed Precision) with GradScaler
• Gradient clipping: 1.0
• Strategy: Overlapping context windows with stride = context_len / 2

⏳ Training stats (duration, steps, loss progression) will be updated after retraining with the new hyperparameters.

Usage

Loading the Model

import torch
from huggingface_hub import hf_hub_download

# Download model weights
model_path = hf_hub_download(
    repo_id="mrvictoru/energydecision-dt",
    filename="aemo_dt_fcas_model.pt",
)

# Load checkpoint
checkpoint = torch.load(model_path, map_location="cpu")

# If it's a wrapped dict (from upload script):
if "model_state_dict" in checkpoint:
    state_dict = checkpoint["model_state_dict"]
    config = checkpoint.get("config", dict())
    best_val_loss = checkpoint.get("val_loss", float("inf"))
else:
    # If it's a raw state dict (from best_model.pt)
    state_dict = checkpoint

Creating the Model

from model import DecisionTransformer

model = DecisionTransformer(
    state_dim=18,
    act_dim=9,
    n_block=8,
    h_dim=384,
    context_len=180,
    n_heads=8,
    drop_p=0.15,
    max_timestep=100000,
)
model.load_state_dict(state_dict)
model.eval()

Inference

# Prepare inputs (batch_size, context_length, dim)
states = torch.randn(1, 180, 18)
actions = torch.zeros(1, 180, 9)  # Past actions (can be zero-padded)
returns_to_go = torch.full((1, 180), 10.0)  # Target return
timesteps = torch.arange(180).unsqueeze(0)

# Get predicted action for the last timestep
with torch.no_grad():
    predicted_action = model.get_action(
        states, actions, returns_to_go, timesteps
    )
    # predicted_action shape: (1, 9)
    # Dim 0: energy dispatch in [-1, 1]
    # Dims 1-8: FCAS bids in [0, 1]

Files

File	Description
aemo_dt_fcas_model.pt	Full checkpoint with config (~230 MB)
aemo_dt_fcas_best_checkpoint.pt	Best model weights only (~230 MB)

Citation

If you use this model, please cite:

@misc{energydecision-dt,
  author = {Victor U},
  title = {EnergyDecision-DT: Decision Transformer for AEMO FCAS Battery Trading},
  year = {2026},
  publisher = {HuggingFace},
  howpublished = {\\url{https://huggingface.co/mrvictoru/energydecision-dt}},
}