resoajoe/loglens-longvideo-drift

Correction (2026-08-05): the conditioning anchor is a generated image, not a photograph — no EXIF, exactly 16:9 at the generated-keyframe resolution, with the text prompt that produced it stored alongside it. No result changes (the anchor is fixed either way), but "real photo" implied a sim-to-real grounding this work does not have. Nothing published here contains a real room, patient, or camera capture.

NEW (2026-08-19): the correction was the error

A stabiliser calibrated for footage whose camera wandered 61 px was pointed at footage whose camera wandered 0.331 px. It removed 28% of the image sharpness to correct a third of a pixel — and the blur gate downstream then discarded 6.7% of the footage as "blurred", footage whose raw sharpness was more than double the gate's threshold.

Warping to a sub-pixel offset resamples every pixel. Bilinear at half a pixel is the average of two neighbours — a two-tap box blur whose transfer is cos(pi*f/f_nyq): unity at DC, zero at Nyquist. Measured on three sources, including real camera footage, so this is a property of the resampler and not of generated video:

offset	nearest	bilinear	bicubic	Lanczos
0.25 px	100%	45.0%	79.3%	89.3%
0.50 px	100%	31.4%	63.2%	79.1%

The two metrics that appear to disagree about this (Laplacian 31%, broad spectral 79%) do not: the loss is concentrated at the top of the band — 98.4% of energy survives below quarter-Nyquist, 16.2% above Nyquist. Quote the band you mean.

The fix is one flag. On real warps: bilinear 72.1% sharpness kept, Lanczos 94.6%, both at zero residual drift; integer-snap 100% with 0.65 px max residual. Applied to the pipeline it came from, it recovered both falsely-rejected chunks and took a rebuild from 65% to 97% of raw sharpness.

A resampling correction costs a fixed amount of high-frequency detail, while its benefit scales with the error it removes. Before applying one, compare the two. And never let a quality gate sit downstream of a correction without checking that the correction is not what the gate is detecting.

Tool: warpcost/ — one file, numpy + opencv, four known-answer self-tests, points at your own frames. Write-up: RESULTS_substab.md · field note

---

how not to believe your own instruments

A field guide from measuring generative video on one edge device. Not a guide to doing research quickly — a guide to the narrower problem of not being fooled by measurements you built yourself, which is where nearly all our wasted effort went and nearly all the saved effort came from.

The record it rests on: As of 2026-08-19: ~18 of 37 sealed predictions wrong across 15 pre-registrations, and 16 instruments wrong before they were right. 1 published caveat that was backwards. 1 headline statistic computed and discarded. 5 pipelines that exited zero and wrote garbage. 9 hours of "the GPU is slow" that were our own leftover processes.

Five practices, ranked by what they actually saved:

1. Literature check before compute — one hour, every time. Ask "what is established, what is the landmine, what is the sharper question" rather than "is this taken".
2. Pre-registration with kill conditions — fired 3 times; one cancelled a four-arm study for ~4 GPU-days before it ran.
3. Known-answer tests — an instrument that cannot report zero on a zero-signal control should not be permitted to report anything. Ours aborts rather than emitting numbers.
4. Open the file — 5 pipelines passed every automated check on pure noise.
5. Check your own leftovers — a timed-out call does not kill the process it started.

Includes the literature-check prompt skeleton (our most portable artifact) and an unexpected staffing finding: the highest-leverage contributions were domain knowledge and measurement discipline, not modelling skill.

Full write-up: RESULTS_practice.md.

---

What a monitoring camera writes down is not what the light did

Two cameras, same room, 30 paired hours. One with exposure and white balance locked (physical ground truth), one on auto (deployed-realistic).

	locked (true light)	auto (what the camera wrote)
luminance dynamic range	1.89x	1.11x
warmth (R/B) dynamic range	1.64x	1.07x
gamma, A ~ L^gamma	1.0 by definition	-0.045
log luminance range removed	—	82.9%
log warmth range removed	—	~86%

The auto camera's response is inverted, not merely flattened — it writes 124.6 at night and 115.4 at midday, brighter when it is darker. And auto white balance destroys colour temperature even more thoroughly than auto exposure destroys luminance: a clean 1.068 -> 1.735 step at the lighting changeover becomes 1.008 -> 1.075, essentially nothing.

This also confirms an earlier finding against ground truth for the first time: an original 220-hour auto-exposure capture measured a 1.1x luminance range, and this pair measures 1.11x for auto while showing the underlying light actually varied by 1.89x.

The fitted response is a rendering layer: synthetic footage with physically correct illumination can be passed through it to produce what a deployed camera would actually have recorded. Without it, synthetic monitoring footage is photometrically correct and unlike any real camera's output.

Full write-up: RESULTS_exposure.md. Reproduce: code/exposure_tf.py.

---

ALSO NEW: every instrument I built this week was wrong at least once

14 of 31 sealed predictions wrong. 13 measurement instruments wrong before they were right, every one caught by a control rather than by inspection. 1 published caveat that turned out backwards. 1 tidy statistic computed and discarded as invalid. 5 pipelines that exited zero and wrote garbage. 9 hours of "the GPU is slow" that were my own leftover processes.

A metric that scored the visibly broken image higher. A harness whose headline signal was a false positive on an empty bed. An instrument invariant to a 64x change in its own subject. A known-answer test that appeared to fail and was right — it had detected an event left inside the control window.

Full write-up: RESULTS_methods.md.

---

Generated video has no stillness

source	quiescent fraction	longest continuous stillness
real fixed-camera (ordinary, barely occupied room), 50 min	0.992	1726 s (28.8 min)
real, second window	0.988	1043 s
real, third window	0.991	849 s
clinical prior, hospital inpatient	0.674	minutes
generated, frontier model clip (superseded — see correction below)	0.320	4 s
generated, ours	0.175	1 s
generated, ours (longer run)	0.104	3 s

Real fixed-camera footage holds perfectly still for 28.8 minutes.

CORRECTED 2026-08-10. This previously read "no generated footage measured here stays still longer than 4 seconds." That was true of the small-model output measured at the time and is false in general. Scoring frontier-model clips on the same protocol:

tier	longest continuous stillness
real bolted camera	1726 s (28.8 min)
frontier generation	16-20 s (two uncensored failures: 16.0 s, 20.0 s)
2B model, raw output	0-2 s

An 86x gap between reality and the frontier, and ~10x between the frontier and a small model. The test stratifies by model class rather than failing everything — which is more useful, since a test everything fails cannot rank anything.

Note on censoring: clips whose stillness equals their duration have not failed, they ran out of video, and must be reported as censored rather than scored.

Why generators fail this at all: they train on trimmed, action-centric clips. Stillness is not in the training distribution.

Pre-registered before measuring (PREREGISTRATION_eventrate.md), and two of four sealed predictions were wrong — generated quiescent fraction came in at 0.10-0.32 against my stated bound of <0.10, and the primary discriminator reaches only 3.1x against the frontier clip, which is my own pre-registered weak-support band, not confirmation. Not rounded up. The run-length result is what survives every framing.

The instrument was rebuilt after the first one proved void. v1's threshold was scale-invariant and pinned every source to ~86% "active"; exposure-normalising the real footage changed the mean frame difference by 64x and moved the reported quiescent fraction by 0.002. v2 uses an absolute threshold (>=12 grey levels) plus a contiguous-area gate (>=0.15% of frame), constants fixed across all sources, and aborts unless it first passes a known-answer test on a provably-static control (reports duty 0.0000, max blob 0 px).

Full write-up: RESULTS_eventrate.md. Reproduce: code/eventrate2.py.

Real-footage rows are aggregate statistics only — no frames displayed, copied or published, source described only as an ordinary barely occupied room. Published with the capture owner's explicit consent.

---

Latent autoencoders implement an undeclared chroma subsampling

Blur only an image's chroma and 80.6% of its colour detail is destroyed — SSIM reports 0.9998. Standard SSIM is computed on luminance, so it is structurally blind; PSNR under-weights chroma by ~11 dB at matched damage. An autoencoder selected against these metrics faces no penalty for discarding chroma.

Measuring the consequence with 6-phase-averaged sinusoidal gratings through encode/decode — largest period at which chroma is >50% destroyed while luma survives intact:

autoencoder	type	chroma destroyed up to
LTX-Video-2B	video AE	8px
SDXL-VAE	image f8	16px
SD-1.5	image f8	24px

Universal, not a quirk of one model. At a 16px period SD-1.5 passes luma at 1.57 and chroma at 0.05 — a 31× disparity at identical spatial frequency. Notably the image autoencoders are worse than the video one, which rules out temporal compression as the cause.

Two things the curves show that a single cutoff number would have hidden:

• These are not low-pass filters. Deep nulls sit at exactly 4px and 8px — periods dividing the f8 lattice — with ~0.93 response either side, and they survive phase averaging. The curves are non-monotonic, so an "MTF50 cutoff" is meaningless here. We computed one first, got tidy 2.57×/3.74× chroma penalties, and discarded them as artifacts of a statistic the data does not support.
• Response above 1.0 is real. A passive optical system cannot amplify contrast; SD-1.5 reaches 1.57 (luma) and 1.73 (chroma). These decoders do not merely lose detail, they synthesise it — latent autoencoders are editorial, not simply lossy.

Consequence: anything carried by small colour features — wristbands, colour-coded labels and tubing, indicator LEDs, status lights, triage tags — does not survive generation, while the metrics used to certify the generator call the reconstruction essentially perfect.

CORRECTION (isoluminant control run): our original grating carried residual luma (17.25 vs 0.13 for a properly isoluminant one). We stated this made the measurement conservative. It was backwards — true isoluminant chroma scores higher at 6-8px (0.42/0.47 vs 0.34/0.21), so the deficit there is smaller than first reported. The headline is unchanged: chroma still >50% destroyed up to an 8px period while luma passes, now confirmed on both red-green and blue-yellow isoluminant axes independently. Scope now closed: the control was re-run on all three autoencoders on both isoluminant axes and every published period is confirmed unchanged — LTX 8px, SDXL 16px, SD-1.5 24px.

Full write-up: chroma/RESULTS_chroma.md. Reproduce: code/vae_mtf.py, code/mtf_crossmodel.py. Raw numbers: data/.

Related: identity has a per-resolution ceiling before any generation happens

Encoding an image to LTX latent space and decoding it — no transformer, no diffusion, no drift — already costs face identity, governed by face pixels, not frame resolution:

face px	identity retained (ArcFace)
54	0.514
75	0.729
95	0.858
110	0.910

But the ceiling is slack: generating at each rung, the full pipeline delivers only −0.03 / 0.20 / 0.13 — nowhere near its ceiling and below the 0.35 same-person threshold. The transformer, not the autoencoder, dominates identity loss. Raising resolution buys almost nothing, so "budget ~100px of face" is necessary but far from sufficient.

Consistent with this, an independently produced 25-second clip from a current frontier video generator (n=1, not ours) with ~77–112px faces scored 0.987 identity at frame 0 and 0.721 by the end — identity present where the pixel budget allows it, and drifting from there.

---

Long-horizon video generation from outside the model

Many leading approaches to long-horizon stabilisation assume access to model internals, custom inference pipelines, or additional training — KV-cache surgery (MAG), temporal attention modification (TiARA), trained memory components. Most practitioners only get black-box image conditioning. This is a measured study of what's actually achievable in that regime, run end-to-end on a single Jetson AGX Orin.

Seven runs, ~36,500 generated frames, four conditioning strategies, one metric harness.

Headline: anchoring frequency is a dial between two failure modes, not a fix for either

strategy	n	drift (end)	drift slope	jitter	spurious/1k
none — re-anchor every chunk	1200	0.092	0.0008	0.017	136
prev — never re-anchor	1200	0.279	0.0111	0.008	46
sink — prev + fixed seed anchor	1200	0.285	0.0011	0.022	222
log (naive) — 13 stacked anchors	1200	0.496	0.0100	0.026	86
tiered — re-anchor every 12	245*	0.205	0.0074	0.030	234

*shorter run; drift accumulates with length, so this row is not comparable on drift. Jitter and spurious/1k are length-normalised and are comparable. A matched-length run is in progress — we are not claiming a win on the unfair axis.

Re-anchor often → low drift, visible seams. Never → smooth but progressively wanders. You choose an operating point; you don't escape the tradeoff.

Four reproducible failure modes — failure is channel-dependent

1. Desaturation / blur (prev) — colour drains to grey-cyan; by chunk 140 the person in the scene has disappeared entirely.
2. Posterization (sink, tiered) — flat saturated colour, hard outlines; the scene becomes an illustration. Structure survives, photorealism doesn't.
3. Catastrophic collapse (log naive) — pure saturated green by ~chunk 40.
4. Jump-cutting (none) — no drift, but every chunk boundary is a cut.

A single scalar metric scores (1) and (4) similarly and misses that one is usable.

The over-conditioning collapse (a gotcha worth knowing)

Stacking N image anchors at the same frame_index with Σstrength = 2.53 drives the latents out of range and collapses to a flat colour field. It degrades with more history — i.e. worse precisely as the method does more of what it exists to do. Normalise Σstrength ≤ 1.0; prefer distributing anchors across frame indices.

Tiered conditioning: match each fact's persistence to a channel's drift-susceptibility

• INVARIANT ("static camera, no pan, bed centre-right, window left") → text prompt. Zero cost and structurally cannot drift. This worked — layout and camera stayed fixed.
• SLOW (appearance) → seed image, periodically re-anchored.
• FAST (motion) → previous frame.

At matched depth, tiered kept every object sharp and identifiable while naive chaining was already smearing faces. But text invariants hold semantic facts and not stylistic ones — output posterized despite "posterized, flat colours, cartoon, illustration" being explicitly in the negative prompt. Text carries what is true; images carry how it looks.

Practical recipe

LTX-Video 2B @ 704×448, 30 steps, CFG 3.0, seeded from a fixed GENERATED anchor image of the room (not a photograph); invariants in the prompt; Σstrength ≤ 1; hard re-anchor every 4–12 chunks depending on whether you prefer seams or wander. ~68 s per 49-frame chunk (≈2 s of video) on an Orin. For long footage, generate many independent 30–60 s segments from one seed rather than one continuous take — embarrassingly parallel and drift-bounded by construction.

Negative results (published rather than buried)

• 13B distilled loads, runs, and outputs noise. Distilled variants need their own timestep schedule, not linear_quadratic → sigmas.
• Disabling use_dynamic_shifting to silence a missing-mu error produces noise while appearing to succeed — correct exit code, plausible timings, files on disk.
• Negative prompts do not prevent progressive style drift.

Method note that cost the most

Five separate pipelines exited zero, wrote files, and produced garbage. Exit codes, timings, file counts, and even image mean/std all passed on pure noise. The only reliable detector was opening a JPEG. If you build an automated generation pipeline, put a human-visible sample check in the loop.

What's here

FINDINGS.md              full write-up
REGISTRY.md              every finding with its score: REPLICATED / MEASURED / CORRECTED / RETRACTED
code/gen_tiered.py       tiered conditioning + periodic re-anchoring
code/gen_long.py         four-arm comparison harness (none/prev/sink/log)
code/drift_metric.py     CLIP-based drift, jitter, spurious-event metrics
results/                 the numbers behind the table

Shipped tools — each a single file, dependency-light, and self-validating. Run the self-test first; an instrument that cannot report a known answer should not be trusted with an unknown one.

tripod/      is this video as still as a real camera?          python3 tripod.py <clip>
warpcost/    what is your stabilisation warp costing you?      python3 warpcost.py --selftest
vae-mtf/     what a latent autoencoder does to colour detail   python3 vae_mtf.py --selftest
nullcheck/   is your metric lying to you?                      python3 nullcheck.py --selftest

Not included

Seed images and generated frames are withheld: they derive from proprietary character and set assets. The code runs on any seed photo — point --anchor-image at your own. No real patients, real footage, or personal data are involved anywhere in this work; the scenes are entirely synthetic.

Limitations

One model family (LTX-Video), one scene, one resolution, single seeds per arm, and the tiered row is not length-matched. The metric is CLIP-embedding based and inherits CLIP's biases. Treat the ordering and the failure taxonomy as the contribution, not the absolute numbers.

---

⚠ This card predates our model-card standard

It does not report: a baseline comparison.

Every model we publish should state (1) the cheap baseline and the margin over it, (2) at least one named failure mode with a measured rate, (3) what the model refuses and why, and (4) label provenance. This card was written before that rule and has not been retrofitted, because the measurements needed no longer exist. Treat its numbers with more caution than our newer cards, and prefer a model that meets the standard where one exists.