Initial release of MiniMax H3 Semantic Bridge v1.0

.gitattributes

@@ -33,3 +33,7 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+examples/01_rooftop_train_chase/native_h3.mp4 filter=lfs diff=lfs merge=lfs -text
+examples/01_rooftop_train_chase/semantic_bridge_alpha_0.15.mp4 filter=lfs diff=lfs merge=lfs -text
+examples/02_glass_table_prompt_adherence/native_h3.mp4 filter=lfs diff=lfs merge=lfs -text
+examples/02_glass_table_prompt_adherence/semantic_bridge_alpha_0.15.mp4 filter=lfs diff=lfs merge=lfs -text

LICENSE.md

@@ -0,0 +1,27 @@
+# Licensing
+
+This repository contains several kinds of artifacts with different upstream relationships.
+
+## Model-derived artifacts
+
+The Semantic Bridge adapter and teacher-side bridge research artifact were developed using MiniMax H3 representations and should be treated in accordance with the upstream **MiniMax H3 Community License Agreement**:
+
+https://huggingface.co/MiniMaxAI/MiniMax-H3/blob/main/LICENSE
+
+The repository metadata therefore uses `license: other` with the MiniMax H3 Community License Agreement as the linked model license rather than describing the model-derived artifacts as MIT or Apache-2.0.
+
+The required MiniMax distribution notice is included in `NOTICE.txt`.
+
+## SenseNova research teacher
+
+SenseNova U1.5 was used during the teacher-assisted research stage. The upstream SenseNova U1.5 model is released under Apache License 2.0:
+
+https://huggingface.co/sensenova/SenseNova-U1.5-8B-MoT
+
+The SenseNova checkpoint itself is not redistributed here.
+
+## Research code and documentation
+
+The repository contains original experimental scripts, reports, documentation, and a ComfyUI integration. No separate permissive license is asserted here over model-derived artifacts. Before reusing or redistributing the package, review the upstream licenses and any terms applicable to your use and territory.
+
+This file is a project summary, not legal advice.

MiniMaxH3_SemanticBridge_v1.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ac0dc8ac05f545ebdee12e2fcebe4515b049f9cfd9558eb4887a9bf3fd6d562e
+size 11023032

MiniMax_H3_Semantic_Bridge_v1.0.zip

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c4866a0cc554237046814c9cb827a5960caecbd2214d5a5a04b0e43716b437bb
+size 4406

NOTICE.txt

@@ -0,0 +1,5 @@
+MiniMax H3 is licensed under the MiniMax H3 Community License Agreement, Copyright © 2026 MiniMax. All Rights Reserved.
+
+MiniMax H3 Semantic Bridge is an independent experimental research project and is not an official MiniMax release.
+
+SenseNova U1.5 was used as an experimental teacher during development. The public Semantic Bridge does not require or redistribute the SenseNova checkpoint.

README.md

@@ -2,4 +2,439 @@
 license: other
 license_name: minimax-h3-community-license-agreement
 license_link: https://huggingface.co/MiniMaxAI/MiniMax-H3/blob/main/LICENSE
+base_model:
+- MiniMaxAI/MiniMax-H3
+tags:
+- minimax-h3
+- comfyui
+- video-generation
+- semantic-adapter
+- representation-learning
+- distillation
+- research
 ---
+
+# MiniMax H3 Semantic Bridge
+
+## Cross-Architecture Semantic Transfer and Distillation for Video Generation
+
+**MiniMax H3 Semantic Bridge** is a compact conditioning-space adapter for the standard **MiniMax H3 FL2VA / text-conditioned generation path**.
+
+It grew out of an experimental cross-architecture representation-transfer project using **SenseNova U1.5** as a semantic teacher. The final released adapter is standalone: **SenseNova is not required at inference time**.
+
+**In one line:**
+
+> cross-architecture semantic transfer → conditioning-space teacher bridge → distillation → a ~11 MB standalone H3 adapter
+
+This is **not** a LoRA, checkpoint merge, or conventional parameter graft. The adapter transforms native H3 conditioning before the video transformer and blends the learned semantic representation back into H3 at a controllable strength.
+
+> **Scope:** v1 is for standard H3 FL2VA / text-conditioned generation. **Ref2VA / reference-conditioned workflows are not supported.** Experimental reference-audio testing showed degraded singing/lip-sync when the adapter was inserted into Ref2VA conditioning.
+
+---
+
+## Quick Start
+
+### Files
+
+- `MiniMaxH3_SemanticBridge_v1.safetensors` — final standalone adapter.
+- `MiniMax_H3_Semantic_Bridge_v1.0.zip` — ComfyUI custom node.
+- `RESEARCH_ARTICLE.md` — full research narrative.
+- `research/` — prompt datasets, raw scripts, reports, and teacher-side research artifact.
+- `examples/` — controlled Native H3 vs Semantic Bridge A/B videos and their exact prompts.
+
+### Installation
+
+1. Extract `MiniMax_H3_Semantic_Bridge_v1.0.zip` into:
+
+```text
+ComfyUI/custom_nodes/
+```
+
+2. Create:
+
+```text
+ComfyUI/models/semantic_bridge/
+```
+
+3. Put:
+
+```text
+MiniMaxH3_SemanticBridge_v1.safetensors
+```
+
+inside that folder.
+
+4. Restart ComfyUI.
+
+### Nodes
+
+- **MiniMax H3 Image to Video + Semantic Bridge**
+- **MiniMax H3 Semantic Bridge**
+- **MiniMax H3 Clear Semantic Bridge Cache**
+
+### Recommended settings
+
+```text
+alpha = 0.10
+magnitude_match = per_token
+```
+
+For the qualitative A/B examples below, `alpha = 0.15` was intentionally used to make the behavioral difference easier to observe.
+
+---
+
+# What problem was this trying to solve?
+
+Generative models can recognize all the concepts in a prompt while still failing to preserve the relationships between those concepts.
+
+For example, a prompt may specify not only a person, table, bottle, mirror, and light source, but also:
+
+- which hand holds which object;
+- which hand must remain still;
+- left/right ordering of several objects;
+- which surfaces are transparent or reflective;
+- how a mirror should correspond to the real scene;
+- how light passes through one material but reflects from another;
+- whether an action is explicitly requested or explicitly *not* requested.
+
+The project therefore focused on **semantic structure and prompt adherence**, rather than adding new visual concepts to H3.
+
+Areas explored during the research included:
+
+- complex composition;
+- spatial relationships;
+- anatomy and body relationships;
+- object counting;
+- text rendering / textual constraints;
+- materials and lighting;
+- reflections, transparency, and occlusion;
+- long prompts with several simultaneous constraints.
+
+---
+
+# Research path
+
+## 1. Direct grafting failed
+
+The project started as a direct grafting experiment between SenseNova U1.5 and MiniMax H3.
+
+The architectures did not expose useful parameter-level correspondences. Exact shape matching, transpose matching, and simple input/output dimensional matching did not provide a meaningful path for direct tensor transplantation.
+
+That negative result changed the question from:
+
+```text
+Which weights can be copied?
+```
+
+to:
+
+```text
+Can the models' internal representations of the same prompt be aligned?
+```
+
+## 2. Hidden-representation alignment
+
+Hidden states from both systems were extracted across a deliberately varied semantic prompt set. Lightweight projections were trained between candidate representation spaces.
+
+A substantially stronger correspondence emerged than the alternatives.
+
+On held-out prompts from the original distribution, the strongest experimental mapping reached approximately:
+
+```text
+validation cosine ≈ 0.904
+```
+
+## 3. Strict OOD test
+
+A separate set of **160 prompts** was constructed to stress harder combinations of anatomy, counting, materials/light, spatial structure, text, architecture/vehicles, reflection/occlusion, and long compositions.
+
+With the bridge frozen, strict OOD similarity was approximately:
+
+```text
+0.749
+```
+
+The drop was real, but the mapping did not collapse. This motivated testing the representation inside the actual H3 generation path.
+
+## 4. Full teacher bridge
+
+An experimental Full Bridge used SenseNova at inference time, projected the teacher-side representation into H3-compatible conditioning, magnitude-aligned it, and blended it with native H3 conditioning.
+
+Conceptually:
+
+```text
+H = native H3 conditioning
+S = mapped teacher semantic representation
+C = H + alpha * (S - H)
+```
+
+The Full Bridge produced coherent H3 generations and visible behavioral changes, demonstrating that the cross-model representation mapping survived the downstream video-generation process.
+
+However, it required the full teacher model at runtime, which was impractical.
+
+## 5. Distillation
+
+The Full Bridge was then treated as a teacher. A compact H3-side student was trained to predict the teacher-derived representation directly from H3's own conditioning.
+
+Early attempts to predict the correction delta directly were weak (best correction similarity around `0.51`). Predicting the teacher-derived representation itself worked dramatically better.
+
+The final result was a small standalone adapter with no SenseNova runtime dependency.
+
+---
+
+# Final distillation results
+
+The preserved V3 report records a **500-prompt training split** and **100-prompt validation split** containing both original-distribution and harder/OOD examples.
+
+| Metric | Result |
+|---|---:|
+| Teacher representation cosine | **0.995890** |
+| Semantic correction cosine | **0.983558** |
+| Main-distribution correction | **0.980888** |
+| OOD correction | **0.989788** |
+| Minimum correction | **0.935910** |
+| Blend cosine, alpha 0.10 | **0.999958** |
+| Blend cosine, alpha 0.20 | **0.999827** |
+| Blend cosine, alpha 0.30 | **0.999602** |
+
+These are **representation-space / distillation metrics**. They do **not** mean that video quality improves by the same percentages, and they are not a substitute for controlled visual evaluation.
+
+### Final correction similarity by semantic category
+
+| Category | Similarity |
+|---|---:|
+| Text | 0.996061 |
+| Reflection / occlusion | 0.995932 |
+| Complex counting | 0.994557 |
+| Complex text | 0.991912 |
+| Reflection / occlusion OOD | 0.991855 |
+| Complex spatial | 0.991125 |
+| Long composition | 0.990637 |
+| Architecture / vehicle | 0.990606 |
+| Complex material / light | 0.987394 |
+| Material | 0.986263 |
+| Complex anatomy | 0.982104 |
+| Anatomy | 0.980773 |
+| Lighting | 0.978327 |
+| Spatial | 0.974002 |
+| Counting | 0.965250 |
+
+---
+
+# Qualitative A/B examples
+
+The following examples use the **same prompt and generation setup within each pair**. The intended comparison is Native H3 versus H3 with Semantic Bridge enabled. The published Bridge examples use `alpha = 0.15` to make the effect easier to inspect visually.
+
+These examples are **qualitative observations**, not a benchmark or proof of universal improvement.
+
+## Example 01 — Rooftop Train Chase
+
+**Focus:** complex motion, anatomy, action sequencing, physical interaction, material response, and spatial continuity.
+
+| Native MiniMax H3 | + Semantic Bridge (`alpha=0.15`) |
+|---|---|
+| [View native video](./examples/01_rooftop_train_chase/native_h3.mp4) | [View bridge video](./examples/01_rooftop_train_chase/semantic_bridge_alpha_0.15.mp4) |
+
+<video controls width="49%" src="https://huggingface.co/speach1sdef178/MiniMax-H3-Semantic-Bridge/resolve/main/examples/01_rooftop_train_chase/native_h3.mp4"></video>
+<video controls width="49%" src="https://huggingface.co/speach1sdef178/MiniMax-H3-Semantic-Bridge/resolve/main/examples/01_rooftop_train_chase/semantic_bridge_alpha_0.15.mp4"></video>
+
+The exact prompt is preserved in:
+
+[`examples/01_rooftop_train_chase/prompt.txt`](./examples/01_rooftop_train_chase/prompt.txt)
+
+This example was designed to stress several constraints simultaneously: two moving characters, pursuit distance, running anatomy, a specific vault interaction, hand contact with the obstacle, landing continuity, moving camera geometry, wet reflective metal, rain, sparks, and a rapidly moving city background.
+
+---
+
+## Example 02 — Prompt Adherence, Materials, Reflection & Transparency
+
+**Focus:** explicit state adherence, hand behavior, material differences, object ordering, reflection, transparency, and text.
+
+| Native MiniMax H3 | + Semantic Bridge (`alpha=0.15`) |
+|---|---|
+| [View native video](./examples/02_glass_table_prompt_adherence/native_h3.mp4) | [View bridge video](./examples/02_glass_table_prompt_adherence/semantic_bridge_alpha_0.15.mp4) |
+
+<video controls width="49%" src="https://huggingface.co/speach1sdef178/MiniMax-H3-Semantic-Bridge/resolve/main/examples/02_glass_table_prompt_adherence/native_h3.mp4"></video>
+<video controls width="49%" src="https://huggingface.co/speach1sdef178/MiniMax-H3-Semantic-Bridge/resolve/main/examples/02_glass_table_prompt_adherence/semantic_bridge_alpha_0.15.mp4"></video>
+
+### Controlled prompt-following observation
+
+A particularly useful instruction in this prompt is:
+
+> **"Her right hand rests flat on the glass tabletop with all five fingers naturally separated and clearly visible."**
+
+In this A/B generation:
+
+- **Native H3** introduces an unrequested action: the right hand moves across the tabletop rather than remaining in the requested resting state.
+- **Semantic Bridge (`alpha=0.15`)** keeps the hand resting on the glass surface, more closely preserving the explicitly requested state.
+
+This observation is important because it is not a subjective claim that one result is simply "prettier." The prompt specifies a directly observable state — **resting** — and the two outputs behave differently with respect to that instruction.
+
+It is still presented only as **qualitative evidence from this controlled pair**, not as a statistical claim that the adapter universally improves prompt adherence.
+
+The exact full prompt is preserved in:
+
+[`examples/02_glass_table_prompt_adherence/prompt.txt`](./examples/02_glass_table_prompt_adherence/prompt.txt)
+
+The same prompt also stresses:
+
+- anatomically coherent hands;
+- a cup held specifically in the left hand;
+- exactly three tabletop objects in a specified left-to-right order;
+- the text `NIGHT SHIFT`;
+- mirror correspondence;
+- transparent glass;
+- refractive bottle behavior;
+- reflective metal;
+- matte ceramic;
+- warm/cool directional lighting interactions.
+
+---
+
+# What Semantic Bridge is — and is not
+
+Semantic Bridge is best described as:
+
+> **a compact conditioning-space adapter distilled from a cross-architecture semantic mapping**
+
+It is **not**:
+
+- a conventional LoRA;
+- a checkpoint merge;
+- a direct parameter graft;
+- a copy of SenseNova weights;
+- a second multimodal model running beside H3.
+
+At inference time the released adapter operates only on H3 conditioning.
+
+Conceptually:
+
+```text
+Prompt
+  ↓
+H3 text conditioning
+  ↓
+Semantic Bridge
+  ↓
+learned semantic representation
+  ↓
+magnitude matching
+  ↓
+controlled residual blend
+  ↓
+MiniMax H3 video generation
+```
+
+SenseNova was used as a teacher during the research process only.
+
+---
+
+# Scope: FL2VA / standard text-conditioned H3 only
+
+This limitation is important.
+
+The released student was distilled from the **standard H3 conditioning path**. It was not trained on the separate multimodal reference-conditioning distribution used by Ref2VA.
+
+An experimental Ref2VA-compatible node was tested by modifying text-designated token positions while preserving visual-reference tokens. In reference-audio singing tests, this produced noticeably worse vocal articulation and stronger mumbling-like lip motion than native Ref2VA.
+
+The practical conclusion for v1 is therefore:
+
+> **Do not use this adapter for Ref2VA / reference-conditioned generation, especially reference-audio singing or lip-sync.**
+
+This does not establish that a semantic bridge can never work with Ref2VA. It suggests that a Ref2VA version should be trained separately on the multimodal conditioning regime it is intended to modify.
+
+A useful lesson from the failed experiment is that **matching tensor dimensionality does not guarantee matching conditioning semantics**.
+
+---
+
+# Research materials included
+
+The repository intentionally includes more than the final adapter so others can inspect the experimental path.
+
+## `research/datasets/`
+
+- `bridge_prompts_480.json` — historical filename; the preserved dataset contains **440** development prompts.
+- `bridge_ood_prompts_160.json` — 160 strict OOD prompts.
+
+## `research/raw_scripts/`
+
+Original research scripts are preserved largely as-run. They include local Windows paths and historical filenames. This is intentional: they are provided as a research snapshot rather than as a polished one-command training framework.
+
+The scripts cover areas such as:
+
+- architecture and tokenizer comparison;
+- H3 and SenseNova hidden-state extraction;
+- layer-pair screening;
+- MoT diagnostic experiments;
+- OOD dataset construction;
+- full bridge evaluation;
+- distillation screening;
+- V2/V3 student training and evaluation.
+
+## `research/reports/`
+
+Raw TXT/CSV outputs from the experiments are included, including negative and intermediate results.
+
+This is deliberate. The unsuccessful directions are part of the research record and may help others avoid repeating the same experiments.
+
+## `research/teacher_artifacts/`
+
+`SN_L32_to_H3_L49_rank128.safetensors` is preserved as an optional research artifact from the teacher-side bridge work.
+
+**It is not required to use the public Semantic Bridge.**
+
+Large extracted hidden-state `.pt` caches are not included. The prompt datasets and extraction scripts are provided so those intermediates can be regenerated.
+
+---
+
+# Limitations
+
+- This is an experimental research adapter, not a universal H3 enhancer.
+- It may help some prompts, do little on others, or occasionally make a result worse.
+- The reported cosine metrics evaluate agreement with the teacher-derived representation, not perceptual video quality.
+- A small conditioning difference can produce a large downstream sampling difference.
+- The current adapter is not validated for Ref2VA/reference-conditioned generation.
+- The project has not yet been evaluated with a large standardized human-preference benchmark.
+- The published A/B videos are illustrative controlled examples, not statistical proof.
+
+---
+
+# Licensing and upstream terms
+
+**Please read this section before using or redistributing the model-derived artifacts.**
+
+MiniMax H3 is released under the **MiniMax H3 Community License Agreement**. The official agreement defines terms for MiniMax H3 and Model Derivatives, including distribution requirements and territorial restrictions. This repository uses `license: other` metadata and points directly to the upstream H3 agreement rather than relabeling the model-derived adapter as Apache/MIT.
+
+Official H3 license:
+
+https://huggingface.co/MiniMaxAI/MiniMax-H3/blob/main/LICENSE
+
+The H3 agreement requires distributions to include a NOTICE. This repository includes `NOTICE.txt`.
+
+SenseNova U1.5, used as the experimental teacher during development, is published under **Apache License 2.0**:
+
+https://huggingface.co/sensenova/SenseNova-U1.5-8B-MoT
+
+This repository does **not** redistribute the SenseNova checkpoint.
+
+See [`LICENSE.md`](./LICENSE.md) and [`UPSTREAM_LICENSES.md`](./UPSTREAM_LICENSES.md) for repository-specific notes and direct upstream references.
+
+> This licensing summary is provided for transparency and is not legal advice. Users and redistributors should review the upstream terms themselves.
+
+---
+
+# Full article
+
+For the complete chronological research write-up, including failed grafting, representation screening, Full Bridge, OOD testing, early distillation, V2/V3 results, and the Ref2VA limitation, see:
+
+**[RESEARCH_ARTICLE.md](./RESEARCH_ARTICLE.md)**
+
+---
+
+# Acknowledgements
+
+This is an independent experimental project built around **MiniMax H3** and teacher-assisted representation studies using **SenseNova U1.5**.
+
+It is not an official MiniMax or SenseNova release.
+
+The value of the project is not only the final adapter, but the possibility that useful semantic behavior may sometimes be transferred between incompatible architectures through **representation alignment and distillation**, even when direct parameter grafting is not meaningful.

RESEARCH_ARTICLE.md

@@ -0,0 +1,1017 @@
+# MiniMax H3 Semantic Bridge
+
+## Cross-Architecture Semantic Transfer and Distillation for Video Generation
+
+### Abstract
+
+**MiniMax H3 Semantic Bridge** is the result of an experimental investigation into whether higher-level semantic behavior can be transferred between fundamentally different generative model architectures without directly merging their weights.
+
+The project began with an attempt to transfer selected capabilities from **SenseNova U1.5** into **MiniMax H3**, particularly in areas such as complex composition, spatial relationships, human anatomy, counting, prompt following, text interpretation, materials, lighting, reflections, transparency, and occlusion.
+
+Direct parameter grafting proved unsuitable because the two models do not expose meaningfully compatible weight structures. This led to a different approach: instead of transferring parameters, I investigated whether the models' internal representations of the **same textual prompt** contained a learnable cross-model relationship.
+
+They did.
+
+An experimental **Full Bridge** was developed in which SenseNova acted as a semantic donor at inference time and its representation was transformed into H3-compatible conditioning. The resulting system produced meaningful changes in actual H3 generations and demonstrated that semantic information could be transferred without modifying H3's generative transformer.
+
+However, the Full Bridge required the complete donor model during inference.
+
+The final stage therefore treated this system as a teacher and distilled its behavior into a compact adapter operating entirely from H3's own representation.
+
+The resulting **MiniMax H3 Semantic Bridge** requires no SenseNova checkpoint, runtime, tokenizer, or additional large model during inference.
+
+Only this final distilled version is included in the public release.
+
+---
+
+# 1. Motivation
+
+Large generative models can possess all of the concepts necessary to satisfy a prompt while still failing to preserve the relationships between those concepts.
+
+A simple prompt such as:
+
+> *A woman holding a glass bottle.*
+
+primarily requires concept recognition.
+
+A more structured instruction is different:
+
+> *A woman stands behind a transparent glass table holding a polished metal cup in her left hand. A mirror behind her reflects the room, while warm side light passes through a glass bottle on the table.*
+
+Now the model must maintain several relationships simultaneously:
+
+```text
+woman
+ └─ left hand
+      └─ polished metal cup
+
+glass table
+ └─ glass bottle
+      └─ warm light passes through it
+
+mirror
+ └─ reflects the room
+```
+
+All of the individual concepts may already be well represented by the model.
+
+The difficult part is their **composition**.
+
+This distinction motivated the project.
+
+Rather than attempting to add new visual knowledge to MiniMax H3, I wanted to investigate whether another model's semantic interpretation could alter how H3 represents complex instructions before generation begins.
+
+The original areas of interest were:
+
+* spatial and compositional reasoning;
+* human anatomy and body relationships;
+* object counting;
+* prompt following;
+* text interpretation;
+* material properties;
+* lighting relationships;
+* reflection and transparency;
+* occlusion;
+* long prompts containing multiple simultaneous constraints.
+
+The question was therefore not simply:
+
+> Can another model be merged into H3?
+
+It was:
+
+> **Can useful semantic structure learned by another architecture be translated into H3's representation space?**
+
+---
+
+# 2. The First Approach: Direct Model Grafting
+
+The project initially began as a model-grafting experiment.
+
+The intuitive approach was to identify semantically relevant components in SenseNova and transfer them directly into MiniMax H3.
+
+Architectural analysis quickly showed why this was unlikely to work.
+
+The two models have substantially different internal structures, dimensionalities and parameter organization. Examination of the major two-dimensional weight matrices did not reveal useful direct correspondences suitable for conventional grafting.
+
+There were no meaningful exact shape matches in the parts investigated, nor did simple transposition or matching input/output dimensions provide a useful mapping.
+
+This is an important negative result.
+
+Two models can potentially encode related semantic knowledge without storing that knowledge in parameter matrices that are directly interchangeable.
+
+A parameter-level question was therefore replaced by a representation-level question.
+
+Instead of looking for:
+
+```text
+SenseNova weights
+       ↓
+matching H3 weights
+```
+
+I began looking for:
+
+```text
+same prompt
+   ↓                 ↓
+SenseNova           H3
+   ↓                 ↓
+internal representation
+        ↓
+learnable relationship?
+```
+
+This turned out to be much more productive.
+
+---
+
+# 3. Searching for Cross-Model Representation Alignment
+
+Both models were presented with the same prompts and their internal textual representations were examined at multiple depths.
+
+The models do not use the same hidden dimensionality, so their representations cannot simply be compared directly.
+
+A learned projection was therefore introduced during the research stage.
+
+The objective was not initially to improve H3 generation. It was simply to answer a more fundamental question:
+
+> **Can a representation produced by one architecture be predictably mapped into a representation produced by another architecture?**
+
+Multiple internal representation combinations were screened.
+
+One correspondence was substantially stronger than the alternatives.
+
+On held-out prompts from the initial dataset, the learned cross-model mapping reached approximately:
+
+```text
+validation cosine similarity ≈ 0.904
+```
+
+This was encouraging, but there was an obvious danger.
+
+The mapping might simply have learned the distribution of prompts used to construct it.
+
+A stricter test was required.
+
+---
+
+# 4. Out-of-Distribution Test
+
+A separate set of **160 new prompts** was constructed.
+
+These prompts were designed to stress combinations such as:
+
+* complex spatial relationships;
+* difficult counting;
+* anatomy;
+* material/light interactions;
+* text;
+* architecture and vehicles;
+* reflection and occlusion;
+* longer multi-object compositions.
+
+The prompts were kept separate from the original fitting set.
+
+When the frozen cross-model bridge was evaluated on these new prompts, representation similarity decreased from approximately:
+
+```text
+0.904
+```
+
+to:
+
+```text
+0.749
+```
+
+This drop was significant.
+
+But the mapping did not collapse.
+
+That distinction was important.
+
+A completely prompt-specific mapping would be expected to fail much more severely once moved outside its fitting distribution. Instead, a substantial relationship remained.
+
+This suggested that the bridge had captured at least part of a genuine cross-model representation alignment rather than simply memorizing sentences.
+
+At this point, numerical similarity alone was no longer sufficient.
+
+The mapping had to be tested inside the actual video-generation pipeline.
+
+---
+
+# 5. The Experimental Full Bridge
+
+The next implementation introduced the mapped SenseNova representation directly into MiniMax H3 conditioning.
+
+Conceptually, the system became:
+
+```text
+Prompt
+  │
+  ├──────────────► H3 Text Encoder
+  │                     │
+  │                     ▼
+  │              native conditioning H
+  │
+  └──────────────► SenseNova
+                        │
+                        ▼
+               semantic representation
+                        │
+                        ▼
+                learned projection
+                        │
+                        ▼
+              H3-compatible semantic S
+```
+
+The transformed semantic representation was magnitude-aligned with native H3 conditioning and combined through a controllable residual blend:
+
+```text
+C = H + α(S - H)
+```
+
+where:
+
+```text
+H = native H3 conditioning
+S = mapped semantic representation
+α = bridge strength
+C = final conditioning supplied to H3
+```
+
+This became the **Full Bridge**.
+
+Crucially, H3 itself was not modified.
+
+The video transformer remained unchanged.
+
+The intervention occurred entirely in the conditioning supplied to it.
+
+---
+
+# 6. From Tensor Similarity to Actual Generation
+
+The Full Bridge was then tested through actual MiniMax H3 generation.
+
+This was the first point at which the experiment became substantially more interesting.
+
+The mapped conditioning did not merely remain numerically valid. It produced coherent H3 generations and observable changes in how prompts were interpreted.
+
+This demonstrated that the transferred representation existed in a region of conditioning space that H3 could meaningfully use.
+
+The Full Bridge therefore served as a proof of concept for three ideas:
+
+**First**, useful information could be transferred between these architectures at the representation level despite the failure of direct parameter grafting.
+
+**Second**, the transferred representation could influence H3 without modifying its generative transformer.
+
+**Third**, the effect could be controlled continuously through the blend strength.
+
+But the solution was impractical.
+
+Every generation required the donor model to participate in prompt processing.
+
+The runtime pipeline therefore contained two large models simply to produce one conditioning tensor.
+
+That was acceptable for research.
+
+It was not a good release architecture.
+
+---
+
+# 7. Removing the Donor
+
+This led to the central distillation experiment.
+
+If the Full Bridge could determine an appropriate semantic correction for H3, perhaps H3's own representation already contained enough information for a smaller model to **predict what the Full Bridge would have done**.
+
+The Full Bridge was therefore converted from an inference solution into a **teacher**.
+
+The objective changed from:
+
+```text
+SenseNova
+   ↓
+map representation into H3
+```
+
+to:
+
+```text
+H3 representation
+       ↓
+small student
+       ↓
+predict teacher-derived
+semantic representation
+```
+
+If successful, SenseNova could disappear completely from inference.
+
+---
+
+# 8. Early Distillation
+
+The first distilled experiments attempted to predict the semantic correction directly.
+
+These results were useful but not sufficiently strong.
+
+The best early single-representation configuration achieved correction prediction similarity of only roughly:
+
+```text
+≈ 0.51
+```
+
+A naïve attempt to combine several H3 representations performed even worse.
+
+This was another useful negative result.
+
+Simply providing the student with more internal features did not automatically produce a better approximation.
+
+The target itself needed to be reconsidered.
+
+Rather than asking the student to directly predict the difference between native and teacher conditioning, the next version learned the **teacher-derived semantic representation itself**.
+
+This changed the result dramatically.
+
+---
+
+# 9. Distilled Student V2
+
+The second student learned to approximate the semantic representation produced by the Full Bridge before the final blend with H3 conditioning.
+
+On the development distribution, representation similarity reached approximately:
+
+```text
+0.995
+```
+
+and semantic-correction similarity reached approximately:
+
+```text
+0.973
+```
+
+This was a major improvement.
+
+However, the strict OOD evaluation revealed a remaining weakness.
+
+Across the previously unseen 160-prompt set, representation similarity remained strong:
+
+```text
+0.941840
+```
+
+but correction similarity fell to:
+
+```text
+0.879840
+```
+
+The model clearly understood much of the teacher transformation, but the generalization gap was still visible.
+
+Long, highly compositional prompts were among the more difficult cases.
+
+This motivated one final training stage.
+
+---
+
+# 10. Final Distillation
+
+The original and difficult prompt distributions were combined and the student was refined while preserving a held-out validation subset.
+
+The final evaluation used:
+
+```text
+500 training prompts
+100 validation prompts
+```
+
+with the validation set containing both original-distribution and strict OOD examples.
+
+The final student produced:
+
+```text
+Teacher representation similarity    0.995890
+Semantic correction similarity       0.983558
+
+Main-distribution correction          0.980888
+OOD correction                        0.989788
+
+Minimum correction similarity         0.935910
+```
+
+The final blended conditioning was even closer to the Full Bridge:
+
+```text
+Bridge strength α = 0.10     0.999958
+Bridge strength α = 0.20     0.999827
+Bridge strength α = 0.30     0.999602
+```
+
+The final evaluation therefore showed not only high approximation of the teacher representation, but strong approximation of the **actual semantic correction** produced by the Full Bridge.
+
+---
+
+# 11. Performance Across Semantic Categories
+
+The final validation set covered multiple semantic categories.
+
+Correction similarity by category was:
+
+| Category                   | Similarity |
+| -------------------------- | ---------: |
+| Text                       |   0.996061 |
+| Reflection / occlusion     |   0.995932 |
+| Complex counting           |   0.994557 |
+| Complex text               |   0.991912 |
+| Reflection / occlusion OOD |   0.991855 |
+| Complex spatial            |   0.991125 |
+| Long composition           |   0.990637 |
+| Architecture / vehicle     |   0.990606 |
+| Complex material / light   |   0.987394 |
+| Material                   |   0.986263 |
+| Complex anatomy            |   0.982104 |
+| Anatomy                    |   0.980773 |
+| Lighting                   |   0.978327 |
+| Spatial                    |   0.974002 |
+| Counting                   |   0.965250 |
+
+One particularly interesting result is the recovery on long-composition prompts.
+
+These had been among the weaker cases during earlier OOD testing, yet the final student reached approximately `0.991` correction similarity for this category.
+
+This suggests that the student capacity itself was not necessarily the primary limitation. The distribution used during distillation was equally important.
+
+---
+
+# 12. The Final Architecture
+
+The research system had started as:
+
+```text
+H3 + SenseNova + cross-model bridge
+```
+
+The final system is simply:
+
+```text
+             MiniMax H3
+                 │
+                 ▼
+          H3 conditioning
+                 │
+                 ▼
+        Semantic Bridge
+                 │
+                 ▼
+      modified conditioning
+                 │
+                 ▼
+             MiniMax H3
+        video generation
+```
+
+SenseNova is no longer involved.
+
+The final adapter operates exclusively on information already produced by H3.
+
+It predicts a learned semantic transformation, magnitude-aligns that representation with native conditioning, and blends the two according to a user-controlled strength.
+
+No weights inside the MiniMax H3 diffusion transformer are modified.
+
+---
+
+# 13. An Unexpected Qualitative Result
+
+The final distilled adapter was originally expected to behave simply as a cheaper approximation of the Full Bridge.
+
+Actual generation produced a more interesting result.
+
+The distilled version preserves the general behavioral effect of the Full Bridge, but the resulting videos are not always visually identical.
+
+In some of my tests, I actually preferred the output of the **distilled adapter** to the original Full Bridge.
+
+This observation should be interpreted cautiously.
+
+It does not prove that the distilled student is objectively superior to its teacher.
+
+One possible explanation is a regularization effect.
+
+The Full Bridge contains the complete mapped donor representation, including components that may not be consistently useful to H3. A compact student cannot reproduce every variation perfectly and may preferentially learn the more predictable structure of the transformation.
+
+In that interpretation, distillation behaves somewhat like a semantic filter:
+
+```text
+Full teacher signal
+        ↓
+distillation
+        ↓
+most reproducible transformation
+        ↓
+H3
+```
+
+This is currently a hypothesis rather than a demonstrated mechanism.
+
+More qualitative testing is needed.
+
+Nevertheless, it was unexpected because the purpose of distillation was originally efficiency—not improvement.
+
+---
+
+# 14. What Semantic Bridge Actually Is
+
+Semantic Bridge is **not a LoRA**.
+
+It is not a checkpoint merge.
+
+It is not a conventional model graft.
+
+And the released file does not contain a second generative model.
+
+A more accurate description is:
+
+> **A compact conditioning-space adapter distilled from a cross-architecture semantic mapping.**
+
+A LoRA modifies effective model weights:
+
+```text
+base weights
+     +
+low-rank weight update
+     ↓
+modified network behavior
+```
+
+Semantic Bridge instead modifies information entering the generative transformer:
+
+```text
+prompt
+   ↓
+H3 representation
+   ↓
+Semantic Bridge
+   ↓
+modified representation
+   ↓
+unchanged H3 generator
+```
+
+This distinction also means that Semantic Bridge and conventional LoRAs are conceptually complementary rather than mutually exclusive.
+
+---
+
+# 15. What Is Being Released
+
+There were two fundamentally different implementations during this research.
+
+### Full SenseNova → H3 Bridge
+
+**Research prototype only — not released.**
+
+This implementation required SenseNova during inference and was used to establish that cross-model semantic transfer could influence H3 generation.
+
+It subsequently became the teacher system for distillation.
+
+### MiniMax H3 Semantic Bridge
+
+**This is the version being released.**
+
+It is the distilled standalone implementation.
+
+It requires:
+
+```text
+MiniMax H3
+H3-compatible text encoder
+MiniMaxH3_SemanticBridge_v1.safetensors
+MiniMax H3 Semantic Bridge custom node
+```
+
+It does **not** require:
+
+```text
+SenseNova checkpoint
+SenseNova runtime
+SenseNova tokenizer
+a second large language/multimodal model
+the experimental Full Bridge
+```
+
+SenseNova was involved in the research and training process only.
+
+**No SenseNova installation is required to use the public release.**
+
+---
+
+# 16. Installation
+
+The release consists of the ComfyUI custom node and the distilled `.safetensors` adapter.
+
+### Custom Node
+
+Copy:
+
+```text
+MiniMax_H3_Semantic_Bridge
+```
+
+to:
+
+```text
+ComfyUI/custom_nodes/
+```
+
+The resulting structure should look like:
+
+```text
+ComfyUI/
+└── custom_nodes/
+    └── MiniMax_H3_Semantic_Bridge/
+        ├── __init__.py
+        ├── nodes.py
+        ├── README.txt
+        ├── ADAPTER_INSTALLATION.txt
+        └── RELEASE_NOTES.txt
+```
+
+### Adapter
+
+Copy:
+
+```text
+MiniMaxH3_SemanticBridge_v1.safetensors
+```
+
+to:
+
+```text
+ComfyUI/models/semantic_bridge/
+```
+
+The final location should therefore be:
+
+```text
+ComfyUI/
+└── models/
+    └── semantic_bridge/
+        └── MiniMaxH3_SemanticBridge_v1.safetensors
+```
+
+Restart ComfyUI after installation.
+
+---
+
+# 17. ComfyUI Nodes
+
+Three nodes are included.
+
+### MiniMax H3 Image to Video + Semantic Bridge
+
+This is the recommended node for normal use.
+
+It replaces the standard MiniMax H3 Image to Video conditioning stage and applies the distilled semantic transformation internally.
+
+It supports first-frame and last-frame conditioning and can therefore be used directly in standard H3 image-to-video workflows.
+
+### MiniMax H3 Semantic Bridge
+
+This is the standalone conditioning version.
+
+It accepts existing MiniMax H3 `CONDITIONING` and applies the Semantic Bridge transformation.
+
+This is useful for custom workflows where H3 conditioning is already being generated elsewhere.
+
+### MiniMax H3 Clear Semantic Bridge Cache
+
+An optional utility for clearing the small adapter from memory.
+
+It is generally unnecessary during normal operation but is provided for workflow and debugging convenience.
+
+---
+
+# 18. Recommended Settings
+
+The recommended starting configuration is:
+
+```text
+semantic_bridge:
+MiniMaxH3_SemanticBridge_v1.safetensors
+
+alpha:
+0.10
+
+magnitude_match:
+per_token
+```
+
+`alpha` controls how strongly the predicted semantic representation influences native H3 conditioning.
+
+Conceptually:
+
+```text
+alpha = 0.00
+│
+│   Native H3
+│
+├── 0.05
+├── 0.10  ← recommended starting point
+├── 0.20
+└── 0.30
+```
+
+A higher value does **not** necessarily mean a better result.
+
+It simply increases the distance from native H3 conditioning.
+
+For controlled comparisons I recommend testing:
+
+```text
+0.00
+0.05
+0.10
+0.20
+0.30
+```
+
+while keeping all other generation parameters unchanged.
+
+---
+
+# 19. How to Evaluate It
+
+Semantic Bridge is intended primarily for prompts where **relationships** matter.
+
+A weak test is:
+
+> *A beautiful woman walking through a city.*
+
+There is little semantic structure for the bridge to affect.
+
+More informative tests involve multiple constraints:
+
+> *A woman holds a transparent bottle in her left hand while pointing toward a red sign with her right hand. A polished metal sphere sits behind the bottle and reflects a blue object outside the frame.*
+
+Useful evaluation areas include:
+
+```text
+left / right relationships
+front / behind
+inside / outside
+multiple people
+specific body parts
+object counts
+reflections
+transparent objects
+material interactions
+lighting direction
+visible text
+long compositions
+multiple simultaneous instructions
+```
+
+For a controlled A/B test, keep the following identical:
+
+```text
+prompt
+seed
+source image
+resolution
+length
+steps
+sampler
+guidance
+```
+
+and compare:
+
+```text
+A — Native MiniMax H3
+B — MiniMax H3 + Semantic Bridge
+```
+
+`alpha = 0.00` can also be used as a native-conditioning baseline within the Semantic Bridge node.
+
+---
+
+# 20. Qualitative A/B Examples
+
+Two controlled A/B pairs are included in the public repository. Within each pair, the prompt and generation setup were kept the same; the intended comparison is native H3 conditioning versus Semantic Bridge. The published Bridge examples use `alpha = 0.15` to make the behavioral difference easier to inspect.
+
+These examples are qualitative observations, not a statistical benchmark.
+
+## Example 01 — Rooftop Train Chase
+
+This test stresses complex motion, running anatomy, pursuit geometry, a specific vault interaction, hand-to-obstacle contact, landing continuity, wet reflective metal, rain, sparks, and camera motion.
+
+The exact prompt and both videos are included under `examples/01_rooftop_train_chase/`.
+
+## Example 02 — Prompt Adherence, Materials, Reflection and Transparency
+
+This test contains a particularly explicit instruction:
+
+> **“Her right hand rests flat on the glass tabletop with all five fingers naturally separated and clearly visible.”**
+
+In the native H3 generation, the character introduces an unrequested action by moving the right hand across the tabletop. In the Semantic Bridge generation at `alpha = 0.15`, the hand remains resting on the surface, more closely matching the explicitly requested state.
+
+This observation is useful because the target is directly inspectable: the requested state is *resting*, rather than moving. It is still presented only as qualitative evidence from this specific controlled pair, not as proof of universal prompt-adherence improvement.
+
+The same prompt also stresses left/right hand roles, exactly three tabletop objects in a specified order, text rendering, transparent glass, reflective metal, mirror correspondence, refraction, and mixed warm/cool lighting.
+
+The exact prompt and both videos are included under `examples/02_glass_table_prompt_adherence/`.
+
+---
+
+# 21. Scope: FL2VA / Standard Text-Conditioned H3 Only
+
+An important limitation emerged during later testing.
+
+The released Semantic Bridge was developed and distilled from the standard H3 text-conditioning path. It was not trained on the separate multimodal reference-conditioning distribution used by Ref2VA.
+
+An experimental Ref2VA-compatible implementation was tested with reference images and reference audio. At a structural level it was possible to preserve visual-reference token positions while applying the adapter to text-designated positions. However, singing and lip-sync tests showed a clear practical regression: vocal articulation became less distinct and more mumbling-like than with native Ref2VA conditioning.
+
+This suggests that preserving reference-image token positions is not sufficient. In Ref2VA, the text representation participates in a larger multimodal alignment involving text, reference media, audio, and visual performance. Altering only the text-designated part of that representation can still disturb the multimodal relationship.
+
+The current release should therefore be treated as designed for:
+
+**MiniMax H3 FL2VA / standard text-conditioned generation.**
+
+It should not currently be considered compatible with:
+
+**MiniMax H3 Ref2VA / reference-conditioned generation**, particularly workflows involving reference audio, singing, voice-driven performance, or lip synchronization.
+
+This negative result does not establish that semantic bridging is fundamentally incompatible with Ref2VA. It indicates that a Ref2VA bridge should likely be trained separately on the multimodal conditioning regime it is intended to modify.
+
+A broader lesson is that **representation compatibility is contextual, not merely dimensional**. Two conditioning tensors can have the same dimensionality while participating in different semantic and temporal alignment mechanisms.
+
+---
+
+# 22. Limitations
+
+This is an experimental research release, not a universal H3 enhancement.
+
+The adapter should not be expected to improve every generation.
+
+MiniMax H3 already has strong semantic capabilities, and many simple prompts do not require any additional transformation.
+
+There may also be prompts for which native conditioning produces a preferable result.
+
+Another important limitation is that the reported numerical metrics measure agreement with the experimental teacher system.
+
+They do **not** directly measure video quality.
+
+A student that perfectly reproduces its teacher is not necessarily a better video generator, and a small difference in conditioning can sometimes result in a substantial visual difference after generative sampling.
+
+For this reason, actual controlled video comparisons remain the most important evaluation.
+
+The current research also focuses on a particular group of semantic behaviors. Other areas may respond differently and have not yet been evaluated systematically.
+
+Finally, Semantic Bridge does not create information that H3 fundamentally cannot represent. It should be viewed as a learned transformation of H3's existing semantic conditioning, not as an independent reasoning system.
+
+---
+
+# 23. Why the Result May Matter Beyond MiniMax H3
+
+The most interesting result of this experiment may not be the released adapter itself.
+
+Direct weight transfer between unrelated architectures is extremely restrictive.
+
+Different hidden dimensions, layer structures, attention implementations and parameter organizations make conventional grafting difficult or meaningless.
+
+Representations are different.
+
+If two models have learned related concepts, their internal spaces do not necessarily need to be structurally identical for a **learnable relationship** to exist between them.
+
+This suggests a more general workflow:
+
+```text
+Model A
+  │
+  │ semantic teacher
+  ▼
+cross-architecture
+representation mapping
+  │
+  ▼
+Model B-compatible
+teacher signal
+  │
+  ▼
+distillation
+  │
+  ▼
+small Model B adapter
+```
+
+After distillation:
+
+```text
+Model A
+  ✕
+  no longer required
+
+
+Model B
+  +
+small distilled adapter
+```
+
+The expensive cross-model system can therefore potentially exist only during research and training.
+
+The deployed model remains essentially its original architecture.
+
+This opens an interesting research direction: instead of asking whether two models can share weights, we can ask whether they can **teach each other representations**.
+
+---
+
+# 24. Conclusion
+
+MiniMax H3 Semantic Bridge began as an attempt to graft semantic capabilities between two incompatible model architectures.
+
+That approach failed.
+
+The failure led to a more interesting question.
+
+Rather than transferring weights, I investigated whether the models' representations of the same prompt contained a learnable relationship.
+
+A measurable cross-model alignment was found.
+
+That alignment was then incorporated into an experimental Full Bridge, which demonstrated that a mapped donor representation could meaningfully influence actual MiniMax H3 generation.
+
+The Full Bridge solved the semantic-transfer problem but introduced an impractical runtime dependency on the donor model.
+
+Distillation removed that dependency.
+
+The final system is therefore much simpler than the research pipeline that produced it:
+
+```text
+MiniMax H3
+     +
+MiniMax H3 Semantic Bridge
+```
+
+The public adapter contains no SenseNova checkpoint and requires no SenseNova installation or inference pass.
+
+It operates entirely from H3's own conditioning and introduces the learned transformation at a controllable strength.
+
+The result suggests a broader possibility:
+
+> **Useful learned behavior may be transferable between incompatible model architectures even when their weights themselves are not.**
+
+The bridge between models may not have to exist in parameter space.
+
+It may exist in representation space.
+
+And once that bridge has taught a sufficiently small student, the original teacher may no longer need to be there at all.
+
+---
+
+## Release Summary
+
+**MiniMax H3 Semantic Bridge v1.0**
+
+Standalone distilled semantic conditioning adapter for MiniMax H3.
+
+**Release includes:**
+
+* ComfyUI custom node
+* distilled Semantic Bridge `.safetensors`
+
+**Release does not require:**
+
+* SenseNova
+* the experimental Full Bridge
+* a second multimodal model at inference
+
+**Recommended starting configuration:**
+
+```text
+alpha = 0.10
+magnitude_match = per_token
+```
+
+**Primary experimental targets:**
+
+```text
+complex composition
+spatial relationships
+anatomy
+counting
+prompt following
+text
+materials
+lighting
+reflection
+transparency
+occlusion
+```
+
+**Status:** Experimental / Research Release.
+
+**Supported path:** MiniMax H3 FL2VA / standard text-conditioned generation.
+
+**Not supported in v1:** Ref2VA / reference-conditioned generation, particularly reference-audio singing and lip-sync.

SHA256SUMS.txt

@@ -0,0 +1,65 @@
+005cd62443f3115ab3d32889c9c999596c5e2ab23be3f04a1c8febfb6f7b964d  research/reports/distillation_source_inspection.txt
+020b21411e7a63a4a6684c0e8a35ef98c5f0f6e55d774b8d0313396402da6c44  research/reports/sensenova_mot_gen_bridge_comparison.txt
+039f12ced4a634c954ed25c22973cdcf95e14299b0eb3c61d362ea489d646f6e  research/reports/h3_sensenova_bridge_report.txt
+07665ef280012a8ff6747db0edb1617a8d63f323277b8130da5042fbefd823aa  research/reports/hidden_bridge_screening_rank128.txt
+0ff88375ec2ae942ad5effa801f36ba700e4cbb07798df6fe495479363ac656f  research/raw_scripts/train_distilled_student_v2.py
+16d0530e314f734c03d52e4c4d6c8838d6df5e7d63f10322554a498112758e14  examples/01_rooftop_train_chase/native_h3.mp4
+1b012bd4ff4d6edc2e5a72a06ce69409c09ca35e6964b5491a6c0ef6f28ce82c  research/reports/hidden_layer_bridge_comparison.txt
+1f9d9d2ba4ef6c692abf791de180f906ee77fc943500fc3e2f9673680a584461  research/raw_scripts/compare_tokenizers.py
+2c19120bfcff95fdf20a58b428ac83dad25f8da8fa3e308aa38e415006f240fb  research/reports/distilled_student_v2_results.csv
+2d6c1f0af8bce0057a6d84bb0041aceb00a1148bd6d20d0fb6a845db1f0f1448  research/raw_scripts/evaluate_distilled_student_v2_ood.py
+3185139654470d93b67e6ce80eb1a7d2594b3c84d94ab93eb85c5de4701d805b  research/raw_scripts/compare_minimax_sensenova.py
+318a764ad464823c134b66e348a1b17bff9e1d4830e2a4ec1cbc455620425912  research/raw_scripts/extract_h3_ood_hidden.py
+35ab00a9d44a75674d2148c67601817d7e0950337b9b82dec228c0c893e6681c  research/datasets/bridge_prompts_480.json
+3ee5db6c7c12f574541b82da93ca11a91037af307dce0e1d0c98d10f5229c429  research/reports/sensenova_mot_gen_bridge_comparison.csv
+3f73f78b2b50a397c7b377a9d2a01aec05885cea3e7dab2fc873432c3b72a3fe  UPSTREAM_LICENSES.md
+47dc42c5292eddd10e3d55c9d169a03a299e275af51aaef5719ffb660cb337bd  research/raw_scripts/make_bridge_ood_prompts.py
+4a01bd07c6333b5bf9172429fd8043f150886a62b07f2c5de117cfd0e0b73b14  research/datasets/bridge_ood_prompts_160.json
+5150010c15ca4042cbb3ed5162957ecb79a0e57f83036bfbad60a6713b717492  research/teacher_artifacts/SN_L32_to_H3_L49_rank128.safetensors
+5333910cda814464c7a147f1de1ccc11281e49b30a98663ab54c23e3bcb9c9b0  research/reports/distilled_student_v2_STRICT_OOD_report.txt
+557b6be384c5f0614a3fa1b12428930d29829edf826127c8b7badfc200f8407c  research/raw_scripts/extract_h3_hidden_states.py
+597990c07be911a937990f30267830bc336ff85316fbd1f81f6f217db7faf7be  research/raw_scripts/extract_sensenova_bridge_dataset.py
+5a10e4a1b9bc0b10d72233101af14ec8aafb2b44bb35b732c334e7275804e288  research/raw_scripts/prepare_sensenova_h3_condition.py
+5c8db67192acfc4666797acc45753092b63185ba4164a1ffc868033468e01635  research/reports/distilled_adapter_screening.csv
+627a9abfefcf0f457ab2d8f3d6f8d5baecabaaed3d48a65f4f1b862c08c71b06  research/reports/distilled_student_v3_report.txt
+635689b0bb6c53ac8add559f4aa32bb5ee66355291771d6c87ead8c857f4ac7a  LICENSE.md
+64e9e9eea85402ab2477cb597e2315d77c2a26d67b23fc800e238efd35ebf421  research/raw_scripts/inspect_distillation_data.py
+6cd2dd305e3f4d3c67c3609976f4be34adecb4b7bd7a51d90c9719ac2ae5ff69  research/raw_scripts/extract_sensenova_hidden_states.py
+7a05ffc62356607eace846fd755b42e7d036b1efa304167941fd183f61c4b3d4  research/raw_scripts/make_bridge_prompts.py
+7e9506dbc9ce6e304aabb991e935a8519e78770d35345496239c522f4b92d7c5  research/raw_scripts/extract_sensenova_hidden_states_stream_v3.py
+7ed600988c2d48f03cf0ad7bee35d8f17427b45aa25433a4eaab55e7b732d221  research/reports/distilled_student_v2_results.txt
+8f30ddf1b781bc9003e235d418754593ff04fcf3e7f673666046a1c6cb3adc1e  README.md
+902580d71cfd1cd1cbf7b6da6254428881ef5af9b2a7d47b7774b3e150a5e52c  research/reports/distilled_student_v3_history.csv
+924ce5be3be8a7f1a06296a89398d0a2e0e714da065b34f961f64e257e1f09dc  RESEARCH_ARTICLE.md
+95e04c8b56655aa9f610292140c4820158c656e89ff08973d75fb5171a2b8499  research/raw_scripts/compare_hidden_layers_bridge.py
+98252ac062a0020208deddef9d99c7b0e2702d4bff9d058196cbd01d6a086af6  research/raw_scripts/train_hidden_bridge_screening.py
+9dfaadf1df0e016473807242e64b87de658d19d780125c7554e4ec4e7d8a725f  research/raw_scripts/inspect_h3_sensenova_bridge.py
+a019fddb4c079c72bc5c393cbcf2a905e57138b4a56e91cee6227147d9be6bed  research/raw_scripts/train_distilled_student_v3.py
+ac0dc8ac05f545ebdee12e2fcebe4515b049f9cfd9558eb4887a9bf3fd6d562e  MiniMaxH3_SemanticBridge_v1.safetensors
+b4fbdce02a44a6fa7d42b1272de404e5de30eb608200fc6f693e1e136aafb0c1  research/raw_scripts/extract_sensenova_ood_hidden.py
+b60c5b367c166751c798024715f37b1b37a09ffbd0fd71402c3b3a48dc8896ce  research/reports/hidden_layer_bridge_comparison.csv
+bb649b0c49e43e82f7ffa8b7be37f20de793d925a6ec55c2ab0d2d503dd1899f  examples/02_glass_table_prompt_adherence/native_h3.mp4
+c1b6452bf0e43d153bd81f45a6688b35ee8ee4df95951c21d23e0ac717a8d67b  research/reports/hidden_bridge_screening_rank128.csv
+c4866a0cc554237046814c9cb827a5960caecbd2214d5a5a04b0e43716b437bb  MiniMax_H3_Semantic_Bridge_v1.0.zip
+c8e799776c9ac010c76ba7eeb11f481b6c130034920030cbbfa584a133f5f404  research/reports/distilled_student_v2_STRICT_OOD_per_prompt.csv
+c90f5447164e3b72e02bdfe5cd484e8d28fcbd2842aae6233c9b840d77d65451  examples/02_glass_table_prompt_adherence/semantic_bridge_alpha_0.15.mp4
+c92aa3822cec5952b59abff26ac426fa4224aa35b94eeb5f0bac903bf22fb5de  research/raw_scripts/extract_sensenova_hidden_states_stream.py
+d06b5cf4154e44e869b007d30528d8ff4e2324245c4be48fa0619e92791ee62f  research/raw_scripts/extract_h3_embeddings_full.py
+d2cca56ac4c8aca46f3d3b4e32dbb7d18e78db7bbab8f3ff5ecd5656ba293a97  NOTICE.txt
+d3efa1ef791d1f9aa6a3fdffc2f0689d4be34b01dd15043aafd52111b31d6cd2  research/raw_scripts/train_distilled_adapter_screening.py
+d5754f336f4dd42f8cd8c53667992475252bf4b187ee73bb928bdd368e03d397  examples/01_rooftop_train_chase/prompt.txt
+d5e10235e90dc379e98852808810dcc92de94d29bd68a40a85cf5816cd64132d  examples/02_glass_table_prompt_adherence/prompt.txt
+d9a8911c11e3d305e0f7af0e0d45a190a33bd1d25ce50b44e298f37fc6b513a8  research/reports/hidden_bridge_OOD_final_report.txt
+e149bd39e003f48c8f5a8c9af31b6aa78e24cb5d0df9cef81d243a8bc08f4c8e  examples/01_rooftop_train_chase/semantic_bridge_alpha_0.15.mp4
+e60b14e081d21982b46aedf28e2f3713616cc5e4f89ac117405dd7c03ab22d8d  research/raw_scripts/make_bridge_ood_prompts_v1.py
+e6c3875c4f82b75c25809667897f61fdccd8001860e9bbb88d50f519323940b7  research/raw_scripts/extract_h3_hidden_states_fast.py
+ea9cfbe8d485093904bf92cd4c1fc2a5c7f38a059e1bf208fe377f66d97a8d2e  research/raw_scripts/extract_h3_embeddings_test.py
+ed67e7fa5f49bbbca9294bc8055d9f571457d03b183b963c40bfffc34fe5f606  research/raw_scripts/evaluate_hidden_bridge_ood.py
+ee773a550f35269187285ae98766493658c91524663c3aa07008616b298a1304  research/raw_scripts/extract_h3_bridge_dataset.py
+f020e87b6f30537be4f219d20bc289480bc93dd9f6f90204f102a08c07db3389  research/raw_scripts/extract_sensenova_hidden_states_local.py
+f40410dd8d9ffaa18bf149d200646ff8268fa5531efcde53b37c34b7fd5f5442  research/reports/hidden_bridge_OOD_per_prompt.csv
+f4186926cb7ac1f4c9e59470a013f780fd21d8a053d39bd9ce74179662d1f190  research/raw_scripts/extract_sensenova_hidden_states_stream_v2.py
+f4be9eb6ba7ffa1207f1d101a1e6690c4c48243df2eb5401161f3a5e5202169f  research/raw_scripts/train_embedding_bridge_test.py
+f6373b1a355f2cfb47e7441c4c43c196ccbb173121c0492793844019059b7fed  research/reports/distilled_adapter_screening.txt
+fbd662aee9e8b11da626956118cc22299aef550d5141b8e90155516698ffaa5f  research/reports/minimax_vs_sensenova_report.txt
+fc540af96938797b4a75da4ffd85b04e69eec4c4ad55f7fac86c7c689f6f6a42  research/raw_scripts/test_sensenova_mot_gen_bridge.py

UPSTREAM_LICENSES.md

@@ -0,0 +1,23 @@
+# Upstream projects and licenses
+
+## MiniMax H3
+
+Project: https://huggingface.co/MiniMaxAI/MiniMax-H3
+
+License: MiniMax H3 Community License Agreement
+
+License text: https://huggingface.co/MiniMaxAI/MiniMax-H3/blob/main/LICENSE
+
+The agreement contains terms covering MiniMax H3 Works and Model Derivatives, distribution requirements, use restrictions, and an Applicable Territory definition. Users should read the official agreement directly.
+
+## SenseNova U1.5
+
+Project: https://huggingface.co/sensenova/SenseNova-U1.5-8B-MoT
+
+License: Apache License 2.0
+
+SenseNova was used as an experimental semantic teacher during development. The public runtime adapter does not require the SenseNova model.
+
+## Qwen3-VL
+
+MiniMax H3 documentation notes that its encoder uses Qwen3-VL-32B, licensed under Apache License 2.0. Refer to the MiniMax H3 license/documentation for the upstream notice and link.

examples/01_rooftop_train_chase/native_h3.mp4

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:16d0530e314f734c03d52e4c4d6c8838d6df5e7d63f10322554a498112758e14
+size 2491328

examples/01_rooftop_train_chase/prompt.txt

@@ -0,0 +1,10 @@
+Scene overview: A cinematic high-speed action scene on the rooftop of a moving train racing through a dense futuristic city at night. A young woman in a black leather jacket and dark cargo pants runs along the roof while being pursued by a masked attacker. Rain pours heavily, making the metal surface wet and reflective. Neon signs, illuminated windows and city lights streak past in the background.
+
+Storyboard:
+[0s-1.5s] The camera tracks backward directly in front of the woman as she sprints toward it along the narrow train roof. Her coat and hair whip violently in the wind. She briefly looks over her shoulder at the masked attacker running several meters behind her. Both characters maintain natural running anatomy and believable foot contact with the moving train.
+
+[1.5s-3.2s] She reaches a low metal obstacle spanning part of the roof, plants her left foot and performs a fast running vault over it, supporting herself momentarily with her right hand. The camera swings dynamically from the front into a low three-quarter side angle while following the movement.
+
+[3.2s-5s] She lands firmly on both feet and immediately continues running. Behind her, the attacker jumps over the same obstacle. At that exact moment the train passes beneath a shower of bright sparks from an overhead electrical structure. The sparks illuminate both figures for an instant while their reflections slide across the rain-covered metal roof.
+
+Fast controlled camera movement, strong sense of forward momentum and physical weight. Realistic human anatomy throughout the running, vaulting and landing motions. Hands and limbs remain anatomically coherent. Accurate interaction between feet, hands and physical surfaces. Wet brushed metal shows sharp moving reflections, leather remains glossy but textured, fabric reacts naturally to wind and motion, rain catches the surrounding neon light. Strong depth and spatial continuity between the woman, attacker, train roof and rapidly moving city background. No slow motion.

examples/01_rooftop_train_chase/semantic_bridge_alpha_0.15.mp4

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e149bd39e003f48c8f5a8c9af31b6aa78e24cb5d0df9cef81d243a8bc08f4c8e
+size 2538282

examples/02_glass_table_prompt_adherence/native_h3.mp4

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bb649b0c49e43e82f7ffa8b7be37f20de793d925a6ec55c2ab0d2d503dd1899f
+size 358133

examples/02_glass_table_prompt_adherence/prompt.txt

@@ -0,0 +1,13 @@
+A cinematic medium-wide shot of a young woman standing behind a transparent glass table in a sophisticated modern apartment at night. She is wearing a fitted black sleeveless dress and a thin silver necklace.
+
+Her left hand holds a polished stainless-steel cup by its handle at chest height. Her right hand rests flat on the glass tabletop with all five fingers naturally separated and clearly visible. Her hands must remain anatomically correct and attached naturally to her arms.
+
+On the table in front of her are three distinct objects arranged from left to right: a clear glass bottle filled halfway with water, a small red ceramic bowl, and a folded white paper card with the words "NIGHT SHIFT" printed in large clean black letters.
+
+A large wall mirror directly behind the woman reflects the back of her head, shoulders, the room behind the camera, and the warm floor lamp standing on the left side of the room. The reflected objects must correspond logically to their real positions.
+
+Warm amber light from the floor lamp passes through the glass bottle, creating subtle refraction and a bright caustic pattern on the tabletop. Cool blue city light enters through a window on the right and produces contrasting highlights along the polished metal cup.
+
+The transparent tabletop must remain visibly transparent, with the woman's lower body partially visible through the glass. The metal cup should have realistic sharp reflections, the bottle should refract the background, the ceramic bowl should remain opaque and matte, and the mirror should behave as a true reflective surface.
+
+Natural human proportions, coherent spatial relationships, physically plausible reflections and transparency, realistic hands, detailed material differences, clear readable text.

examples/02_glass_table_prompt_adherence/semantic_bridge_alpha_0.15.mp4

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c90f5447164e3b72e02bdfe5cd484e8d28fcbd2842aae6233c9b840d77d65451
+size 329053

research/datasets/bridge_ood_prompts_160.json

@@ -0,0 +1,642 @@
+[
+  {
+    "category": "complex_counting",
+    "prompt": "Exactly 3 glass bottles form two staggered rows, while exactly 2 blue books occupy the background. None of the objects is completely hidden, and one small yellow marker is located beside object number 1. Counting variant 1."
+  },
+  {
+    "category": "complex_text",
+    "prompt": "Inside a minimalist living room, the exact phrase \"PLATFORM 11\" is printed clearly on a rectangular sign. A person stands partly in front of the sign without covering any letters. A reflective surface beside the sign shows a reversed reflection of part of the scene while the original text remains completely readable. Scene variant 13."
+  },
+  {
+    "category": "complex_counting",
+    "prompt": "Exactly 3 silver spheres form a curved row, while exactly 4 red cylinders occupy the background. None of the objects is completely hidden, and one small yellow marker is located beside object number 1. Counting variant 16."
+  },
+  {
+    "category": "architecture_vehicle",
+    "prompt": "A white city bus passes beside a railway platform. A staircase rises on the left side and turns at an upper landing. Three architectural openings are visible at different depths, and a glass facade reflects a building across the street. Composition variant 17."
+  },
+  {
+    "category": "complex_anatomy",
+    "prompt": "A young woman raises the left hand while holding a cup in the right hand. A woman wearing a dark jacket stands beside them and points toward the object with the right hand. Both people's hands and feet remain visible. Anatomy variant 1."
+  },
+  {
+    "category": "architecture_vehicle",
+    "prompt": "A silver tram passes beside a industrial studio. A staircase rises on the left side and turns at an upper landing. Three architectural openings are visible at different depths, and a glass facade reflects a building across the street. Composition variant 15."
+  },
+  {
+    "category": "complex_spatial",
+    "prompt": "A small lamp stands behind a ceramic cup, while a blue book is positioned to their left. The ceramic cup partially overlaps the small lamp from the camera viewpoint. Exactly 3 major objects are visible in the composition. Spatial variant 9."
+  },
+  {
+    "category": "complex_material_light",
+    "prompt": "A ceramic cup made from dark polished wood rests on a surface made from brushed aluminum, illuminated by two opposing light sources. The scene clearly shows reflection, roughness, transparency, or subsurface behavior appropriate to both materials. A small colored object is reflected near the edge of the surface. Material-light variant 8."
+  },
+  {
+    "category": "reflection_occlusion_ood",
+    "prompt": "A man wearing a gray coat stands behind a transparent glass panel. A ceramic cup in the foreground partially occludes the torso while the face and both hands remain visible. A mirror behind the person reflects a transparent cube located outside the direct camera frame. The glass also contains a faint reflection of an overhead light at position 12. Reflection variant 12."
+  },
+  {
+    "category": "long_composition",
+    "prompt": "An elderly man stands at the left side of a glass table while a man wearing a gray coat sits to the right. The standing person holds a red cylinder in the left hand and points toward a sign reading exactly \"FINAL STOP\" with the right hand. A ceramic cup and a transparent cube lie on the table in that order from left to right. The scene is illuminated by cool light entering from the right. A large mirror behind both people reflects the seated person's back and one object outside the direct camera frame. Exactly 6 major foreground objects should remain clearly distinguishable. Long-scene variant 10."
+  },
+  {
+    "category": "complex_spatial",
+    "prompt": "A ceramic cup stands behind a blue book, while a black vase is positioned to their right. The blue book partially overlaps the ceramic cup from the camera viewpoint. Exactly 4 major objects are visible in the composition. Spatial variant 2."
+  },
+  {
+    "category": "architecture_vehicle",
+    "prompt": "A red compact car passes beside a modern kitchen. A staircase rises on the right side and turns at an upper landing. Three architectural openings are visible at different depths, and a glass facade reflects a building across the street. Composition variant 16."
+  },
+  {
+    "category": "long_composition",
+    "prompt": "A tall man stands at the left side of a glass table while a woman with short hair sits to the right. The standing person holds a metal box in the left hand and points toward a sign reading exactly \"PLATFORM 11\" with the right hand. A blue book and a red cylinder lie on the table in that order from left to right. The scene is illuminated by strong backlight. A large mirror behind both people reflects the seated person's back and one object outside the direct camera frame. Exactly 7 major foreground objects should remain clearly distinguishable. Long-scene variant 3."
+  },
+  {
+    "category": "complex_spatial",
+    "prompt": "A small lamp stands behind a ceramic cup, while a blue book is positioned to their left. The ceramic cup partially overlaps the small lamp from the camera viewpoint. Exactly 5 major objects are visible in the composition. Spatial variant 19."
+  },
+  {
+    "category": "complex_text",
+    "prompt": "Inside a underground station, the exact phrase \"WEST EXIT\" is printed clearly on a rectangular sign. A person stands partly in front of the sign without covering any letters. A reflective surface beside the sign shows a reversed reflection of part of the scene while the original text remains completely readable. Scene variant 4."
+  },
+  {
+    "category": "architecture_vehicle",
+    "prompt": "A white city bus passes beside a industrial studio. A staircase rises on the left side and turns at an upper landing. Three architectural openings are visible at different depths, and a glass facade reflects a building across the street. Composition variant 7."
+  },
+  {
+    "category": "reflection_occlusion_ood",
+    "prompt": "A woman wearing a dark jacket stands behind a transparent glass panel. A ceramic cup in the foreground partially occludes the torso while the face and both hands remain visible. A mirror behind the person reflects a transparent cube located outside the direct camera frame. The glass also contains a faint reflection of an overhead light at position 2. Reflection variant 2."
+  },
+  {
+    "category": "architecture_vehicle",
+    "prompt": "A blue bicycle passes beside a railway platform. A staircase rises on the left side and turns at an upper landing. Three architectural openings are visible at different depths, and a glass facade reflects a building across the street. Composition variant 9."
+  },
+  {
+    "category": "complex_text",
+    "prompt": "Inside a industrial studio, the exact phrase \"STUDIO C\" is printed clearly on a rectangular sign. A person stands partly in front of the sign without covering any letters. A reflective surface beside the sign shows a reversed reflection of part of the scene while the original text remains completely readable. Scene variant 6."
+  },
+  {
+    "category": "complex_anatomy",
+    "prompt": "A man wearing a gray coat holds a bottle with both hands directly in front of the chest. A young woman stands behind them and points toward the object with the left hand. Both people's hands and feet remain visible. Anatomy variant 14."
+  },
+  {
+    "category": "complex_spatial",
+    "prompt": "A silver sphere stands behind a small lamp, while a ceramic cup is positioned to their right. The small lamp partially overlaps the silver sphere from the camera viewpoint. Exactly 6 major objects are visible in the composition. Spatial variant 16."
+  },
+  {
+    "category": "complex_anatomy",
+    "prompt": "A tall man holds a bottle with both hands directly in front of the chest. A man wearing a gray coat stands beside them and points toward the object with the right hand. Both people's hands and feet remain visible. Anatomy variant 19."
+  },
+  {
+    "category": "complex_text",
+    "prompt": "Inside a small workshop, the exact phrase \"ROOM 314\" is printed clearly on a rectangular sign. A person stands partly in front of the sign without covering any letters. A reflective surface beside the sign shows a reversed reflection of part of the scene while the original text remains completely readable. Scene variant 2."
+  },
+  {
+    "category": "complex_material_light",
+    "prompt": "A ceramic cup made from frosted glass rests on a surface made from translucent amber acrylic, illuminated by cool light entering from the right. The scene clearly shows reflection, roughness, transparency, or subsurface behavior appropriate to both materials. A small colored object is reflected near the edge of the surface. Material-light variant 18."
+  },
+  {
+    "category": "complex_spatial",
+    "prompt": "A silver sphere stands behind a small lamp, while a ceramic cup is positioned to their right. The small lamp partially overlaps the silver sphere from the camera viewpoint. Exactly 4 major objects are visible in the composition. Spatial variant 6."
+  },
+  {
+    "category": "architecture_vehicle",
+    "prompt": "A red compact car passes beside a railway platform. A staircase rises on the left side and turns at an upper landing. Three architectural openings are visible at different depths, and a glass facade reflects a building across the street. Composition variant 1."
+  },
+  {
+    "category": "complex_counting",
+    "prompt": "Exactly 7 blue books form two staggered rows, while exactly 2 small lamps occupy the background. None of the objects is completely hidden, and one small yellow marker is located beside object number 5. Counting variant 5."
+  },
+  {
+    "category": "complex_counting",
+    "prompt": "Exactly 3 glass bottles form two staggered rows, while exactly 2 blue books occupy the background. None of the objects is completely hidden, and one small yellow marker is located beside object number 2. Counting variant 11."
+  },
+  {
+    "category": "complex_spatial",
+    "prompt": "A wooden chair stands behind a transparent cube, while a red cylinder is positioned to their right. The transparent cube partially overlaps the wooden chair from the camera viewpoint. Exactly 4 major objects are visible in the composition. Spatial variant 14."
+  },
+  {
+    "category": "long_composition",
+    "prompt": "An elderly man stands at the left side of a glass table while a man wearing a gray coat sits to the right. The standing person holds a black vase in the left hand and points toward a sign reading exactly \"CAFE LEVEL 2\" with the right hand. A red cylinder and a blue book lie on the table in that order from left to right. The scene is illuminated by cool light entering from the right. A large mirror behind both people reflects the seated person's back and one object outside the direct camera frame. Exactly 6 major foreground objects should remain clearly distinguishable. Long-scene variant 18."
+  },
+  {
+    "category": "complex_anatomy",
+    "prompt": "A seated woman reaches forward with the right arm while keeping the left hand behind the back. An elderly woman stands beside them and points toward the object with the right hand. Both people's hands and feet remain visible. Anatomy variant 13."
+  },
+  {
+    "category": "long_composition",
+    "prompt": "A woman wearing a dark jacket stands at the left side of a glass table while an elderly woman sits to the right. The standing person holds a red cylinder in the left hand and points toward a sign reading exactly \"FINAL STOP\" with the right hand. A ceramic cup and a transparent cube lie on the table in that order from left to right. The scene is illuminated by a narrow spotlight from above. A large mirror behind both people reflects the seated person's back and one object outside the direct camera frame. Exactly 8 major foreground objects should remain clearly distinguishable. Long-scene variant 20."
+  },
+  {
+    "category": "complex_material_light",
+    "prompt": "A black vase made from glossy ceramic rests on a surface made from polished chrome, illuminated by hard side lighting. The scene clearly shows reflection, roughness, transparency, or subsurface behavior appropriate to both materials. A small colored object is reflected near the edge of the surface. Material-light variant 14."
+  },
+  {
+    "category": "complex_material_light",
+    "prompt": "A transparent cube made from brushed aluminum rests on a surface made from glossy ceramic, illuminated by strong backlight. The scene clearly shows reflection, roughness, transparency, or subsurface behavior appropriate to both materials. A small colored object is reflected near the edge of the surface. Material-light variant 3."
+  },
+  {
+    "category": "complex_counting",
+    "prompt": "Exactly 3 silver spheres form a curved row, while exactly 4 red cylinders occupy the background. None of the objects is completely hidden, and one small yellow marker is located beside object number 3. Counting variant 6."
+  },
+  {
+    "category": "architecture_vehicle",
+    "prompt": "A black motorcycle passes beside a small workshop. A staircase rises on the left side and turns at an upper landing. Three architectural openings are visible at different depths, and a glass facade reflects a building across the street. Composition variant 3."
+  },
+  {
+    "category": "complex_anatomy",
+    "prompt": "An elderly man reaches forward with the right arm while keeping the left hand behind the back. A seated woman stands behind them and points toward the object with the left hand. Both people's hands and feet remain visible. Anatomy variant 18."
+  },
+  {
+    "category": "complex_spatial",
+    "prompt": "A wooden chair stands behind a transparent cube, while a red cylinder is positioned to their right. The transparent cube partially overlaps the wooden chair from the camera viewpoint. Exactly 6 major objects are visible in the composition. Spatial variant 4."
+  },
+  {
+    "category": "complex_anatomy",
+    "prompt": "A man wearing a gray coat raises the left hand while holding a cup in the right hand. A young woman stands behind them and points toward the object with the left hand. Both people's hands and feet remain visible. Anatomy variant 6."
+  },
+  {
+    "category": "complex_material_light",
+    "prompt": "A metal box made from polished chrome rests on a surface made from wet black stone, illuminated by warm light entering from the left. The scene clearly shows reflection, roughness, transparency, or subsurface behavior appropriate to both materials. A small colored object is reflected near the edge of the surface. Material-light variant 9."
+  },
+  {
+    "category": "architecture_vehicle",
+    "prompt": "A white city bus passes beside a glass office lobby. A staircase rises on the right side and turns at an upper landing. Three architectural openings are visible at different depths, and a glass facade reflects a building across the street. Composition variant 2."
+  },
+  {
+    "category": "architecture_vehicle",
+    "prompt": "A black motorcycle passes beside a underground station. A staircase rises on the left side and turns at an upper landing. Three architectural openings are visible at different depths, and a glass facade reflects a building across the street. Composition variant 13."
+  },
+  {
+    "category": "reflection_occlusion_ood",
+    "prompt": "An elderly man stands behind a transparent glass panel. A silver sphere in the foreground partially occludes the torso while the face and both hands remain visible. A mirror behind the person reflects a glass bottle located outside the direct camera frame. The glass also contains a faint reflection of an overhead light at position 16. Reflection variant 16."
+  },
+  {
+    "category": "long_composition",
+    "prompt": "A seated woman stands at the left side of a glass table while a young woman sits to the right. The standing person holds a blue book in the left hand and points toward a sign reading exactly \"OPEN 24 HOURS\" with the right hand. A transparent cube and a ceramic cup lie on the table in that order from left to right. The scene is illuminated by soft diffused window light. A large mirror behind both people reflects the seated person's back and one object outside the direct camera frame. Exactly 5 major foreground objects should remain clearly distinguishable. Long-scene variant 5."
+  },
+  {
+    "category": "long_composition",
+    "prompt": "A woman with short hair stands at the left side of a glass table while a tall man sits to the right. The standing person holds a transparent cube in the left hand and points toward a sign reading exactly \"RIVER HOTEL\" with the right hand. A small lamp and a wooden chair lie on the table in that order from left to right. The scene is illuminated by warm reflected light from below. A large mirror behind both people reflects the seated person's back and one object outside the direct camera frame. Exactly 7 major foreground objects should remain clearly distinguishable. Long-scene variant 7."
+  },
+  {
+    "category": "complex_material_light",
+    "prompt": "A glass bottle made from rough concrete rests on a surface made from frosted glass, illuminated by warm reflected light from below. The scene clearly shows reflection, roughness, transparency, or subsurface behavior appropriate to both materials. A small colored object is reflected near the edge of the surface. Material-light variant 7."
+  },
+  {
+    "category": "reflection_occlusion_ood",
+    "prompt": "A young woman stands behind a transparent glass panel. A transparent cube in the foreground partially occludes the torso while the face and both hands remain visible. A mirror behind the person reflects a ceramic cup located outside the direct camera frame. The glass also contains a faint reflection of an overhead light at position 7. Reflection variant 7."
+  },
+  {
+    "category": "reflection_occlusion_ood",
+    "prompt": "A tall man stands behind a transparent glass panel. A glass bottle in the foreground partially occludes the torso while the face and both hands remain visible. A mirror behind the person reflects a silver sphere located outside the direct camera frame. The glass also contains a faint reflection of an overhead light at position 1. Reflection variant 1."
+  },
+  {
+    "category": "reflection_occlusion_ood",
+    "prompt": "A woman with short hair stands behind a transparent glass panel. A metal box in the foreground partially occludes the torso while the face and both hands remain visible. A mirror behind the person reflects a black vase located outside the direct camera frame. The glass also contains a faint reflection of an overhead light at position 13. Reflection variant 13."
+  },
+  {
+    "category": "complex_material_light",
+    "prompt": "A silver sphere made from wet black stone rests on a surface made from rough concrete, illuminated by a narrow spotlight from above. The scene clearly shows reflection, roughness, transparency, or subsurface behavior appropriate to both materials. A small colored object is reflected near the edge of the surface. Material-light variant 12."
+  },
+  {
+    "category": "complex_text",
+    "prompt": "Inside a minimalist living room, the exact phrase \"OPEN 24 HOURS\" is printed clearly on a rectangular sign. A person stands partly in front of the sign without covering any letters. A reflective surface beside the sign shows a reversed reflection of part of the scene while the original text remains completely readable. Scene variant 5."
+  },
+  {
+    "category": "complex_counting",
+    "prompt": "Exactly 5 metal boxs form two staggered rows, while exactly 2 transparent cubes occupy the background. None of the objects is completely hidden, and one small yellow marker is located beside object number 3. Counting variant 3."
+  },
+  {
+    "category": "complex_material_light",
+    "prompt": "A wooden chair made from frosted glass rests on a surface made from translucent amber acrylic, illuminated by cool light entering from the right. The scene clearly shows reflection, roughness, transparency, or subsurface behavior appropriate to both materials. A small colored object is reflected near the edge of the surface. Material-light variant 10."
+  },
+  {
+    "category": "complex_text",
+    "prompt": "Inside a underground station, the exact phrase \"ROOM 314\" is printed clearly on a rectangular sign. A person stands partly in front of the sign without covering any letters. A reflective surface beside the sign shows a reversed reflection of part of the scene while the original text remains completely readable. Scene variant 12."
+  },
+  {
+    "category": "complex_material_light",
+    "prompt": "A blue book made from brushed aluminum rests on a surface made from glossy ceramic, illuminated by strong backlight. The scene clearly shows reflection, roughness, transparency, or subsurface behavior appropriate to both materials. A small colored object is reflected near the edge of the surface. Material-light variant 11."
+  },
+  {
+    "category": "complex_anatomy",
+    "prompt": "A young woman crosses the right leg over the left while looking over the left shoulder. A woman wearing a dark jacket stands beside them and points toward the object with the left hand. Both people's hands and feet remain visible. Anatomy variant 17."
+  },
+  {
+    "category": "complex_counting",
+    "prompt": "Exactly 6 wooden chairs form a curved row, while exactly 4 black vases occupy the background. None of the objects is completely hidden, and one small yellow marker is located beside object number 2. Counting variant 14."
+  },
+  {
+    "category": "complex_text",
+    "prompt": "Inside a underground station, the exact phrase \"FINAL STOP\" is printed clearly on a rectangular sign. A person stands partly in front of the sign without covering any letters. A reflective surface beside the sign shows a reversed reflection of part of the scene while the original text remains completely readable. Scene variant 20."
+  },
+  {
+    "category": "long_composition",
+    "prompt": "An elderly man stands at the left side of a glass table while a man wearing a gray coat sits to the right. The standing person holds a ceramic cup in the left hand and points toward a sign reading exactly \"ROOM 314\" with the right hand. A wooden chair and a small lamp lie on the table in that order from left to right. The scene is illuminated by cool light entering from the right. A large mirror behind both people reflects the seated person's back and one object outside the direct camera frame. Exactly 6 major foreground objects should remain clearly distinguishable. Long-scene variant 2."
+  },
+  {
+    "category": "complex_counting",
+    "prompt": "Exactly 6 small lamps form two staggered rows, while exactly 2 metal boxs occupy the background. None of the objects is completely hidden, and one small yellow marker is located beside object number 3. Counting variant 9."
+  },
+  {
+    "category": "complex_spatial",
+    "prompt": "A glass bottle stands behind a wooden chair, while a transparent cube is positioned to their left. The wooden chair partially overlaps the glass bottle from the camera viewpoint. Exactly 3 major objects are visible in the composition. Spatial variant 1."
+  },
+  {
+    "category": "long_composition",
+    "prompt": "A woman wearing a dark jacket stands at the left side of a glass table while an elderly woman sits to the right. The standing person holds a wooden chair in the left hand and points toward a sign reading exactly \"WEST EXIT\" with the right hand. A silver sphere and a glass bottle lie on the table in that order from left to right. The scene is illuminated by a narrow spotlight from above. A large mirror behind both people reflects the seated person's back and one object outside the direct camera frame. Exactly 8 major foreground objects should remain clearly distinguishable. Long-scene variant 4."
+  },
+  {
+    "category": "complex_spatial",
+    "prompt": "A ceramic cup stands behind a blue book, while a black vase is positioned to their right. The blue book partially overlaps the ceramic cup from the camera viewpoint. Exactly 6 major objects are visible in the composition. Spatial variant 12."
+  },
+  {
+    "category": "complex_spatial",
+    "prompt": "A blue book stands behind a black vase, while a glass bottle is positioned to their left. The black vase partially overlaps the blue book from the camera viewpoint. Exactly 5 major objects are visible in the composition. Spatial variant 15."
+  },
+  {
+    "category": "complex_spatial",
+    "prompt": "A transparent cube stands behind a red cylinder, while a metal box is positioned to their left. The red cylinder partially overlaps the transparent cube from the camera viewpoint. Exactly 5 major objects are visible in the composition. Spatial variant 7."
+  },
+  {
+    "category": "long_composition",
+    "prompt": "A seated woman stands at the left side of a glass table while a young woman sits to the right. The standing person holds a metal box in the left hand and points toward a sign reading exactly \"PLATFORM 11\" with the right hand. A blue book and a red cylinder lie on the table in that order from left to right. The scene is illuminated by soft diffused window light. A large mirror behind both people reflects the seated person's back and one object outside the direct camera frame. Exactly 5 major foreground objects should remain clearly distinguishable. Long-scene variant 13."
+  },
+  {
+    "category": "reflection_occlusion_ood",
+    "prompt": "A woman with short hair stands behind a transparent glass panel. A blue book in the foreground partially occludes the torso while the face and both hands remain visible. A mirror behind the person reflects a red cylinder located outside the direct camera frame. The glass also contains a faint reflection of an overhead light at position 5. Reflection variant 5."
+  },
+  {
+    "category": "reflection_occlusion_ood",
+    "prompt": "A man wearing a gray coat stands behind a transparent glass panel. A wooden chair in the foreground partially occludes the torso while the face and both hands remain visible. A mirror behind the person reflects a small lamp located outside the direct camera frame. The glass also contains a faint reflection of an overhead light at position 4. Reflection variant 4."
+  },
+  {
+    "category": "complex_text",
+    "prompt": "Inside a modern kitchen, the exact phrase \"RIVER HOTEL\" is printed clearly on a rectangular sign. A person stands partly in front of the sign without covering any letters. A reflective surface beside the sign shows a reversed reflection of part of the scene while the original text remains completely readable. Scene variant 7."
+  },
+  {
+    "category": "complex_spatial",
+    "prompt": "A metal box stands behind a silver sphere, while a small lamp is positioned to their left. The silver sphere partially overlaps the metal box from the camera viewpoint. Exactly 3 major objects are visible in the composition. Spatial variant 13."
+  },
+  {
+    "category": "complex_material_light",
+    "prompt": "A glass bottle made from polished chrome rests on a surface made from wet black stone, illuminated by warm light entering from the left. The scene clearly shows reflection, roughness, transparency, or subsurface behavior appropriate to both materials. A small colored object is reflected near the edge of the surface. Material-light variant 17."
+  },
+  {
+    "category": "architecture_vehicle",
+    "prompt": "A blue bicycle passes beside a small workshop. A staircase rises on the left side and turns at an upper landing. Three architectural openings are visible at different depths, and a glass facade reflects a building across the street. Composition variant 19."
+  },
+  {
+    "category": "complex_spatial",
+    "prompt": "A black vase stands behind a glass bottle, while a wooden chair is positioned to their right. The glass bottle partially overlaps the black vase from the camera viewpoint. Exactly 6 major objects are visible in the composition. Spatial variant 8."
+  },
+  {
+    "category": "long_composition",
+    "prompt": "A woman wearing a dark jacket stands at the left side of a glass table while an elderly woman sits to the right. The standing person holds a ceramic cup in the left hand and points toward a sign reading exactly \"ROOM 314\" with the right hand. A wooden chair and a small lamp lie on the table in that order from left to right. The scene is illuminated by a narrow spotlight from above. A large mirror behind both people reflects the seated person's back and one object outside the direct camera frame. Exactly 8 major foreground objects should remain clearly distinguishable. Long-scene variant 12."
+  },
+  {
+    "category": "complex_counting",
+    "prompt": "Exactly 6 small lamps form two staggered rows, while exactly 2 metal boxs occupy the background. None of the objects is completely hidden, and one small yellow marker is located beside object number 1. Counting variant 19."
+  },
+  {
+    "category": "complex_anatomy",
+    "prompt": "An elderly man crosses the right leg over the left while looking over the left shoulder. A seated woman stands behind them and points toward the object with the left hand. Both people's hands and feet remain visible. Anatomy variant 2."
+  },
+  {
+    "category": "complex_anatomy",
+    "prompt": "A tall man reaches forward with the right arm while keeping the left hand behind the back. A man wearing a gray coat stands beside them and points toward the object with the left hand. Both people's hands and feet remain visible. Anatomy variant 3."
+  },
+  {
+    "category": "complex_counting",
+    "prompt": "Exactly 4 transparent cubes form two staggered rows, while exactly 2 glass bottles occupy the background. None of the objects is completely hidden, and one small yellow marker is located beside object number 1. Counting variant 17."
+  },
+  {
+    "category": "architecture_vehicle",
+    "prompt": "A red compact car passes beside a small workshop. A staircase rises on the left side and turns at an upper landing. Three architectural openings are visible at different depths, and a glass facade reflects a building across the street. Composition variant 11."
+  },
+  {
+    "category": "long_composition",
+    "prompt": "A man wearing a gray coat stands at the left side of a glass table while an elderly man sits to the right. The standing person holds a silver sphere in the left hand and points toward a sign reading exactly \"STUDIO C\" with the right hand. A black vase and a metal box lie on the table in that order from left to right. The scene is illuminated by hard side lighting. A large mirror behind both people reflects the seated person's back and one object outside the direct camera frame. Exactly 6 major foreground objects should remain clearly distinguishable. Long-scene variant 6."
+  },
+  {
+    "category": "complex_spatial",
+    "prompt": "A glass bottle stands behind a wooden chair, while a transparent cube is positioned to their left. The wooden chair partially overlaps the glass bottle from the camera viewpoint. Exactly 5 major objects are visible in the composition. Spatial variant 11."
+  },
+  {
+    "category": "complex_spatial",
+    "prompt": "A red cylinder stands behind a metal box, while a silver sphere is positioned to their right. The metal box partially overlaps the red cylinder from the camera viewpoint. Exactly 6 major objects are visible in the composition. Spatial variant 20."
+  },
+  {
+    "category": "long_composition",
+    "prompt": "A man wearing a gray coat stands at the left side of a glass table while an elderly man sits to the right. The standing person holds a wooden chair in the left hand and points toward a sign reading exactly \"WEST EXIT\" with the right hand. A silver sphere and a glass bottle lie on the table in that order from left to right. The scene is illuminated by hard side lighting. A large mirror behind both people reflects the seated person's back and one object outside the direct camera frame. Exactly 6 major foreground objects should remain clearly distinguishable. Long-scene variant 14."
+  },
+  {
+    "category": "long_composition",
+    "prompt": "An elderly woman stands at the left side of a glass table while a woman wearing a dark jacket sits to the right. The standing person holds a silver sphere in the left hand and points toward a sign reading exactly \"STUDIO C\" with the right hand. A black vase and a metal box lie on the table in that order from left to right. The scene is illuminated by two opposing light sources. A large mirror behind both people reflects the seated person's back and one object outside the direct camera frame. Exactly 8 major foreground objects should remain clearly distinguishable. Long-scene variant 16."
+  },
+  {
+    "category": "reflection_occlusion_ood",
+    "prompt": "A young woman stands behind a transparent glass panel. A blue book in the foreground partially occludes the torso while the face and both hands remain visible. A mirror behind the person reflects a red cylinder located outside the direct camera frame. The glass also contains a faint reflection of an overhead light at position 15. Reflection variant 15."
+  },
+  {
+    "category": "complex_anatomy",
+    "prompt": "An elderly woman reaches forward with the right arm while keeping the left hand behind the back. A tall man stands behind them and points toward the object with the left hand. Both people's hands and feet remain visible. Anatomy variant 8."
+  },
+  {
+    "category": "architecture_vehicle",
+    "prompt": "A silver tram passes beside a glass office lobby. A staircase rises on the right side and turns at an upper landing. Three architectural openings are visible at different depths, and a glass facade reflects a building across the street. Composition variant 10."
+  },
+  {
+    "category": "complex_text",
+    "prompt": "Inside a industrial studio, the exact phrase \"WEST EXIT\" is printed clearly on a rectangular sign. A person stands partly in front of the sign without covering any letters. A reflective surface beside the sign shows a reversed reflection of part of the scene while the original text remains completely readable. Scene variant 14."
+  },
+  {
+    "category": "complex_counting",
+    "prompt": "Exactly 5 black vases form a curved row, while exactly 4 ceramic cups occupy the background. None of the objects is completely hidden, and one small yellow marker is located beside object number 3. Counting variant 18."
+  },
+  {
+    "category": "complex_material_light",
+    "prompt": "A silver sphere made from frosted glass rests on a surface made from translucent amber acrylic, illuminated by cool light entering from the right. The scene clearly shows reflection, roughness, transparency, or subsurface behavior appropriate to both materials. A small colored object is reflected near the edge of the surface. Material-light variant 2."
+  },
+  {
+    "category": "complex_material_light",
+    "prompt": "A small lamp made from translucent amber acrylic rests on a surface made from dark polished wood, illuminated by soft diffused window light. The scene clearly shows reflection, roughness, transparency, or subsurface behavior appropriate to both materials. A small colored object is reflected near the edge of the surface. Material-light variant 5."
+  },
+  {
+    "category": "reflection_occlusion_ood",
+    "prompt": "An elderly man stands behind a transparent glass panel. A black vase in the foreground partially occludes the torso while the face and both hands remain visible. A mirror behind the person reflects a metal box located outside the direct camera frame. The glass also contains a faint reflection of an overhead light at position 8. Reflection variant 8."
+  },
+  {
+    "category": "complex_counting",
+    "prompt": "Exactly 5 metal boxs form two staggered rows, while exactly 2 transparent cubes occupy the background. None of the objects is completely hidden, and one small yellow marker is located beside object number 3. Counting variant 13."
+  },
+  {
+    "category": "complex_text",
+    "prompt": "Inside a modern kitchen, the exact phrase \"OPEN 24 HOURS\" is printed clearly on a rectangular sign. A person stands partly in front of the sign without covering any letters. A reflective surface beside the sign shows a reversed reflection of part of the scene while the original text remains completely readable. Scene variant 15."
+  },
+  {
+    "category": "reflection_occlusion_ood",
+    "prompt": "A seated woman stands behind a transparent glass panel. A glass bottle in the foreground partially occludes the torso while the face and both hands remain visible. A mirror behind the person reflects a silver sphere located outside the direct camera frame. The glass also contains a faint reflection of an overhead light at position 11. Reflection variant 11."
+  },
+  {
+    "category": "reflection_occlusion_ood",
+    "prompt": "A man wearing a gray coat stands behind a transparent glass panel. A red cylinder in the foreground partially occludes the torso while the face and both hands remain visible. A mirror behind the person reflects a blue book located outside the direct camera frame. The glass also contains a faint reflection of an overhead light at position 20. Reflection variant 20."
+  },
+  {
+    "category": "long_composition",
+    "prompt": "A young woman stands at the left side of a glass table while a seated woman sits to the right. The standing person holds a small lamp in the left hand and points toward a sign reading exactly \"AUTHORIZED ENTRY\" with the right hand. A glass bottle and a silver sphere lie on the table in that order from left to right. The scene is illuminated by warm light entering from the left. A large mirror behind both people reflects the seated person's back and one object outside the direct camera frame. Exactly 5 major foreground objects should remain clearly distinguishable. Long-scene variant 9."
+  },
+  {
+    "category": "architecture_vehicle",
+    "prompt": "A black motorcycle passes beside a glass office lobby. A staircase rises on the right side and turns at an upper landing. Three architectural openings are visible at different depths, and a glass facade reflects a building across the street. Composition variant 18."
+  },
+  {
+    "category": "complex_text",
+    "prompt": "Inside a small workshop, the exact phrase \"CAFE LEVEL 2\" is printed clearly on a rectangular sign. A person stands partly in front of the sign without covering any letters. A reflective surface beside the sign shows a reversed reflection of part of the scene while the original text remains completely readable. Scene variant 18."
+  },
+  {
+    "category": "complex_text",
+    "prompt": "Inside a glass office lobby, the exact phrase \"RIVER HOTEL\" is printed clearly on a rectangular sign. A person stands partly in front of the sign without covering any letters. A reflective surface beside the sign shows a reversed reflection of part of the scene while the original text remains completely readable. Scene variant 17."
+  },
+  {
+    "category": "complex_counting",
+    "prompt": "Exactly 5 black vases form a curved row, while exactly 4 ceramic cups occupy the background. None of the objects is completely hidden, and one small yellow marker is located beside object number 3. Counting variant 8."
+  },
+  {
+    "category": "complex_anatomy",
+    "prompt": "A tall man raises the left hand while holding a cup in the right hand. A man wearing a gray coat stands beside them and points toward the object with the left hand. Both people's hands and feet remain visible. Anatomy variant 11."
+  },
+  {
+    "category": "reflection_occlusion_ood",
+    "prompt": "An elderly woman stands behind a transparent glass panel. A wooden chair in the foreground partially occludes the torso while the face and both hands remain visible. A mirror behind the person reflects a small lamp located outside the direct camera frame. The glass also contains a faint reflection of an overhead light at position 14. Reflection variant 14."
+  },
+  {
+    "category": "complex_text",
+    "prompt": "Inside a railway platform, the exact phrase \"STUDIO C\" is printed clearly on a rectangular sign. A person stands partly in front of the sign without covering any letters. A reflective surface beside the sign shows a reversed reflection of part of the scene while the original text remains completely readable. Scene variant 16."
+  },
+  {
+    "category": "architecture_vehicle",
+    "prompt": "A silver tram passes beside a underground station. A staircase rises on the left side and turns at an upper landing. Three architectural openings are visible at different depths, and a glass facade reflects a building across the street. Composition variant 5."
+  },
+  {
+    "category": "complex_anatomy",
+    "prompt": "A woman wearing a dark jacket holds a bottle with both hands directly in front of the chest. A woman with short hair stands behind them and points toward the object with the right hand. Both people's hands and feet remain visible. Anatomy variant 4."
+  },
+  {
+    "category": "architecture_vehicle",
+    "prompt": "A black motorcycle passes beside a modern kitchen. A staircase rises on the right side and turns at an upper landing. Three architectural openings are visible at different depths, and a glass facade reflects a building across the street. Composition variant 8."
+  },
+  {
+    "category": "complex_text",
+    "prompt": "Inside a glass office lobby, the exact phrase \"NORTH GATE\" is printed clearly on a rectangular sign. A person stands partly in front of the sign without covering any letters. A reflective surface beside the sign shows a reversed reflection of part of the scene while the original text remains completely readable. Scene variant 1."
+  },
+  {
+    "category": "complex_text",
+    "prompt": "Inside a hotel entrance, the exact phrase \"AUTHORIZED ENTRY\" is printed clearly on a rectangular sign. A person stands partly in front of the sign without covering any letters. A reflective surface beside the sign shows a reversed reflection of part of the scene while the original text remains completely readable. Scene variant 19."
+  },
+  {
+    "category": "reflection_occlusion_ood",
+    "prompt": "A seated woman stands behind a transparent glass panel. A small lamp in the foreground partially occludes the torso while the face and both hands remain visible. A mirror behind the person reflects a wooden chair located outside the direct camera frame. The glass also contains a faint reflection of an overhead light at position 19. Reflection variant 19."
+  },
+  {
+    "category": "long_composition",
+    "prompt": "A young woman stands at the left side of a glass table while a seated woman sits to the right. The standing person holds a glass bottle in the left hand and points toward a sign reading exactly \"NORTH GATE\" with the right hand. A metal box and a black vase lie on the table in that order from left to right. The scene is illuminated by warm light entering from the left. A large mirror behind both people reflects the seated person's back and one object outside the direct camera frame. Exactly 5 major foreground objects should remain clearly distinguishable. Long-scene variant 1."
+  },
+  {
+    "category": "complex_material_light",
+    "prompt": "A red cylinder made from dark polished wood rests on a surface made from brushed aluminum, illuminated by two opposing light sources. The scene clearly shows reflection, roughness, transparency, or subsurface behavior appropriate to both materials. A small colored object is reflected near the edge of the surface. Material-light variant 16."
+  },
+  {
+    "category": "complex_anatomy",
+    "prompt": "A woman with short hair crosses the right leg over the left while looking over the left shoulder. An elderly man stands beside them and points toward the object with the right hand. Both people's hands and feet remain visible. Anatomy variant 7."
+  },
+  {
+    "category": "reflection_occlusion_ood",
+    "prompt": "A tall man stands behind a transparent glass panel. A transparent cube in the foreground partially occludes the torso while the face and both hands remain visible. A mirror behind the person reflects a ceramic cup located outside the direct camera frame. The glass also contains a faint reflection of an overhead light at position 17. Reflection variant 17."
+  },
+  {
+    "category": "complex_counting",
+    "prompt": "Exactly 7 blue books form two staggered rows, while exactly 2 small lamps occupy the background. None of the objects is completely hidden, and one small yellow marker is located beside object number 1. Counting variant 15."
+  },
+  {
+    "category": "complex_counting",
+    "prompt": "Exactly 7 red cylinders form a curved row, while exactly 4 wooden chairs occupy the background. None of the objects is completely hidden, and one small yellow marker is located beside object number 6. Counting variant 20."
+  },
+  {
+    "category": "architecture_vehicle",
+    "prompt": "A white city bus passes beside a hotel entrance. A staircase rises on the right side and turns at an upper landing. Three architectural openings are visible at different depths, and a glass facade reflects a building across the street. Composition variant 12."
+  },
+  {
+    "category": "long_composition",
+    "prompt": "A woman with short hair stands at the left side of a glass table while a tall man sits to the right. The standing person holds a blue book in the left hand and points toward a sign reading exactly \"OPEN 24 HOURS\" with the right hand. A transparent cube and a ceramic cup lie on the table in that order from left to right. The scene is illuminated by warm reflected light from below. A large mirror behind both people reflects the seated person's back and one object outside the direct camera frame. Exactly 7 major foreground objects should remain clearly distinguishable. Long-scene variant 15."
+  },
+  {
+    "category": "complex_material_light",
+    "prompt": "A transparent cube made from translucent amber acrylic rests on a surface made from dark polished wood, illuminated by soft diffused window light. The scene clearly shows reflection, roughness, transparency, or subsurface behavior appropriate to both materials. A small colored object is reflected near the edge of the surface. Material-light variant 13."
+  },
+  {
+    "category": "complex_anatomy",
+    "prompt": "A seated woman turns the torso right while the head remains facing left. An elderly woman stands beside them and points toward the object with the left hand. Both people's hands and feet remain visible. Anatomy variant 5."
+  },
+  {
+    "category": "reflection_occlusion_ood",
+    "prompt": "A woman wearing a dark jacket stands behind a transparent glass panel. A black vase in the foreground partially occludes the torso while the face and both hands remain visible. A mirror behind the person reflects a metal box located outside the direct camera frame. The glass also contains a faint reflection of an overhead light at position 18. Reflection variant 18."
+  },
+  {
+    "category": "complex_spatial",
+    "prompt": "A metal box stands behind a silver sphere, while a small lamp is positioned to their left. The silver sphere partially overlaps the metal box from the camera viewpoint. Exactly 5 major objects are visible in the composition. Spatial variant 3."
+  },
+  {
+    "category": "complex_text",
+    "prompt": "Inside a glass office lobby, the exact phrase \"AUTHORIZED ENTRY\" is printed clearly on a rectangular sign. A person stands partly in front of the sign without covering any letters. A reflective surface beside the sign shows a reversed reflection of part of the scene while the original text remains completely readable. Scene variant 9."
+  },
+  {
+    "category": "complex_anatomy",
+    "prompt": "A young woman holds a bottle with both hands directly in front of the chest. A woman wearing a dark jacket stands beside them and points toward the object with the left hand. Both people's hands and feet remain visible. Anatomy variant 9."
+  },
+  {
+    "category": "complex_material_light",
+    "prompt": "A blue book made from polished chrome rests on a surface made from wet black stone, illuminated by warm light entering from the left. The scene clearly shows reflection, roughness, transparency, or subsurface behavior appropriate to both materials. A small colored object is reflected near the edge of the surface. Material-light variant 1."
+  },
+  {
+    "category": "long_composition",
+    "prompt": "An elderly woman stands at the left side of a glass table while a woman wearing a dark jacket sits to the right. The standing person holds a black vase in the left hand and points toward a sign reading exactly \"CAFE LEVEL 2\" with the right hand. A red cylinder and a blue book lie on the table in that order from left to right. The scene is illuminated by two opposing light sources. A large mirror behind both people reflects the seated person's back and one object outside the direct camera frame. Exactly 8 major foreground objects should remain clearly distinguishable. Long-scene variant 8."
+  },
+  {
+    "category": "complex_spatial",
+    "prompt": "A transparent cube stands behind a red cylinder, while a metal box is positioned to their left. The red cylinder partially overlaps the transparent cube from the camera viewpoint. Exactly 3 major objects are visible in the composition. Spatial variant 17."
+  },
+  {
+    "category": "complex_material_light",
+    "prompt": "A red cylinder made from glossy ceramic rests on a surface made from polished chrome, illuminated by hard side lighting. The scene clearly shows reflection, roughness, transparency, or subsurface behavior appropriate to both materials. A small colored object is reflected near the edge of the surface. Material-light variant 6."
+  },
+  {
+    "category": "complex_counting",
+    "prompt": "Exactly 4 ceramic cups form a curved row, while exactly 4 silver spheres occupy the background. None of the objects is completely hidden, and one small yellow marker is located beside object number 2. Counting variant 2."
+  },
+  {
+    "category": "long_composition",
+    "prompt": "A tall man stands at the left side of a glass table while a woman with short hair sits to the right. The standing person holds a glass bottle in the left hand and points toward a sign reading exactly \"NORTH GATE\" with the right hand. A metal box and a black vase lie on the table in that order from left to right. The scene is illuminated by strong backlight. A large mirror behind both people reflects the seated person's back and one object outside the direct camera frame. Exactly 7 major foreground objects should remain clearly distinguishable. Long-scene variant 11."
+  },
+  {
+    "category": "reflection_occlusion_ood",
+    "prompt": "A seated woman stands behind a transparent glass panel. A metal box in the foreground partially occludes the torso while the face and both hands remain visible. A mirror behind the person reflects a black vase located outside the direct camera frame. The glass also contains a faint reflection of an overhead light at position 3. Reflection variant 3."
+  },
+  {
+    "category": "complex_spatial",
+    "prompt": "A red cylinder stands behind a metal box, while a silver sphere is positioned to their right. The metal box partially overlaps the red cylinder from the camera viewpoint. Exactly 4 major objects are visible in the composition. Spatial variant 10."
+  },
+  {
+    "category": "complex_counting",
+    "prompt": "Exactly 4 transparent cubes form two staggered rows, while exactly 2 glass bottles occupy the background. None of the objects is completely hidden, and one small yellow marker is located beside object number 3. Counting variant 7."
+  },
+  {
+    "category": "complex_anatomy",
+    "prompt": "An elderly woman raises the left hand while holding a cup in the right hand. A tall man stands behind them and points toward the object with the right hand. Both people's hands and feet remain visible. Anatomy variant 16."
+  },
+  {
+    "category": "reflection_occlusion_ood",
+    "prompt": "A woman wearing a dark jacket stands behind a transparent glass panel. A red cylinder in the foreground partially occludes the torso while the face and both hands remain visible. A mirror behind the person reflects a blue book located outside the direct camera frame. The glass also contains a faint reflection of an overhead light at position 10. Reflection variant 10."
+  },
+  {
+    "category": "long_composition",
+    "prompt": "A young woman stands at the left side of a glass table while a seated woman sits to the right. The standing person holds a transparent cube in the left hand and points toward a sign reading exactly \"RIVER HOTEL\" with the right hand. A small lamp and a wooden chair lie on the table in that order from left to right. The scene is illuminated by warm light entering from the left. A large mirror behind both people reflects the seated person's back and one object outside the direct camera frame. Exactly 5 major foreground objects should remain clearly distinguishable. Long-scene variant 17."
+  },
+  {
+    "category": "complex_material_light",
+    "prompt": "A black vase made from wet black stone rests on a surface made from rough concrete, illuminated by a narrow spotlight from above. The scene clearly shows reflection, roughness, transparency, or subsurface behavior appropriate to both materials. A small colored object is reflected near the edge of the surface. Material-light variant 4."
+  },
+  {
+    "category": "complex_text",
+    "prompt": "Inside a railway platform, the exact phrase \"CAFE LEVEL 2\" is printed clearly on a rectangular sign. A person stands partly in front of the sign without covering any letters. A reflective surface beside the sign shows a reversed reflection of part of the scene while the original text remains completely readable. Scene variant 8."
+  },
+  {
+    "category": "complex_anatomy",
+    "prompt": "An elderly man turns the torso right while the head remains facing left. A seated woman stands behind them and points toward the object with the right hand. Both people's hands and feet remain visible. Anatomy variant 10."
+  },
+  {
+    "category": "complex_spatial",
+    "prompt": "A black vase stands behind a glass bottle, while a wooden chair is positioned to their right. The glass bottle partially overlaps the black vase from the camera viewpoint. Exactly 4 major objects are visible in the composition. Spatial variant 18."
+  },
+  {
+    "category": "complex_text",
+    "prompt": "Inside a hotel entrance, the exact phrase \"NORTH GATE\" is printed clearly on a rectangular sign. A person stands partly in front of the sign without covering any letters. A reflective surface beside the sign shows a reversed reflection of part of the scene while the original text remains completely readable. Scene variant 11."
+  },
+  {
+    "category": "architecture_vehicle",
+    "prompt": "A blue bicycle passes beside a hotel entrance. A staircase rises on the right side and turns at an upper landing. Three architectural openings are visible at different depths, and a glass facade reflects a building across the street. Composition variant 4."
+  },
+  {
+    "category": "long_composition",
+    "prompt": "A tall man stands at the left side of a glass table while a woman with short hair sits to the right. The standing person holds a small lamp in the left hand and points toward a sign reading exactly \"AUTHORIZED ENTRY\" with the right hand. A glass bottle and a silver sphere lie on the table in that order from left to right. The scene is illuminated by strong backlight. A large mirror behind both people reflects the seated person's back and one object outside the direct camera frame. Exactly 7 major foreground objects should remain clearly distinguishable. Long-scene variant 19."
+  },
+  {
+    "category": "complex_spatial",
+    "prompt": "A blue book stands behind a black vase, while a glass bottle is positioned to their left. The black vase partially overlaps the blue book from the camera viewpoint. Exactly 3 major objects are visible in the composition. Spatial variant 5."
+  },
+  {
+    "category": "architecture_vehicle",
+    "prompt": "A red compact car passes beside a minimalist living room. A staircase rises on the right side and turns at an upper landing. Three architectural openings are visible at different depths, and a glass facade reflects a building across the street. Composition variant 6."
+  },
+  {
+    "category": "complex_material_light",
+    "prompt": "A metal box made from brushed aluminum rests on a surface made from glossy ceramic, illuminated by strong backlight. The scene clearly shows reflection, roughness, transparency, or subsurface behavior appropriate to both materials. A small colored object is reflected near the edge of the surface. Material-light variant 19."
+  },
+  {
+    "category": "complex_material_light",
+    "prompt": "A small lamp made from rough concrete rests on a surface made from frosted glass, illuminated by warm reflected light from below. The scene clearly shows reflection, roughness, transparency, or subsurface behavior appropriate to both materials. A small colored object is reflected near the edge of the surface. Material-light variant 15."
+  },
+  {
+    "category": "complex_anatomy",
+    "prompt": "A woman wearing a dark jacket crosses the right leg over the left while looking over the left shoulder. A woman with short hair stands behind them and points toward the object with the left hand. Both people's hands and feet remain visible. Anatomy variant 12."
+  },
+  {
+    "category": "complex_counting",
+    "prompt": "Exactly 4 ceramic cups form a curved row, while exactly 4 silver spheres occupy the background. None of the objects is completely hidden, and one small yellow marker is located beside object number 4. Counting variant 12."
+  },
+  {
+    "category": "complex_counting",
+    "prompt": "Exactly 7 red cylinders form a curved row, while exactly 4 wooden chairs occupy the background. None of the objects is completely hidden, and one small yellow marker is located beside object number 3. Counting variant 10."
+  },
+  {
+    "category": "complex_anatomy",
+    "prompt": "A woman with short hair turns the torso right while the head remains facing left. An elderly man stands beside them and points toward the object with the left hand. Both people's hands and feet remain visible. Anatomy variant 15."
+  },
+  {
+    "category": "reflection_occlusion_ood",
+    "prompt": "An elderly woman stands behind a transparent glass panel. A silver sphere in the foreground partially occludes the torso while the face and both hands remain visible. A mirror behind the person reflects a glass bottle located outside the direct camera frame. The glass also contains a faint reflection of an overhead light at position 6. Reflection variant 6."
+  },
+  {
+    "category": "architecture_vehicle",
+    "prompt": "A blue bicycle passes beside a minimalist living room. A staircase rises on the right side and turns at an upper landing. Three architectural openings are visible at different depths, and a glass facade reflects a building across the street. Composition variant 14."
+  },
+  {
+    "category": "complex_text",
+    "prompt": "Inside a hotel entrance, the exact phrase \"PLATFORM 11\" is printed clearly on a rectangular sign. A person stands partly in front of the sign without covering any letters. A reflective surface beside the sign shows a reversed reflection of part of the scene while the original text remains completely readable. Scene variant 3."
+  },
+  {
+    "category": "reflection_occlusion_ood",
+    "prompt": "A tall man stands behind a transparent glass panel. A small lamp in the foreground partially occludes the torso while the face and both hands remain visible. A mirror behind the person reflects a wooden chair located outside the direct camera frame. The glass also contains a faint reflection of an overhead light at position 9. Reflection variant 9."
+  },
+  {
+    "category": "complex_counting",
+    "prompt": "Exactly 6 wooden chairs form a curved row, while exactly 4 black vases occupy the background. None of the objects is completely hidden, and one small yellow marker is located beside object number 4. Counting variant 4."
+  },
+  {
+    "category": "complex_material_light",
+    "prompt": "A wooden chair made from wet black stone rests on a surface made from rough concrete, illuminated by a narrow spotlight from above. The scene clearly shows reflection, roughness, transparency, or subsurface behavior appropriate to both materials. A small colored object is reflected near the edge of the surface. Material-light variant 20."
+  },
+  {
+    "category": "complex_anatomy",
+    "prompt": "A woman wearing a dark jacket turns the torso right while the head remains facing left. A woman with short hair stands behind them and points toward the object with the left hand. Both people's hands and feet remain visible. Anatomy variant 20."
+  },
+  {
+    "category": "complex_text",
+    "prompt": "Inside a small workshop, the exact phrase \"FINAL STOP\" is printed clearly on a rectangular sign. A person stands partly in front of the sign without covering any letters. A reflective surface beside the sign shows a reversed reflection of part of the scene while the original text remains completely readable. Scene variant 10."
+  },
+  {
+    "category": "architecture_vehicle",
+    "prompt": "A silver tram passes beside a hotel entrance. A staircase rises on the right side and turns at an upper landing. Three architectural openings are visible at different depths, and a glass facade reflects a building across the street. Composition variant 20."
+  }
+]

research/datasets/bridge_prompts_480.json

@@ -0,0 +1,1762 @@
+[
+  {
+    "category": "lighting",
+    "prompt": "a cup illuminated by strong backlight, with clearly visible light direction and shadow"
+  },
+  {
+    "category": "anatomy",
+    "prompt": "an elderly man reaching forward with their right arm, full body clearly visible"
+  },
+  {
+    "category": "reflection_occlusion",
+    "prompt": "one person partially occluded by another person"
+  },
+  {
+    "category": "anatomy",
+    "prompt": "a woman turning their head to the left, full body clearly visible"
+  },
+  {
+    "category": "reflection_occlusion",
+    "prompt": "a woman reflected accurately in a wall mirror"
+  },
+  {
+    "category": "spatial",
+    "prompt": "a white book positioned below a black bottle"
+  },
+  {
+    "category": "lighting",
+    "prompt": "a glass illuminated by strong backlight, with clearly visible light direction and shadow"
+  },
+  {
+    "category": "spatial",
+    "prompt": "a red book positioned below a yellow vase"
+  },
+  {
+    "category": "reflection_occlusion",
+    "prompt": "a translucent curtain illuminated from behind"
+  },
+  {
+    "category": "counting",
+    "prompt": "2 wooden blocks arranged in the foreground with 3 bottles behind them"
+  },
+  {
+    "category": "spatial",
+    "prompt": "a green book positioned below a red glass"
+  },
+  {
+    "category": "spatial",
+    "prompt": "a green chair positioned beside a orange sphere"
+  },
+  {
+    "category": "anatomy",
+    "prompt": "an elderly man standing with one arm behind their back, full body clearly visible"
+  },
+  {
+    "category": "counting",
+    "prompt": "5 lamps arranged in the foreground with 3 books behind them"
+  },
+  {
+    "category": "counting",
+    "prompt": "4 tables arranged in the foreground with 4 mirrors behind them"
+  },
+  {
+    "category": "spatial",
+    "prompt": "a red cube positioned partially behind a white metal cylinder"
+  },
+  {
+    "category": "lighting",
+    "prompt": "a vase illuminated by warm light reflected from below, with clearly visible light direction and shadow"
+  },
+  {
+    "category": "lighting",
+    "prompt": "a bottle illuminated by strong backlight, with clearly visible light direction and shadow"
+  },
+  {
+    "category": "anatomy",
+    "prompt": "a man standing with one arm behind their back, full body clearly visible"
+  },
+  {
+    "category": "reflection_occlusion",
+    "prompt": "one person partially occluded by another person"
+  },
+  {
+    "category": "anatomy",
+    "prompt": "an elderly woman touching their face with their left hand, full body clearly visible"
+  },
+  {
+    "category": "text",
+    "prompt": "the word \"STUDIO\" printed clearly and correctly on a white rectangular sign"
+  },
+  {
+    "category": "lighting",
+    "prompt": "a mirror illuminated by soft light from the left, with clearly visible light direction and shadow"
+  },
+  {
+    "category": "spatial",
+    "prompt": "a red vase positioned partially behind a golden lamp"
+  },
+  {
+    "category": "material",
+    "prompt": "a table made of polished chrome, showing realistic surface properties and reflections"
+  },
+  {
+    "category": "text",
+    "prompt": "the word \"EXIT\" printed clearly and correctly on a metal street sign"
+  },
+  {
+    "category": "reflection_occlusion",
+    "prompt": "one person partially occluded by another person"
+  },
+  {
+    "category": "spatial",
+    "prompt": "a white wooden block positioned to the right of a purple mirror"
+  },
+  {
+    "category": "counting",
+    "prompt": "5 cubes arranged in the foreground with 4 plates behind them"
+  },
+  {
+    "category": "lighting",
+    "prompt": "a metal cylinder illuminated by warm overhead light, with clearly visible light direction and shadow"
+  },
+  {
+    "category": "material",
+    "prompt": "a glass made of wet concrete, showing realistic surface properties and reflections"
+  },
+  {
+    "category": "lighting",
+    "prompt": "a box illuminated by cool side lighting, with clearly visible light direction and shadow"
+  },
+  {
+    "category": "spatial",
+    "prompt": "a purple metal cylinder positioned beside a red wooden block"
+  },
+  {
+    "category": "material",
+    "prompt": "a chair made of polished chrome, showing realistic surface properties and reflections"
+  },
+  {
+    "category": "lighting",
+    "prompt": "a glass illuminated by soft diffused daylight, with clearly visible light direction and shadow"
+  },
+  {
+    "category": "anatomy",
+    "prompt": "an elderly man sitting with both feet visible, full body clearly visible"
+  },
+  {
+    "category": "lighting",
+    "prompt": "a sphere illuminated by warm light reflected from below, with clearly visible light direction and shadow"
+  },
+  {
+    "category": "lighting",
+    "prompt": "a cube illuminated by soft diffused daylight, with clearly visible light direction and shadow"
+  },
+  {
+    "category": "text",
+    "prompt": "the word \"STOP\" printed clearly and correctly on a white rectangular sign"
+  },
+  {
+    "category": "counting",
+    "prompt": "4 metal cylinders arranged in the foreground with 4 cups behind them"
+  },
+  {
+    "category": "text",
+    "prompt": "the word \"CLOSED\" printed clearly and correctly on a paper label"
+  },
+  {
+    "category": "lighting",
+    "prompt": "a metal cylinder illuminated by warm light reflected from below, with clearly visible light direction and shadow"
+  },
+  {
+    "category": "anatomy",
+    "prompt": "an elderly man holding an object between both hands, full body clearly visible"
+  },
+  {
+    "category": "lighting",
+    "prompt": "a box illuminated by cool side lighting, with clearly visible light direction and shadow"
+  },
+  {
+    "category": "text",
+    "prompt": "the word \"HELLO\" printed clearly and correctly on a white rectangular sign"
+  },
+  {
+    "category": "spatial",
+    "prompt": "a purple cup positioned to the left of a silver glass"
+  },
+  {
+    "category": "lighting",
+    "prompt": "a box illuminated by hard light from the right, with clearly visible light direction and shadow"
+  },
+  {
+    "category": "text",
+    "prompt": "the word \"HELLO\" printed clearly and correctly on a black poster"
+  },
+  {
+    "category": "reflection_occlusion",
+    "prompt": "a mirror showing the back of a person facing away"
+  },
+  {
+    "category": "text",
+    "prompt": "the word \"EXIT\" printed clearly and correctly on a metal street sign"
+  },
+  {
+    "category": "text",
+    "prompt": "the word \"HELLO\" printed clearly and correctly on a glass storefront"
+  },
+  {
+    "category": "anatomy",
+    "prompt": "an elderly woman holding an object between both hands, full body clearly visible"
+  },
+  {
+    "category": "anatomy",
+    "prompt": "an elderly woman holding both hands in front of their chest, full body clearly visible"
+  },
+  {
+    "category": "reflection_occlusion",
+    "prompt": "a transparent object casting a faint shadow"
+  },
+  {
+    "category": "anatomy",
+    "prompt": "an elderly man holding both hands in front of their chest, full body clearly visible"
+  },
+  {
+    "category": "counting",
+    "prompt": "2 boxs arranged in the foreground with 4 spheres behind them"
+  },
+  {
+    "category": "spatial",
+    "prompt": "a white mirror positioned above a golden vase"
+  },
+  {
+    "category": "material",
+    "prompt": "a chair made of glossy ceramic, showing realistic surface properties and reflections"
+  },
+  {
+    "category": "text",
+    "prompt": "the word \"EXIT\" printed clearly and correctly on a black poster"
+  },
+  {
+    "category": "anatomy",
+    "prompt": "an elderly woman raising their left hand above their head, full body clearly visible"
+  },
+  {
+    "category": "counting",
+    "prompt": "5 glasss arranged in the foreground with 3 wooden blocks behind them"
+  },
+  {
+    "category": "anatomy",
+    "prompt": "a young woman holding an object between both hands, full body clearly visible"
+  },
+  {
+    "category": "counting",
+    "prompt": "5 chairs arranged in the foreground with 4 wooden blocks behind them"
+  },
+  {
+    "category": "lighting",
+    "prompt": "a wooden block illuminated by warm light reflected from below, with clearly visible light direction and shadow"
+  },
+  {
+    "category": "lighting",
+    "prompt": "a sphere illuminated by hard light from the right, with clearly visible light direction and shadow"
+  },
+  {
+    "category": "lighting",
+    "prompt": "a vase illuminated by soft light from the left, with clearly visible light direction and shadow"
+  },
+  {
+    "category": "spatial",
+    "prompt": "a black wooden block positioned behind a white glass"
+  },
+  {
+    "category": "material",
+    "prompt": "a cup made of rough stone, showing realistic surface properties and reflections"
+  },
+  {
+    "category": "spatial",
+    "prompt": "a golden mirror positioned to the right of a red metal cylinder"
+  },
+  {
+    "category": "spatial",
+    "prompt": "a purple cup positioned to the left of a blue cube"
+  },
+  {
+    "category": "lighting",
+    "prompt": "a sphere illuminated by soft diffused daylight, with clearly visible light direction and shadow"
+  },
+  {
+    "category": "material",
+    "prompt": "a box made of translucent acrylic, showing realistic surface properties and reflections"
+  },
+  {
+    "category": "text",
+    "prompt": "the word \"OPEN\" printed clearly and correctly on a white rectangular sign"
+  },
+  {
+    "category": "lighting",
+    "prompt": "a vase illuminated by soft diffused daylight, with clearly visible light direction and shadow"
+  },
+  {
+    "category": "spatial",
+    "prompt": "a purple bottle positioned above a green box"
+  },
+  {
+    "category": "lighting",
+    "prompt": "a vase illuminated by a narrow beam of light from above, with clearly visible light direction and shadow"
+  },
+  {
+    "category": "text",
+    "prompt": "the word \"STUDIO\" printed clearly and correctly on a glass storefront"
+  },
+  {
+    "category": "anatomy",
+    "prompt": "a woman holding both hands in front of their chest, full body clearly visible"
+  },
+  {
+    "category": "material",
+    "prompt": "a box made of dark wood, showing realistic surface properties and reflections"
+  },
+  {
+    "category": "text",
+    "prompt": "the word \"CLOSED\" printed clearly and correctly on a black poster"
+  },
+  {
+    "category": "material",
+    "prompt": "a vase made of dark wood, showing realistic surface properties and reflections"
+  },
+  {
+    "category": "lighting",
+    "prompt": "a chair illuminated by strong backlight, with clearly visible light direction and shadow"
+  },
+  {
+    "category": "lighting",
+    "prompt": "a lamp illuminated by warm overhead light, with clearly visible light direction and shadow"
+  },
+  {
+    "category": "reflection_occlusion",
+    "prompt": "a glass sphere resting on a reflective metal surface"
+  },
+  {
+    "category": "lighting",
+    "prompt": "a mirror illuminated by hard light from the right, with clearly visible light direction and shadow"
+  },
+  {
+    "category": "spatial",
+    "prompt": "a black wooden block positioned to the right of a silver vase"
+  },
+  {
+    "category": "lighting",
+    "prompt": "a lamp illuminated by cool side lighting, with clearly visible light direction and shadow"
+  },
+  {
+    "category": "reflection_occlusion",
+    "prompt": "a mirror showing the back of a person facing away"
+  },
+  {
+    "category": "material",
+    "prompt": "a vase made of rough stone, showing realistic surface properties and reflections"
+  },
+  {
+    "category": "spatial",
+    "prompt": "a silver metal cylinder positioned above a green sphere"
+  },
+  {
+    "category": "lighting",
+    "prompt": "a cup illuminated by soft diffused daylight, with clearly visible light direction and shadow"
+  },
+  {
+    "category": "spatial",
+    "prompt": "a yellow bottle positioned below a red lamp"
+  },
+  {
+    "category": "reflection_occlusion",
+    "prompt": "a glass sphere resting on a reflective metal surface"
+  },
+  {
+    "category": "anatomy",
+    "prompt": "a woman reaching forward with their right arm, full body clearly visible"
+  },
+  {
+    "category": "spatial",
+    "prompt": "a green vase positioned behind a blue lamp"
+  },
+  {
+    "category": "reflection_occlusion",
+    "prompt": "a shiny metal object reflecting a nearby red object"
+  },
+  {
+    "category": "spatial",
+    "prompt": "a white sphere positioned to the left of a yellow metal cylinder"
+  },
+  {
+    "category": "spatial",
+    "prompt": "a golden book positioned below a yellow plate"
+  },
+  {
+    "category": "counting",
+    "prompt": "2 vases arranged in the foreground with 4 wooden blocks behind them"
+  },
+  {
+    "category": "spatial",
+    "prompt": "a white book positioned above a purple table"
+  },
+  {
+    "category": "anatomy",
+    "prompt": "a young man reaching forward with their right arm, full body clearly visible"
+  },
+  {
+    "category": "text",
+    "prompt": "the word \"CLOSED\" printed clearly and correctly on a paper label"
+  },
+  {
+    "category": "material",
+    "prompt": "a sphere made of translucent acrylic, showing realistic surface properties and reflections"
+  },
+  {
+    "category": "lighting",
+    "prompt": "a bottle illuminated by a narrow beam of light from above, with clearly visible light direction and shadow"
+  },
+  {
+    "category": "lighting",
+    "prompt": "a metal cylinder illuminated by strong backlight, with clearly visible light direction and shadow"
+  },
+  {
+    "category": "material",
+    "prompt": "a bottle made of glossy ceramic, showing realistic surface properties and reflections"
+  },
+  {
+    "category": "material",
+    "prompt": "a bottle made of transparent glass, showing realistic surface properties and reflections"
+  },
+  {
+    "category": "spatial",
+    "prompt": "a black metal cylinder positioned beside a orange chair"
+  },
+  {
+    "category": "counting",
+    "prompt": "2 bottles arranged in the foreground with 2 mirrors behind them"
+  },
+  {
+    "category": "material",
+    "prompt": "a lamp made of wet concrete, showing realistic surface properties and reflections"
+  },
+  {
+    "category": "text",
+    "prompt": "the word \"CAFE\" printed clearly and correctly on a black poster"
+  },
+  {
+    "category": "anatomy",
+    "prompt": "an elderly man holding both hands in front of their chest, full body clearly visible"
+  },
+  {
+    "category": "reflection_occlusion",
+    "prompt": "a shiny metal object reflecting a nearby red object"
+  },
+  {
+    "category": "reflection_occlusion",
+    "prompt": "a transparent glass bottle in front of a person's face"
+  },
+  {
+    "category": "spatial",
+    "prompt": "a orange vase positioned to the right of a green table"
+  },
+  {
+    "category": "counting",
+    "prompt": "3 mirrors arranged in the foreground with 2 glasss behind them"
+  },
+  {
+    "category": "counting",
+    "prompt": "3 boxs arranged in the foreground with 4 cubes behind them"
+  },
+  {
+    "category": "lighting",
+    "prompt": "a plate illuminated by cool side lighting, with clearly visible light direction and shadow"
+  },
+  {
+    "category": "lighting",
+    "prompt": "a cube illuminated by a narrow beam of light from above, with clearly visible light direction and shadow"
+  },
+  {
+    "category": "counting",
+    "prompt": "4 mirrors arranged in the foreground with 1 cubes behind them"
+  },
+  {
+    "category": "anatomy",
+    "prompt": "an elderly man raising their left hand above their head, full body clearly visible"
+  },
+  {
+    "category": "anatomy",
+    "prompt": "a woman crossing their right leg over their left leg, full body clearly visible"
+  },
+  {
+    "category": "material",
+    "prompt": "a plate made of matte plastic, showing realistic surface properties and reflections"
+  },
+  {
+    "category": "text",
+    "prompt": "the word \"HOTEL\" printed clearly and correctly on a white rectangular sign"
+  },
+  {
+    "category": "lighting",
+    "prompt": "a box illuminated by hard light from the right, with clearly visible light direction and shadow"
+  },
+  {
+    "category": "spatial",
+    "prompt": "a red bottle positioned behind a green mirror"
+  },
+  {
+    "category": "spatial",
+    "prompt": "a black book positioned partially behind a red bottle"
+  },
+  {
+    "category": "spatial",
+    "prompt": "a white metal cylinder positioned partially behind a black plate"
+  },
+  {
+    "category": "counting",
+    "prompt": "4 boxs arranged in the foreground with 2 lamps behind them"
+  },
+  {
+    "category": "anatomy",
+    "prompt": "a young man touching their face with their left hand, full body clearly visible"
+  },
+  {
+    "category": "counting",
+    "prompt": "2 chairs arranged in the foreground with 1 plates behind them"
+  },
+  {
+    "category": "reflection_occlusion",
+    "prompt": "a hand visible through a transparent glass panel"
+  },
+  {
+    "category": "lighting",
+    "prompt": "a mirror illuminated by soft light from the left, with clearly visible light direction and shadow"
+  },
+  {
+    "category": "spatial",
+    "prompt": "a blue box positioned above a black book"
+  },
+  {
+    "category": "text",
+    "prompt": "the word \"BLUE\" printed clearly and correctly on a metal street sign"
+  },
+  {
+    "category": "counting",
+    "prompt": "3 cups arranged in the foreground with 4 mirrors behind them"
+  },
+  {
+    "category": "lighting",
+    "prompt": "a cup illuminated by cool side lighting, with clearly visible light direction and shadow"
+  },
+  {
+    "category": "text",
+    "prompt": "the word \"NORTH\" printed clearly and correctly on a white rectangular sign"
+  },
+  {
+    "category": "material",
+    "prompt": "a wooden block made of wet concrete, showing realistic surface properties and reflections"
+  },
+  {
+    "category": "spatial",
+    "prompt": "a purple bottle positioned to the right of a green lamp"
+  },
+  {
+    "category": "text",
+    "prompt": "the word \"CAFE\" printed clearly and correctly on a paper label"
+  },
+  {
+    "category": "lighting",
+    "prompt": "a vase illuminated by strong backlight, with clearly visible light direction and shadow"
+  },
+  {
+    "category": "lighting",
+    "prompt": "a glass illuminated by a narrow beam of light from above, with clearly visible light direction and shadow"
+  },
+  {
+    "category": "lighting",
+    "prompt": "a metal cylinder illuminated by cool side lighting, with clearly visible light direction and shadow"
+  },
+  {
+    "category": "anatomy",
+    "prompt": "an elderly woman holding both hands in front of their chest, full body clearly visible"
+  },
+  {
+    "category": "spatial",
+    "prompt": "a black sphere positioned beside a blue vase"
+  },
+  {
+    "category": "anatomy",
+    "prompt": "a man holding a bottle in their right hand, full body clearly visible"
+  },
+  {
+    "category": "material",
+    "prompt": "a metal cylinder made of glossy ceramic, showing realistic surface properties and reflections"
+  },
+  {
+    "category": "anatomy",
+    "prompt": "a young man touching their face with their left hand, full body clearly visible"
+  },
+  {
+    "category": "anatomy",
+    "prompt": "an elderly man holding an object between both hands, full body clearly visible"
+  },
+  {
+    "category": "counting",
+    "prompt": "2 boxs arranged in the foreground with 4 lamps behind them"
+  },
+  {
+    "category": "spatial",
+    "prompt": "a yellow glass positioned beside a white vase"
+  },
+  {
+    "category": "material",
+    "prompt": "a box made of rough stone, showing realistic surface properties and reflections"
+  },
+  {
+    "category": "material",
+    "prompt": "a box made of dark wood, showing realistic surface properties and reflections"
+  },
+  {
+    "category": "reflection_occlusion",
+    "prompt": "a transparent object casting a faint shadow"
+  },
+  {
+    "category": "reflection_occlusion",
+    "prompt": "a mirror showing the back of a person facing away"
+  },
+  {
+    "category": "material",
+    "prompt": "a chair made of wet concrete, showing realistic surface properties and reflections"
+  },
+  {
+    "category": "lighting",
+    "prompt": "a lamp illuminated by hard light from the right, with clearly visible light direction and shadow"
+  },
+  {
+    "category": "spatial",
+    "prompt": "a black plate positioned above a yellow box"
+  },
+  {
+    "category": "text",
+    "prompt": "the word \"OPEN\" printed clearly and correctly on a paper label"
+  },
+  {
+    "category": "lighting",
+    "prompt": "a chair illuminated by soft diffused daylight, with clearly visible light direction and shadow"
+  },
+  {
+    "category": "spatial",
+    "prompt": "a white box positioned above a yellow metal cylinder"
+  },
+  {
+    "category": "material",
+    "prompt": "a vase made of rough stone, showing realistic surface properties and reflections"
+  },
+  {
+    "category": "counting",
+    "prompt": "4 bottles arranged in the foreground with 2 boxs behind them"
+  },
+  {
+    "category": "spatial",
+    "prompt": "a orange plate positioned to the left of a green lamp"
+  },
+  {
+    "category": "anatomy",
+    "prompt": "a man sitting with both feet visible, full body clearly visible"
+  },
+  {
+    "category": "counting",
+    "prompt": "4 cubes arranged in the foreground with 1 mirrors behind them"
+  },
+  {
+    "category": "reflection_occlusion",
+    "prompt": "a mirror showing the back of a person facing away"
+  },
+  {
+    "category": "spatial",
+    "prompt": "a silver mirror positioned in front of a orange cube"
+  },
+  {
+    "category": "counting",
+    "prompt": "4 glasss arranged in the foreground with 2 wooden blocks behind them"
+  },
+  {
+    "category": "counting",
+    "prompt": "3 metal cylinders arranged in the foreground with 1 lamps behind them"
+  },
+  {
+    "category": "text",
+    "prompt": "the word \"NORTH\" printed clearly and correctly on a black poster"
+  },
+  {
+    "category": "material",
+    "prompt": "a plate made of wet concrete, showing realistic surface properties and reflections"
+  },
+  {
+    "category": "reflection_occlusion",
+    "prompt": "a shiny metal object reflecting a nearby red object"
+  },
+  {
+    "category": "counting",
+    "prompt": "5 chairs arranged in the foreground with 4 cups behind them"
+  },
+  {
+    "category": "material",
+    "prompt": "a mirror made of dark wood, showing realistic surface properties and reflections"
+  },
+  {
+    "category": "material",
+    "prompt": "a chair made of wet concrete, showing realistic surface properties and reflections"
+  },
+  {
+    "category": "anatomy",
+    "prompt": "an elderly man holding an object between both hands, full body clearly visible"
+  },
+  {
+    "category": "spatial",
+    "prompt": "a white lamp positioned in front of a black wooden block"
+  },
+  {
+    "category": "counting",
+    "prompt": "3 chairs arranged in the foreground with 4 lamps behind them"
+  },
+  {
+    "category": "lighting",
+    "prompt": "a plate illuminated by cool side lighting, with clearly visible light direction and shadow"
+  },
+  {
+    "category": "spatial",
+    "prompt": "a orange wooden block positioned to the left of a red cube"
+  },
+  {
+    "category": "spatial",
+    "prompt": "a silver chair positioned above a blue wooden block"
+  },
+  {
+    "category": "reflection_occlusion",
+    "prompt": "a transparent glass bottle in front of a person's face"
+  },
+  {
+    "category": "spatial",
+    "prompt": "a yellow bottle positioned to the left of a black vase"
+  },
+  {
+    "category": "text",
+    "prompt": "the word \"HELLO\" printed clearly and correctly on a black poster"
+  },
+  {
+    "category": "anatomy",
+    "prompt": "a young man sitting with both feet visible, full body clearly visible"
+  },
+  {
+    "category": "text",
+    "prompt": "the word \"CLOSED\" printed clearly and correctly on a paper label"
+  },
+  {
+    "category": "text",
+    "prompt": "the word \"CAFE\" printed clearly and correctly on a paper label"
+  },
+  {
+    "category": "text",
+    "prompt": "the word \"HELLO\" printed clearly and correctly on a white rectangular sign"
+  },
+  {
+    "category": "spatial",
+    "prompt": "a golden plate positioned below a yellow lamp"
+  },
+  {
+    "category": "reflection_occlusion",
+    "prompt": "a mirror showing the back of a person facing away"
+  },
+  {
+    "category": "reflection_occlusion",
+    "prompt": "a translucent curtain illuminated from behind"
+  },
+  {
+    "category": "text",
+    "prompt": "the word \"OPEN\" printed clearly and correctly on a glass storefront"
+  },
+  {
+    "category": "anatomy",
+    "prompt": "a young man holding both hands in front of their chest, full body clearly visible"
+  },
+  {
+    "category": "anatomy",
+    "prompt": "an elderly woman holding an object between both hands, full body clearly visible"
+  },
+  {
+    "category": "lighting",
+    "prompt": "a table illuminated by cool side lighting, with clearly visible light direction and shadow"
+  },
+  {
+    "category": "spatial",
+    "prompt": "a white table positioned beside a black vase"
+  },
+  {
+    "category": "reflection_occlusion",
+    "prompt": "a transparent object casting a faint shadow"
+  },
+  {
+    "category": "material",
+    "prompt": "a lamp made of glossy ceramic, showing realistic surface properties and reflections"
+  },
+  {
+    "category": "reflection_occlusion",
+    "prompt": "a chrome sphere reflecting a room around it"
+  },
+  {
+    "category": "spatial",
+    "prompt": "a purple table positioned behind a blue metal cylinder"
+  },
+  {
+    "category": "anatomy",
+    "prompt": "an elderly woman holding an object between both hands, full body clearly visible"
+  },
+  {
+    "category": "anatomy",
+    "prompt": "a young woman crossing their right leg over their left leg, full body clearly visible"
+  },
+  {
+    "category": "anatomy",
+    "prompt": "an elderly woman holding an object between both hands, full body clearly visible"
+  },
+  {
+    "category": "spatial",
+    "prompt": "a red cube positioned behind a blue cup"
+  },
+  {
+    "category": "material",
+    "prompt": "a table made of translucent acrylic, showing realistic surface properties and reflections"
+  },
+  {
+    "category": "anatomy",
+    "prompt": "a young woman holding a bottle in their right hand, full body clearly visible"
+  },
+  {
+    "category": "spatial",
+    "prompt": "a blue metal cylinder positioned to the right of a red lamp"
+  },
+  {
+    "category": "lighting",
+    "prompt": "a book illuminated by warm light reflected from below, with clearly visible light direction and shadow"
+  },
+  {
+    "category": "counting",
+    "prompt": "5 books arranged in the foreground with 1 mirrors behind them"
+  },
+  {
+    "category": "spatial",
+    "prompt": "a golden metal cylinder positioned partially behind a purple vase"
+  },
+  {
+    "category": "spatial",
+    "prompt": "a white book positioned beside a yellow glass"
+  },
+  {
+    "category": "spatial",
+    "prompt": "a blue mirror positioned partially behind a silver glass"
+  },
+  {
+    "category": "material",
+    "prompt": "a cup made of translucent acrylic, showing realistic surface properties and reflections"
+  },
+  {
+    "category": "text",
+    "prompt": "the word \"STOP\" printed clearly and correctly on a white rectangular sign"
+  },
+  {
+    "category": "anatomy",
+    "prompt": "a woman holding an object between both hands, full body clearly visible"
+  },
+  {
+    "category": "material",
+    "prompt": "a cup made of rough stone, showing realistic surface properties and reflections"
+  },
+  {
+    "category": "anatomy",
+    "prompt": "a woman standing with one arm behind their back, full body clearly visible"
+  },
+  {
+    "category": "text",
+    "prompt": "the word \"STOP\" printed clearly and correctly on a white rectangular sign"
+  },
+  {
+    "category": "anatomy",
+    "prompt": "a woman touching their face with their left hand, full body clearly visible"
+  },
+  {
+    "category": "spatial",
+    "prompt": "a red plate positioned partially behind a silver sphere"
+  },
+  {
+    "category": "counting",
+    "prompt": "3 books arranged in the foreground with 1 boxs behind them"
+  },
+  {
+    "category": "counting",
+    "prompt": "5 cups arranged in the foreground with 2 metal cylinders behind them"
+  },
+  {
+    "category": "lighting",
+    "prompt": "a cube illuminated by cool side lighting, with clearly visible light direction and shadow"
+  },
+  {
+    "category": "material",
+    "prompt": "a box made of rough stone, showing realistic surface properties and reflections"
+  },
+  {
+    "category": "anatomy",
+    "prompt": "an elderly man reaching forward with their right arm, full body clearly visible"
+  },
+  {
+    "category": "text",
+    "prompt": "the word \"OPEN\" printed clearly and correctly on a glass storefront"
+  },
+  {
+    "category": "anatomy",
+    "prompt": "a man holding a bottle in their right hand, full body clearly visible"
+  },
+  {
+    "category": "reflection_occlusion",
+    "prompt": "a transparent object casting a faint shadow"
+  },
+  {
+    "category": "text",
+    "prompt": "the word \"STUDIO\" printed clearly and correctly on a white rectangular sign"
+  },
+  {
+    "category": "counting",
+    "prompt": "3 spheres arranged in the foreground with 3 lamps behind them"
+  },
+  {
+    "category": "material",
+    "prompt": "a plate made of matte plastic, showing realistic surface properties and reflections"
+  },
+  {
+    "category": "counting",
+    "prompt": "3 glasss arranged in the foreground with 1 vases behind them"
+  },
+  {
+    "category": "counting",
+    "prompt": "5 books arranged in the foreground with 3 chairs behind them"
+  },
+  {
+    "category": "anatomy",
+    "prompt": "an elderly woman holding an object between both hands, full body clearly visible"
+  },
+  {
+    "category": "spatial",
+    "prompt": "a green sphere positioned behind a black plate"
+  },
+  {
+    "category": "lighting",
+    "prompt": "a bottle illuminated by soft diffused daylight, with clearly visible light direction and shadow"
+  },
+  {
+    "category": "text",
+    "prompt": "the word \"NORTH\" printed clearly and correctly on a metal street sign"
+  },
+  {
+    "category": "counting",
+    "prompt": "2 metal cylinders arranged in the foreground with 3 mirrors behind them"
+  },
+  {
+    "category": "text",
+    "prompt": "the word \"STOP\" printed clearly and correctly on a white rectangular sign"
+  },
+  {
+    "category": "text",
+    "prompt": "the word \"BLUE\" printed clearly and correctly on a white rectangular sign"
+  },
+  {
+    "category": "counting",
+    "prompt": "2 boxs arranged in the foreground with 3 chairs behind them"
+  },
+  {
+    "category": "lighting",
+    "prompt": "a cube illuminated by warm overhead light, with clearly visible light direction and shadow"
+  },
+  {
+    "category": "text",
+    "prompt": "the word \"CLOSED\" printed clearly and correctly on a white rectangular sign"
+  },
+  {
+    "category": "spatial",
+    "prompt": "a orange plate positioned partially behind a silver vase"
+  },
+  {
+    "category": "material",
+    "prompt": "a vase made of wet concrete, showing realistic surface properties and reflections"
+  },
+  {
+    "category": "anatomy",
+    "prompt": "a woman touching their face with their left hand, full body clearly visible"
+  },
+  {
+    "category": "anatomy",
+    "prompt": "a man holding an object between both hands, full body clearly visible"
+  },
+  {
+    "category": "reflection_occlusion",
+    "prompt": "a shiny metal object reflecting a nearby red object"
+  },
+  {
+    "category": "spatial",
+    "prompt": "a purple metal cylinder positioned partially behind a white table"
+  },
+  {
+    "category": "anatomy",
+    "prompt": "an elderly man touching their face with their left hand, full body clearly visible"
+  },
+  {
+    "category": "reflection_occlusion",
+    "prompt": "a translucent curtain illuminated from behind"
+  },
+  {
+    "category": "spatial",
+    "prompt": "a black box positioned below a yellow wooden block"
+  },
+  {
+    "category": "spatial",
+    "prompt": "a purple table positioned partially behind a yellow wooden block"
+  },
+  {
+    "category": "counting",
+    "prompt": "4 cups arranged in the foreground with 2 glasss behind them"
+  },
+  {
+    "category": "counting",
+    "prompt": "3 wooden blocks arranged in the foreground with 3 books behind them"
+  },
+  {
+    "category": "lighting",
+    "prompt": "a lamp illuminated by hard light from the right, with clearly visible light direction and shadow"
+  },
+  {
+    "category": "reflection_occlusion",
+    "prompt": "a mirror showing the back of a person facing away"
+  },
+  {
+    "category": "spatial",
+    "prompt": "a silver wooden block positioned partially behind a green table"
+  },
+  {
+    "category": "spatial",
+    "prompt": "a red book positioned behind a white sphere"
+  },
+  {
+    "category": "reflection_occlusion",
+    "prompt": "a mirror showing the back of a person facing away"
+  },
+  {
+    "category": "spatial",
+    "prompt": "a silver metal cylinder positioned partially behind a blue sphere"
+  },
+  {
+    "category": "spatial",
+    "prompt": "a yellow sphere positioned above a red box"
+  },
+  {
+    "category": "text",
+    "prompt": "the word \"STUDIO\" printed clearly and correctly on a paper label"
+  },
+  {
+    "category": "anatomy",
+    "prompt": "an elderly man holding an object between both hands, full body clearly visible"
+  },
+  {
+    "category": "text",
+    "prompt": "the word \"BLUE\" printed clearly and correctly on a metal street sign"
+  },
+  {
+    "category": "spatial",
+    "prompt": "a yellow plate positioned above a orange table"
+  },
+  {
+    "category": "lighting",
+    "prompt": "a mirror illuminated by soft diffused daylight, with clearly visible light direction and shadow"
+  },
+  {
+    "category": "anatomy",
+    "prompt": "a woman reaching forward with their right arm, full body clearly visible"
+  },
+  {
+    "category": "lighting",
+    "prompt": "a cup illuminated by a narrow beam of light from above, with clearly visible light direction and shadow"
+  },
+  {
+    "category": "material",
+    "prompt": "a book made of wet concrete, showing realistic surface properties and reflections"
+  },
+  {
+    "category": "reflection_occlusion",
+    "prompt": "a translucent curtain illuminated from behind"
+  },
+  {
+    "category": "text",
+    "prompt": "the word \"CLOSED\" printed clearly and correctly on a metal street sign"
+  },
+  {
+    "category": "counting",
+    "prompt": "4 boxs arranged in the foreground with 2 glasss behind them"
+  },
+  {
+    "category": "lighting",
+    "prompt": "a vase illuminated by warm light reflected from below, with clearly visible light direction and shadow"
+  },
+  {
+    "category": "reflection_occlusion",
+    "prompt": "a shiny metal object reflecting a nearby red object"
+  },
+  {
+    "category": "lighting",
+    "prompt": "a book illuminated by soft diffused daylight, with clearly visible light direction and shadow"
+  },
+  {
+    "category": "spatial",
+    "prompt": "a green wooden block positioned behind a purple chair"
+  },
+  {
+    "category": "counting",
+    "prompt": "4 cups arranged in the foreground with 4 cubes behind them"
+  },
+  {
+    "category": "anatomy",
+    "prompt": "a young man touching their face with their left hand, full body clearly visible"
+  },
+  {
+    "category": "counting",
+    "prompt": "3 vases arranged in the foreground with 2 tables behind them"
+  },
+  {
+    "category": "counting",
+    "prompt": "2 wooden blocks arranged in the foreground with 1 glasss behind them"
+  },
+  {
+    "category": "lighting",
+    "prompt": "a table illuminated by hard light from the right, with clearly visible light direction and shadow"
+  },
+  {
+    "category": "reflection_occlusion",
+    "prompt": "a hand visible through a transparent glass panel"
+  },
+  {
+    "category": "anatomy",
+    "prompt": "a man holding a bottle in their right hand, full body clearly visible"
+  },
+  {
+    "category": "anatomy",
+    "prompt": "an elderly woman sitting with both feet visible, full body clearly visible"
+  },
+  {
+    "category": "spatial",
+    "prompt": "a black book positioned below a orange cup"
+  },
+  {
+    "category": "text",
+    "prompt": "the word \"OPEN\" printed clearly and correctly on a metal street sign"
+  },
+  {
+    "category": "material",
+    "prompt": "a wooden block made of glossy ceramic, showing realistic surface properties and reflections"
+  },
+  {
+    "category": "reflection_occlusion",
+    "prompt": "one person partially occluded by another person"
+  },
+  {
+    "category": "counting",
+    "prompt": "2 books arranged in the foreground with 3 cubes behind them"
+  },
+  {
+    "category": "material",
+    "prompt": "a glass made of rough stone, showing realistic surface properties and reflections"
+  },
+  {
+    "category": "anatomy",
+    "prompt": "a woman holding a bottle in their right hand, full body clearly visible"
+  },
+  {
+    "category": "reflection_occlusion",
+    "prompt": "a translucent curtain illuminated from behind"
+  },
+  {
+    "category": "text",
+    "prompt": "the word \"CAFE\" printed clearly and correctly on a black poster"
+  },
+  {
+    "category": "text",
+    "prompt": "the word \"OPEN\" printed clearly and correctly on a paper label"
+  },
+  {
+    "category": "counting",
+    "prompt": "3 vases arranged in the foreground with 4 lamps behind them"
+  },
+  {
+    "category": "counting",
+    "prompt": "5 boxs arranged in the foreground with 4 metal cylinders behind them"
+  },
+  {
+    "category": "spatial",
+    "prompt": "a white book positioned beside a blue lamp"
+  },
+  {
+    "category": "anatomy",
+    "prompt": "a man reaching forward with their right arm, full body clearly visible"
+  },
+  {
+    "category": "reflection_occlusion",
+    "prompt": "a glass sphere resting on a reflective metal surface"
+  },
+  {
+    "category": "material",
+    "prompt": "a bottle made of wet concrete, showing realistic surface properties and reflections"
+  },
+  {
+    "category": "lighting",
+    "prompt": "a cube illuminated by warm overhead light, with clearly visible light direction and shadow"
+  },
+  {
+    "category": "spatial",
+    "prompt": "a blue cube positioned beside a white table"
+  },
+  {
+    "category": "lighting",
+    "prompt": "a vase illuminated by soft light from the left, with clearly visible light direction and shadow"
+  },
+  {
+    "category": "anatomy",
+    "prompt": "a young man crossing their right leg over their left leg, full body clearly visible"
+  },
+  {
+    "category": "spatial",
+    "prompt": "a yellow bottle positioned beside a green book"
+  },
+  {
+    "category": "reflection_occlusion",
+    "prompt": "a translucent curtain illuminated from behind"
+  },
+  {
+    "category": "material",
+    "prompt": "a vase made of dark wood, showing realistic surface properties and reflections"
+  },
+  {
+    "category": "material",
+    "prompt": "a plate made of polished chrome, showing realistic surface properties and reflections"
+  },
+  {
+    "category": "spatial",
+    "prompt": "a black metal cylinder positioned below a orange sphere"
+  },
+  {
+    "category": "spatial",
+    "prompt": "a black glass positioned below a green lamp"
+  },
+  {
+    "category": "counting",
+    "prompt": "5 bottles arranged in the foreground with 4 books behind them"
+  },
+  {
+    "category": "material",
+    "prompt": "a glass made of translucent acrylic, showing realistic surface properties and reflections"
+  },
+  {
+    "category": "anatomy",
+    "prompt": "a young man raising their left hand above their head, full body clearly visible"
+  },
+  {
+    "category": "anatomy",
+    "prompt": "an elderly woman holding both hands in front of their chest, full body clearly visible"
+  },
+  {
+    "category": "anatomy",
+    "prompt": "a young man crossing their right leg over their left leg, full body clearly visible"
+  },
+  {
+    "category": "lighting",
+    "prompt": "a sphere illuminated by warm overhead light, with clearly visible light direction and shadow"
+  },
+  {
+    "category": "lighting",
+    "prompt": "a plate illuminated by cool side lighting, with clearly visible light direction and shadow"
+  },
+  {
+    "category": "lighting",
+    "prompt": "a box illuminated by hard light from the right, with clearly visible light direction and shadow"
+  },
+  {
+    "category": "spatial",
+    "prompt": "a red book positioned in front of a white sphere"
+  },
+  {
+    "category": "counting",
+    "prompt": "4 mirrors arranged in the foreground with 3 wooden blocks behind them"
+  },
+  {
+    "category": "material",
+    "prompt": "a table made of rough stone, showing realistic surface properties and reflections"
+  },
+  {
+    "category": "counting",
+    "prompt": "5 wooden blocks arranged in the foreground with 2 bottles behind them"
+  },
+  {
+    "category": "anatomy",
+    "prompt": "an elderly woman raising their left hand above their head, full body clearly visible"
+  },
+  {
+    "category": "spatial",
+    "prompt": "a blue sphere positioned to the left of a silver vase"
+  },
+  {
+    "category": "spatial",
+    "prompt": "a green glass positioned below a white cup"
+  },
+  {
+    "category": "text",
+    "prompt": "the word \"BLUE\" printed clearly and correctly on a paper label"
+  },
+  {
+    "category": "spatial",
+    "prompt": "a white wooden block positioned partially behind a green mirror"
+  },
+  {
+    "category": "counting",
+    "prompt": "3 chairs arranged in the foreground with 4 plates behind them"
+  },
+  {
+    "category": "counting",
+    "prompt": "2 spheres arranged in the foreground with 4 mirrors behind them"
+  },
+  {
+    "category": "counting",
+    "prompt": "4 books arranged in the foreground with 4 wooden blocks behind them"
+  },
+  {
+    "category": "reflection_occlusion",
+    "prompt": "a chrome sphere reflecting a room around it"
+  },
+  {
+    "category": "counting",
+    "prompt": "4 spheres arranged in the foreground with 2 metal cylinders behind them"
+  },
+  {
+    "category": "anatomy",
+    "prompt": "a young woman holding both hands in front of their chest, full body clearly visible"
+  },
+  {
+    "category": "anatomy",
+    "prompt": "an elderly man holding both hands in front of their chest, full body clearly visible"
+  },
+  {
+    "category": "lighting",
+    "prompt": "a bottle illuminated by warm light reflected from below, with clearly visible light direction and shadow"
+  },
+  {
+    "category": "lighting",
+    "prompt": "a metal cylinder illuminated by soft diffused daylight, with clearly visible light direction and shadow"
+  },
+  {
+    "category": "spatial",
+    "prompt": "a orange glass positioned partially behind a yellow sphere"
+  },
+  {
+    "category": "counting",
+    "prompt": "4 plates arranged in the foreground with 4 vases behind them"
+  },
+  {
+    "category": "spatial",
+    "prompt": "a green cup positioned to the right of a silver vase"
+  },
+  {
+    "category": "spatial",
+    "prompt": "a silver sphere positioned to the right of a golden metal cylinder"
+  },
+  {
+    "category": "reflection_occlusion",
+    "prompt": "a woman reflected accurately in a wall mirror"
+  },
+  {
+    "category": "spatial",
+    "prompt": "a golden bottle positioned partially behind a purple book"
+  },
+  {
+    "category": "reflection_occlusion",
+    "prompt": "a shiny metal object reflecting a nearby red object"
+  },
+  {
+    "category": "material",
+    "prompt": "a vase made of dark wood, showing realistic surface properties and reflections"
+  },
+  {
+    "category": "material",
+    "prompt": "a wooden block made of transparent glass, showing realistic surface properties and reflections"
+  },
+  {
+    "category": "reflection_occlusion",
+    "prompt": "a translucent curtain illuminated from behind"
+  },
+  {
+    "category": "counting",
+    "prompt": "4 chairs arranged in the foreground with 2 wooden blocks behind them"
+  },
+  {
+    "category": "anatomy",
+    "prompt": "an elderly woman standing with one arm behind their back, full body clearly visible"
+  },
+  {
+    "category": "reflection_occlusion",
+    "prompt": "a woman reflected accurately in a wall mirror"
+  },
+  {
+    "category": "text",
+    "prompt": "the word \"STOP\" printed clearly and correctly on a white rectangular sign"
+  },
+  {
+    "category": "material",
+    "prompt": "a book made of transparent glass, showing realistic surface properties and reflections"
+  },
+  {
+    "category": "counting",
+    "prompt": "2 mirrors arranged in the foreground with 1 tables behind them"
+  },
+  {
+    "category": "text",
+    "prompt": "the word \"STUDIO\" printed clearly and correctly on a white rectangular sign"
+  },
+  {
+    "category": "spatial",
+    "prompt": "a yellow cup positioned behind a golden cube"
+  },
+  {
+    "category": "lighting",
+    "prompt": "a cup illuminated by warm light reflected from below, with clearly visible light direction and shadow"
+  },
+  {
+    "category": "anatomy",
+    "prompt": "a man crossing their right leg over their left leg, full body clearly visible"
+  },
+  {
+    "category": "material",
+    "prompt": "a mirror made of glossy ceramic, showing realistic surface properties and reflections"
+  },
+  {
+    "category": "material",
+    "prompt": "a glass made of dark wood, showing realistic surface properties and reflections"
+  },
+  {
+    "category": "material",
+    "prompt": "a cube made of wet concrete, showing realistic surface properties and reflections"
+  },
+  {
+    "category": "spatial",
+    "prompt": "a green bottle positioned behind a black chair"
+  },
+  {
+    "category": "counting",
+    "prompt": "2 metal cylinders arranged in the foreground with 4 spheres behind them"
+  },
+  {
+    "category": "anatomy",
+    "prompt": "a young woman standing with one arm behind their back, full body clearly visible"
+  },
+  {
+    "category": "spatial",
+    "prompt": "a blue wooden block positioned behind a white book"
+  },
+  {
+    "category": "text",
+    "prompt": "the word \"OPEN\" printed clearly and correctly on a glass storefront"
+  },
+  {
+    "category": "counting",
+    "prompt": "2 chairs arranged in the foreground with 2 cups behind them"
+  },
+  {
+    "category": "text",
+    "prompt": "the word \"OPEN\" printed clearly and correctly on a paper label"
+  },
+  {
+    "category": "anatomy",
+    "prompt": "an elderly man turning their head to the left, full body clearly visible"
+  },
+  {
+    "category": "material",
+    "prompt": "a cup made of transparent glass, showing realistic surface properties and reflections"
+  },
+  {
+    "category": "text",
+    "prompt": "the word \"NORTH\" printed clearly and correctly on a glass storefront"
+  },
+  {
+    "category": "material",
+    "prompt": "a cube made of rough stone, showing realistic surface properties and reflections"
+  },
+  {
+    "category": "spatial",
+    "prompt": "a silver vase positioned to the right of a black glass"
+  },
+  {
+    "category": "lighting",
+    "prompt": "a box illuminated by strong backlight, with clearly visible light direction and shadow"
+  },
+  {
+    "category": "material",
+    "prompt": "a cup made of polished chrome, showing realistic surface properties and reflections"
+  },
+  {
+    "category": "reflection_occlusion",
+    "prompt": "one person partially occluded by another person"
+  },
+  {
+    "category": "material",
+    "prompt": "a glass made of glossy ceramic, showing realistic surface properties and reflections"
+  },
+  {
+    "category": "anatomy",
+    "prompt": "an elderly woman reaching forward with their right arm, full body clearly visible"
+  },
+  {
+    "category": "reflection_occlusion",
+    "prompt": "a shiny metal object reflecting a nearby red object"
+  },
+  {
+    "category": "spatial",
+    "prompt": "a red mirror positioned beside a orange glass"
+  },
+  {
+    "category": "material",
+    "prompt": "a chair made of brushed metal, showing realistic surface properties and reflections"
+  },
+  {
+    "category": "spatial",
+    "prompt": "a silver mirror positioned below a purple metal cylinder"
+  },
+  {
+    "category": "material",
+    "prompt": "a wooden block made of transparent glass, showing realistic surface properties and reflections"
+  },
+  {
+    "category": "counting",
+    "prompt": "5 mirrors arranged in the foreground with 1 chairs behind them"
+  },
+  {
+    "category": "spatial",
+    "prompt": "a golden cube positioned to the right of a black table"
+  },
+  {
+    "category": "lighting",
+    "prompt": "a bottle illuminated by a narrow beam of light from above, with clearly visible light direction and shadow"
+  },
+  {
+    "category": "anatomy",
+    "prompt": "a woman raising their left hand above their head, full body clearly visible"
+  },
+  {
+    "category": "text",
+    "prompt": "the word \"STUDIO\" printed clearly and correctly on a black poster"
+  },
+  {
+    "category": "material",
+    "prompt": "a cup made of wet concrete, showing realistic surface properties and reflections"
+  },
+  {
+    "category": "counting",
+    "prompt": "5 mirrors arranged in the foreground with 3 cups behind them"
+  },
+  {
+    "category": "counting",
+    "prompt": "3 glasss arranged in the foreground with 3 plates behind them"
+  },
+  {
+    "category": "anatomy",
+    "prompt": "a young woman crossing their right leg over their left leg, full body clearly visible"
+  },
+  {
+    "category": "reflection_occlusion",
+    "prompt": "a hand visible through a transparent glass panel"
+  },
+  {
+    "category": "spatial",
+    "prompt": "a golden chair positioned above a white cup"
+  },
+  {
+    "category": "reflection_occlusion",
+    "prompt": "one person partially occluded by another person"
+  },
+  {
+    "category": "spatial",
+    "prompt": "a purple sphere positioned to the right of a green vase"
+  },
+  {
+    "category": "anatomy",
+    "prompt": "a man sitting with both feet visible, full body clearly visible"
+  },
+  {
+    "category": "spatial",
+    "prompt": "a silver plate positioned above a purple metal cylinder"
+  },
+  {
+    "category": "material",
+    "prompt": "a cup made of matte plastic, showing realistic surface properties and reflections"
+  },
+  {
+    "category": "material",
+    "prompt": "a sphere made of glossy ceramic, showing realistic surface properties and reflections"
+  },
+  {
+    "category": "spatial",
+    "prompt": "a purple lamp positioned partially behind a green cube"
+  },
+  {
+    "category": "material",
+    "prompt": "a bottle made of polished chrome, showing realistic surface properties and reflections"
+  },
+  {
+    "category": "reflection_occlusion",
+    "prompt": "a glass sphere resting on a reflective metal surface"
+  },
+  {
+    "category": "material",
+    "prompt": "a mirror made of matte plastic, showing realistic surface properties and reflections"
+  },
+  {
+    "category": "counting",
+    "prompt": "2 metal cylinders arranged in the foreground with 3 bottles behind them"
+  },
+  {
+    "category": "anatomy",
+    "prompt": "an elderly woman raising their left hand above their head, full body clearly visible"
+  },
+  {
+    "category": "anatomy",
+    "prompt": "a woman holding both hands in front of their chest, full body clearly visible"
+  },
+  {
+    "category": "lighting",
+    "prompt": "a bottle illuminated by soft diffused daylight, with clearly visible light direction and shadow"
+  },
+  {
+    "category": "reflection_occlusion",
+    "prompt": "a transparent object casting a faint shadow"
+  },
+  {
+    "category": "spatial",
+    "prompt": "a golden mirror positioned below a blue cube"
+  },
+  {
+    "category": "material",
+    "prompt": "a glass made of translucent acrylic, showing realistic surface properties and reflections"
+  },
+  {
+    "category": "lighting",
+    "prompt": "a wooden block illuminated by cool side lighting, with clearly visible light direction and shadow"
+  },
+  {
+    "category": "counting",
+    "prompt": "3 cubes arranged in the foreground with 3 bottles behind them"
+  },
+  {
+    "category": "text",
+    "prompt": "the word \"STOP\" printed clearly and correctly on a metal street sign"
+  },
+  {
+    "category": "text",
+    "prompt": "the word \"EXIT\" printed clearly and correctly on a white rectangular sign"
+  },
+  {
+    "category": "counting",
+    "prompt": "3 boxs arranged in the foreground with 4 glasss behind them"
+  },
+  {
+    "category": "reflection_occlusion",
+    "prompt": "a glass sphere resting on a reflective metal surface"
+  },
+  {
+    "category": "spatial",
+    "prompt": "a black bottle positioned to the right of a red box"
+  },
+  {
+    "category": "anatomy",
+    "prompt": "a woman standing with one arm behind their back, full body clearly visible"
+  },
+  {
+    "category": "text",
+    "prompt": "the word \"CAFE\" printed clearly and correctly on a glass storefront"
+  },
+  {
+    "category": "anatomy",
+    "prompt": "a young woman raising their left hand above their head, full body clearly visible"
+  },
+  {
+    "category": "reflection_occlusion",
+    "prompt": "a glass sphere resting on a reflective metal surface"
+  },
+  {
+    "category": "anatomy",
+    "prompt": "a young woman raising their left hand above their head, full body clearly visible"
+  },
+  {
+    "category": "reflection_occlusion",
+    "prompt": "a woman reflected accurately in a wall mirror"
+  },
+  {
+    "category": "anatomy",
+    "prompt": "a woman holding a bottle in their right hand, full body clearly visible"
+  },
+  {
+    "category": "spatial",
+    "prompt": "a black lamp positioned partially behind a orange box"
+  },
+  {
+    "category": "spatial",
+    "prompt": "a golden plate positioned to the left of a orange wooden block"
+  },
+  {
+    "category": "counting",
+    "prompt": "4 bottles arranged in the foreground with 2 wooden blocks behind them"
+  },
+  {
+    "category": "spatial",
+    "prompt": "a orange lamp positioned to the left of a black cube"
+  },
+  {
+    "category": "counting",
+    "prompt": "5 lamps arranged in the foreground with 2 glasss behind them"
+  },
+  {
+    "category": "spatial",
+    "prompt": "a yellow metal cylinder positioned to the right of a silver cube"
+  },
+  {
+    "category": "material",
+    "prompt": "a bottle made of matte plastic, showing realistic surface properties and reflections"
+  },
+  {
+    "category": "spatial",
+    "prompt": "a golden vase positioned in front of a white table"
+  },
+  {
+    "category": "reflection_occlusion",
+    "prompt": "a transparent glass bottle in front of a person's face"
+  },
+  {
+    "category": "material",
+    "prompt": "a table made of glossy ceramic, showing realistic surface properties and reflections"
+  },
+  {
+    "category": "material",
+    "prompt": "a cup made of dark wood, showing realistic surface properties and reflections"
+  },
+  {
+    "category": "anatomy",
+    "prompt": "a woman holding a bottle in their right hand, full body clearly visible"
+  },
+  {
+    "category": "text",
+    "prompt": "the word \"HELLO\" printed clearly and correctly on a paper label"
+  },
+  {
+    "category": "material",
+    "prompt": "a plate made of rough stone, showing realistic surface properties and reflections"
+  }
+]

research/raw_scripts/compare_hidden_layers_bridge.py

@@ -0,0 +1,439 @@
+import os
+import csv
+import torch
+
+from safetensors.torch import load_file
+
+
+ROOT = r"E:\Models SD XL\UNET\Minimax_H3"
+
+H3_FILE = os.path.join(
+    ROOT,
+    "h3_hidden_states_fast.pt"
+)
+
+SN_FILE = os.path.join(
+    ROOT,
+    "sensenova_hidden_states_stream_v3.pt"
+)
+
+BRIDGE_FILE = os.path.join(
+    ROOT,
+    "sensenova_to_h3_embedding_bridge_rank512.safetensors"
+)
+
+OUT_TXT = os.path.join(
+    ROOT,
+    "hidden_layer_bridge_comparison.txt"
+)
+
+OUT_CSV = os.path.join(
+    ROOT,
+    "hidden_layer_bridge_comparison.csv"
+)
+
+
+SN_LAYERS = [8, 16, 24, 32, 41]
+H3_LAYERS = [8, 16, 24, 32, 40, 49]
+
+
+print("=" * 80)
+print("SenseNova -> H3 hidden-layer bridge comparison")
+print("=" * 80)
+
+
+# ============================================================
+# LOAD
+# ============================================================
+
+print("\nLoading hidden states...")
+
+h3_data = torch.load(
+    H3_FILE,
+    map_location="cpu",
+    weights_only=False,
+)
+
+sn_data = torch.load(
+    SN_FILE,
+    map_location="cpu",
+    weights_only=False,
+)
+
+print("H3 prompts:", len(h3_data["results"]))
+print("SN prompts:", len(sn_data["results"]))
+
+
+print("\nLoading embedding bridge...")
+
+bridge = load_file(
+    BRIDGE_FILE,
+    device="cpu"
+)
+
+A = bridge["proj_in.weight"].float()
+B = bridge["proj_out.weight"].float()
+
+print("A:", tuple(A.shape))
+print("B:", tuple(B.shape))
+
+# Expected:
+# A = [512, 4096]
+# B = [5120, 512]
+
+
+# ============================================================
+# PROJECTOR
+# ============================================================
+
+def project_sn(x):
+    """
+    x:
+      [..., 4096]
+
+    A:
+      [512, 4096]
+
+    B:
+      [5120, 512]
+
+    output:
+      [..., 5120]
+    """
+
+    x = x.float()
+
+    z = torch.nn.functional.linear(
+        x,
+        A
+    )
+
+    y = torch.nn.functional.linear(
+        z,
+        B
+    )
+
+    return y
+
+
+def cosine_stats(a, b):
+
+    a = a.float()
+    b = b.float()
+
+    cos = torch.nn.functional.cosine_similarity(
+        a,
+        b,
+        dim=-1
+    )
+
+    return {
+        "mean": cos.mean().item(),
+        "min": cos.min().item(),
+        "max": cos.max().item(),
+        "std": cos.std().item()
+        if cos.numel() > 1 else 0.0,
+    }
+
+
+# ============================================================
+# VALIDATE PROMPT / TOKEN ALIGNMENT
+# ============================================================
+
+print("\nValidating prompt and token alignment...")
+
+if len(h3_data["results"]) != len(sn_data["results"]):
+    raise RuntimeError(
+        "Different number of prompts."
+    )
+
+
+for i, (h3_item, sn_item) in enumerate(
+    zip(
+        h3_data["results"],
+        sn_data["results"]
+    )
+):
+
+    if h3_item["prompt"] != sn_item["prompt"]:
+        raise RuntimeError(
+            f"Prompt mismatch at index {i}"
+        )
+
+    h_ids = h3_item["input_ids"]
+    s_ids = sn_item["input_ids"]
+
+    if not torch.equal(h_ids, s_ids):
+        raise RuntimeError(
+            f"Token IDs mismatch for:\n"
+            f"{h3_item['prompt']}"
+        )
+
+
+print("Prompt/token alignment: PERFECT")
+
+
+# ============================================================
+# COMPARE
+# ============================================================
+
+rows = []
+
+print()
+print("=" * 80)
+print("PAIRWISE LAYER COMPARISON")
+print("=" * 80)
+
+
+for sn_layer in SN_LAYERS:
+
+    for h3_layer in H3_LAYERS:
+
+        all_projected = []
+        all_h3 = []
+
+        prompt_scores = []
+
+        for h3_item, sn_item in zip(
+            h3_data["results"],
+            sn_data["results"]
+        ):
+
+            sn_x = sn_item["layers"][sn_layer]
+            h3_y = h3_item["layers"][h3_layer]
+
+            if sn_x.shape[:2] != h3_y.shape[:2]:
+                raise RuntimeError(
+                    f"Sequence mismatch "
+                    f"SN L{sn_layer} vs "
+                    f"H3 L{h3_layer}"
+                )
+
+            projected = project_sn(sn_x)
+
+            stats = cosine_stats(
+                projected,
+                h3_y
+            )
+
+            prompt_scores.append(
+                stats["mean"]
+            )
+
+            all_projected.append(
+                projected.reshape(
+                    -1,
+                    projected.shape[-1]
+                )
+            )
+
+            all_h3.append(
+                h3_y.reshape(
+                    -1,
+                    h3_y.shape[-1]
+                )
+            )
+
+
+        P = torch.cat(
+            all_projected,
+            dim=0
+        )
+
+        H = torch.cat(
+            all_h3,
+            dim=0
+        )
+
+        stats = cosine_stats(P, H)
+
+        # Magnitudes are not directly comparable,
+        # but useful diagnostically.
+        proj_norm = P.norm(
+            dim=-1
+        ).mean().item()
+
+        h3_norm = H.norm(
+            dim=-1
+        ).mean().item()
+
+        norm_ratio = (
+            proj_norm / h3_norm
+            if h3_norm != 0 else 0
+        )
+
+        row = {
+            "sn_layer": sn_layer,
+            "h3_layer": h3_layer,
+
+            "cosine_mean": stats["mean"],
+            "cosine_std": stats["std"],
+            "cosine_min": stats["min"],
+            "cosine_max": stats["max"],
+
+            "projected_norm": proj_norm,
+            "h3_norm": h3_norm,
+            "norm_ratio": norm_ratio,
+
+            "prompt_1": prompt_scores[0],
+            "prompt_2": prompt_scores[1],
+            "prompt_3": prompt_scores[2],
+        }
+
+        rows.append(row)
+
+        print(
+            f"SN L{sn_layer:02d} "
+            f"-> H3 L{h3_layer:02d} | "
+            f"cos={stats['mean']:.6f} | "
+            f"norm_ratio={norm_ratio:.4f}"
+        )
+
+
+# ============================================================
+# SORT
+# ============================================================
+
+rows_sorted = sorted(
+    rows,
+    key=lambda x: x["cosine_mean"],
+    reverse=True
+)
+
+
+# ============================================================
+# SAVE CSV
+# ============================================================
+
+with open(
+    OUT_CSV,
+    "w",
+    newline="",
+    encoding="utf-8"
+) as f:
+
+    writer = csv.DictWriter(
+        f,
+        fieldnames=rows_sorted[0].keys()
+    )
+
+    writer.writeheader()
+    writer.writerows(rows_sorted)
+
+
+# ============================================================
+# SAVE TXT
+# ============================================================
+
+with open(
+    OUT_TXT,
+    "w",
+    encoding="utf-8"
+) as f:
+
+    f.write(
+        "SenseNova -> H3 hidden-layer "
+        "bridge comparison\n"
+    )
+
+    f.write("=" * 80 + "\n\n")
+
+    f.write(
+        "The projector was trained ONLY "
+        "on token embeddings.\n"
+    )
+
+    f.write(
+        "Hidden states were unseen during "
+        "bridge training.\n\n"
+    )
+
+    f.write(
+        "TOP LAYER PAIRS\n"
+    )
+
+    f.write("-" * 80 + "\n")
+
+    for rank, row in enumerate(
+        rows_sorted,
+        1
+    ):
+
+        f.write(
+            f"{rank:02d}. "
+            f"SN L{row['sn_layer']:02d} "
+            f"-> H3 L{row['h3_layer']:02d} | "
+            f"cos={row['cosine_mean']:.6f} | "
+            f"std={row['cosine_std']:.6f} | "
+            f"norm_ratio={row['norm_ratio']:.4f}\n"
+        )
+
+        f.write(
+            "    prompts: "
+            f"{row['prompt_1']:.6f}, "
+            f"{row['prompt_2']:.6f}, "
+            f"{row['prompt_3']:.6f}\n"
+        )
+
+
+# ============================================================
+# DISPLAY TOP
+# ============================================================
+
+print()
+print("=" * 80)
+print("TOP 15")
+print("=" * 80)
+
+for rank, row in enumerate(
+    rows_sorted[:15],
+    1
+):
+
+    print(
+        f"{rank:02d}. "
+        f"SN L{row['sn_layer']:02d} "
+        f"-> H3 L{row['h3_layer']:02d} | "
+        f"cos={row['cosine_mean']:.6f} | "
+        f"norm_ratio={row['norm_ratio']:.4f}"
+    )
+
+
+# ============================================================
+# BEST H3 TARGET FOR EACH SN LAYER
+# ============================================================
+
+print()
+print("=" * 80)
+print("BEST TARGET FOR EACH SENSENOVA LAYER")
+print("=" * 80)
+
+for sn_layer in SN_LAYERS:
+
+    candidates = [
+        r for r in rows
+        if r["sn_layer"] == sn_layer
+    ]
+
+    best = max(
+        candidates,
+        key=lambda x: x["cosine_mean"]
+    )
+
+    print(
+        f"SN L{sn_layer:02d} "
+        f"-> H3 L{best['h3_layer']:02d} | "
+        f"cos={best['cosine_mean']:.6f}"
+    )
+
+
+print()
+print("=" * 80)
+print("DONE")
+print()
+print("TXT:")
+print(OUT_TXT)
+print()
+print("CSV:")
+print(OUT_CSV)
+print("=" * 80)

research/raw_scripts/compare_minimax_sensenova.py

@@ -0,0 +1,335 @@
+from safetensors import safe_open
+from collections import defaultdict
+import os
+import re
+import json
+
+MINIMAX = r"E:\Models SD XL\UNET\Minimax_H3\minimax_h3_fl2va_pruned_int8_convrot.safetensors"
+SENSENOVA = r"D:\ComfyUI_Python312\ComfyUI\models\unet\SenseNova-U1.5-8B-MoT-pruned-bf16.safetensors"
+
+OUT = r"E:\Models SD XL\UNET\Minimax_H3\minimax_vs_sensenova_report.txt"
+
+
+def load_manifest(path):
+    info = {}
+    metadata = {}
+
+    with safe_open(path, framework="pt", device="cpu") as f:
+        metadata = f.metadata() or {}
+
+        for key in f.keys():
+            t = f.get_tensor(key)
+            info[key] = {
+                "shape": tuple(t.shape),
+                "dtype": str(t.dtype),
+                "ndim": t.ndim,
+            }
+
+    return info, metadata
+
+
+def base_module_name(key):
+    suffixes = [
+        ".weight",
+        ".bias",
+        ".weight_scale",
+        ".input_scale",
+        ".comfy_quant",
+    ]
+    for s in suffixes:
+        if key.endswith(s):
+            return key[:-len(s)]
+    return key
+
+
+def logical_weights(manifest):
+    """
+    Берём только настоящие weight tensors.
+    Для Minimax INT8 ConvRot игнорируем metadata/scales.
+    """
+    result = {}
+
+    for key, v in manifest.items():
+        if not key.endswith(".weight"):
+            continue
+
+        if v["ndim"] != 2:
+            continue
+
+        result[key] = v
+
+    return result
+
+
+def classify(key):
+    k = key.lower()
+
+    if any(x in k for x in [
+        "q_proj", ".query", ".q.", "to_q", "wq"
+    ]):
+        return "attention_q"
+
+    if any(x in k for x in [
+        "k_proj", ".key", ".k.", "to_k", "wk"
+    ]):
+        return "attention_k"
+
+    if any(x in k for x in [
+        "v_proj", ".value", ".v.", "to_v", "wv"
+    ]):
+        return "attention_v"
+
+    if any(x in k for x in [
+        "o_proj", "out_proj", "to_out", "wo"
+    ]):
+        return "attention_out"
+
+    if "qkv" in k:
+        return "attention_qkv"
+
+    if any(x in k for x in [
+        "gate_proj", "w1", "linear_fc1", "fc1"
+    ]):
+        return "mlp_gate_or_fc1"
+
+    if any(x in k for x in [
+        "up_proj", "w3"
+    ]):
+        return "mlp_up"
+
+    if any(x in k for x in [
+        "down_proj", "w2", "linear_fc2", "fc2"
+    ]):
+        return "mlp_down_or_fc2"
+
+    if any(x in k for x in [
+        "proj", "projection"
+    ]):
+        return "projection"
+
+    return "other_2d"
+
+
+def layer_index(key):
+    patterns = [
+        r"layers\.(\d+)",
+        r"blocks\.(\d+)",
+        r"transformer_blocks\.(\d+)",
+        r"double_blocks\.(\d+)",
+        r"single_blocks\.(\d+)",
+    ]
+
+    for p in patterns:
+        m = re.search(p, key)
+        if m:
+            return int(m.group(1))
+
+    return None
+
+
+def get_quantized_modules(manifest):
+    modules = set()
+
+    for key in manifest:
+        if key.endswith(".comfy_quant"):
+            modules.add(key[:-len(".comfy_quant")])
+
+    return modules
+
+
+print("Loading manifests...")
+mini, mini_meta = load_manifest(MINIMAX)
+sense, sense_meta = load_manifest(SENSENOVA)
+
+mini_w = logical_weights(mini)
+sense_w = logical_weights(sense)
+
+mini_quant = get_quantized_modules(mini)
+
+exact = []
+transpose = []
+same_input = []
+same_output = []
+typed_exact = []
+
+sense_by_shape = defaultdict(list)
+sense_by_transposed_shape = defaultdict(list)
+sense_by_in = defaultdict(list)
+sense_by_out = defaultdict(list)
+
+for k, v in sense_w.items():
+    shape = v["shape"]
+    sense_by_shape[shape].append(k)
+    sense_by_transposed_shape[(shape[1], shape[0])].append(k)
+    sense_by_in[shape[1]].append(k)
+    sense_by_out[shape[0]].append(k)
+
+for mk, mv in mini_w.items():
+    ms = mv["shape"]
+
+    for sk in sense_by_shape.get(ms, []):
+        exact.append((mk, sk, ms))
+
+        if classify(mk) == classify(sk):
+            typed_exact.append((mk, sk, ms, classify(mk)))
+
+    for sk in sense_by_transposed_shape.get(ms, []):
+        ss = sense_w[sk]["shape"]
+        if ss != ms:
+            transpose.append((mk, sk, ms, ss))
+
+    for sk in sense_by_in.get(ms[1], []):
+        if sense_w[sk]["shape"] != ms:
+            same_input.append((mk, sk, ms, sense_w[sk]["shape"]))
+
+    for sk in sense_by_out.get(ms[0], []):
+        if sense_w[sk]["shape"] != ms:
+            same_output.append((mk, sk, ms, sense_w[sk]["shape"]))
+
+
+mini_classes = defaultdict(int)
+sense_classes = defaultdict(int)
+
+for k in mini_w:
+    mini_classes[classify(k)] += 1
+
+for k in sense_w:
+    sense_classes[classify(k)] += 1
+
+
+with open(OUT, "w", encoding="utf-8") as f:
+    def w(x=""):
+        f.write(str(x) + "\n")
+
+    w("=" * 100)
+    w("MINIMAX H3 vs SENSENOVA STRUCTURAL COMPARISON")
+    w("=" * 100)
+    w()
+
+    w("FILES")
+    w("-" * 100)
+    w(f"MINIMAX:   {MINIMAX}")
+    w(f"Size GB:   {os.path.getsize(MINIMAX) / 1024**3:.3f}")
+    w(f"Tensors:   {len(mini)}")
+    w(f"2D weights:{len(mini_w)}")
+    w(f"Quantized modules detected: {len(mini_quant)}")
+    w()
+    w(f"SENSENOVA: {SENSENOVA}")
+    w(f"Size GB:   {os.path.getsize(SENSENOVA) / 1024**3:.3f}")
+    w(f"Tensors:   {len(sense)}")
+    w(f"2D weights:{len(sense_w)}")
+    w()
+
+    w("=" * 100)
+    w("LAYER TYPE COUNTS")
+    w("=" * 100)
+    all_types = sorted(set(mini_classes) | set(sense_classes))
+    for t in all_types:
+        w(f"{t:24s}  MiniMax={mini_classes[t]:5d}   SenseNova={sense_classes[t]:5d}")
+    w()
+
+    w("=" * 100)
+    w("EXACT SHAPE MATCHES")
+    w("=" * 100)
+    w(f"Count: {len(exact)}")
+    for mk, sk, shape in exact[:2000]:
+        w(f"{shape} | {mk}  <=>  {sk}")
+    w()
+
+    w("=" * 100)
+    w("EXACT SHAPE + SAME SEMANTIC TYPE")
+    w("=" * 100)
+    w(f"Count: {len(typed_exact)}")
+    for mk, sk, shape, typ in typed_exact[:2000]:
+        w(f"[{typ}] {shape}")
+        w(f"  H3:    {mk}")
+        w(f"  Sense: {sk}")
+    w()
+
+    w("=" * 100)
+    w("TRANSPOSE-COMPATIBLE SHAPES")
+    w("=" * 100)
+    w(f"Count: {len(transpose)}")
+    for mk, sk, ms, ss in transpose[:1000]:
+        w(f"H3 {ms} | Sense {ss}")
+        w(f"  H3:    {mk}")
+        w(f"  Sense: {sk}")
+    w()
+
+    w("=" * 100)
+    w("SAME INPUT DIMENSION")
+    w("=" * 100)
+    w(f"Count: {len(same_input)}")
+    for mk, sk, ms, ss in same_input[:1000]:
+        w(f"in={ms[1]} | H3 {ms} | Sense {ss}")
+        w(f"  H3:    {mk}")
+        w(f"  Sense: {sk}")
+    w()
+
+    w("=" * 100)
+    w("SAME OUTPUT DIMENSION")
+    w("=" * 100)
+    w(f"Count: {len(same_output)}")
+    for mk, sk, ms, ss in same_output[:1000]:
+        w(f"out={ms[0]} | H3 {ms} | Sense {ss}")
+        w(f"  H3:    {mk}")
+        w(f"  Sense: {sk}")
+    w()
+
+    w("=" * 100)
+    w("MINIMAX QUANTIZED MODULES")
+    w("=" * 100)
+    for k in sorted(mini_quant):
+        shape = mini.get(k + ".weight", {}).get("shape")
+        w(f"{k} | {shape}")
+    w()
+
+    w("=" * 100)
+    w("MINIMAX 2D WEIGHTS")
+    w("=" * 100)
+    for k, v in sorted(mini_w.items()):
+        w(f"{k} | {v['shape']} | {v['dtype']} | type={classify(k)} | layer={layer_index(k)}")
+    w()
+
+    w("=" * 100)
+    w("SENSENOVA 2D WEIGHTS")
+    w("=" * 100)
+    for k, v in sorted(sense_w.items()):
+        w(f"{k} | {v['shape']} | {v['dtype']} | type={classify(k)} | layer={layer_index(k)}")
+    w()
+
+    w("=" * 100)
+    w("MINIMAX METADATA")
+    w("=" * 100)
+    for k, v in mini_meta.items():
+        if k == "_quantization_metadata":
+            try:
+                parsed = json.loads(v)
+                w("_quantization_metadata:")
+                w(json.dumps(parsed, indent=2))
+            except Exception:
+                w(f"{k} = {v}")
+        else:
+            w(f"{k} = {v}")
+    w()
+
+    w("=" * 100)
+    w("SENSENOVA METADATA")
+    w("=" * 100)
+    for k, v in sense_meta.items():
+        w(f"{k} = {v}")
+
+
+print()
+print("Done.")
+print("Report:")
+print(OUT)
+print()
+print("Summary:")
+print("MiniMax 2D weights:", len(mini_w))
+print("SenseNova 2D weights:", len(sense_w))
+print("Exact shape matches:", len(exact))
+print("Exact + same semantic type:", len(typed_exact))
+print("Transpose matches:", len(transpose))
+print("Same input-dim candidates:", len(same_input))
+print("Same output-dim candidates:", len(same_output))

research/raw_scripts/compare_tokenizers.py

@@ -0,0 +1,145 @@
+import os
+import sys
+import hashlib
+
+COMFY_ROOT = r"D:\ComfyUI_Python312\ComfyUI"
+
+sys.path.insert(0, COMFY_ROOT)
+os.chdir(COMFY_ROOT)
+
+from transformers import AutoTokenizer
+
+print("=" * 80)
+print("Tokenizer compatibility test")
+print("=" * 80)
+
+# SenseNova tokenizer — скачиваются только tokenizer/config файлы,
+# а НЕ 45 GB weights.
+print("\nLoading SenseNova tokenizer...")
+
+sense_tok = AutoTokenizer.from_pretrained(
+    "sensenova/SenseNova-U1.5-8B-MoT",
+    trust_remote_code=True,
+)
+
+# MiniMax H3 uses Qwen3-VL-32B tokenizer.
+print("Loading Qwen3-VL tokenizer used by H3...")
+
+h3_tok = AutoTokenizer.from_pretrained(
+    "Qwen/Qwen3-VL-32B-Instruct",
+    trust_remote_code=True,
+)
+
+print("\nSenseNova vocab size:", len(sense_tok))
+print("H3/Qwen vocab size:  ", len(h3_tok))
+
+sense_vocab = sense_tok.get_vocab()
+h3_vocab = h3_tok.get_vocab()
+
+common_tokens = set(sense_vocab) & set(h3_vocab)
+
+same_id = 0
+different_id = []
+
+for token in common_tokens:
+    a = sense_vocab[token]
+    b = h3_vocab[token]
+
+    if a == b:
+        same_id += 1
+    else:
+        different_id.append((token, a, b))
+
+print()
+print("Common token strings:", len(common_tokens))
+print("Same token ID:       ", same_id)
+print("Different token ID:  ", len(different_id))
+
+if common_tokens:
+    print(
+        "ID agreement:",
+        f"{100.0 * same_id / len(common_tokens):.6f}%"
+    )
+
+print("\nFirst ID mismatches:")
+
+for item in different_id[:30]:
+    print(item)
+
+# -----------------------------------------------------------
+# Direct ID -> token comparison
+# -----------------------------------------------------------
+
+max_compare = min(len(sense_tok), len(h3_tok))
+
+id_mismatches = []
+
+for i in range(max_compare):
+    a = sense_tok.convert_ids_to_tokens(i)
+    b = h3_tok.convert_ids_to_tokens(i)
+
+    if a != b:
+        id_mismatches.append((i, a, b))
+
+print()
+print("ID->token mismatches:", len(id_mismatches))
+
+for x in id_mismatches[:30]:
+    print(x)
+
+# -----------------------------------------------------------
+# Real prompt test
+# -----------------------------------------------------------
+
+prompts = [
+    "a red cube on a blue sphere",
+    "a woman holding a transparent glass bottle",
+    "three people standing behind a wooden table",
+    "the word HELLO printed on a white sign",
+    "a metallic robot illuminated from the left",
+    "a hand holding a glass of water",
+    "two cars parked beside a brick building",
+    "a woman reflected in a mirror",
+]
+
+print("\n" + "=" * 80)
+print("PROMPT TOKENIZATION")
+print("=" * 80)
+
+all_prompt_equal = True
+
+for text in prompts:
+
+    s = sense_tok.encode(
+        text,
+        add_special_tokens=False,
+    )
+
+    h = h3_tok.encode(
+        text,
+        add_special_tokens=False,
+    )
+
+    equal = s == h
+
+    if not equal:
+        all_prompt_equal = False
+
+    print()
+    print(text)
+    print("equal:", equal)
+    print("SenseNova:", s)
+    print("H3:       ", h)
+
+print("\n" + "=" * 80)
+
+if (
+    len(different_id) == 0
+    and len(id_mismatches) == 0
+    and all_prompt_equal
+):
+    print("RESULT: TOKENIZERS ARE IDENTICAL FOR TESTED VOCAB")
+else:
+    print("RESULT: TOKENIZERS DIFFER")
+
+print("=" * 80)

research/raw_scripts/evaluate_distilled_student_v2_ood.py

@@ -0,0 +1,1343 @@
+import os
+import csv
+import torch
+import torch.nn as nn
+import torch.nn.functional as F
+
+from safetensors.torch import load_file
+
+
+# ============================================================
+# PATHS
+# ============================================================
+
+ROOT = r"E:\Models SD XL\UNET\Minimax_H3"
+
+H3_OOD_FILE = os.path.join(
+    ROOT,
+    "h3_ood_hidden_160.pt"
+)
+
+SN_OOD_FILE = os.path.join(
+    ROOT,
+    "sensenova_ood_hidden_160.pt"
+)
+
+TEACHER_BRIDGE = (
+    r"D:\ComfyUI_Krea2\ComfyUI\models\bridge"
+    r"\SN_L32_to_H3_L49_rank128.safetensors"
+)
+
+STUDENT_FILE = os.path.join(
+    ROOT,
+    "distilled_student_v2",
+    "H3_L49_to_projected_SN32_v2.safetensors"
+)
+
+OUT_DIR = os.path.join(
+    ROOT,
+    "distilled_student_v2_ood"
+)
+
+os.makedirs(
+    OUT_DIR,
+    exist_ok=True
+)
+
+REPORT_FILE = os.path.join(
+    OUT_DIR,
+    "distilled_student_v2_STRICT_OOD_report.txt"
+)
+
+CSV_FILE = os.path.join(
+    OUT_DIR,
+    "distilled_student_v2_STRICT_OOD_per_prompt.csv"
+)
+
+
+# ============================================================
+# SETTINGS
+# ============================================================
+
+DEVICE = (
+    "cuda"
+    if torch.cuda.is_available()
+    else "cpu"
+)
+
+ALPHAS = [
+    0.05,
+    0.10,
+    0.20,
+    0.30,
+]
+
+HIDDEN_DIM = 512
+
+
+# ============================================================
+# STUDENT ARCHITECTURE
+# MUST MATCH TRAINING V2 EXACTLY
+# ============================================================
+
+class SemanticStudent(
+    nn.Module
+):
+    def __init__(
+        self,
+        input_dim=5120,
+        hidden_dim=512,
+        output_dim=5120,
+    ):
+        super().__init__()
+
+        self.fc1 = nn.Linear(
+            input_dim,
+            hidden_dim,
+            bias=True,
+        )
+
+        self.fc2 = nn.Linear(
+            hidden_dim,
+            hidden_dim,
+            bias=True,
+        )
+
+        self.fc3 = nn.Linear(
+            hidden_dim,
+            output_dim,
+            bias=True,
+        )
+
+        self.act = nn.SiLU()
+
+
+    def forward(
+        self,
+        x,
+    ):
+        x = self.fc1(x)
+
+        x = self.act(x)
+
+        x = self.fc2(x)
+
+        x = self.act(x)
+
+        x = self.fc3(x)
+
+        return x
+
+
+# ============================================================
+# HELPERS
+# ============================================================
+
+def rms_normalize(x):
+
+    x = x.float()
+
+    rms = torch.sqrt(
+        x.pow(2).mean(
+            dim=-1,
+            keepdim=True
+        ) + 1e-6
+    )
+
+    return x / rms
+
+
+def magnitude_match(
+    source,
+    target,
+):
+
+    source = source.float()
+    target = target.float()
+
+    source_rms = torch.sqrt(
+        source.pow(2).mean(
+            dim=-1,
+            keepdim=True
+        ) + 1e-8
+    )
+
+    target_rms = torch.sqrt(
+        target.pow(2).mean(
+            dim=-1,
+            keepdim=True
+        ) + 1e-8
+    )
+
+    return (
+        source
+        * (
+            target_rms
+            / source_rms
+        )
+    )
+
+
+def cosine_tensor(
+    a,
+    b,
+):
+    return F.cosine_similarity(
+        a.float(),
+        b.float(),
+        dim=-1,
+    )
+
+
+def cosine_mean(
+    a,
+    b,
+):
+    return (
+        cosine_tensor(
+            a,
+            b,
+        )
+        .mean()
+        .item()
+    )
+
+
+# ============================================================
+# START
+# ============================================================
+
+print("=" * 100)
+
+print(
+    "DISTILLED STUDENT V2"
+)
+
+print(
+    "STRICT 160-PROMPT OOD EVALUATION"
+)
+
+print("=" * 100)
+
+print(
+    "Device:",
+    DEVICE
+)
+
+
+# ============================================================
+# CHECK FILES
+# ============================================================
+
+for path in [
+    H3_OOD_FILE,
+    SN_OOD_FILE,
+    TEACHER_BRIDGE,
+    STUDENT_FILE,
+]:
+    if not os.path.exists(path):
+        raise FileNotFoundError(
+            f"Required file not found:\n{path}"
+        )
+
+
+# ============================================================
+# LOAD OOD DATA
+# ============================================================
+
+print(
+    "\nLoading H3 strict OOD states..."
+)
+
+h3_data = torch.load(
+    H3_OOD_FILE,
+    map_location="cpu",
+    weights_only=False,
+)
+
+print(
+    "H3 prompts:",
+    len(
+        h3_data["results"]
+    )
+)
+
+
+print(
+    "\nLoading SenseNova strict OOD states..."
+)
+
+sn_data = torch.load(
+    SN_OOD_FILE,
+    map_location="cpu",
+    weights_only=False,
+)
+
+print(
+    "SenseNova prompts:",
+    len(
+        sn_data["results"]
+    )
+)
+
+
+h3_results = h3_data[
+    "results"
+]
+
+sn_results = sn_data[
+    "results"
+]
+
+
+if len(h3_results) != 160:
+    raise RuntimeError(
+        f"Expected 160 H3 OOD prompts, "
+        f"got {len(h3_results)}"
+    )
+
+if len(sn_results) != 160:
+    raise RuntimeError(
+        f"Expected 160 SenseNova OOD prompts, "
+        f"got {len(sn_results)}"
+    )
+
+
+# ============================================================
+# ALIGNMENT
+# ============================================================
+
+print(
+    "\nChecking strict OOD alignment..."
+)
+
+for i, (
+    h3_item,
+    sn_item,
+) in enumerate(
+    zip(
+        h3_results,
+        sn_results,
+    )
+):
+
+    if (
+        h3_item["prompt"]
+        != sn_item["prompt"]
+    ):
+        raise RuntimeError(
+            f"Prompt mismatch at {i}"
+        )
+
+    if (
+        h3_item["category"]
+        != sn_item["category"]
+    ):
+        raise RuntimeError(
+            f"Category mismatch at {i}"
+        )
+
+    if not torch.equal(
+        h3_item["input_ids"],
+        sn_item["input_ids"],
+    ):
+        raise RuntimeError(
+            f"Token mismatch at {i}"
+        )
+
+
+print(
+    "Alignment: PERFECT"
+)
+
+
+# ============================================================
+# LOAD TEACHER BRIDGE
+# ============================================================
+
+print(
+    "\nLoading Full Bridge teacher..."
+)
+
+teacher = load_file(
+    TEACHER_BRIDGE,
+    device="cpu",
+)
+
+teacher_down = (
+    teacher[
+        "down.weight"
+    ]
+    .float()
+)
+
+teacher_up = (
+    teacher[
+        "up.weight"
+    ]
+    .float()
+)
+
+print(
+    "Teacher down:",
+    tuple(
+        teacher_down.shape
+    )
+)
+
+print(
+    "Teacher up:",
+    tuple(
+        teacher_up.shape
+    )
+)
+
+
+if tuple(
+    teacher_down.shape
+) != (
+    128,
+    4096,
+):
+    raise RuntimeError(
+        "Unexpected teacher down shape."
+    )
+
+if tuple(
+    teacher_up.shape
+) != (
+    5120,
+    128,
+):
+    raise RuntimeError(
+        "Unexpected teacher up shape."
+    )
+
+
+# ============================================================
+# LOAD DISTILLED STUDENT
+# ============================================================
+
+print(
+    "\nLoading frozen Distilled Student..."
+)
+
+student_weights = load_file(
+    STUDENT_FILE,
+    device="cpu",
+)
+
+student = SemanticStudent(
+    input_dim=5120,
+    hidden_dim=HIDDEN_DIM,
+    output_dim=5120,
+)
+
+
+with torch.no_grad():
+
+    student.fc1.weight.copy_(
+        student_weights[
+            "fc1.weight"
+        ].float()
+    )
+
+    student.fc1.bias.copy_(
+        student_weights[
+            "fc1.bias"
+        ].float()
+    )
+
+    student.fc2.weight.copy_(
+        student_weights[
+            "fc2.weight"
+        ].float()
+    )
+
+    student.fc2.bias.copy_(
+        student_weights[
+            "fc2.bias"
+        ].float()
+    )
+
+    student.fc3.weight.copy_(
+        student_weights[
+            "fc3.weight"
+        ].float()
+    )
+
+    student.fc3.bias.copy_(
+        student_weights[
+            "fc3.bias"
+        ].float()
+    )
+
+
+student = student.to(
+    DEVICE
+)
+
+student.eval()
+
+
+param_count = sum(
+    p.numel()
+    for p in student.parameters()
+)
+
+print(
+    "Student parameters:",
+    f"{param_count:,}"
+)
+
+print(
+    "Approx FP16 size:",
+    f"{param_count * 2 / 1024**2:.2f} MiB"
+)
+
+
+# ============================================================
+# STORAGE
+# ============================================================
+
+rows = []
+
+teacher_cos_all = []
+
+correction_cos_all = []
+
+blend_scores = {
+    alpha: []
+    for alpha
+    in ALPHAS
+}
+
+category_scores = {}
+
+category_correction = {}
+
+category_blend = {
+    alpha: {}
+    for alpha
+    in ALPHAS
+}
+
+
+# ============================================================
+# STRICT OOD EVALUATION
+# ============================================================
+
+print()
+
+print("=" * 100)
+
+print(
+    "RUNNING FROZEN STUDENT ON STRICT OOD"
+)
+
+print("=" * 100)
+
+
+with torch.inference_mode():
+
+    for i, (
+        h3_item,
+        sn_item,
+    ) in enumerate(
+        zip(
+            h3_results,
+            sn_results,
+        )
+    ):
+
+        prompt = h3_item[
+            "prompt"
+        ]
+
+        category = h3_item[
+            "category"
+        ]
+
+
+        # ====================================================
+        # OOD files contain only selected hidden layer
+        # ====================================================
+
+        h49 = (
+            h3_item[
+                "hidden"
+            ]
+            .float()
+        )
+
+        sn32 = (
+            sn_item[
+                "hidden"
+            ]
+            .float()
+        )
+
+
+        if h49.shape[-1] != 5120:
+            raise RuntimeError(
+                f"Bad H3 shape at {i}: "
+                f"{tuple(h49.shape)}"
+            )
+
+        if sn32.shape[-1] != 4096:
+            raise RuntimeError(
+                f"Bad SenseNova shape at {i}: "
+                f"{tuple(sn32.shape)}"
+            )
+
+
+        # ====================================================
+        # FULL BRIDGE TEACHER
+        # SenseNova L32 -> 4096 -> 128 -> 5120
+        # ====================================================
+
+        sn_norm = rms_normalize(
+            sn32
+        )
+
+        teacher_rank = F.linear(
+            sn_norm,
+            teacher_down,
+        )
+
+        teacher_projected = F.linear(
+            teacher_rank,
+            teacher_up,
+        )
+
+
+        # ====================================================
+        # DISTILLED STUDENT
+        # H3 L49 -> nonlinear 5120->512->512->5120
+        # ====================================================
+
+        student_input = rms_normalize(
+            h49
+        )
+
+        student_projected = (
+            student(
+                student_input.to(
+                    DEVICE
+                )
+            )
+            .cpu()
+        )
+
+
+        # ====================================================
+        # METRIC 1:
+        # STUDENT vs PROJECTED SENSENOVA
+        # ====================================================
+
+        teacher_token_cos = cosine_tensor(
+            student_projected,
+            teacher_projected,
+        )
+
+        teacher_cos = (
+            teacher_token_cos
+            .mean()
+            .item()
+        )
+
+
+        # ====================================================
+        # SAME PER-TOKEN MAGNITUDE MATCH USED BY FULL BRIDGE
+        # ====================================================
+
+        teacher_scaled = magnitude_match(
+            teacher_projected,
+            h49,
+        )
+
+        student_scaled = magnitude_match(
+            student_projected,
+            h49,
+        )
+
+
+        # ====================================================
+        # METRIC 2:
+        # DOES STUDENT REPRODUCE THE ACTUAL CORRECTION?
+        # ====================================================
+
+        teacher_delta = (
+            teacher_scaled
+            - h49
+        )
+
+        student_delta = (
+            student_scaled
+            - h49
+        )
+
+        correction_token_cos = cosine_tensor(
+            student_delta,
+            teacher_delta,
+        )
+
+        correction_cos = (
+            correction_token_cos
+            .mean()
+            .item()
+        )
+
+
+        # ====================================================
+        # METRIC 3:
+        # FULL CONDITIONING vs DISTILLED CONDITIONING
+        # ====================================================
+
+        prompt_blends = {}
+
+        for alpha in ALPHAS:
+
+            full_conditioning = (
+                h49
+                + alpha
+                * teacher_delta
+            )
+
+            distilled_conditioning = (
+                h49
+                + alpha
+                * student_delta
+            )
+
+            blend_cos = cosine_mean(
+                distilled_conditioning,
+                full_conditioning,
+            )
+
+            prompt_blends[
+                alpha
+            ] = blend_cos
+
+            blend_scores[
+                alpha
+            ].append(
+                blend_cos
+            )
+
+            category_blend[
+                alpha
+            ].setdefault(
+                category,
+                []
+            ).append(
+                blend_cos
+            )
+
+
+        # ====================================================
+        # GLOBAL / CATEGORY
+        # ====================================================
+
+        teacher_cos_all.append(
+            teacher_cos
+        )
+
+        correction_cos_all.append(
+            correction_cos
+        )
+
+        category_scores.setdefault(
+            category,
+            []
+        ).append(
+            teacher_cos
+        )
+
+        category_correction.setdefault(
+            category,
+            []
+        ).append(
+            correction_cos
+        )
+
+
+        # ====================================================
+        # CSV ROW
+        # ====================================================
+
+        row = {
+            "index":
+                i,
+
+            "category":
+                category,
+
+            "token_count":
+                h49.shape[1],
+
+            "teacher_rep_cos":
+                teacher_cos,
+
+            "correction_cos":
+                correction_cos,
+
+            "prompt":
+                prompt,
+        }
+
+        for alpha in ALPHAS:
+
+            row[
+                f"blend_alpha_{alpha}"
+            ] = (
+                prompt_blends[
+                    alpha
+                ]
+            )
+
+        rows.append(
+            row
+        )
+
+
+        # ====================================================
+        # PROGRESS
+        # ====================================================
+
+        if (
+            i == 0
+            or (i + 1) % 20 == 0
+            or i + 1 == 160
+        ):
+
+            print(
+                f"{i + 1:3d}/160 | "
+                f"teacher={teacher_cos:.6f} | "
+                f"correction={correction_cos:.6f} | "
+                f"blend@0.20="
+                f"{prompt_blends[0.20]:.6f} | "
+                f"{category}"
+            )
+
+
+# ============================================================
+# GLOBAL METRICS
+# ============================================================
+
+teacher_tensor = torch.tensor(
+    teacher_cos_all,
+    dtype=torch.float32,
+)
+
+correction_tensor = torch.tensor(
+    correction_cos_all,
+    dtype=torch.float32,
+)
+
+
+teacher_mean = (
+    teacher_tensor.mean().item()
+)
+
+teacher_std = (
+    teacher_tensor.std().item()
+)
+
+teacher_min = (
+    teacher_tensor.min().item()
+)
+
+teacher_max = (
+    teacher_tensor.max().item()
+)
+
+
+correction_mean = (
+    correction_tensor.mean().item()
+)
+
+correction_std = (
+    correction_tensor.std().item()
+)
+
+correction_min = (
+    correction_tensor.min().item()
+)
+
+correction_max = (
+    correction_tensor.max().item()
+)
+
+
+global_blend = {}
+
+for alpha in ALPHAS:
+
+    values = torch.tensor(
+        blend_scores[
+            alpha
+        ],
+        dtype=torch.float32,
+    )
+
+    global_blend[
+        alpha
+    ] = {
+        "mean":
+            values.mean().item(),
+
+        "std":
+            values.std().item(),
+
+        "min":
+            values.min().item(),
+
+        "max":
+            values.max().item(),
+    }
+
+
+# ============================================================
+# CATEGORY RESULTS
+# ============================================================
+
+categories = sorted(
+    category_scores.keys()
+)
+
+category_results = {}
+
+for category in categories:
+
+    t = torch.tensor(
+        category_scores[
+            category
+        ],
+        dtype=torch.float32,
+    )
+
+    c = torch.tensor(
+        category_correction[
+            category
+        ],
+        dtype=torch.float32,
+    )
+
+    category_results[
+        category
+    ] = {
+        "teacher":
+            t.mean().item(),
+
+        "correction":
+            c.mean().item(),
+    }
+
+    for alpha in ALPHAS:
+
+        b = torch.tensor(
+            category_blend[
+                alpha
+            ][
+                category
+            ],
+            dtype=torch.float32,
+        )
+
+        category_results[
+            category
+        ][
+            f"blend_{alpha}"
+        ] = (
+            b.mean().item()
+        )
+
+
+# ============================================================
+# SAVE CSV
+# ============================================================
+
+fields = [
+    "index",
+    "category",
+    "token_count",
+    "teacher_rep_cos",
+    "correction_cos",
+]
+
+for alpha in ALPHAS:
+
+    fields.append(
+        f"blend_alpha_{alpha}"
+    )
+
+fields.append(
+    "prompt"
+)
+
+
+with open(
+    CSV_FILE,
+    "w",
+    newline="",
+    encoding="utf-8",
+) as f:
+
+    writer = csv.DictWriter(
+        f,
+        fieldnames=fields,
+    )
+
+    writer.writeheader()
+
+    writer.writerows(
+        rows
+    )
+
+
+# ============================================================
+# WORST / BEST
+# ============================================================
+
+sorted_correction = sorted(
+    rows,
+    key=lambda x:
+        x[
+            "correction_cos"
+        ]
+)
+
+worst = sorted_correction[
+    :10
+]
+
+best = list(
+    reversed(
+        sorted_correction[
+            -10:
+        ]
+    )
+)
+
+
+# ============================================================
+# SAVE REPORT
+# ============================================================
+
+with open(
+    REPORT_FILE,
+    "w",
+    encoding="utf-8",
+) as f:
+
+    f.write(
+        "DISTILLED STUDENT V2 - STRICT OOD EVALUATION\n"
+    )
+
+    f.write(
+        "=" * 100
+        + "\n\n"
+    )
+
+    f.write(
+        "Student was trained on the previous 440-prompt dataset.\n"
+    )
+
+    f.write(
+        "These 160 prompts were not used for student training.\n\n"
+    )
+
+    f.write(
+        "Student input: H3 L49\n"
+    )
+
+    f.write(
+        "Teacher: SenseNova L32 -> rank128 Full Bridge\n"
+    )
+
+    f.write(
+        "Student architecture: "
+        "5120 -> 512 -> 512 -> 5120, SiLU\n\n"
+    )
+
+
+    f.write(
+        "PROJECTED TEACHER REPRESENTATION\n"
+    )
+
+    f.write(
+        "-" * 100
+        + "\n"
+    )
+
+    f.write(
+        f"mean = {teacher_mean:.6f}\n"
+        f"std  = {teacher_std:.6f}\n"
+        f"min  = {teacher_min:.6f}\n"
+        f"max  = {teacher_max:.6f}\n\n"
+    )
+
+
+    f.write(
+        "FULL-BRIDGE CORRECTION\n"
+    )
+
+    f.write(
+        "-" * 100
+        + "\n"
+    )
+
+    f.write(
+        f"mean = {correction_mean:.6f}\n"
+        f"std  = {correction_std:.6f}\n"
+        f"min  = {correction_min:.6f}\n"
+        f"max  = {correction_max:.6f}\n\n"
+    )
+
+
+    f.write(
+        "FULL vs DISTILLED BLENDED CONDITIONING\n"
+    )
+
+    f.write(
+        "-" * 100
+        + "\n"
+    )
+
+    for alpha in ALPHAS:
+
+        r = global_blend[
+            alpha
+        ]
+
+        f.write(
+            f"alpha={alpha:.2f} | "
+            f"mean={r['mean']:.6f} | "
+            f"std={r['std']:.6f} | "
+            f"min={r['min']:.6f} | "
+            f"max={r['max']:.6f}\n"
+        )
+
+
+    f.write(
+        "\nCATEGORY RESULTS\n"
+    )
+
+    f.write(
+        "-" * 100
+        + "\n"
+    )
+
+    for category in categories:
+
+        r = category_results[
+            category
+        ]
+
+        f.write(
+            f"{category:28s} | "
+            f"teacher={r['teacher']:.6f} | "
+            f"correction={r['correction']:.6f} | "
+            f"blend@0.20={r['blend_0.2']:.6f}\n"
+        )
+
+
+    f.write(
+        "\nTOP 10 WORST CORRECTION PROMPTS\n"
+    )
+
+    f.write(
+        "-" * 100
+        + "\n"
+    )
+
+    for i, row in enumerate(
+        worst,
+        1,
+    ):
+
+        f.write(
+            f"{i:02d}. "
+            f"correction={row['correction_cos']:.6f} | "
+            f"teacher={row['teacher_rep_cos']:.6f} | "
+            f"{row['category']}\n"
+        )
+
+        f.write(
+            f"    {row['prompt']}\n"
+        )
+
+
+    f.write(
+        "\nTOP 10 BEST CORRECTION PROMPTS\n"
+    )
+
+    f.write(
+        "-" * 100
+        + "\n"
+    )
+
+    for i, row in enumerate(
+        best,
+        1,
+    ):
+
+        f.write(
+            f"{i:02d}. "
+            f"correction={row['correction_cos']:.6f} | "
+            f"teacher={row['teacher_rep_cos']:.6f} | "
+            f"{row['category']}\n"
+        )
+
+        f.write(
+            f"    {row['prompt']}\n"
+        )
+
+
+# ============================================================
+# CONSOLE RESULT
+# ============================================================
+
+print()
+
+print("=" * 100)
+
+print(
+    "STRICT OOD FINAL RESULT"
+)
+
+print("=" * 100)
+
+print()
+
+print(
+    "Student -> projected SenseNova:"
+)
+
+print(
+    f"  mean: {teacher_mean:.6f}"
+)
+
+print(
+    f"  min : {teacher_min:.6f}"
+)
+
+
+print()
+
+print(
+    "Student -> Full Bridge correction:"
+)
+
+print(
+    f"  mean: {correction_mean:.6f}"
+)
+
+print(
+    f"  min : {correction_min:.6f}"
+)
+
+
+print()
+
+print(
+    "Full vs Distilled conditioning:"
+)
+
+for alpha in ALPHAS:
+
+    r = global_blend[
+        alpha
+    ]
+
+    print(
+        f"  alpha={alpha:.2f}: "
+        f"{r['mean']:.6f}"
+    )
+
+
+print()
+
+print(
+    "BY CATEGORY:"
+)
+
+for category in categories:
+
+    r = category_results[
+        category
+    ]
+
+    print(
+        f"{category:28s} "
+        f"teacher={r['teacher']:.6f} "
+        f"correction={r['correction']:.6f}"
+    )
+
+
+print()
+
+print("=" * 100)
+
+print(
+    "INTERPRETATION"
+)
+
+print("=" * 100)
+
+
+if correction_mean >= 0.90:
+
+    print(
+        "EXCELLENT: distilled student generalizes strongly "
+        "to strict OOD prompts."
+    )
+
+elif correction_mean >= 0.80:
+
+    print(
+        "VERY GOOD: distilled student preserves most of "
+        "the Full Bridge correction on strict OOD prompts."
+    )
+
+elif correction_mean >= 0.70:
+
+    print(
+        "PROMISING: meaningful generalization exists, "
+        "but additional distillation data may help."
+    )
+
+else:
+
+    print(
+        "WEAK OOD GENERALIZATION: student needs a broader "
+        "training/distillation dataset."
+    )
+
+
+print()
+
+print(
+    "Report:"
+)
+
+print(
+    REPORT_FILE
+)
+
+print()
+
+print(
+    "CSV:"
+)
+
+print(
+    CSV_FILE
+)
+
+print()
+
+print(
+    "DONE"
+)

research/raw_scripts/evaluate_hidden_bridge_ood.py

@@ -0,0 +1,700 @@
+import os
+import csv
+import torch
+import torch.nn.functional as F
+
+from safetensors.torch import load_file
+
+
+# ============================================================
+# PATHS
+# ============================================================
+
+ROOT = r"E:\Models SD XL\UNET\Minimax_H3"
+
+SN_FILE = os.path.join(
+    ROOT,
+    "sensenova_ood_hidden_160.pt"
+)
+
+H3_FILE = os.path.join(
+    ROOT,
+    "h3_ood_hidden_160.pt"
+)
+
+BRIDGE_FILE = os.path.join(
+    ROOT,
+    "hidden_bridge_probes_rank128",
+    "SN_L32_to_H3_L49_rank128.safetensors"
+)
+
+OUT_TXT = os.path.join(
+    ROOT,
+    "hidden_bridge_OOD_final_report.txt"
+)
+
+OUT_CSV = os.path.join(
+    ROOT,
+    "hidden_bridge_OOD_per_prompt.csv"
+)
+
+
+# ============================================================
+# LOAD
+# ============================================================
+
+print("=" * 90)
+print("SenseNova L32 -> H3 L49")
+print("STRICT OOD FROZEN-BRIDGE EVALUATION")
+print("=" * 90)
+
+print("\nLoading SenseNova OOD states...")
+
+sn_data = torch.load(
+    SN_FILE,
+    map_location="cpu",
+    weights_only=False
+)
+
+print("SenseNova prompts:", len(sn_data["results"]))
+
+print("\nLoading H3 OOD states...")
+
+h3_data = torch.load(
+    H3_FILE,
+    map_location="cpu",
+    weights_only=False
+)
+
+print("H3 prompts:", len(h3_data["results"]))
+
+print("\nLoading frozen bridge...")
+
+bridge = load_file(
+    BRIDGE_FILE,
+    device="cpu"
+)
+
+down = bridge["down.weight"].float()
+up = bridge["up.weight"].float()
+
+print("down:", tuple(down.shape))
+print("up:  ", tuple(up.shape))
+
+
+# ============================================================
+# EXPECTED SHAPES
+# ============================================================
+
+if tuple(down.shape) != (128, 4096):
+    raise RuntimeError(
+        f"Unexpected down shape: {tuple(down.shape)}"
+    )
+
+if tuple(up.shape) != (5120, 128):
+    raise RuntimeError(
+        f"Unexpected up shape: {tuple(up.shape)}"
+    )
+
+
+# ============================================================
+# CHECK DATA ALIGNMENT
+# ============================================================
+
+sn_results = sn_data["results"]
+h3_results = h3_data["results"]
+
+if len(sn_results) != 160:
+    raise RuntimeError(
+        f"Expected 160 SenseNova prompts, got {len(sn_results)}"
+    )
+
+if len(h3_results) != 160:
+    raise RuntimeError(
+        f"Expected 160 H3 prompts, got {len(h3_results)}"
+    )
+
+print("\nChecking prompt/token alignment...")
+
+for i, (s, h) in enumerate(
+    zip(sn_results, h3_results)
+):
+
+    if s["prompt"] != h["prompt"]:
+        raise RuntimeError(
+            f"Prompt mismatch at index {i}"
+        )
+
+    if s["category"] != h["category"]:
+        raise RuntimeError(
+            f"Category mismatch at index {i}"
+        )
+
+    if not torch.equal(
+        s["input_ids"],
+        h["input_ids"]
+    ):
+        raise RuntimeError(
+            f"Token mismatch at index {i}:\n"
+            f"{s['prompt']}"
+        )
+
+print("Alignment: PERFECT")
+
+
+# ============================================================
+# SAME NORMALIZATION AS TRAINING
+# ============================================================
+
+def normalize_input(x):
+
+    x = x.float()
+
+    rms = torch.sqrt(
+        x.pow(2).mean(
+            dim=-1,
+            keepdim=True
+        ) + 1e-6
+    )
+
+    return x / rms
+
+
+# ============================================================
+# FROZEN BRIDGE
+# ============================================================
+
+def project(x):
+
+    x = normalize_input(x)
+
+    z = F.linear(
+        x,
+        down
+    )
+
+    y = F.linear(
+        z,
+        up
+    )
+
+    return y
+
+
+# ============================================================
+# EVALUATE
+# ============================================================
+
+all_token_cos = []
+all_prompt_rows = []
+
+category_token_scores = {}
+category_prompt_scores = {}
+
+print()
+print("=" * 90)
+print("EVALUATING 160 STRICT OOD PROMPTS")
+print("=" * 90)
+
+with torch.no_grad():
+
+    for i, (s, h) in enumerate(
+        zip(sn_results, h3_results)
+    ):
+
+        prompt = s["prompt"]
+        category = s["category"]
+
+        sn_hidden = s["hidden"].float()
+        h3_hidden = h["hidden"].float()
+
+        if sn_hidden.shape[-1] != 4096:
+            raise RuntimeError(
+                f"Bad SN dim at {i}: {sn_hidden.shape}"
+            )
+
+        if h3_hidden.shape[-1] != 5120:
+            raise RuntimeError(
+                f"Bad H3 dim at {i}: {h3_hidden.shape}"
+            )
+
+        if sn_hidden.shape[:2] != h3_hidden.shape[:2]:
+            raise RuntimeError(
+                f"Sequence mismatch at {i}"
+            )
+
+        pred = project(
+            sn_hidden
+        )
+
+        cos = F.cosine_similarity(
+            pred,
+            h3_hidden,
+            dim=-1
+        )
+
+        # [1,T] -> [T]
+        cos = cos.squeeze(0)
+
+        prompt_cos = (
+            cos.mean().item()
+        )
+
+        token_min = (
+            cos.min().item()
+        )
+
+        token_max = (
+            cos.max().item()
+        )
+
+        token_std = (
+            cos.std().item()
+            if cos.numel() > 1
+            else 0.0
+        )
+
+        pred_norm = (
+            pred.norm(
+                dim=-1
+            ).mean().item()
+        )
+
+        h3_norm = (
+            h3_hidden.norm(
+                dim=-1
+            ).mean().item()
+        )
+
+        norm_ratio = (
+            pred_norm / h3_norm
+            if h3_norm != 0
+            else 0.0
+        )
+
+
+        # ====================================================
+        # GLOBAL TOKEN SCORES
+        # ====================================================
+
+        all_token_cos.append(
+            cos.cpu()
+        )
+
+
+        # ====================================================
+        # CATEGORY SCORES
+        # ====================================================
+
+        category_token_scores.setdefault(
+            category,
+            []
+        ).extend(
+            cos.cpu().tolist()
+        )
+
+        category_prompt_scores.setdefault(
+            category,
+            []
+        ).append(
+            prompt_cos
+        )
+
+
+        # ====================================================
+        # PER-PROMPT ROW
+        # ====================================================
+
+        all_prompt_rows.append({
+            "index": i,
+            "category": category,
+            "token_count": cos.numel(),
+            "cosine_mean": prompt_cos,
+            "cosine_std": token_std,
+            "cosine_min": token_min,
+            "cosine_max": token_max,
+            "norm_ratio": norm_ratio,
+            "prompt": prompt,
+        })
+
+
+        if (
+            i == 0
+            or (i + 1) % 20 == 0
+            or i + 1 == 160
+        ):
+
+            print(
+                f"{i + 1:3d}/160 | "
+                f"cos={prompt_cos:.6f} | "
+                f"{category}"
+            )
+
+
+# ============================================================
+# GLOBAL RESULTS
+# ============================================================
+
+all_token_cos = torch.cat(
+    all_token_cos
+)
+
+global_mean = (
+    all_token_cos.mean().item()
+)
+
+global_std = (
+    all_token_cos.std().item()
+)
+
+global_min = (
+    all_token_cos.min().item()
+)
+
+global_max = (
+    all_token_cos.max().item()
+)
+
+
+prompt_values = torch.tensor(
+    [
+        x["cosine_mean"]
+        for x in all_prompt_rows
+    ],
+    dtype=torch.float32
+)
+
+prompt_mean = (
+    prompt_values.mean().item()
+)
+
+prompt_std = (
+    prompt_values.std().item()
+)
+
+prompt_min = (
+    prompt_values.min().item()
+)
+
+prompt_max = (
+    prompt_values.max().item()
+)
+
+
+# ============================================================
+# CATEGORY RESULTS
+# ============================================================
+
+category_results = {}
+
+for category in sorted(
+    category_token_scores
+):
+
+    token_values = torch.tensor(
+        category_token_scores[
+            category
+        ],
+        dtype=torch.float32
+    )
+
+    prompt_values_cat = torch.tensor(
+        category_prompt_scores[
+            category
+        ],
+        dtype=torch.float32
+    )
+
+    category_results[
+        category
+    ] = {
+        "token_cosine":
+            token_values.mean().item(),
+
+        "prompt_cosine":
+            prompt_values_cat.mean().item(),
+
+        "prompt_std":
+            prompt_values_cat.std().item(),
+
+        "prompt_min":
+            prompt_values_cat.min().item(),
+
+        "prompt_max":
+            prompt_values_cat.max().item(),
+
+        "prompt_count":
+            len(
+                category_prompt_scores[
+                    category
+                ]
+            ),
+    }
+
+
+# ============================================================
+# SAVE CSV
+# ============================================================
+
+with open(
+    OUT_CSV,
+    "w",
+    newline="",
+    encoding="utf-8"
+) as f:
+
+    fields = [
+        "index",
+        "category",
+        "token_count",
+        "cosine_mean",
+        "cosine_std",
+        "cosine_min",
+        "cosine_max",
+        "norm_ratio",
+        "prompt",
+    ]
+
+    writer = csv.DictWriter(
+        f,
+        fieldnames=fields
+    )
+
+    writer.writeheader()
+
+    writer.writerows(
+        all_prompt_rows
+    )
+
+
+# ============================================================
+# SORT BEST / WORST PROMPTS
+# ============================================================
+
+best_prompts = sorted(
+    all_prompt_rows,
+    key=lambda x: x[
+        "cosine_mean"
+    ],
+    reverse=True
+)
+
+worst_prompts = sorted(
+    all_prompt_rows,
+    key=lambda x: x[
+        "cosine_mean"
+    ]
+)
+
+
+# ============================================================
+# SAVE REPORT
+# ============================================================
+
+with open(
+    OUT_TXT,
+    "w",
+    encoding="utf-8"
+) as f:
+
+    f.write(
+        "SenseNova L32 -> MiniMax H3 L49\n"
+    )
+
+    f.write(
+        "STRICT OOD FROZEN-BRIDGE EVALUATION\n"
+    )
+
+    f.write("=" * 90 + "\n\n")
+
+    f.write(
+        "Bridge was trained on the previous "
+        "440-prompt dataset.\n"
+    )
+
+    f.write(
+        "No training or adaptation was performed "
+        "on these 160 OOD prompts.\n\n"
+    )
+
+    f.write(
+        "Bridge: "
+        "SN_L32_to_H3_L49_rank128\n\n"
+    )
+
+    f.write(
+        "GLOBAL TOKEN-LEVEL\n"
+    )
+
+    f.write("-" * 90 + "\n")
+
+    f.write(
+        f"cosine_mean = {global_mean:.6f}\n"
+        f"cosine_std  = {global_std:.6f}\n"
+        f"cosine_min  = {global_min:.6f}\n"
+        f"cosine_max  = {global_max:.6f}\n\n"
+    )
+
+    f.write(
+        "GLOBAL PROMPT-LEVEL\n"
+    )
+
+    f.write("-" * 90 + "\n")
+
+    f.write(
+        f"cosine_mean = {prompt_mean:.6f}\n"
+        f"cosine_std  = {prompt_std:.6f}\n"
+        f"cosine_min  = {prompt_min:.6f}\n"
+        f"cosine_max  = {prompt_max:.6f}\n\n"
+    )
+
+    f.write(
+        "CATEGORY RESULTS\n"
+    )
+
+    f.write("-" * 90 + "\n")
+
+    for category, r in (
+        category_results.items()
+    ):
+
+        f.write(
+            f"{category:28s} | "
+            f"prompt_cos={r['prompt_cosine']:.6f} | "
+            f"token_cos={r['token_cosine']:.6f} | "
+            f"std={r['prompt_std']:.6f} | "
+            f"min={r['prompt_min']:.6f} | "
+            f"max={r['prompt_max']:.6f}\n"
+        )
+
+
+    f.write("\n")
+    f.write(
+        "TOP 10 BEST OOD PROMPTS\n"
+    )
+
+    f.write("-" * 90 + "\n")
+
+    for i, row in enumerate(
+        best_prompts[:10],
+        1
+    ):
+
+        f.write(
+            f"{i:02d}. "
+            f"{row['cosine_mean']:.6f} | "
+            f"{row['category']}\n"
+        )
+
+        f.write(
+            f"    {row['prompt']}\n"
+        )
+
+
+    f.write("\n")
+    f.write(
+        "TOP 10 WORST OOD PROMPTS\n"
+    )
+
+    f.write("-" * 90 + "\n")
+
+    for i, row in enumerate(
+        worst_prompts[:10],
+        1
+    ):
+
+        f.write(
+            f"{i:02d}. "
+            f"{row['cosine_mean']:.6f} | "
+            f"{row['category']}\n"
+        )
+
+        f.write(
+            f"    {row['prompt']}\n"
+        )
+
+
+# ============================================================
+# CONSOLE REPORT
+# ============================================================
+
+print()
+print("=" * 90)
+print("FINAL STRICT OOD RESULT")
+print("=" * 90)
+
+print()
+print(
+    "Token-level cosine:",
+    f"{global_mean:.6f}"
+)
+
+print(
+    "Prompt-level cosine:",
+    f"{prompt_mean:.6f}"
+)
+
+print()
+print("BY CATEGORY:")
+print()
+
+for category, r in (
+    category_results.items()
+):
+
+    print(
+        f"{category:28s} "
+        f"{r['prompt_cosine']:.6f}"
+    )
+
+
+print()
+print("=" * 90)
+print("INTERPRETATION")
+print("=" * 90)
+
+if prompt_mean >= 0.85:
+
+    print(
+        "EXCELLENT: bridge generalizes extremely well "
+        "to strict OOD prompts."
+    )
+
+elif prompt_mean >= 0.80:
+
+    print(
+        "VERY STRONG: bridge generalizes well "
+        "outside the training prompt distribution."
+    )
+
+elif prompt_mean >= 0.70:
+
+    print(
+        "GOOD: substantial cross-model alignment "
+        "survives OOD generalization."
+    )
+
+elif prompt_mean >= 0.55:
+
+    print(
+        "MODERATE: real alignment exists, "
+        "but generalization is limited."
+    )
+
+else:
+
+    print(
+        "WEAK: training-set alignment did not "
+        "generalize sufficiently."
+    )
+
+
+print()
+print("Report:")
+print(OUT_TXT)
+
+print()
+print("Per-prompt CSV:")
+print(OUT_CSV)
+
+print("=" * 90)

research/raw_scripts/extract_h3_bridge_dataset.py

@@ -0,0 +1,272 @@
+import os
+import sys
+import json
+import gc
+import torch
+
+ROOT = r"E:\Models SD XL\UNET\Minimax_H3"
+
+COMFY_ROOT = r"D:\ComfyUI_Python312\ComfyUI"
+
+H3_TE = (
+    r"D:\ComfyUI_Python312\ComfyUI\models\text_encoders"
+    r"\qwen3vl_32b_minimax_h3_nvfp4_awq.safetensors"
+)
+
+PROMPT_FILE = os.path.join(
+    ROOT,
+    "bridge_prompts_480.json"
+)
+
+OUT = os.path.join(
+    ROOT,
+    "h3_bridge_hidden_480.pt"
+)
+
+TARGET_LAYERS = [
+    8, 16, 24, 32, 40, 49
+]
+
+
+# ============================================================
+# COMFY
+# ============================================================
+
+sys.path.insert(0, COMFY_ROOT)
+os.chdir(COMFY_ROOT)
+
+import comfy.sd
+
+from transformers import AutoTokenizer
+
+
+print("=" * 80)
+print("H3 bridge dataset extractor")
+print("=" * 80)
+
+
+# ============================================================
+# PROMPTS
+# ============================================================
+
+with open(
+    PROMPT_FILE,
+    "r",
+    encoding="utf-8"
+) as f:
+
+    dataset = json.load(f)
+
+print("Prompts:", len(dataset))
+
+
+# ============================================================
+# TOKENIZER
+# ============================================================
+
+print("\nLoading tokenizer...")
+
+tokenizer = AutoTokenizer.from_pretrained(
+    "Qwen/Qwen3-VL-32B-Instruct",
+    trust_remote_code=True,
+)
+
+
+# ============================================================
+# H3 TE
+# ============================================================
+
+print("Loading H3 TE...")
+
+clip = comfy.sd.load_clip(
+    ckpt_paths=[H3_TE],
+    embedding_directory=None,
+    clip_type=comfy.sd.CLIPType.MINIMAX,
+    model_options={
+        "load_device": torch.device("cuda"),
+        "offload_device": torch.device("cpu"),
+    },
+)
+
+root = clip.cond_stage_model
+
+
+# ============================================================
+# FIND TRANSFORMER
+# ============================================================
+
+transformer = None
+
+for name, module in root.named_modules():
+
+    if name.endswith(
+        "qwen3vl_32b.transformer"
+    ):
+
+        transformer = module
+
+        print(
+            "Transformer:",
+            name
+        )
+
+        break
+
+
+if transformer is None:
+    raise RuntimeError(
+        "H3 transformer not found"
+    )
+
+
+# ============================================================
+# HOOKS
+# ============================================================
+
+captured = {}
+hooks = {}
+
+
+def make_hook(idx):
+
+    def hook(module, inputs, output):
+
+        if isinstance(output, tuple):
+            x = output[0]
+        else:
+            x = output
+
+        captured[idx] = (
+            x.detach()
+            .to(
+                device="cpu",
+                dtype=torch.float16
+            )
+        )
+
+    return hook
+
+
+for idx in TARGET_LAYERS:
+
+    target_name = (
+        f"model.layers.{idx}"
+    )
+
+    target = None
+
+    for name, module in transformer.named_modules():
+
+        if name == target_name:
+            target = module
+            break
+
+    if target is None:
+        raise RuntimeError(
+            f"Layer {idx} not found"
+        )
+
+    hooks[idx] = (
+        target.register_forward_hook(
+            make_hook(idx)
+        )
+    )
+
+
+print(
+    "Hooks:",
+    TARGET_LAYERS
+)
+
+
+# ============================================================
+# EXTRACT
+# ============================================================
+
+results = []
+
+for i, row in enumerate(dataset):
+
+    prompt = row["prompt"]
+
+    tokens = tokenizer(
+        prompt,
+        return_tensors="pt",
+        add_special_tokens=False,
+    )
+
+    input_ids = (
+        tokens["input_ids"]
+        .to("cuda")
+    )
+
+    captured.clear()
+
+    with torch.inference_mode():
+
+        out = transformer(
+            input_ids=input_ids
+        )
+
+    item = {
+        "prompt": prompt,
+        "category": row["category"],
+        "input_ids": (
+            input_ids.detach()
+            .cpu()
+        ),
+        "layers": {}
+    }
+
+    for idx in TARGET_LAYERS:
+
+        if idx not in captured:
+            raise RuntimeError(
+                f"Missing H3 layer {idx}"
+            )
+
+        item["layers"][idx] = (
+            captured[idx]
+        )
+
+    results.append(item)
+
+    del out
+    del input_ids
+
+    if (i + 1) % 10 == 0:
+
+        print(
+            f"{i + 1:4d}/"
+            f"{len(dataset)}"
+        )
+
+    if (i + 1) % 25 == 0:
+
+        torch.cuda.empty_cache()
+
+
+# ============================================================
+# SAVE
+# ============================================================
+
+for h in hooks.values():
+    h.remove()
+
+
+torch.save(
+    {
+        "target_layers": TARGET_LAYERS,
+        "results": results,
+    },
+    OUT
+)
+
+print()
+print("=" * 80)
+print("SUCCESS")
+print(OUT)
+print("=" * 80)
+
+gc.collect()
+torch.cuda.empty_cache()

research/raw_scripts/extract_h3_embeddings_full.py

@@ -0,0 +1,188 @@
+import os
+import sys
+import gc
+import torch
+
+from safetensors.torch import save_file
+
+COMFY_ROOT = r"D:\ComfyUI_Python312\ComfyUI"
+
+H3_TE = r"D:\ComfyUI_Python312\ComfyUI\models\text_encoders\qwen3vl_32b_minimax_h3_nvfp4_awq.safetensors"
+
+OUT = r"E:\Models SD XL\UNET\Minimax_H3\h3_qwen3vl32b_embeddings_bf16.safetensors"
+
+VOCAB_SIZE = 151936
+HIDDEN_SIZE = 5120
+
+# Количество token vectors за один проход.
+# 256 очень консервативно по памяти.
+BATCH = 256
+
+
+# ---------------------------------------------------------
+# Load ComfyUI
+# ---------------------------------------------------------
+
+sys.path.insert(0, COMFY_ROOT)
+os.chdir(COMFY_ROOT)
+
+import comfy.sd
+
+
+print("=" * 80)
+print("MiniMax H3 embedding extractor")
+print("=" * 80)
+
+print("Loading H3 TE...")
+
+clip = comfy.sd.load_clip(
+    ckpt_paths=[H3_TE],
+    embedding_directory=None,
+    clip_type=comfy.sd.CLIPType.MINIMAX,
+    model_options={
+        "load_device": torch.device("cpu"),
+        "offload_device": torch.device("cpu"),
+    },
+)
+
+root = clip.cond_stage_model
+
+
+# ---------------------------------------------------------
+# Locate embed_tokens
+# ---------------------------------------------------------
+
+embed_module = None
+embed_name = None
+
+for name, module in root.named_modules():
+    if "embed_tokens" in name.lower():
+        embed_module = module
+        embed_name = name
+        break
+
+if embed_module is None:
+    raise RuntimeError("embed_tokens module not found")
+
+
+print("Embedding module:")
+print(embed_name)
+
+print("Weight shape:")
+print(tuple(embed_module.weight.shape))
+
+if tuple(embed_module.weight.shape) != (VOCAB_SIZE, HIDDEN_SIZE):
+    raise RuntimeError(
+        f"Unexpected embedding shape: {tuple(embed_module.weight.shape)}"
+    )
+
+
+# ---------------------------------------------------------
+# Allocate final BF16 matrix
+# ---------------------------------------------------------
+
+print()
+print("Allocating output matrix...")
+print(f"{VOCAB_SIZE} x {HIDDEN_SIZE} BF16")
+
+output = torch.empty(
+    (VOCAB_SIZE, HIDDEN_SIZE),
+    dtype=torch.bfloat16,
+    device="cpu",
+)
+
+
+# ---------------------------------------------------------
+# Extract in batches
+# ---------------------------------------------------------
+
+print()
+print("Extracting...")
+
+with torch.no_grad():
+
+    for start in range(0, VOCAB_SIZE, BATCH):
+
+        end = min(start + BATCH, VOCAB_SIZE)
+
+        ids = torch.arange(
+            start,
+            end,
+            dtype=torch.long,
+        ).unsqueeze(0)
+
+        emb = embed_module(ids)
+
+        # [1, batch, 5120] -> [batch, 5120]
+        emb = emb.squeeze(0)
+
+        output[start:end].copy_(
+            emb.to(dtype=torch.bfloat16, device="cpu")
+        )
+
+        if start % (BATCH * 20) == 0 or end == VOCAB_SIZE:
+            pct = 100.0 * end / VOCAB_SIZE
+
+            print(
+                f"{end:6d}/{VOCAB_SIZE} "
+                f"({pct:6.2f}%)"
+            )
+
+
+# ---------------------------------------------------------
+# Validation
+# ---------------------------------------------------------
+
+print()
+print("Validating...")
+
+finite = torch.isfinite(output).all().item()
+
+print("Shape :", tuple(output.shape))
+print("Dtype :", output.dtype)
+print("Finite:", finite)
+
+sample = output.float()
+
+print("Min :", sample.min().item())
+print("Max :", sample.max().item())
+print("Mean:", sample.mean().item())
+print("Std :", sample.std().item())
+
+del sample
+
+
+if not finite:
+    raise RuntimeError("Non-finite values detected!")
+
+
+# ---------------------------------------------------------
+# Save
+# ---------------------------------------------------------
+
+print()
+print("Saving:")
+print(OUT)
+
+save_file(
+    {
+        "model.embed_tokens.weight": output
+    },
+    OUT,
+    metadata={
+        "source": os.path.basename(H3_TE),
+        "description": "Dequantized MiniMax H3 Qwen3-VL-32B token embeddings",
+        "vocab_size": str(VOCAB_SIZE),
+        "hidden_size": str(HIDDEN_SIZE),
+        "dtype": "bfloat16",
+    },
+)
+
+print()
+print("DONE")
+print(OUT)
+
+# cleanup
+del output
+del clip
+gc.collect()

research/raw_scripts/extract_h3_embeddings_test.py

@@ -0,0 +1,139 @@
+import os
+import sys
+import torch
+
+COMFY_ROOT = r"D:\ComfyUI_Python312\ComfyUI"
+
+H3_TE = r"D:\ComfyUI_Python312\ComfyUI\models\text_encoders\qwen3vl_32b_minimax_h3_nvfp4_awq.safetensors"
+
+OUT = r"E:\Models SD XL\UNET\Minimax_H3\h3_embedding_test.pt"
+
+# ---------------------------------------------------------
+# Add ComfyUI to Python path
+# ---------------------------------------------------------
+
+sys.path.insert(0, COMFY_ROOT)
+os.chdir(COMFY_ROOT)
+
+import comfy.sd
+
+
+print("=" * 80)
+print("Loading MiniMax H3 text encoder through ComfyUI")
+print("=" * 80)
+
+clip = comfy.sd.load_clip(
+    ckpt_paths=[H3_TE],
+    embedding_directory=None,
+    clip_type=comfy.sd.CLIPType.MINIMAX,
+    model_options={
+        # CPU keeps VRAM usage low for this test.
+        # Change to cuda later if needed.
+        "load_device": torch.device("cpu"),
+        "offload_device": torch.device("cpu"),
+    },
+)
+
+print("CLIP loaded.")
+print("clip type:", type(clip))
+
+
+# ---------------------------------------------------------
+# Inspect module tree
+# ---------------------------------------------------------
+
+root = clip.cond_stage_model
+
+print("\ncond_stage_model:", type(root))
+
+candidates = []
+
+for name, module in root.named_modules():
+    lname = name.lower()
+
+    if "embed_tokens" in lname:
+        candidates.append((name, module))
+
+print("\nFound embed_tokens candidates:", len(candidates))
+
+for name, module in candidates:
+    print()
+    print("MODULE:", name)
+    print("TYPE:  ", type(module))
+
+    if hasattr(module, "weight"):
+        try:
+            print("WEIGHT TYPE:", type(module.weight))
+            print("WEIGHT SHAPE:", tuple(module.weight.shape))
+            print("WEIGHT DTYPE:", module.weight.dtype)
+            print("WEIGHT DEVICE:", module.weight.device)
+        except Exception as e:
+            print("Could not inspect weight directly:", repr(e))
+
+
+if not candidates:
+    print("\nNo embed_tokens module found.")
+    print("Dumping likely embedding/module names:")
+
+    for name, module in root.named_modules():
+        lname = name.lower()
+
+        if (
+            "embed" in lname
+            or "qwen" in lname
+            or "transformer" in lname
+        ):
+            print(name, type(module))
+
+    raise SystemExit(1)
+
+
+# ---------------------------------------------------------
+# Test embedding lookup
+# ---------------------------------------------------------
+
+name, embed_module = candidates[0]
+
+print("\nUsing:")
+print(name)
+
+# Several safe token IDs spread through the vocabulary.
+test_ids = torch.tensor(
+    [[0, 1, 10, 100, 1000, 10000, 50000, 100000, 151000]],
+    dtype=torch.long,
+)
+
+print("\nRunning embedding lookup...")
+
+with torch.no_grad():
+    embeddings = embed_module(test_ids)
+
+print("OUTPUT SHAPE:", tuple(embeddings.shape))
+print("OUTPUT DTYPE:", embeddings.dtype)
+print("OUTPUT DEVICE:", embeddings.device)
+
+embeddings_cpu = embeddings.detach().float().cpu()
+
+print("\nVector statistics:")
+print("min :", embeddings_cpu.min().item())
+print("max :", embeddings_cpu.max().item())
+print("mean:", embeddings_cpu.mean().item())
+print("std :", embeddings_cpu.std().item())
+
+finite = torch.isfinite(embeddings_cpu).all().item()
+print("all finite:", finite)
+
+
+torch.save(
+    {
+        "token_ids": test_ids.cpu(),
+        "embeddings": embeddings_cpu,
+        "module_name": name,
+    },
+    OUT,
+)
+
+print("\nSaved:")
+print(OUT)
+
+print("\nSUCCESS")

research/raw_scripts/extract_h3_hidden_states.py

@@ -0,0 +1,189 @@
+import os
+import sys
+import torch
+
+COMFY_ROOT = r"D:\ComfyUI_Python312\ComfyUI"
+H3_TE = r"D:\ComfyUI_Python312\ComfyUI\models\text_encoders\qwen3vl_32b_minimax_h3_nvfp4_awq.safetensors"
+
+OUT = r"E:\Models SD XL\UNET\Minimax_H3\h3_hidden_states.pt"
+
+sys.path.insert(0, COMFY_ROOT)
+os.chdir(COMFY_ROOT)
+
+import comfy.sd
+from transformers import AutoTokenizer
+
+
+PROMPTS = [
+    "a red cube on a blue sphere",
+    "a woman holding a transparent glass bottle",
+    "three people standing behind a wooden table",
+    "the word HELLO printed on a white sign",
+    "a metallic robot illuminated from the left",
+    "a hand holding a glass of water",
+    "two cars parked beside a brick building",
+    "a woman reflected in a mirror",
+    "a person standing behind another person",
+    "a glass sphere resting on a metal cube",
+    "a red chair to the left of a blue table",
+    "four candles arranged in a square",
+    "a hand with five clearly visible fingers",
+    "a woman sitting with crossed legs",
+    "a person holding an umbrella above their head",
+    "the word OPEN written in large black letters",
+    "a transparent bottle filled halfway with water",
+    "polished chrome reflecting a warm light",
+    "rough stone illuminated by soft side light",
+    "a wooden table casting a shadow to the right",
+]
+
+
+print("Loading tokenizer...")
+tokenizer = AutoTokenizer.from_pretrained(
+    "Qwen/Qwen3-VL-32B-Instruct",
+    trust_remote_code=True,
+)
+
+print("Loading H3 TE...")
+
+clip = comfy.sd.load_clip(
+    ckpt_paths=[H3_TE],
+    embedding_directory=None,
+    clip_type=comfy.sd.CLIPType.MINIMAX,
+    model_options={
+        "load_device": torch.device("cpu"),
+        "offload_device": torch.device("cpu"),
+    },
+)
+
+root = clip.cond_stage_model
+
+# Find underlying transformer
+transformer = None
+
+for name, module in root.named_modules():
+    if name.endswith("qwen3vl_32b.transformer"):
+        transformer = module
+        print("Found transformer:", name)
+        break
+
+if transformer is None:
+    print("Could not find exact transformer name.")
+    print("Candidates:")
+    for name, module in root.named_modules():
+        if "transformer" in name.lower():
+            print(name, type(module))
+    raise SystemExit(1)
+
+
+results = []
+
+hooks = {}
+captured = {}
+
+# Capture selected layers.
+TARGET_LAYERS = [8, 16, 24, 32, 40, 49]
+
+for layer_idx in TARGET_LAYERS:
+    layer_name = f"model.layers.{layer_idx}"
+
+    target_module = None
+
+    for name, module in transformer.named_modules():
+        if name == layer_name:
+            target_module = module
+            break
+
+    if target_module is None:
+        print("Layer not found:", layer_name)
+        continue
+
+    def make_hook(idx):
+        def hook(module, inputs, output):
+            if isinstance(output, tuple):
+                x = output[0]
+            else:
+                x = output
+
+            captured[idx] = x.detach().float().cpu()
+        return hook
+
+    hooks[layer_idx] = target_module.register_forward_hook(
+        make_hook(layer_idx)
+    )
+
+
+print("Registered hooks:", list(hooks.keys()))
+
+
+for i, prompt in enumerate(PROMPTS):
+
+    print()
+    print(f"[{i+1}/{len(PROMPTS)}]")
+    print(prompt)
+
+    tokens = tokenizer(
+        prompt,
+        return_tensors="pt",
+        add_special_tokens=False,
+    )
+
+    input_ids = tokens["input_ids"]
+
+    captured.clear()
+
+    # Use embedding/transformer directly.
+    with torch.no_grad():
+        try:
+            out = transformer(
+                input_ids=input_ids,
+                output_hidden_states=True,
+                return_dict=True,
+            )
+        except TypeError:
+            out = transformer(
+                input_ids=input_ids,
+            )
+
+    item = {
+        "prompt": prompt,
+        "input_ids": input_ids.cpu(),
+        "layers": {},
+    }
+
+    for idx in TARGET_LAYERS:
+        if idx in captured:
+            item["layers"][idx] = captured[idx]
+            print(
+                f"  layer {idx}:",
+                tuple(captured[idx].shape)
+            )
+
+    if hasattr(out, "last_hidden_state"):
+        item["last_hidden_state"] = (
+            out.last_hidden_state.detach().float().cpu()
+        )
+        print(
+            "  final:",
+            tuple(item["last_hidden_state"].shape)
+        )
+
+    results.append(item)
+
+
+for h in hooks.values():
+    h.remove()
+
+torch.save(
+    {
+        "prompts": PROMPTS,
+        "target_layers": TARGET_LAYERS,
+        "results": results,
+    },
+    OUT,
+)
+
+print()
+print("Saved:")
+print(OUT)
+print("DONE")

research/raw_scripts/extract_h3_hidden_states_fast.py

@@ -0,0 +1,266 @@
+import os
+import sys
+import torch
+
+COMFY_ROOT = r"D:\ComfyUI_Python312\ComfyUI"
+H3_TE = r"D:\ComfyUI_Python312\ComfyUI\models\text_encoders\qwen3vl_32b_minimax_h3_nvfp4_awq.safetensors"
+
+OUT = r"E:\Models SD XL\UNET\Minimax_H3\h3_hidden_states_fast.pt"
+
+sys.path.insert(0, COMFY_ROOT)
+os.chdir(COMFY_ROOT)
+
+import comfy.sd
+from transformers import AutoTokenizer
+
+
+PROMPTS = [
+    "a red cube on a blue sphere",
+    "a woman holding a transparent glass bottle",
+    "three people standing behind a wooden table",
+]
+
+TARGET_LAYERS = [8, 16, 24, 32, 40, 49]
+
+
+print("=" * 80)
+print("H3 hidden state extractor - FAST CUDA TEST")
+print("=" * 80)
+
+print("\nLoading tokenizer...")
+
+tokenizer = AutoTokenizer.from_pretrained(
+    "Qwen/Qwen3-VL-32B-Instruct",
+    trust_remote_code=True,
+)
+
+print("Loading H3 TE on CUDA...")
+
+clip = comfy.sd.load_clip(
+    ckpt_paths=[H3_TE],
+    embedding_directory=None,
+    clip_type=comfy.sd.CLIPType.MINIMAX,
+    model_options={
+        "load_device": torch.device("cuda"),
+        "offload_device": torch.device("cpu"),
+    },
+)
+
+root = clip.cond_stage_model
+
+print("CLIP loaded.")
+
+
+# =========================================================
+# FIND TRANSFORMER
+# =========================================================
+
+transformer = None
+transformer_name = None
+
+for name, module in root.named_modules():
+    if name.endswith("qwen3vl_32b.transformer"):
+        transformer = module
+        transformer_name = name
+        break
+
+if transformer is None:
+    print("\nCould not find exact transformer.")
+    print("Possible transformer modules:")
+
+    for name, module in root.named_modules():
+        if "transformer" in name.lower():
+            print(name, type(module))
+
+    raise SystemExit(1)
+
+print("Found transformer:", transformer_name)
+
+
+# =========================================================
+# REGISTER HOOKS
+# =========================================================
+
+captured = {}
+hooks = {}
+
+
+def make_hook(idx):
+    def hook(module, inputs, output):
+
+        if isinstance(output, tuple):
+            x = output[0]
+        else:
+            x = output
+
+        # Move immediately to CPU so VRAM does not accumulate.
+        captured[idx] = x.detach().to(
+            device="cpu",
+            dtype=torch.float32
+        )
+
+    return hook
+
+
+for layer_idx in TARGET_LAYERS:
+
+    layer_name = f"model.layers.{layer_idx}"
+
+    target_module = None
+
+    for name, module in transformer.named_modules():
+        if name == layer_name:
+            target_module = module
+            break
+
+    if target_module is None:
+        print("Layer not found:", layer_name)
+        continue
+
+    hooks[layer_idx] = target_module.register_forward_hook(
+        make_hook(layer_idx)
+    )
+
+
+print("Registered hooks:", sorted(hooks.keys()))
+
+if len(hooks) == 0:
+    raise RuntimeError("No hooks registered.")
+
+
+# =========================================================
+# RUN PROMPTS
+# =========================================================
+
+results = []
+
+for i, prompt in enumerate(PROMPTS):
+
+    print()
+    print("=" * 80)
+    print(f"[{i + 1}/{len(PROMPTS)}]")
+    print(prompt)
+    print("=" * 80)
+
+    tokens = tokenizer(
+        prompt,
+        return_tensors="pt",
+        add_special_tokens=False,
+    )
+
+    input_ids = tokens["input_ids"].to("cuda")
+
+    print("Token count:", input_ids.shape[1])
+
+    captured.clear()
+
+    torch.cuda.empty_cache()
+
+    with torch.inference_mode():
+
+        # We do NOT request output_hidden_states.
+        # Hooks capture only selected layers.
+        out = transformer(
+            input_ids=input_ids,
+        )
+
+    item = {
+        "prompt": prompt,
+        "input_ids": input_ids.detach().cpu(),
+        "layers": {},
+    }
+
+    for idx in TARGET_LAYERS:
+
+        if idx in captured:
+            item["layers"][idx] = captured[idx]
+
+            print(
+                f"layer {idx:2d}: "
+                f"shape={tuple(captured[idx].shape)} "
+                f"dtype={captured[idx].dtype}"
+            )
+
+        else:
+            print(f"layer {idx:2d}: NOT CAPTURED")
+
+    results.append(item)
+
+    del out
+    del input_ids
+
+    torch.cuda.empty_cache()
+
+
+# =========================================================
+# REMOVE HOOKS
+# =========================================================
+
+for h in hooks.values():
+    h.remove()
+
+
+# =========================================================
+# VALIDATION
+# =========================================================
+
+print()
+print("=" * 80)
+print("VALIDATION")
+print("=" * 80)
+
+all_ok = True
+
+for item in results:
+
+    print()
+    print(item["prompt"])
+
+    for idx in TARGET_LAYERS:
+
+        if idx not in item["layers"]:
+            print(f"  layer {idx}: missing")
+            all_ok = False
+            continue
+
+        x = item["layers"][idx]
+
+        finite = torch.isfinite(x).all().item()
+
+        print(
+            f"  layer {idx}: "
+            f"{tuple(x.shape)} "
+            f"finite={finite} "
+            f"mean={x.mean().item():.6f} "
+            f"std={x.std().item():.6f}"
+        )
+
+        if not finite:
+            all_ok = False
+
+
+# =========================================================
+# SAVE
+# =========================================================
+
+torch.save(
+    {
+        "prompts": PROMPTS,
+        "target_layers": TARGET_LAYERS,
+        "results": results,
+    },
+    OUT,
+)
+
+print()
+print("=" * 80)
+
+if all_ok:
+    print("SUCCESS")
+else:
+    print("COMPLETED WITH WARNINGS")
+
+print("Saved:")
+print(OUT)
+
+print("=" * 80)

research/raw_scripts/extract_h3_ood_hidden.py

@@ -0,0 +1,318 @@
+import os
+import sys
+import json
+import gc
+import torch
+
+# ============================================================
+# PATHS
+# ============================================================
+
+ROOT = r"E:\Models SD XL\UNET\Minimax_H3"
+
+COMFY_ROOT = r"D:\ComfyUI_Python312\ComfyUI"
+
+PROMPTS_FILE = os.path.join(
+    ROOT,
+    "bridge_ood_prompts_160.json"
+)
+
+OUT_FILE = os.path.join(
+    ROOT,
+    "h3_ood_hidden_160.pt"
+)
+
+H3_TE = (
+    r"D:\ComfyUI_Python312\ComfyUI\models\text_encoders"
+    r"\qwen3vl_32b_minimax_h3_nvfp4_awq.safetensors"
+)
+
+LAYER = 49
+
+
+# ============================================================
+# COMFY IMPORT
+# ============================================================
+
+sys.path.insert(0, COMFY_ROOT)
+os.chdir(COMFY_ROOT)
+
+import comfy.sd
+from transformers import AutoTokenizer
+
+
+print("=" * 80)
+print("MiniMax H3 OOD hidden-state extraction")
+print("=" * 80)
+
+print("Prompts:", PROMPTS_FILE)
+print("Output :", OUT_FILE)
+print("Layer  :", LAYER)
+
+if not torch.cuda.is_available():
+    raise RuntimeError("CUDA is required.")
+
+print("GPU:", torch.cuda.get_device_name(0))
+
+
+# ============================================================
+# LOAD PROMPTS
+# ============================================================
+
+with open(
+    PROMPTS_FILE,
+    "r",
+    encoding="utf-8"
+) as f:
+    prompts = json.load(f)
+
+print("Prompt count:", len(prompts))
+
+if len(prompts) != 160:
+    raise RuntimeError(
+        f"Expected 160 prompts, got {len(prompts)}"
+    )
+
+
+# ============================================================
+# TOKENIZER
+# ============================================================
+
+print("\nLoading tokenizer...")
+
+tokenizer = AutoTokenizer.from_pretrained(
+    "Qwen/Qwen3-VL-32B-Instruct",
+    trust_remote_code=True,
+)
+
+
+# ============================================================
+# LOAD H3 TEXT ENCODER
+# ============================================================
+
+print("\nLoading H3 text encoder through ComfyUI...")
+
+clip = comfy.sd.load_clip(
+    ckpt_paths=[H3_TE],
+    embedding_directory=None,
+    clip_type=comfy.sd.CLIPType.MINIMAX,
+    model_options={
+        "load_device": torch.device("cuda"),
+        "offload_device": torch.device("cpu"),
+    },
+)
+
+print("CLIP loaded.")
+
+
+# ============================================================
+# FIND TRANSFORMER
+# ============================================================
+
+root = clip.cond_stage_model
+
+transformer = None
+transformer_name = None
+
+for name, module in root.named_modules():
+
+    if name.endswith("qwen3vl_32b.transformer"):
+        transformer = module
+        transformer_name = name
+        break
+
+if transformer is None:
+
+    print("\nCould not find exact transformer.")
+    print("Transformer-like modules:")
+
+    for name, module in root.named_modules():
+        if "transformer" in name.lower():
+            print(name, type(module))
+
+    raise RuntimeError("H3 transformer not found.")
+
+print("Transformer:", transformer_name)
+
+
+# ============================================================
+# FIND TARGET LAYER
+# ============================================================
+
+target_layer = None
+target_name = f"model.layers.{LAYER}"
+
+for name, module in transformer.named_modules():
+
+    if name == target_name:
+        target_layer = module
+        break
+
+if target_layer is None:
+    raise RuntimeError(
+        f"H3 layer {LAYER} not found."
+    )
+
+print("Target layer:", target_name)
+
+
+# ============================================================
+# HOOK
+# ============================================================
+
+captured = {}
+
+
+def hook_fn(module, inputs, output):
+
+    if isinstance(output, tuple):
+        hidden = output[0]
+    else:
+        hidden = output
+
+    if not torch.is_tensor(hidden):
+        raise RuntimeError(
+            f"Unexpected H3 layer output type: {type(hidden)}"
+        )
+
+    captured["hidden"] = (
+        hidden.detach()
+        .to(
+            device="cpu",
+            dtype=torch.float16
+        )
+        .clone()
+    )
+
+
+handle = target_layer.register_forward_hook(
+    hook_fn
+)
+
+
+# ============================================================
+# EXTRACT
+# ============================================================
+
+results = []
+
+print("\nStarting extraction...\n")
+
+with torch.inference_mode():
+
+    for index, item in enumerate(prompts):
+
+        prompt = item["prompt"]
+        category = item["category"]
+
+        encoded = tokenizer(
+            prompt,
+            return_tensors="pt",
+            add_special_tokens=False,
+        )
+
+        input_ids_cpu = (
+            encoded["input_ids"]
+            .detach()
+            .cpu()
+        )
+
+        input_ids = (
+            encoded["input_ids"]
+            .to("cuda")
+        )
+
+        captured.clear()
+
+        output = transformer(
+            input_ids=input_ids
+        )
+
+        if "hidden" not in captured:
+            raise RuntimeError(
+                f"Layer hook failed at prompt {index}"
+            )
+
+        hidden = captured["hidden"]
+
+        if hidden.ndim != 3:
+            raise RuntimeError(
+                f"Unexpected hidden shape: {hidden.shape}"
+            )
+
+        if hidden.shape[-1] != 5120:
+            raise RuntimeError(
+                f"Expected H3 hidden dim 5120, "
+                f"got {hidden.shape[-1]}"
+            )
+
+        if not torch.isfinite(
+            hidden.float()
+        ).all():
+            raise RuntimeError(
+                f"Non-finite H3 state at prompt {index}"
+            )
+
+        results.append({
+            "prompt": prompt,
+            "category": category,
+            "input_ids": input_ids_cpu,
+            "hidden": hidden,
+        })
+
+        if (
+            index == 0
+            or (index + 1) % 10 == 0
+            or index + 1 == len(prompts)
+        ):
+            print(
+                f"{index + 1:3d}/{len(prompts)} | "
+                f"tokens={hidden.shape[1]:3d} | "
+                f"shape={tuple(hidden.shape)}"
+            )
+
+        del output
+        del input_ids
+
+        if (index + 1) % 20 == 0:
+            gc.collect()
+            torch.cuda.empty_cache()
+
+
+# ============================================================
+# CLEANUP
+# ============================================================
+
+handle.remove()
+
+
+# ============================================================
+# SAVE
+# ============================================================
+
+payload = {
+    "model": "MiniMax H3 Qwen3-VL TE",
+    "layer": LAYER,
+    "hidden_dim": 5120,
+    "prompt_file": os.path.basename(
+        PROMPTS_FILE
+    ),
+    "results": results,
+}
+
+torch.save(
+    payload,
+    OUT_FILE
+)
+
+
+print()
+print("=" * 80)
+print("SUCCESS")
+print("=" * 80)
+
+print("Prompts:", len(results))
+print("Layer  :", LAYER)
+
+print("Output:")
+print(OUT_FILE)

research/raw_scripts/extract_sensenova_bridge_dataset.py

@@ -0,0 +1,425 @@
+import os
+import sys
+import gc
+import json
+import shutil
+import torch
+
+
+# ============================================================
+# PATHS
+# ============================================================
+
+ROOT = r"E:\Models SD XL\UNET\Minimax_H3"
+
+SENSENOVA_SRC = os.path.join(
+    ROOT,
+    "SenseNova-U1",
+    "src"
+)
+
+COMPAT_FILE = os.path.join(
+    SENSENOVA_SRC,
+    "sensenova_u1",
+    "models",
+    "neo_unify",
+    "transformers_compat.py"
+)
+
+MODEL_FILE = (
+    r"D:\ComfyUI_Python312\ComfyUI\models\unet"
+    r"\SenseNova-U1.5-8B-MoT-pruned-bf16.safetensors"
+)
+
+PROMPT_FILE = os.path.join(
+    ROOT,
+    "bridge_prompts_480.json"
+)
+
+OUT = os.path.join(
+    ROOT,
+    "sensenova_bridge_hidden_480.pt"
+)
+
+HF_REPO = "sensenova/SenseNova-U1.5-8B-MoT"
+
+TARGET_LAYERS = [8, 16, 24, 32, 41]
+
+
+# ============================================================
+# PATCH TRANSFORMERS 4.57 COMPAT
+# ============================================================
+
+print("=" * 80)
+print("SenseNova U1.5 bridge dataset extractor")
+print("=" * 80)
+
+print("\nChecking Transformers compatibility patch...")
+
+with open(COMPAT_FILE, "r", encoding="utf-8") as f:
+    text = f.read()
+
+OLD = (
+    "    from transformers.utils.generic import "
+    "check_model_inputs as model_input_compat"
+)
+
+NEW = (
+    "    from transformers.utils.generic import check_model_inputs\n"
+    "    def model_input_compat(func):\n"
+    "        return check_model_inputs()(func)"
+)
+
+if OLD in text:
+
+    backup = COMPAT_FILE + ".bak"
+
+    if not os.path.exists(backup):
+        shutil.copy2(COMPAT_FILE, backup)
+
+    text = text.replace(OLD, NEW)
+
+    with open(COMPAT_FILE, "w", encoding="utf-8") as f:
+        f.write(text)
+
+    print("Patch applied.")
+
+else:
+    print("Patch already present / not needed.")
+
+
+# ============================================================
+# IMPORTS
+# ============================================================
+
+sys.path.insert(0, SENSENOVA_SRC)
+
+from transformers import AutoTokenizer
+from accelerate import init_empty_weights, load_checkpoint_and_dispatch
+
+from sensenova_u1.models.neo_unify.configuration_neo_chat import (
+    NEOChatConfig,
+)
+
+from sensenova_u1.models.neo_unify.modeling_neo_chat import (
+    NEOChatModel,
+)
+
+
+# ============================================================
+# LOAD PROMPTS
+# ============================================================
+
+print("\nLoading prompt dataset...")
+
+with open(
+    PROMPT_FILE,
+    "r",
+    encoding="utf-8"
+) as f:
+
+    dataset = json.load(f)
+
+print("Prompts:", len(dataset))
+
+
+# ============================================================
+# TOKENIZER
+# ============================================================
+
+print("\nLoading tokenizer...")
+
+tokenizer = AutoTokenizer.from_pretrained(
+    HF_REPO,
+    trust_remote_code=True,
+)
+
+
+# ============================================================
+# CONFIG
+# ============================================================
+
+print("Loading SenseNova config...")
+
+config = NEOChatConfig.from_pretrained(
+    HF_REPO
+)
+
+config.llm_config._attn_implementation = "eager"
+
+print("Hidden size:", config.llm_config.hidden_size)
+print("Layers:", config.llm_config.num_hidden_layers)
+
+
+# ============================================================
+# META ARCHITECTURE
+# ============================================================
+
+print("\nCreating META architecture...")
+
+with init_empty_weights():
+
+    model = NEOChatModel(config)
+
+print("META architecture created.")
+
+
+# ============================================================
+# LOAD CHECKPOINT
+# ============================================================
+
+print("\nLoading BF16 checkpoint with Accelerate...")
+
+model = load_checkpoint_and_dispatch(
+    model,
+    checkpoint=MODEL_FILE,
+    device_map="auto",
+    max_memory={
+        0: "19GiB",
+        "cpu": "70GiB",
+    },
+    dtype=torch.bfloat16,
+    no_split_module_classes=[
+        "Qwen3DecoderLayer",
+        "Qwen3MoeDecoderLayer",
+    ],
+)
+
+model.eval()
+
+print("Checkpoint loaded.")
+
+
+# ============================================================
+# LANGUAGE MODEL
+# ============================================================
+
+language_model = model.language_model
+qwen = language_model.model
+layers = qwen.layers
+
+print("\nDetected language layers:", len(layers))
+
+
+# ============================================================
+# INPUT DEVICE
+# ============================================================
+
+device_map = getattr(model, "hf_device_map", {})
+
+INPUT_DEVICE = torch.device("cuda:0")
+
+for name, dev in device_map.items():
+
+    if name.endswith("language_model.model.embed_tokens"):
+
+        if str(dev) == "cpu":
+            INPUT_DEVICE = torch.device("cpu")
+        else:
+            INPUT_DEVICE = torch.device("cuda:0")
+
+        break
+
+print("Input device:", INPUT_DEVICE)
+
+
+# ============================================================
+# HOOKS
+# ============================================================
+
+captured = {}
+hooks = {}
+
+
+def make_hook(idx):
+
+    def hook(module, inputs, output):
+
+        if isinstance(output, tuple):
+            x = output[0]
+        else:
+            x = output
+
+        if not torch.is_tensor(x):
+            raise RuntimeError(
+                f"Unexpected output type at layer {idx}: {type(x)}"
+            )
+
+        # float16 saves a lot of disk space.
+        captured[idx] = (
+            x.detach()
+            .to(
+                device="cpu",
+                dtype=torch.float16
+            )
+        )
+
+    return hook
+
+
+print("\nRegistering hooks...")
+
+for idx in TARGET_LAYERS:
+
+    hooks[idx] = layers[idx].register_forward_hook(
+        make_hook(idx)
+    )
+
+    print(" layer", idx)
+
+
+# ============================================================
+# EXTRACT
+# ============================================================
+
+results = []
+
+print()
+print("=" * 80)
+print("EXTRACTING")
+print("=" * 80)
+
+for i, row in enumerate(dataset):
+
+    prompt = row["prompt"]
+
+    encoded = tokenizer(
+        prompt,
+        return_tensors="pt",
+        add_special_tokens=False,
+    )
+
+    input_ids = encoded["input_ids"].to(
+        INPUT_DEVICE
+    )
+
+    captured.clear()
+
+    # SenseNova keeps an internal index for generation paths.
+    # Reset it for every independent prompt.
+    qwen.current_index = -1
+
+    with torch.inference_mode():
+
+        outputs = qwen(
+            input_ids=input_ids,
+            image_gen_indicators=None,
+            attention_mask=None,
+            use_cache=False,
+        )
+
+    item = {
+        "prompt": prompt,
+        "category": row["category"],
+        "input_ids": (
+            input_ids.detach()
+            .cpu()
+        ),
+        "layers": {},
+    }
+
+    for idx in TARGET_LAYERS:
+
+        if idx not in captured:
+
+            raise RuntimeError(
+                f"Missing SenseNova layer {idx} "
+                f"at prompt {i}"
+            )
+
+        item["layers"][idx] = (
+            captured[idx]
+        )
+
+    results.append(item)
+
+    del outputs
+    del input_ids
+
+    if (i + 1) % 10 == 0:
+
+        print(
+            f"{i + 1:4d}/"
+            f"{len(dataset)}"
+        )
+
+    if (i + 1) % 20 == 0:
+
+        gc.collect()
+        torch.cuda.empty_cache()
+
+
+# ============================================================
+# REMOVE HOOKS
+# ============================================================
+
+for h in hooks.values():
+    h.remove()
+
+
+# ============================================================
+# FINAL VALIDATION
+# ============================================================
+
+print()
+print("=" * 80)
+print("VALIDATION")
+print("=" * 80)
+
+if len(results) != len(dataset):
+
+    raise RuntimeError(
+        "Result count mismatch."
+    )
+
+for i, item in enumerate(results):
+
+    for idx in TARGET_LAYERS:
+
+        x = item["layers"][idx]
+
+        if x.ndim != 3:
+
+            raise RuntimeError(
+                f"Bad ndim at prompt {i}, layer {idx}"
+            )
+
+        if x.shape[-1] != 4096:
+
+            raise RuntimeError(
+                f"Bad hidden size at prompt {i}, layer {idx}: "
+                f"{x.shape}"
+            )
+
+        if not torch.isfinite(x).all():
+
+            raise RuntimeError(
+                f"Non-finite values at prompt {i}, layer {idx}"
+            )
+
+print("All hidden states valid.")
+
+
+# ============================================================
+# SAVE
+# ============================================================
+
+print("\nSaving...")
+
+torch.save(
+    {
+        "target_layers": TARGET_LAYERS,
+        "results": results,
+        "source_model": MODEL_FILE,
+    },
+    OUT,
+)
+
+print()
+print("=" * 80)
+print("SUCCESS")
+print(OUT)
+print("=" * 80)
+
+gc.collect()
+torch.cuda.empty_cache()

research/raw_scripts/extract_sensenova_hidden_states.py

@@ -0,0 +1,321 @@
+import os
+import torch
+
+from transformers import AutoConfig, AutoModelForCausalLM, AutoTokenizer
+from safetensors import safe_open
+
+
+MODEL_FILE = r"D:\ComfyUI_Python312\ComfyUI\models\unet\SenseNova-U1.5-8B-MoT-pruned-bf16.safetensors"
+
+OUT = r"E:\Models SD XL\UNET\Minimax_H3\sensenova_hidden_states_fast.pt"
+
+REPO = "sensenova/SenseNova-U1.5-8B-MoT"
+
+PROMPTS = [
+    "a red cube on a blue sphere",
+    "a woman holding a transparent glass bottle",
+    "three people standing behind a wooden table",
+]
+
+TARGET_LAYERS = [8, 16, 24, 32, 41]
+
+
+print("=" * 80)
+print("SenseNova hidden-state extractor")
+print("=" * 80)
+
+print("\nLoading tokenizer...")
+
+tokenizer = AutoTokenizer.from_pretrained(
+    REPO,
+    trust_remote_code=True,
+)
+
+
+# ============================================================
+# CONFIG
+# ============================================================
+
+print("Loading SenseNova config...")
+
+config = AutoConfig.from_pretrained(
+    REPO,
+    trust_remote_code=True,
+)
+
+print("Config type:", type(config))
+
+
+# ============================================================
+# LOAD MODEL
+#
+# We create the official architecture, then load YOUR local
+# pruned BF16 checkpoint.
+# ============================================================
+
+print("\nCreating SenseNova model architecture...")
+
+model = AutoModelForCausalLM.from_config(
+    config,
+    trust_remote_code=True,
+)
+
+print("Architecture created.")
+
+
+# ============================================================
+# LOAD LOCAL SAFETENSORS
+# ============================================================
+
+print("\nLoading local checkpoint:")
+print(MODEL_FILE)
+
+state = {}
+
+with safe_open(MODEL_FILE, framework="pt", device="cpu") as f:
+
+    keys = list(f.keys())
+
+    # Only language-model tensors are needed for this test.
+    lm_keys = [
+        k for k in keys
+        if k.startswith("language_model.")
+    ]
+
+    print("Language-model tensors:", len(lm_keys))
+
+    for i, key in enumerate(lm_keys):
+
+        state[key] = f.get_tensor(key)
+
+        if (i + 1) % 100 == 0:
+            print(f"Loaded {i + 1}/{len(lm_keys)}")
+
+
+print("\nApplying state dict...")
+
+missing, unexpected = model.load_state_dict(
+    state,
+    strict=False,
+)
+
+print("Missing keys:", len(missing))
+print("Unexpected keys:", len(unexpected))
+
+del state
+
+
+# ============================================================
+# FIND LANGUAGE MODEL
+# ============================================================
+
+language_model = None
+
+candidates = [
+    "language_model",
+    "model.language_model",
+]
+
+for path in candidates:
+
+    obj = model
+
+    try:
+        for part in path.split("."):
+            obj = getattr(obj, part)
+
+        language_model = obj
+        print("Found language model:", path)
+        break
+
+    except Exception:
+        pass
+
+
+if language_model is None:
+
+    print("\nCould not find language model directly.")
+    print("Candidate modules:")
+
+    for name, module in model.named_modules():
+        if "language_model" in name.lower():
+            print(name, type(module))
+
+    raise RuntimeError("language_model not found")
+
+
+# ============================================================
+# MOVE / OFFLOAD
+# ============================================================
+
+print("\nPreparing model...")
+
+model.eval()
+
+# Keep whole architecture on CPU initially.
+model.to("cpu")
+
+# We will let PyTorch move the language model to GPU if possible.
+# If this raises OOM, we will switch to layer-by-layer streaming.
+try:
+
+    language_model.to("cuda")
+
+    DEVICE = "cuda"
+
+    print("Language model moved to CUDA.")
+
+except torch.cuda.OutOfMemoryError:
+
+    print("Language model does not fit in VRAM.")
+    print("Falling back to CPU.")
+
+    torch.cuda.empty_cache()
+
+    DEVICE = "cpu"
+
+
+# ============================================================
+# HOOKS
+# ============================================================
+
+captured = {}
+hooks = {}
+
+
+def make_hook(idx):
+
+    def hook(module, inputs, output):
+
+        if isinstance(output, tuple):
+            x = output[0]
+        else:
+            x = output
+
+        captured[idx] = (
+            x.detach()
+             .float()
+             .cpu()
+        )
+
+    return hook
+
+
+for idx in TARGET_LAYERS:
+
+    target = None
+
+    expected_suffix = f"layers.{idx}"
+
+    for name, module in language_model.named_modules():
+
+        if name.endswith(expected_suffix):
+            target = module
+            print("Hook:", idx, "->", name)
+            break
+
+    if target is not None:
+        hooks[idx] = target.register_forward_hook(
+            make_hook(idx)
+        )
+
+    else:
+        print("Layer not found:", idx)
+
+
+print("\nRegistered hooks:", sorted(hooks.keys()))
+
+
+# ============================================================
+# RUN
+# ============================================================
+
+results = []
+
+for i, prompt in enumerate(PROMPTS):
+
+    print()
+    print("=" * 80)
+    print(f"[{i + 1}/{len(PROMPTS)}]")
+    print(prompt)
+    print("=" * 80)
+
+    encoded = tokenizer(
+        prompt,
+        return_tensors="pt",
+        add_special_tokens=False,
+    )
+
+    input_ids = encoded["input_ids"].to(DEVICE)
+
+    print("Token count:", input_ids.shape[1])
+
+    captured.clear()
+
+    with torch.inference_mode():
+
+        output = language_model(
+            input_ids=input_ids,
+            use_cache=False,
+        )
+
+    item = {
+        "prompt": prompt,
+        "input_ids": input_ids.cpu(),
+        "layers": {},
+    }
+
+    for idx in TARGET_LAYERS:
+
+        if idx in captured:
+
+            x = captured[idx]
+
+            item["layers"][idx] = x
+
+            print(
+                f"layer {idx:2d}: "
+                f"shape={tuple(x.shape)} "
+                f"mean={x.mean().item():.6f} "
+                f"std={x.std().item():.6f} "
+                f"finite={torch.isfinite(x).all().item()}"
+            )
+
+        else:
+
+            print(
+                f"layer {idx:2d}: NOT CAPTURED"
+            )
+
+    results.append(item)
+
+    del output
+
+    if DEVICE == "cuda":
+        torch.cuda.empty_cache()
+
+
+# ============================================================
+# CLEANUP
+# ============================================================
+
+for hook in hooks.values():
+    hook.remove()
+
+
+torch.save(
+    {
+        "prompts": PROMPTS,
+        "target_layers": TARGET_LAYERS,
+        "results": results,
+    },
+    OUT,
+)
+
+
+print()
+print("=" * 80)
+print("SUCCESS")
+print("Saved:")
+print(OUT)
+print("=" * 80)

research/raw_scripts/extract_sensenova_hidden_states_local.py

@@ -0,0 +1,421 @@
+import os
+import sys
+import torch
+from safetensors import safe_open
+from transformers import AutoTokenizer
+
+# ============================================================
+# PATHS
+# ============================================================
+
+ROOT = r"E:\Models SD XL\UNET\Minimax_H3"
+SENSENOVA_REPO = os.path.join(ROOT, "SenseNova-U1")
+SENSENOVA_SRC = os.path.join(SENSENOVA_REPO, "src")
+
+MODEL_FILE = r"D:\ComfyUI_Python312\ComfyUI\models\unet\SenseNova-U1.5-8B-MoT-pruned-bf16.safetensors"
+
+OUT = os.path.join(
+    ROOT,
+    "sensenova_hidden_states_fast.pt"
+)
+
+HF_REPO = "sensenova/SenseNova-U1.5-8B-MoT"
+
+PROMPTS = [
+    "a red cube on a blue sphere",
+    "a woman holding a transparent glass bottle",
+    "three people standing behind a wooden table",
+]
+
+TARGET_LAYERS = [8, 16, 24, 32, 41]
+
+
+# ============================================================
+# LOCAL SENSENOVA CODE
+# ============================================================
+
+sys.path.insert(0, SENSENOVA_SRC)
+
+print("=" * 80)
+print("SenseNova U1.5 hidden-state extractor - LOCAL CODE")
+print("=" * 80)
+
+print("\nSenseNova src:")
+print(SENSENOVA_SRC)
+
+
+# ============================================================
+# IMPORT CUSTOM CLASSES
+# ============================================================
+
+print("\nImporting local SenseNova classes...")
+
+from sensenova_u1.models.neo_unify.configuration_neo_chat import NEOChatConfig
+
+try:
+    from sensenova_u1.models.neo_unify.modeling_neo_chat import NEOChatModel
+    MODEL_CLASS = NEOChatModel
+    print("Using model class: NEOChatModel")
+
+except ImportError:
+    try:
+        from sensenova_u1.models.neo_unify.modeling_neo_chat import NEOChatForCausalLM
+        MODEL_CLASS = NEOChatForCausalLM
+        print("Using model class: NEOChatForCausalLM")
+
+    except ImportError:
+        print("\nCould not import expected model class.")
+        print("Available names containing 'Chat' or 'Model':")
+
+        import sensenova_u1.models.neo_unify.modeling_neo_chat as mod
+
+        for name in dir(mod):
+            if "Chat" in name or "Model" in name:
+                print(" ", name)
+
+        raise
+
+
+# ============================================================
+# TOKENIZER
+# ============================================================
+
+print("\nLoading tokenizer...")
+
+tokenizer = AutoTokenizer.from_pretrained(
+    HF_REPO,
+    trust_remote_code=True,
+)
+
+
+# ============================================================
+# CONFIG
+# ============================================================
+
+print("Loading config using LOCAL NEOChatConfig...")
+
+config = NEOChatConfig.from_pretrained(
+    HF_REPO,
+)
+
+print("Config loaded.")
+print("Config class:", type(config))
+
+
+# ============================================================
+# CREATE MODEL ARCHITECTURE
+# ============================================================
+
+print("\nCreating SenseNova architecture...")
+
+model = MODEL_CLASS(config)
+
+print("Architecture created.")
+print("Model type:", type(model))
+
+
+# ============================================================
+# LOAD ONLY LANGUAGE MODEL WEIGHTS
+# ============================================================
+
+print("\nScanning local checkpoint...")
+
+with safe_open(MODEL_FILE, framework="pt", device="cpu") as f:
+    keys = list(f.keys())
+
+lm_keys = [
+    k for k in keys
+    if k.startswith("language_model.")
+]
+
+print("Total checkpoint tensors:", len(keys))
+print("Language-model tensors:", len(lm_keys))
+
+
+print("\nLoading language-model tensors...")
+
+state = {}
+
+with safe_open(MODEL_FILE, framework="pt", device="cpu") as f:
+
+    for i, key in enumerate(lm_keys):
+        state[key] = f.get_tensor(key)
+
+        if (i + 1) % 100 == 0:
+            print(
+                f"  loaded {i + 1}/{len(lm_keys)}"
+            )
+
+
+# ============================================================
+# APPLY
+# ============================================================
+
+print("\nApplying state dict...")
+
+result = model.load_state_dict(
+    state,
+    strict=False,
+)
+
+print("Missing keys:", len(result.missing_keys))
+print("Unexpected keys:", len(result.unexpected_keys))
+
+if len(result.unexpected_keys) > 0:
+    print("\nFirst unexpected keys:")
+    for k in result.unexpected_keys[:20]:
+        print(" ", k)
+
+del state
+
+
+# ============================================================
+# FIND LANGUAGE MODEL
+# ============================================================
+
+print("\nFinding language model...")
+
+language_model = None
+language_model_name = None
+
+possible_paths = [
+    "language_model",
+    "model.language_model",
+    "language_model.model",
+]
+
+for path in possible_paths:
+
+    obj = model
+
+    try:
+        for part in path.split("."):
+            obj = getattr(obj, part)
+
+        language_model = obj
+        language_model_name = path
+        break
+
+    except Exception:
+        pass
+
+
+if language_model is None:
+
+    print("Could not locate language model automatically.")
+    print("\nCandidate modules:")
+
+    for name, module in model.named_modules():
+        if "language_model" in name.lower():
+            print(name, type(module))
+
+    raise RuntimeError("language_model not found")
+
+
+print("Language model:", language_model_name)
+print("Type:", type(language_model))
+
+
+# ============================================================
+# FIND TRANSFORMER LAYERS
+# ============================================================
+
+print("\nSearching for transformer layers...")
+
+for name, module in language_model.named_modules():
+
+    if name.endswith("layers.0"):
+        print("Example layer path:", name)
+        break
+
+
+# ============================================================
+# CUDA
+# ============================================================
+
+model.eval()
+
+print("\nTrying language model on CUDA...")
+
+DEVICE = "cuda"
+
+try:
+
+    language_model.to(
+        device="cuda",
+        dtype=torch.bfloat16,
+    )
+
+    print("Language model moved to CUDA.")
+
+except torch.cuda.OutOfMemoryError:
+
+    print("\nCUDA OOM while moving language model.")
+    print("Falling back to CPU for diagnostic run.")
+
+    torch.cuda.empty_cache()
+
+    language_model.to("cpu")
+
+    DEVICE = "cpu"
+
+
+# ============================================================
+# HOOKS
+# ============================================================
+
+captured = {}
+hooks = {}
+
+
+def make_hook(idx):
+
+    def hook(module, inputs, output):
+
+        if isinstance(output, tuple):
+            x = output[0]
+        else:
+            x = output
+
+        captured[idx] = (
+            x.detach()
+            .float()
+            .cpu()
+        )
+
+    return hook
+
+
+print("\nRegistering hooks...")
+
+for idx in TARGET_LAYERS:
+
+    target = None
+    target_name = None
+
+    suffix = f"layers.{idx}"
+
+    for name, module in language_model.named_modules():
+
+        if name.endswith(suffix):
+            target = module
+            target_name = name
+            break
+
+    if target is None:
+        print(f"  layer {idx}: NOT FOUND")
+        continue
+
+    hooks[idx] = target.register_forward_hook(
+        make_hook(idx)
+    )
+
+    print(
+        f"  layer {idx}: {target_name}"
+    )
+
+
+print("\nRegistered hooks:", sorted(hooks.keys()))
+
+if not hooks:
+    raise RuntimeError("No transformer layers found")
+
+
+# ============================================================
+# INFERENCE
+# ============================================================
+
+results = []
+
+for i, prompt in enumerate(PROMPTS):
+
+    print()
+    print("=" * 80)
+    print(f"[{i+1}/{len(PROMPTS)}]")
+    print(prompt)
+    print("=" * 80)
+
+    encoded = tokenizer(
+        prompt,
+        return_tensors="pt",
+        add_special_tokens=False,
+    )
+
+    input_ids = encoded["input_ids"].to(DEVICE)
+
+    print("Token count:", input_ids.shape[1])
+
+    captured.clear()
+
+    with torch.inference_mode():
+
+        try:
+            out = language_model(
+                input_ids=input_ids,
+                use_cache=False,
+            )
+
+        except TypeError:
+
+            out = language_model(
+                input_ids=input_ids,
+            )
+
+    item = {
+        "prompt": prompt,
+        "input_ids": input_ids.detach().cpu(),
+        "layers": {},
+    }
+
+    for idx in TARGET_LAYERS:
+
+        if idx not in captured:
+            print(
+                f"layer {idx:2d}: NOT CAPTURED"
+            )
+            continue
+
+        x = captured[idx]
+
+        item["layers"][idx] = x
+
+        print(
+            f"layer {idx:2d}: "
+            f"shape={tuple(x.shape)} "
+            f"mean={x.mean().item():.6f} "
+            f"std={x.std().item():.6f} "
+            f"finite={torch.isfinite(x).all().item()}"
+        )
+
+    results.append(item)
+
+    del out
+    del input_ids
+
+    if DEVICE == "cuda":
+        torch.cuda.empty_cache()
+
+
+# ============================================================
+# SAVE
+# ============================================================
+
+for h in hooks.values():
+    h.remove()
+
+
+torch.save(
+    {
+        "prompts": PROMPTS,
+        "target_layers": TARGET_LAYERS,
+        "results": results,
+    },
+    OUT,
+)
+
+print()
+print("=" * 80)
+print("SUCCESS")
+print("Saved:")
+print(OUT)
+print("=" * 80)

research/raw_scripts/extract_sensenova_hidden_states_stream.py

@@ -0,0 +1,352 @@
+import os
+import sys
+import gc
+import torch
+
+from transformers import AutoTokenizer
+from accelerate import init_empty_weights, load_checkpoint_and_dispatch
+
+# ============================================================
+# PATHS
+# ============================================================
+
+ROOT = r"E:\Models SD XL\UNET\Minimax_H3"
+
+SENSENOVA_SRC = os.path.join(
+    ROOT,
+    "SenseNova-U1",
+    "src"
+)
+
+MODEL_FILE = (
+    r"D:\ComfyUI_Python312\ComfyUI\models\unet"
+    r"\SenseNova-U1.5-8B-MoT-pruned-bf16.safetensors"
+)
+
+OUT = os.path.join(
+    ROOT,
+    "sensenova_hidden_states_stream.pt"
+)
+
+HF_REPO = "sensenova/SenseNova-U1.5-8B-MoT"
+
+PROMPTS = [
+    "a red cube on a blue sphere",
+    "a woman holding a transparent glass bottle",
+    "three people standing behind a wooden table",
+]
+
+TARGET_LAYERS = [8, 16, 24, 32, 41]
+
+# Не отдаём GPU все 24 GB, оставляем запас.
+MAX_MEMORY = {
+    0: "20GiB",
+    "cpu": "70GiB",
+}
+
+
+# ============================================================
+# LOCAL SENSENOVA CODE
+# ============================================================
+
+sys.path.insert(0, SENSENOVA_SRC)
+
+from sensenova_u1.models.neo_unify.configuration_neo_chat import (
+    NEOChatConfig,
+)
+from sensenova_u1.models.neo_unify.modeling_neo_chat import (
+    NEOChatModel,
+)
+
+
+print("=" * 80)
+print("SenseNova U1.5 hidden-state extractor")
+print("MEMORY-EFFICIENT / ACCELERATE")
+print("=" * 80)
+
+print("\nGPU:")
+print(torch.cuda.get_device_name(0))
+
+print(
+    "VRAM:",
+    round(
+        torch.cuda.get_device_properties(0).total_memory / 1024**3,
+        2
+    ),
+    "GB"
+)
+
+
+# ============================================================
+# TOKENIZER + CONFIG
+# ============================================================
+
+print("\nLoading tokenizer...")
+
+tokenizer = AutoTokenizer.from_pretrained(
+    HF_REPO,
+    trust_remote_code=True,
+)
+
+print("Loading LOCAL SenseNova config...")
+
+config = NEOChatConfig.from_pretrained(
+    HF_REPO,
+)
+
+print("Config loaded.")
+print("LLM hidden size:", config.llm_config.hidden_size)
+print("LLM layers:", config.llm_config.num_hidden_layers)
+
+
+# ============================================================
+# CREATE MODEL ON META
+#
+# Ключевой момент:
+# здесь НЕ выделяется 45+ GB RAM под пустую модель.
+# ============================================================
+
+print()
+print("Creating architecture on META device...")
+
+with init_empty_weights():
+    model = NEOChatModel(config)
+
+print("META architecture created.")
+
+
+# ============================================================
+# LOAD + DISPATCH CHECKPOINT DIRECTLY
+#
+# Нет state = {...}
+# Нет второй полной копии checkpoint в RAM.
+# ============================================================
+
+print()
+print("Loading checkpoint directly with Accelerate...")
+print("This can take a few minutes, but RAM should stay reasonable.")
+
+model = load_checkpoint_and_dispatch(
+    model,
+    checkpoint=MODEL_FILE,
+    device_map="auto",
+    max_memory=MAX_MEMORY,
+    dtype=torch.bfloat16,
+
+    # Decoder layer нельзя разрезать между устройствами.
+    no_split_module_classes=[
+        "Qwen3DecoderLayer",
+        "Qwen3MoeDecoderLayer",
+    ],
+)
+
+model.eval()
+
+print()
+print("Checkpoint loaded successfully.")
+
+
+# ============================================================
+# FIND LANGUAGE MODEL
+# ============================================================
+
+language_model = model.language_model
+
+print("Language model:")
+print(type(language_model))
+
+print("\nDevice map summary:")
+
+device_map = getattr(model, "hf_device_map", None)
+
+if device_map:
+    gpu_count = 0
+    cpu_count = 0
+    disk_count = 0
+
+    for name, dev in device_map.items():
+
+        s = str(dev)
+
+        if s in ("0", "cuda", "cuda:0"):
+            gpu_count += 1
+        elif s == "cpu":
+            cpu_count += 1
+        elif s == "disk":
+            disk_count += 1
+
+    print("GPU modules :", gpu_count)
+    print("CPU modules :", cpu_count)
+    print("Disk modules:", disk_count)
+
+    print("\nFirst device-map entries:")
+    for i, (name, dev) in enumerate(device_map.items()):
+        print(f"  {name:60s} -> {dev}")
+
+        if i >= 19:
+            break
+
+
+# ============================================================
+# FIND TARGET LAYERS
+# ============================================================
+
+layers = language_model.model.layers
+
+print()
+print("Detected language layers:", len(layers))
+
+
+# ============================================================
+# HOOKS
+# ============================================================
+
+captured = {}
+hooks = {}
+
+
+def make_hook(idx):
+
+    def hook(module, inputs, output):
+
+        if isinstance(output, tuple):
+            x = output[0]
+        else:
+            x = output
+
+        captured[idx] = (
+            x.detach()
+            .float()
+            .cpu()
+        )
+
+    return hook
+
+
+print("\nRegistering hooks...")
+
+for idx in TARGET_LAYERS:
+
+    if idx >= len(layers):
+        print(f"layer {idx}: OUT OF RANGE")
+        continue
+
+    hooks[idx] = layers[idx].register_forward_hook(
+        make_hook(idx)
+    )
+
+    print(
+        f"layer {idx}:",
+        type(layers[idx]),
+    )
+
+
+# ============================================================
+# RUN PROMPTS
+# ============================================================
+
+results = []
+
+for i, prompt in enumerate(PROMPTS):
+
+    print()
+    print("=" * 80)
+    print(f"[{i + 1}/{len(PROMPTS)}]")
+    print(prompt)
+    print("=" * 80)
+
+    encoded = tokenizer(
+        prompt,
+        return_tensors="pt",
+        add_special_tokens=False,
+    )
+
+    # Input должен попасть туда, где embedding.
+    embed_device = language_model.model.embed_tokens.weight.device
+
+    # При accelerate иногда weight показывает meta;
+    # тогда безопасно используем cuda:0.
+    if embed_device.type == "meta":
+        embed_device = torch.device("cuda:0")
+
+    input_ids = encoded["input_ids"].to(embed_device)
+
+    print("Tokens:", input_ids.tolist()[0])
+    print("Token count:", input_ids.shape[1])
+    print("Input device:", input_ids.device)
+
+    captured.clear()
+
+    with torch.inference_mode():
+
+        outputs = language_model.model(
+            input_ids=input_ids,
+            use_cache=False,
+        )
+
+    item = {
+        "prompt": prompt,
+        "input_ids": input_ids.detach().cpu(),
+        "layers": {},
+    }
+
+    for idx in TARGET_LAYERS:
+
+        if idx not in captured:
+
+            print(
+                f"layer {idx:2d}: NOT CAPTURED"
+            )
+
+            continue
+
+        x = captured[idx]
+
+        item["layers"][idx] = x
+
+        print(
+            f"layer {idx:2d}: "
+            f"shape={tuple(x.shape)} "
+            f"mean={x.mean().item():.6f} "
+            f"std={x.std().item():.6f} "
+            f"finite={torch.isfinite(x).all().item()}"
+        )
+
+    results.append(item)
+
+    del outputs
+    del input_ids
+
+    torch.cuda.empty_cache()
+
+
+# ============================================================
+# CLEANUP HOOKS
+# ============================================================
+
+for h in hooks.values():
+    h.remove()
+
+
+# ============================================================
+# SAVE
+# ============================================================
+
+torch.save(
+    {
+        "prompts": PROMPTS,
+        "target_layers": TARGET_LAYERS,
+        "results": results,
+    },
+    OUT,
+)
+
+print()
+print("=" * 80)
+print("SUCCESS")
+print("Saved:")
+print(OUT)
+print("=" * 80)
+
+gc.collect()
+torch.cuda.empty_cache()

research/raw_scripts/extract_sensenova_hidden_states_stream_v2.py

@@ -0,0 +1,586 @@
+import os
+import sys
+import gc
+import inspect
+import torch
+
+from transformers import AutoTokenizer
+from accelerate import init_empty_weights, load_checkpoint_and_dispatch
+
+
+# ============================================================
+# PATHS
+# ============================================================
+
+ROOT = r"E:\Models SD XL\UNET\Minimax_H3"
+
+SENSENOVA_SRC = os.path.join(
+    ROOT,
+    "SenseNova-U1",
+    "src"
+)
+
+MODEL_FILE = (
+    r"D:\ComfyUI_Python312\ComfyUI\models\unet"
+    r"\SenseNova-U1.5-8B-MoT-pruned-bf16.safetensors"
+)
+
+OUT = os.path.join(
+    ROOT,
+    "sensenova_hidden_states_stream_v2.pt"
+)
+
+HF_REPO = "sensenova/SenseNova-U1.5-8B-MoT"
+
+
+# ============================================================
+# TEST PROMPTS
+# ============================================================
+
+PROMPTS = [
+    "a red cube on a blue sphere",
+    "a woman holding a transparent glass bottle",
+    "three people standing behind a wooden table",
+]
+
+TARGET_LAYERS = [8, 16, 24, 32, 41]
+
+
+# ============================================================
+# MEMORY
+# ============================================================
+
+# Оставляем запас VRAM для forward.
+MAX_MEMORY = {
+    0: "19GiB",
+    "cpu": "70GiB",
+}
+
+
+# ============================================================
+# IMPORT LOCAL SENSENOVA CODE
+# ============================================================
+
+sys.path.insert(0, SENSENOVA_SRC)
+
+from sensenova_u1.models.neo_unify.configuration_neo_chat import (
+    NEOChatConfig,
+)
+
+from sensenova_u1.models.neo_unify.modeling_neo_chat import (
+    NEOChatModel,
+)
+
+
+print("=" * 80)
+print("SenseNova U1.5 hidden-state extractor V2")
+print("Transformers 4.57 compatibility-wrapper bypass")
+print("=" * 80)
+
+print("\nPyTorch:", torch.__version__)
+
+import transformers
+import accelerate
+
+print("Transformers:", transformers.__version__)
+print("Accelerate:", accelerate.__version__)
+
+print("\nGPU:")
+print(torch.cuda.get_device_name(0))
+
+print(
+    "VRAM:",
+    round(
+        torch.cuda.get_device_properties(0).total_memory
+        / 1024**3,
+        2
+    ),
+    "GB"
+)
+
+
+# ============================================================
+# TOKENIZER
+# ============================================================
+
+print("\nLoading tokenizer...")
+
+tokenizer = AutoTokenizer.from_pretrained(
+    HF_REPO,
+    trust_remote_code=True,
+)
+
+
+# ============================================================
+# CONFIG
+# ============================================================
+
+print("Loading LOCAL SenseNova config...")
+
+config = NEOChatConfig.from_pretrained(
+    HF_REPO,
+)
+
+print("Config loaded.")
+print("LLM hidden size:", config.llm_config.hidden_size)
+print("LLM layers:", config.llm_config.num_hidden_layers)
+
+
+# ============================================================
+# META MODEL
+# ============================================================
+
+print()
+print("Creating architecture on META device...")
+
+with init_empty_weights():
+    model = NEOChatModel(config)
+
+print("META architecture created.")
+
+
+# ============================================================
+# LOAD CHECKPOINT
+# ============================================================
+
+print()
+print("Loading local BF16 checkpoint with Accelerate...")
+print("No duplicate full state_dict will be created.")
+
+model = load_checkpoint_and_dispatch(
+    model,
+    checkpoint=MODEL_FILE,
+    device_map="auto",
+    max_memory=MAX_MEMORY,
+    dtype=torch.bfloat16,
+    no_split_module_classes=[
+        "Qwen3DecoderLayer",
+        "Qwen3MoeDecoderLayer",
+    ],
+)
+
+model.eval()
+
+print()
+print("Checkpoint loaded successfully.")
+
+
+# ============================================================
+# LANGUAGE MODEL
+# ============================================================
+
+language_model = model.language_model
+qwen_model = language_model.model
+layers = qwen_model.layers
+
+print()
+print("Language model:")
+print(type(language_model))
+
+print("Base model:")
+print(type(qwen_model))
+
+print("Detected language layers:", len(layers))
+
+
+# ============================================================
+# DEVICE MAP
+# ============================================================
+
+device_map = getattr(model, "hf_device_map", None)
+
+if device_map:
+
+    gpu_count = 0
+    cpu_count = 0
+    disk_count = 0
+
+    for name, dev in device_map.items():
+
+        s = str(dev)
+
+        if s in ("0", "cuda", "cuda:0"):
+            gpu_count += 1
+
+        elif s == "cpu":
+            cpu_count += 1
+
+        elif s == "disk":
+            disk_count += 1
+
+    print()
+    print("Device map:")
+    print("  GPU modules :", gpu_count)
+    print("  CPU modules :", cpu_count)
+    print("  Disk modules:", disk_count)
+
+
+# ============================================================
+# BYPASS TRANSFORMERS 4.57 WRAPPER
+# ============================================================
+
+print()
+print("Preparing raw SenseNova forward...")
+
+# Берём forward непосредственно с класса,
+# затем снимаем ВСЕ decorators через inspect.unwrap().
+#
+# Это позволяет обойти check_model_inputs из Transformers,
+# который ломает прямой вызов SenseNova в 4.57.x.
+
+decorated_forward = type(qwen_model).forward
+raw_forward = inspect.unwrap(decorated_forward)
+
+print("Decorated forward:")
+print(decorated_forward)
+
+print()
+print("Unwrapped forward:")
+print(raw_forward)
+
+try:
+    print()
+    print("Raw forward signature:")
+    print(inspect.signature(raw_forward))
+except Exception:
+    pass
+
+
+# ============================================================
+# HOOKS
+# ============================================================
+
+captured = {}
+hooks = {}
+
+
+def make_hook(idx):
+
+    def hook(module, inputs, output):
+
+        if isinstance(output, tuple):
+            x = output[0]
+
+        elif hasattr(output, "hidden_states"):
+            x = output.hidden_states
+
+        else:
+            x = output
+
+        if not torch.is_tensor(x):
+            print(
+                f"WARNING: layer {idx} returned "
+                f"{type(x)} instead of Tensor"
+            )
+            return
+
+        captured[idx] = (
+            x.detach()
+            .to(
+                device="cpu",
+                dtype=torch.float32,
+            )
+        )
+
+    return hook
+
+
+print()
+print("Registering hooks...")
+
+for idx in TARGET_LAYERS:
+
+    if idx >= len(layers):
+        print(f"  layer {idx}: OUT OF RANGE")
+        continue
+
+    hooks[idx] = layers[idx].register_forward_hook(
+        make_hook(idx)
+    )
+
+    print(
+        f"  layer {idx}:",
+        type(layers[idx]),
+    )
+
+
+if not hooks:
+    raise RuntimeError("No hooks were registered.")
+
+
+# ============================================================
+# FIND INPUT DEVICE
+# ============================================================
+
+def get_embedding_device():
+
+    try:
+        d = qwen_model.embed_tokens.weight.device
+
+        if d.type != "meta":
+            return d
+
+    except Exception:
+        pass
+
+    # По device_map embedding у нас обычно на GPU.
+    if device_map:
+
+        for name, dev in device_map.items():
+
+            if name.endswith(
+                "language_model.model.embed_tokens"
+            ):
+
+                if str(dev) == "0":
+                    return torch.device("cuda:0")
+
+                return torch.device(str(dev))
+
+    return torch.device("cuda:0")
+
+
+INPUT_DEVICE = get_embedding_device()
+
+print()
+print("Input device:", INPUT_DEVICE)
+
+
+# ============================================================
+# SAFE RAW FORWARD
+# ============================================================
+
+def run_sensenova_forward(input_ids):
+
+    """
+    Вызывает настоящий Qwen3Model.forward без
+    Transformers compatibility decorator.
+    """
+
+    kwargs = {
+        "input_ids": input_ids,
+        "use_cache": False,
+
+        # Для обычного text/understanding path.
+        # MoT-generation path будем тестировать отдельно.
+        "image_gen_indicators": None,
+    }
+
+    # Первый вариант: полный ожидаемый вызов.
+    try:
+
+        return raw_forward(
+            qwen_model,
+            **kwargs,
+        )
+
+    except TypeError as e1:
+
+        print()
+        print("First raw-forward variant failed:")
+        print(repr(e1))
+
+        print("Retrying minimal raw forward...")
+
+        # На случай небольших отличий сигнатуры.
+        try:
+
+            return raw_forward(
+                qwen_model,
+                input_ids=input_ids,
+                use_cache=False,
+            )
+
+        except TypeError as e2:
+
+            print()
+            print("Minimal raw forward also failed:")
+            print(repr(e2))
+
+            print()
+            print("Raw signature:")
+            print(inspect.signature(raw_forward))
+
+            raise
+
+
+# ============================================================
+# RUN PROMPTS
+# ============================================================
+
+results = []
+
+for i, prompt in enumerate(PROMPTS):
+
+    print()
+    print("=" * 80)
+    print(f"[{i + 1}/{len(PROMPTS)}]")
+    print(prompt)
+    print("=" * 80)
+
+    encoded = tokenizer(
+        prompt,
+        return_tensors="pt",
+        add_special_tokens=False,
+    )
+
+    input_ids = encoded["input_ids"].to(
+        INPUT_DEVICE
+    )
+
+    print(
+        "Tokens:",
+        input_ids.detach().cpu().tolist()[0]
+    )
+
+    print(
+        "Token count:",
+        input_ids.shape[1]
+    )
+
+    print(
+        "Input device:",
+        input_ids.device
+    )
+
+    captured.clear()
+
+    with torch.inference_mode():
+
+        outputs = run_sensenova_forward(
+            input_ids
+        )
+
+    item = {
+        "prompt": prompt,
+        "input_ids": (
+            input_ids.detach()
+            .cpu()
+        ),
+        "layers": {},
+    }
+
+    print()
+
+    for idx in TARGET_LAYERS:
+
+        if idx not in captured:
+
+            print(
+                f"layer {idx:2d}: NOT CAPTURED"
+            )
+
+            continue
+
+        x = captured[idx]
+
+        item["layers"][idx] = x
+
+        finite = torch.isfinite(x).all().item()
+
+        print(
+            f"layer {idx:2d}: "
+            f"shape={tuple(x.shape)} "
+            f"dtype={x.dtype} "
+            f"mean={x.mean().item():.6f} "
+            f"std={x.std().item():.6f} "
+            f"finite={finite}"
+        )
+
+    results.append(item)
+
+    del outputs
+    del input_ids
+
+    gc.collect()
+    torch.cuda.empty_cache()
+
+
+# ============================================================
+# REMOVE HOOKS
+# ============================================================
+
+for h in hooks.values():
+    h.remove()
+
+
+# ============================================================
+# VALIDATION
+# ============================================================
+
+print()
+print("=" * 80)
+print("VALIDATION")
+print("=" * 80)
+
+all_ok = True
+
+for item in results:
+
+    print()
+    print(item["prompt"])
+
+    for idx in TARGET_LAYERS:
+
+        if idx not in item["layers"]:
+
+            print(
+                f"  layer {idx}: missing"
+            )
+
+            all_ok = False
+            continue
+
+        x = item["layers"][idx]
+
+        correct_dim = (
+            x.ndim == 3
+            and x.shape[-1] == 4096
+        )
+
+        finite = torch.isfinite(x).all().item()
+
+        print(
+            f"  layer {idx}: "
+            f"{tuple(x.shape)} "
+            f"dim_ok={correct_dim} "
+            f"finite={finite}"
+        )
+
+        if not correct_dim or not finite:
+            all_ok = False
+
+
+# ============================================================
+# SAVE
+# ============================================================
+
+torch.save(
+    {
+        "prompts": PROMPTS,
+        "target_layers": TARGET_LAYERS,
+        "results": results,
+        "source_model": MODEL_FILE,
+        "transformers_version": transformers.__version__,
+    },
+    OUT,
+)
+
+
+print()
+print("=" * 80)
+
+if all_ok:
+    print("SUCCESS")
+else:
+    print("COMPLETED WITH WARNINGS")
+
+print()
+print("Saved:")
+print(OUT)
+
+print("=" * 80)
+
+
+gc.collect()
+torch.cuda.empty_cache()

research/raw_scripts/extract_sensenova_hidden_states_stream_v3.py

@@ -0,0 +1,546 @@
+import os
+import sys
+import gc
+import shutil
+import torch
+
+
+# ============================================================
+# PATHS
+# ============================================================
+
+ROOT = r"E:\Models SD XL\UNET\Minimax_H3"
+
+SENSENOVA_REPO = os.path.join(
+    ROOT,
+    "SenseNova-U1"
+)
+
+SENSENOVA_SRC = os.path.join(
+    SENSENOVA_REPO,
+    "src"
+)
+
+COMPAT_FILE = os.path.join(
+    SENSENOVA_SRC,
+    "sensenova_u1",
+    "models",
+    "neo_unify",
+    "transformers_compat.py"
+)
+
+MODEL_FILE = (
+    r"D:\ComfyUI_Python312\ComfyUI\models\unet"
+    r"\SenseNova-U1.5-8B-MoT-pruned-bf16.safetensors"
+)
+
+OUT = os.path.join(
+    ROOT,
+    "sensenova_hidden_states_stream_v3.pt"
+)
+
+HF_REPO = "sensenova/SenseNova-U1.5-8B-MoT"
+
+
+PROMPTS = [
+    "a red cube on a blue sphere",
+    "a woman holding a transparent glass bottle",
+    "three people standing behind a wooden table",
+]
+
+TARGET_LAYERS = [8, 16, 24, 32, 41]
+
+
+# ============================================================
+# PATCH TRANSFORMERS 4.57 COMPATIBILITY BUG
+# ============================================================
+
+print("=" * 80)
+print("SenseNova U1.5 hidden-state extractor V3")
+print("=" * 80)
+
+print("\nChecking SenseNova Transformers compatibility...")
+
+with open(COMPAT_FILE, "r", encoding="utf-8") as f:
+    text = f.read()
+
+OLD = (
+    "    from transformers.utils.generic import "
+    "check_model_inputs as model_input_compat"
+)
+
+NEW = (
+    "    from transformers.utils.generic import check_model_inputs\n"
+    "    def model_input_compat(func: Callable[..., Any]) -> Callable[..., Any]:\n"
+    "        return check_model_inputs()(func)"
+)
+
+if OLD in text:
+
+    backup = COMPAT_FILE + ".bak"
+
+    if not os.path.exists(backup):
+        shutil.copy2(COMPAT_FILE, backup)
+        print("Backup created:")
+        print(backup)
+
+    text = text.replace(OLD, NEW)
+
+    with open(COMPAT_FILE, "w", encoding="utf-8") as f:
+        f.write(text)
+
+    print("Compatibility patch APPLIED.")
+
+elif "return check_model_inputs()(func)" in text:
+
+    print("Compatibility patch already present.")
+
+else:
+
+    print("WARNING: expected compatibility line not found.")
+    print("No automatic source modification performed.")
+
+
+# ============================================================
+# IMPORTANT:
+# Import SenseNova only AFTER patching source.
+# ============================================================
+
+sys.path.insert(0, SENSENOVA_SRC)
+
+from transformers import AutoTokenizer
+from accelerate import (
+    init_empty_weights,
+    load_checkpoint_and_dispatch,
+)
+
+from sensenova_u1.models.neo_unify.configuration_neo_chat import (
+    NEOChatConfig,
+)
+
+from sensenova_u1.models.neo_unify.modeling_neo_chat import (
+    NEOChatModel,
+)
+
+
+import transformers
+import accelerate
+
+print()
+print("PyTorch:", torch.__version__)
+print("Transformers:", transformers.__version__)
+print("Accelerate:", accelerate.__version__)
+
+print()
+print("GPU:")
+print(torch.cuda.get_device_name(0))
+
+print(
+    "VRAM:",
+    round(
+        torch.cuda.get_device_properties(0).total_memory
+        / 1024**3,
+        2
+    ),
+    "GB"
+)
+
+
+# ============================================================
+# TOKENIZER / CONFIG
+# ============================================================
+
+print("\nLoading tokenizer...")
+
+tokenizer = AutoTokenizer.from_pretrained(
+    HF_REPO,
+    trust_remote_code=True,
+)
+
+print("Loading LOCAL SenseNova config...")
+
+config = NEOChatConfig.from_pretrained(
+    HF_REPO,
+)
+
+# Force eager because SenseNova forward_und expects it.
+config.llm_config._attn_implementation = "eager"
+
+print("Config loaded.")
+print("LLM hidden size:", config.llm_config.hidden_size)
+print("LLM layers:", config.llm_config.num_hidden_layers)
+print(
+    "Attention implementation:",
+    config.llm_config._attn_implementation
+)
+
+
+# ============================================================
+# META MODEL
+# ============================================================
+
+print()
+print("Creating architecture on META device...")
+
+with init_empty_weights():
+
+    model = NEOChatModel(config)
+
+print("META architecture created.")
+
+
+# ============================================================
+# LOAD CHECKPOINT
+# ============================================================
+
+MAX_MEMORY = {
+    0: "19GiB",
+    "cpu": "70GiB",
+}
+
+print()
+print("Loading local BF16 checkpoint with Accelerate...")
+
+model = load_checkpoint_and_dispatch(
+    model,
+    checkpoint=MODEL_FILE,
+    device_map="auto",
+    max_memory=MAX_MEMORY,
+    dtype=torch.bfloat16,
+    no_split_module_classes=[
+        "Qwen3DecoderLayer",
+        "Qwen3MoeDecoderLayer",
+    ],
+)
+
+model.eval()
+
+print("Checkpoint loaded successfully.")
+
+
+# ============================================================
+# LANGUAGE MODEL
+# ============================================================
+
+language_model = model.language_model
+qwen_model = language_model.model
+layers = qwen_model.layers
+
+print()
+print("Language model:")
+print(type(language_model))
+
+print("Base model:")
+print(type(qwen_model))
+
+print("Detected language layers:", len(layers))
+
+
+# ============================================================
+# CHECK FORWARD SIGNATURE
+# ============================================================
+
+import inspect
+
+print()
+print("Qwen3Model.forward signature:")
+
+try:
+    print(inspect.signature(qwen_model.forward))
+except Exception as e:
+    print("Could not inspect:", e)
+
+# This MUST contain input_ids now.
+sig = str(inspect.signature(qwen_model.forward))
+
+if "input_ids" not in sig:
+
+    raise RuntimeError(
+        "SenseNova compatibility patch did not fix Qwen3Model.forward. "
+        f"Current signature: {sig}"
+    )
+
+print("Forward compatibility: OK")
+
+
+# ============================================================
+# DEVICE MAP
+# ============================================================
+
+device_map = getattr(model, "hf_device_map", {})
+
+gpu_count = 0
+cpu_count = 0
+disk_count = 0
+
+for _, dev in device_map.items():
+
+    s = str(dev)
+
+    if s in ("0", "cuda", "cuda:0"):
+        gpu_count += 1
+
+    elif s == "cpu":
+        cpu_count += 1
+
+    elif s == "disk":
+        disk_count += 1
+
+print()
+print("Device map:")
+print(" GPU modules :", gpu_count)
+print(" CPU modules :", cpu_count)
+print(" Disk modules:", disk_count)
+
+
+# ============================================================
+# INPUT DEVICE
+# ============================================================
+
+INPUT_DEVICE = torch.device("cuda:0")
+
+for name, dev in device_map.items():
+
+    if name.endswith("language_model.model.embed_tokens"):
+
+        if str(dev) == "0":
+            INPUT_DEVICE = torch.device("cuda:0")
+
+        elif str(dev) == "cpu":
+            INPUT_DEVICE = torch.device("cpu")
+
+        break
+
+print()
+print("Input device:", INPUT_DEVICE)
+
+
+# ============================================================
+# HOOKS
+# ============================================================
+
+captured = {}
+hooks = {}
+
+
+def make_hook(idx):
+
+    def hook(module, inputs, output):
+
+        if isinstance(output, tuple):
+            x = output[0]
+        else:
+            x = output
+
+        if not torch.is_tensor(x):
+            print(
+                f"WARNING layer {idx}: "
+                f"unexpected output type {type(x)}"
+            )
+            return
+
+        captured[idx] = (
+            x.detach()
+             .float()
+             .cpu()
+        )
+
+    return hook
+
+
+print()
+print("Registering hooks...")
+
+for idx in TARGET_LAYERS:
+
+    if idx >= len(layers):
+        print("layer", idx, ": OUT OF RANGE")
+        continue
+
+    hooks[idx] = layers[idx].register_forward_hook(
+        make_hook(idx)
+    )
+
+    print(
+        f"layer {idx}:",
+        type(layers[idx])
+    )
+
+
+# ============================================================
+# RUN
+# ============================================================
+
+results = []
+
+for i, prompt in enumerate(PROMPTS):
+
+    print()
+    print("=" * 80)
+    print(f"[{i + 1}/{len(PROMPTS)}]")
+    print(prompt)
+    print("=" * 80)
+
+    encoded = tokenizer(
+        prompt,
+        return_tensors="pt",
+        add_special_tokens=False,
+    )
+
+    input_ids = encoded["input_ids"].to(
+        INPUT_DEVICE
+    )
+
+    print(
+        "Tokens:",
+        input_ids.detach().cpu().tolist()[0]
+    )
+
+    print("Token count:", input_ids.shape[1])
+    print("Input device:", input_ids.device)
+
+    captured.clear()
+
+    # Reset SenseNova internal prompt index.
+    qwen_model.current_index = -1
+
+    with torch.inference_mode():
+
+        outputs = qwen_model(
+            input_ids=input_ids,
+            image_gen_indicators=None,
+            attention_mask=None,
+            use_cache=False,
+        )
+
+    item = {
+        "prompt": prompt,
+        "input_ids": (
+            input_ids.detach()
+            .cpu()
+        ),
+        "layers": {},
+    }
+
+    print()
+
+    for idx in TARGET_LAYERS:
+
+        if idx not in captured:
+
+            print(
+                f"layer {idx:2d}: NOT CAPTURED"
+            )
+            continue
+
+        x = captured[idx]
+
+        item["layers"][idx] = x
+
+        finite = torch.isfinite(x).all().item()
+
+        print(
+            f"layer {idx:2d}: "
+            f"shape={tuple(x.shape)} "
+            f"dtype={x.dtype} "
+            f"mean={x.mean().item():.6f} "
+            f"std={x.std().item():.6f} "
+            f"finite={finite}"
+        )
+
+    results.append(item)
+
+    del outputs
+    del input_ids
+
+    gc.collect()
+    torch.cuda.empty_cache()
+
+
+# ============================================================
+# REMOVE HOOKS
+# ============================================================
+
+for h in hooks.values():
+    h.remove()
+
+
+# ============================================================
+# VALIDATE
+# ============================================================
+
+print()
+print("=" * 80)
+print("VALIDATION")
+print("=" * 80)
+
+all_ok = True
+
+for item in results:
+
+    print()
+    print(item["prompt"])
+
+    for idx in TARGET_LAYERS:
+
+        if idx not in item["layers"]:
+
+            print(
+                f" layer {idx}: missing"
+            )
+
+            all_ok = False
+            continue
+
+        x = item["layers"][idx]
+
+        dim_ok = (
+            x.ndim == 3
+            and x.shape[-1] == 4096
+        )
+
+        finite = torch.isfinite(x).all().item()
+
+        print(
+            f" layer {idx}: "
+            f"{tuple(x.shape)} "
+            f"dim_ok={dim_ok} "
+            f"finite={finite}"
+        )
+
+        if not dim_ok or not finite:
+            all_ok = False
+
+
+# ============================================================
+# SAVE
+# ============================================================
+
+torch.save(
+    {
+        "prompts": PROMPTS,
+        "target_layers": TARGET_LAYERS,
+        "results": results,
+        "source_model": MODEL_FILE,
+        "transformers_version": transformers.__version__,
+    },
+    OUT,
+)
+
+
+print()
+print("=" * 80)
+
+if all_ok:
+    print("SUCCESS")
+else:
+    print("COMPLETED WITH WARNINGS")
+
+print()
+print("Saved:")
+print(OUT)
+
+print("=" * 80)
+
+gc.collect()
+torch.cuda.empty_cache()

research/raw_scripts/extract_sensenova_ood_hidden.py

@@ -0,0 +1,633 @@
+import os
+import sys
+import json
+import gc
+import shutil
+import torch
+
+
+# ============================================================
+# PATHS
+# ============================================================
+
+ROOT = r"E:\Models SD XL\UNET\Minimax_H3"
+
+SENSENOVA_SRC = os.path.join(
+    ROOT,
+    "SenseNova-U1",
+    "src"
+)
+
+COMPAT_FILE = os.path.join(
+    SENSENOVA_SRC,
+    "sensenova_u1",
+    "models",
+    "neo_unify",
+    "transformers_compat.py"
+)
+
+CHECKPOINT = (
+    r"D:\ComfyUI_Python312\ComfyUI\models\unet"
+    r"\SenseNova-U1.5-8B-MoT-pruned-bf16.safetensors"
+)
+
+PROMPTS_FILE = os.path.join(
+    ROOT,
+    "bridge_ood_prompts_160.json"
+)
+
+OUT_FILE = os.path.join(
+    ROOT,
+    "sensenova_ood_hidden_160.pt"
+)
+
+HF_REPO = "sensenova/SenseNova-U1.5-8B-MoT"
+
+LAYER = 32
+
+
+# ============================================================
+# COMPATIBILITY PATCH
+# ============================================================
+
+print("=" * 80)
+print("SenseNova U1.5 OOD hidden-state extraction")
+print("=" * 80)
+
+print("\nChecking Transformers compatibility patch...")
+
+with open(
+    COMPAT_FILE,
+    "r",
+    encoding="utf-8"
+) as f:
+    text = f.read()
+
+OLD = (
+    "    from transformers.utils.generic import "
+    "check_model_inputs as model_input_compat"
+)
+
+NEW = (
+    "    from transformers.utils.generic import check_model_inputs\n"
+    "    def model_input_compat(func):\n"
+    "        return check_model_inputs()(func)"
+)
+
+if OLD in text:
+
+    backup = COMPAT_FILE + ".bak"
+
+    if not os.path.exists(backup):
+        shutil.copy2(
+            COMPAT_FILE,
+            backup
+        )
+
+    text = text.replace(
+        OLD,
+        NEW
+    )
+
+    with open(
+        COMPAT_FILE,
+        "w",
+        encoding="utf-8"
+    ) as f:
+        f.write(text)
+
+    print("Compatibility patch applied.")
+
+elif "return check_model_inputs()(func)" in text:
+
+    print("Compatibility patch already present.")
+
+else:
+
+    print(
+        "WARNING: expected compatibility code "
+        "not found."
+    )
+
+
+# ============================================================
+# IMPORT LOCAL SENSENOVA CODE
+# ============================================================
+
+sys.path.insert(
+    0,
+    SENSENOVA_SRC
+)
+
+from transformers import AutoTokenizer
+from accelerate import (
+    init_empty_weights,
+    load_checkpoint_and_dispatch
+)
+
+from sensenova_u1.models.neo_unify.configuration_neo_chat import (
+    NEOChatConfig
+)
+
+from sensenova_u1.models.neo_unify.modeling_neo_chat import (
+    NEOChatModel
+)
+
+
+# ============================================================
+# SYSTEM INFO
+# ============================================================
+
+if not torch.cuda.is_available():
+    raise RuntimeError(
+        "CUDA is required."
+    )
+
+print("\nGPU:")
+print(torch.cuda.get_device_name(0))
+
+print(
+    "VRAM:",
+    round(
+        torch.cuda.get_device_properties(0).total_memory
+        / 1024**3,
+        2
+    ),
+    "GB"
+)
+
+
+# ============================================================
+# LOAD OOD PROMPTS
+# ============================================================
+
+print("\nLoading OOD prompts...")
+
+with open(
+    PROMPTS_FILE,
+    "r",
+    encoding="utf-8"
+) as f:
+    prompts = json.load(f)
+
+print("Prompt count:", len(prompts))
+
+if len(prompts) != 160:
+    raise RuntimeError(
+        f"Expected 160 prompts, got {len(prompts)}"
+    )
+
+
+# ============================================================
+# TOKENIZER
+# ============================================================
+
+print("\nLoading tokenizer...")
+
+tokenizer = AutoTokenizer.from_pretrained(
+    HF_REPO,
+    trust_remote_code=True,
+)
+
+
+# ============================================================
+# CONFIG
+# ============================================================
+
+print("Loading SenseNova U1.5 config...")
+
+config = NEOChatConfig.from_pretrained(
+    HF_REPO
+)
+
+config.llm_config._attn_implementation = "eager"
+
+print("Config loaded.")
+print(
+    "LLM hidden size:",
+    config.llm_config.hidden_size
+)
+print(
+    "LLM layers:",
+    config.llm_config.num_hidden_layers
+)
+print(
+    "Attention:",
+    config.llm_config._attn_implementation
+)
+
+
+# ============================================================
+# CREATE MODEL ON META
+# ============================================================
+
+print()
+print("Creating META architecture...")
+
+with init_empty_weights():
+
+    model = NEOChatModel(
+        config
+    )
+
+print("META architecture created.")
+
+
+# ============================================================
+# LOAD CHECKPOINT
+# ============================================================
+
+print()
+print(
+    "Loading BF16 checkpoint with Accelerate..."
+)
+
+model = load_checkpoint_and_dispatch(
+    model,
+    checkpoint=CHECKPOINT,
+    device_map="auto",
+    max_memory={
+        0: "19GiB",
+        "cpu": "70GiB",
+    },
+    dtype=torch.bfloat16,
+    no_split_module_classes=[
+        "Qwen3DecoderLayer",
+        "Qwen3MoeDecoderLayer",
+    ],
+)
+
+model.eval()
+
+print("Checkpoint loaded successfully.")
+
+
+# ============================================================
+# LANGUAGE MODEL
+# ============================================================
+
+language_model = model.language_model
+qwen = language_model.model
+layers = qwen.layers
+
+print()
+print(
+    "Detected language layers:",
+    len(layers)
+)
+
+if LAYER >= len(layers):
+    raise RuntimeError(
+        f"Layer {LAYER} does not exist."
+    )
+
+
+# ============================================================
+# DEVICE MAP
+# ============================================================
+
+device_map = getattr(
+    model,
+    "hf_device_map",
+    {}
+)
+
+gpu_count = 0
+cpu_count = 0
+disk_count = 0
+
+for _, dev in device_map.items():
+
+    s = str(dev)
+
+    if s in (
+        "0",
+        "cuda",
+        "cuda:0"
+    ):
+        gpu_count += 1
+
+    elif s == "cpu":
+        cpu_count += 1
+
+    elif s == "disk":
+        disk_count += 1
+
+
+print()
+print("Device map:")
+print(" GPU modules :", gpu_count)
+print(" CPU modules :", cpu_count)
+print(" Disk modules:", disk_count)
+
+
+# ============================================================
+# FIND INPUT DEVICE
+# ============================================================
+
+INPUT_DEVICE = torch.device(
+    "cuda:0"
+)
+
+for name, dev in device_map.items():
+
+    if name.endswith(
+        "language_model.model.embed_tokens"
+    ):
+
+        if str(dev) == "cpu":
+            INPUT_DEVICE = torch.device(
+                "cpu"
+            )
+
+        else:
+            INPUT_DEVICE = torch.device(
+                "cuda:0"
+            )
+
+        break
+
+
+print()
+print("Input device:", INPUT_DEVICE)
+
+
+# ============================================================
+# HOOK ONLY LAYER 32
+# ============================================================
+
+captured = {}
+
+
+def hook_fn(
+    module,
+    inputs,
+    output
+):
+
+    if isinstance(
+        output,
+        tuple
+    ):
+        hidden = output[0]
+
+    else:
+        hidden = output
+
+    if not torch.is_tensor(
+        hidden
+    ):
+        raise RuntimeError(
+            "Unexpected layer output type: "
+            f"{type(hidden)}"
+        )
+
+    captured["hidden"] = (
+        hidden.detach()
+        .to(
+            device="cpu",
+            dtype=torch.float16
+        )
+        .clone()
+    )
+
+
+handle = (
+    layers[LAYER]
+    .register_forward_hook(
+        hook_fn
+    )
+)
+
+print()
+print(
+    "Hook registered on "
+    f"SenseNova layer {LAYER}"
+)
+
+
+# ============================================================
+# EXTRACTION
+# ============================================================
+
+results = []
+
+print()
+print("=" * 80)
+print("STARTING EXTRACTION")
+print("=" * 80)
+print()
+
+with torch.inference_mode():
+
+    for index, item in enumerate(
+        prompts
+    ):
+
+        prompt = item["prompt"]
+        category = item["category"]
+
+        encoded = tokenizer(
+            prompt,
+            return_tensors="pt",
+            add_special_tokens=False,
+        )
+
+        input_ids_cpu = (
+            encoded["input_ids"]
+            .detach()
+            .cpu()
+        )
+
+        input_ids = (
+            encoded["input_ids"]
+            .to(INPUT_DEVICE)
+        )
+
+        captured.clear()
+
+        # Same reset used in our successful V3.
+        if hasattr(
+            qwen,
+            "current_index"
+        ):
+            qwen.current_index = -1
+
+        outputs = qwen(
+            input_ids=input_ids,
+
+            # IMPORTANT:
+            # understanding branch, NOT MoT-gen.
+            image_gen_indicators=None,
+
+            attention_mask=None,
+            use_cache=False,
+        )
+
+        if "hidden" not in captured:
+
+            raise RuntimeError(
+                f"Layer hook failed "
+                f"at prompt {index}"
+            )
+
+        hidden = captured[
+            "hidden"
+        ]
+
+        # ====================================================
+        # VALIDATE
+        # ====================================================
+
+        if hidden.ndim != 3:
+
+            raise RuntimeError(
+                "Unexpected hidden shape "
+                f"at prompt {index}: "
+                f"{hidden.shape}"
+            )
+
+        if hidden.shape[-1] != 4096:
+
+            raise RuntimeError(
+                f"Expected hidden dim 4096, "
+                f"got {hidden.shape[-1]}"
+            )
+
+        if not torch.isfinite(
+            hidden.float()
+        ).all():
+
+            raise RuntimeError(
+                "Non-finite SenseNova "
+                f"hidden state at prompt {index}"
+            )
+
+
+        # ====================================================
+        # STORE
+        # ====================================================
+
+        results.append({
+            "prompt": prompt,
+            "category": category,
+            "input_ids": input_ids_cpu,
+            "hidden": hidden,
+        })
+
+
+        # ====================================================
+        # PROGRESS
+        # ====================================================
+
+        if (
+            index == 0
+            or (index + 1) % 10 == 0
+            or index + 1 == len(prompts)
+        ):
+
+            print(
+                f"{index + 1:3d}/"
+                f"{len(prompts)} | "
+                f"tokens={hidden.shape[1]:3d} | "
+                f"shape={tuple(hidden.shape)}"
+            )
+
+
+        del outputs
+        del input_ids
+
+        if (
+            index + 1
+        ) % 20 == 0:
+
+            gc.collect()
+            torch.cuda.empty_cache()
+
+
+# ============================================================
+# REMOVE HOOK
+# ============================================================
+
+handle.remove()
+
+
+# ============================================================
+# FINAL CROSS-CHECK
+# ============================================================
+
+if len(results) != 160:
+
+    raise RuntimeError(
+        "Wrong number of extracted prompts: "
+        f"{len(results)}"
+    )
+
+
+for i, result in enumerate(
+    results
+):
+
+    expected = prompts[i]
+
+    if (
+        result["prompt"]
+        != expected["prompt"]
+    ):
+
+        raise RuntimeError(
+            f"Prompt mismatch at {i}"
+        )
+
+
+# ============================================================
+# SAVE
+# ============================================================
+
+print()
+print("Saving OOD hidden states...")
+
+payload = {
+    "model": (
+        "SenseNova-U1.5-8B-MoT"
+    ),
+    "branch": "understanding",
+    "layer": LAYER,
+    "hidden_dim": 4096,
+    "prompt_file": os.path.basename(
+        PROMPTS_FILE
+    ),
+    "results": results,
+}
+
+torch.save(
+    payload,
+    OUT_FILE
+)
+
+
+print()
+print("=" * 80)
+print("SUCCESS")
+print("=" * 80)
+
+print(
+    "Prompts:",
+    len(results)
+)
+
+print(
+    "Layer:",
+    LAYER
+)
+
+print(
+    "Branch: understanding"
+)
+
+print("\nOutput:")
+print(OUT_FILE)
+
+
+gc.collect()
+torch.cuda.empty_cache()

research/raw_scripts/inspect_distillation_data.py

@@ -0,0 +1,466 @@
+import os
+import torch
+from collections import Counter, defaultdict
+
+ROOT = r"E:\Models SD XL\UNET\Minimax_H3"
+
+H3_FILE = os.path.join(
+    ROOT,
+    "h3_bridge_hidden_480.pt"
+)
+
+SN_FILE = os.path.join(
+    ROOT,
+    "sensenova_bridge_hidden_480.pt"
+)
+
+BRIDGE_FILE = (
+    r"D:\ComfyUI_Krea2\ComfyUI\models\bridge"
+    r"\SN_L32_to_H3_L49_rank128.safetensors"
+)
+
+REPORT_FILE = os.path.join(
+    ROOT,
+    "distillation_source_inspection.txt"
+)
+
+
+# ============================================================
+# HELPERS
+# ============================================================
+
+def tensor_info(t):
+    return (
+        f"shape={tuple(t.shape)} "
+        f"dtype={t.dtype} "
+        f"device={t.device}"
+    )
+
+
+def walk(obj, path="root", out=None, depth=0, max_depth=8):
+    if out is None:
+        out = []
+
+    if depth > max_depth:
+        return out
+
+    if torch.is_tensor(obj):
+        out.append({
+            "path": path,
+            "shape": tuple(obj.shape),
+            "dtype": str(obj.dtype),
+            "tensor": obj,
+        })
+        return out
+
+    if isinstance(obj, dict):
+        for key, value in obj.items():
+            walk(
+                value,
+                f"{path}.{key}",
+                out,
+                depth + 1,
+                max_depth,
+            )
+
+    elif isinstance(obj, (list, tuple)):
+        for i, value in enumerate(obj):
+            walk(
+                value,
+                f"{path}[{i}]",
+                out,
+                depth + 1,
+                max_depth,
+            )
+
+    return out
+
+
+def print_structure(obj, name, lines, depth=0, max_depth=4, max_items=4):
+    indent = "  " * depth
+
+    if depth > max_depth:
+        lines.append(f"{indent}...")
+        return
+
+    if torch.is_tensor(obj):
+        lines.append(
+            f"{indent}{tensor_info(obj)}"
+        )
+        return
+
+    if isinstance(obj, dict):
+        lines.append(
+            f"{indent}dict keys={len(obj)}"
+        )
+
+        for i, (key, value) in enumerate(obj.items()):
+            if i >= max_items:
+                lines.append(
+                    f"{indent}  ... ({len(obj) - max_items} more)"
+                )
+                break
+
+            lines.append(
+                f"{indent}  KEY: {repr(key)}"
+            )
+
+            print_structure(
+                value,
+                name,
+                lines,
+                depth + 2,
+                max_depth,
+                max_items,
+            )
+
+    elif isinstance(obj, (list, tuple)):
+        lines.append(
+            f"{indent}{type(obj).__name__} len={len(obj)}"
+        )
+
+        for i, value in enumerate(obj[:max_items]):
+            lines.append(
+                f"{indent}  ITEM [{i}]"
+            )
+
+            print_structure(
+                value,
+                name,
+                lines,
+                depth + 2,
+                max_depth,
+                max_items,
+            )
+
+        if len(obj) > max_items:
+            lines.append(
+                f"{indent}  ... ({len(obj) - max_items} more)"
+            )
+
+    else:
+        value_repr = repr(obj)
+
+        if len(value_repr) > 180:
+            value_repr = value_repr[:180] + "..."
+
+        lines.append(
+            f"{indent}{type(obj).__name__}: {value_repr}"
+        )
+
+
+def analyze_file(label, path, lines):
+    lines.append("")
+    lines.append("=" * 100)
+    lines.append(label)
+    lines.append("=" * 100)
+    lines.append(f"Path: {path}")
+    lines.append(
+        f"Size: {os.path.getsize(path) / (1024 ** 2):.2f} MiB"
+    )
+
+    print(f"\nLoading {label}...")
+    data = torch.load(
+        path,
+        map_location="cpu",
+        weights_only=False,
+    )
+
+    print(f"{label} loaded.")
+
+    lines.append("")
+    lines.append("TOP-LEVEL STRUCTURE")
+    lines.append("-" * 100)
+
+    print_structure(
+        data,
+        label,
+        lines,
+    )
+
+    tensors = walk(data)
+
+    lines.append("")
+    lines.append(
+        f"Total tensors found recursively: {len(tensors)}"
+    )
+
+    shape_counter = Counter(
+        item["shape"]
+        for item in tensors
+    )
+
+    lines.append("")
+    lines.append("MOST COMMON TENSOR SHAPES")
+    lines.append("-" * 100)
+
+    for shape, count in shape_counter.most_common(20):
+        lines.append(
+            f"{count:6d} x {shape}"
+        )
+
+    # ========================================================
+    # HIDDEN-STATE CANDIDATES
+    # ========================================================
+
+    candidates_4096 = []
+    candidates_5120 = []
+
+    for item in tensors:
+        shape = item["shape"]
+
+        if len(shape) >= 1:
+            if shape[-1] == 4096:
+                candidates_4096.append(item)
+
+            if shape[-1] == 5120:
+                candidates_5120.append(item)
+
+    lines.append("")
+    lines.append("4096-DIM CANDIDATES")
+    lines.append("-" * 100)
+    lines.append(
+        f"Count: {len(candidates_4096)}"
+    )
+
+    for item in candidates_4096[:30]:
+        lines.append(
+            f"{item['path']} | "
+            f"{item['shape']} | "
+            f"{item['dtype']}"
+        )
+
+    if len(candidates_4096) > 30:
+        lines.append(
+            f"... {len(candidates_4096) - 30} more"
+        )
+
+    lines.append("")
+    lines.append("5120-DIM CANDIDATES")
+    lines.append("-" * 100)
+    lines.append(
+        f"Count: {len(candidates_5120)}"
+    )
+
+    for item in candidates_5120[:30]:
+        lines.append(
+            f"{item['path']} | "
+            f"{item['shape']} | "
+            f"{item['dtype']}"
+        )
+
+    if len(candidates_5120) > 30:
+        lines.append(
+            f"... {len(candidates_5120) - 30} more"
+        )
+
+    # ========================================================
+    # PATHS THAT LOOK LIKE OUR TARGET LAYERS
+    # ========================================================
+
+    lines.append("")
+    lines.append("PATHS CONTAINING LAYER 32 / L32")
+    lines.append("-" * 100)
+
+    layer32 = [
+        item
+        for item in tensors
+        if (
+            "32" in item["path"].lower()
+            or "l32" in item["path"].lower()
+            or "layer_32" in item["path"].lower()
+            or "layer32" in item["path"].lower()
+        )
+    ]
+
+    lines.append(
+        f"Count: {len(layer32)}"
+    )
+
+    for item in layer32[:40]:
+        lines.append(
+            f"{item['path']} | "
+            f"{item['shape']}"
+        )
+
+    lines.append("")
+    lines.append("PATHS CONTAINING LAYER 49 / L49")
+    lines.append("-" * 100)
+
+    layer49 = [
+        item
+        for item in tensors
+        if (
+            "49" in item["path"].lower()
+            or "l49" in item["path"].lower()
+            or "layer_49" in item["path"].lower()
+            or "layer49" in item["path"].lower()
+        )
+    ]
+
+    lines.append(
+        f"Count: {len(layer49)}"
+    )
+
+    for item in layer49[:40]:
+        lines.append(
+            f"{item['path']} | "
+            f"{item['shape']}"
+        )
+
+    return data, tensors
+
+
+# ============================================================
+# MAIN
+# ============================================================
+
+print("=" * 100)
+print("DISTILLATION SOURCE INSPECTION")
+print("=" * 100)
+
+for path in [
+    H3_FILE,
+    SN_FILE,
+    BRIDGE_FILE,
+]:
+    if not os.path.exists(path):
+        raise FileNotFoundError(
+            f"Missing required file:\n{path}"
+        )
+
+print("H3:")
+print(H3_FILE)
+
+print("\nSenseNova:")
+print(SN_FILE)
+
+print("\nBridge:")
+print(BRIDGE_FILE)
+
+lines = []
+
+lines.append(
+    "SenseNova -> MiniMax H3 Distillation Source Inspection"
+)
+
+lines.append(
+    "=" * 100
+)
+
+h3_data, h3_tensors = analyze_file(
+    "MINIMAX H3 DATA",
+    H3_FILE,
+    lines,
+)
+
+sn_data, sn_tensors = analyze_file(
+    "SENSENOVA DATA",
+    SN_FILE,
+    lines,
+)
+
+
+# ============================================================
+# BRIDGE
+# ============================================================
+
+lines.append("")
+lines.append("=" * 100)
+lines.append("BRIDGE FILE")
+lines.append("=" * 100)
+lines.append(f"Path: {BRIDGE_FILE}")
+
+from safetensors.torch import load_file
+
+bridge = load_file(
+    BRIDGE_FILE,
+    device="cpu",
+)
+
+for key, tensor in bridge.items():
+    lines.append(
+        f"{key:40s} "
+        f"{tuple(tensor.shape)} "
+        f"{tensor.dtype}"
+    )
+
+
+# ============================================================
+# SUMMARY COUNTS
+# ============================================================
+
+h3_5120 = [
+    x for x in h3_tensors
+    if (
+        len(x["shape"]) > 0
+        and x["shape"][-1] == 5120
+    )
+]
+
+sn_4096 = [
+    x for x in sn_tensors
+    if (
+        len(x["shape"]) > 0
+        and x["shape"][-1] == 4096
+    )
+]
+
+lines.append("")
+lines.append("=" * 100)
+lines.append("FINAL SUMMARY")
+lines.append("=" * 100)
+
+lines.append(
+    f"H3 5120-dim tensors found:       {len(h3_5120)}"
+)
+
+lines.append(
+    f"SenseNova 4096-dim tensors found: {len(sn_4096)}"
+)
+
+lines.append("")
+lines.append(
+    "The next script will use this report to identify the exact "
+    "H3 input representation and SenseNova teacher target without "
+    "guessing the .pt layout."
+)
+
+
+# ============================================================
+# SAVE
+# ============================================================
+
+with open(
+    REPORT_FILE,
+    "w",
+    encoding="utf-8",
+) as f:
+    f.write(
+        "\n".join(lines)
+    )
+
+print()
+print("=" * 100)
+print("SUCCESS")
+print("=" * 100)
+
+print("Report saved:")
+print(REPORT_FILE)
+
+print()
+print(
+    "H3 5120-dim tensors:",
+    len(h3_5120)
+)
+
+print(
+    "SenseNova 4096-dim tensors:",
+    len(sn_4096)
+)
+
+print()
+print(
+    "Send me the contents of "
+    "distillation_source_inspection.txt"
+)

research/raw_scripts/inspect_h3_sensenova_bridge.py

@@ -0,0 +1,274 @@
+from safetensors import safe_open
+from collections import Counter
+import os
+import re
+
+H3_TE = r"D:\ComfyUI_Python312\ComfyUI\models\text_encoders\qwen3vl_32b_minimax_h3_nvfp4_awq.safetensors"
+SENSENOVA = r"D:\ComfyUI_Python312\ComfyUI\models\unet\SenseNova-U1.5-8B-MoT-pruned-bf16.safetensors"
+
+OUT = r"E:\Models SD XL\UNET\Minimax_H3\h3_sensenova_bridge_report.txt"
+
+
+def inspect(path):
+    data = {}
+    meta = {}
+
+    with safe_open(path, framework="pt", device="cpu") as f:
+        meta = f.metadata() or {}
+
+        for k in f.keys():
+            s = f.get_slice(k)
+            shape = tuple(s.get_shape())
+
+            data[k] = {
+                "shape": shape,
+                "ndim": len(shape),
+            }
+
+    return data, meta
+
+
+def get_layer_number(k):
+    patterns = [
+        r"layers\.(\d+)",
+        r"blocks\.(\d+)",
+        r"transformer_blocks\.(\d+)",
+    ]
+
+    for p in patterns:
+        m = re.search(p, k)
+        if m:
+            return int(m.group(1))
+
+    return None
+
+
+def likely_embedding(k):
+    x = k.lower()
+    return any(s in x for s in [
+        "embed_tokens",
+        "token_embedding",
+        "word_embeddings",
+        "tok_embeddings",
+    ])
+
+
+def likely_norm(k):
+    return "norm" in k.lower()
+
+
+def likely_attention(k):
+    x = k.lower()
+    return any(s in x for s in [
+        "q_proj", "k_proj", "v_proj", "o_proj",
+        "qkv", "attention", "attn"
+    ])
+
+
+print("Reading H3 TE...")
+h3, h3_meta = inspect(H3_TE)
+
+print("Reading SenseNova...")
+sn, sn_meta = inspect(SENSENOVA)
+
+
+with open(OUT, "w", encoding="utf-8") as f:
+
+    def w(x=""):
+        f.write(str(x) + "\n")
+
+    w("=" * 100)
+    w("H3 TE <-> SENSENOVA BRIDGE DIAGNOSTIC")
+    w("=" * 100)
+    w()
+
+    w("FILES")
+    w("-" * 100)
+    w(f"H3 TE:      {H3_TE}")
+    w(f"Size:       {os.path.getsize(H3_TE) / 1024**3:.3f} GB")
+    w(f"Tensors:    {len(h3)}")
+    w()
+    w(f"SenseNova:  {SENSENOVA}")
+    w(f"Size:       {os.path.getsize(SENSENOVA) / 1024**3:.3f} GB")
+    w(f"Tensors:    {len(sn)}")
+    w()
+
+    # -------------------------------------------------------
+    # H3 dimensions
+    # -------------------------------------------------------
+
+    w("=" * 100)
+    w("H3 TE MATRICES CONTAINING DIMENSION 5120")
+    w("=" * 100)
+
+    h3_5120 = []
+
+    for k, v in h3.items():
+        if 5120 in v["shape"]:
+            h3_5120.append((k, v["shape"]))
+            w(f"{k} | {v['shape']}")
+
+    w()
+    w(f"COUNT: {len(h3_5120)}")
+    w()
+
+    # -------------------------------------------------------
+    # SenseNova dimensions
+    # -------------------------------------------------------
+
+    w("=" * 100)
+    w("SENSENOVA MATRICES CONTAINING DIMENSION 4096")
+    w("=" * 100)
+
+    sn_4096 = []
+
+    for k, v in sn.items():
+        if 4096 in v["shape"]:
+            sn_4096.append((k, v["shape"]))
+            w(f"{k} | {v['shape']}")
+
+    w()
+    w(f"COUNT: {len(sn_4096)}")
+    w()
+
+    # -------------------------------------------------------
+    # Embeddings
+    # -------------------------------------------------------
+
+    w("=" * 100)
+    w("LIKELY TOKEN EMBEDDINGS")
+    w("=" * 100)
+
+    w("H3:")
+    for k, v in h3.items():
+        if likely_embedding(k):
+            w(f"  {k} | {v['shape']}")
+
+    w()
+    w("SENSENOVA:")
+    for k, v in sn.items():
+        if likely_embedding(k):
+            w(f"  {k} | {v['shape']}")
+
+    w()
+
+    # -------------------------------------------------------
+    # Layer counts
+    # -------------------------------------------------------
+
+    h3_layers = sorted(set(
+        x for x in (get_layer_number(k) for k in h3)
+        if x is not None
+    ))
+
+    sn_layers = sorted(set(
+        x for x in (get_layer_number(k) for k in sn)
+        if x is not None
+    ))
+
+    w("=" * 100)
+    w("DETECTED LAYERS")
+    w("=" * 100)
+
+    w(f"H3 layer indexes ({len(h3_layers)}):")
+    w(str(h3_layers))
+
+    w()
+
+    w(f"SenseNova layer indexes ({len(sn_layers)}):")
+    w(str(sn_layers))
+
+    w()
+
+    # -------------------------------------------------------
+    # Selected H3 late layers
+    # -------------------------------------------------------
+
+    w("=" * 100)
+    w("H3 LAST DETECTED LANGUAGE-LIKE LAYERS")
+    w("=" * 100)
+
+    if h3_layers:
+        selected = set(h3_layers[-5:])
+
+        for k, v in h3.items():
+            n = get_layer_number(k)
+            if n in selected:
+                w(f"{k} | {v['shape']}")
+
+    w()
+
+    # -------------------------------------------------------
+    # SenseNova late layers
+    # -------------------------------------------------------
+
+    w("=" * 100)
+    w("SENSENOVA LAST DETECTED LANGUAGE-LIKE LAYERS")
+    w("=" * 100)
+
+    if sn_layers:
+        selected = set(sn_layers[-5:])
+
+        for k, v in sn.items():
+            n = get_layer_number(k)
+            if n in selected:
+                w(f"{k} | {v['shape']}")
+
+    w()
+
+    # -------------------------------------------------------
+    # Attention candidates
+    # -------------------------------------------------------
+
+    w("=" * 100)
+    w("H3 ATTENTION-LIKE MATRICES WITH 5120")
+    w("=" * 100)
+
+    for k, v in h3.items():
+        if v["ndim"] == 2 and 5120 in v["shape"] and likely_attention(k):
+            w(f"{k} | {v['shape']}")
+
+    w()
+
+    w("=" * 100)
+    w("SENSENOVA ATTENTION-LIKE MATRICES WITH 4096")
+    w("=" * 100)
+
+    for k, v in sn.items():
+        if v["ndim"] == 2 and 4096 in v["shape"] and likely_attention(k):
+            w(f"{k} | {v['shape']}")
+
+    w()
+
+    # -------------------------------------------------------
+    # Metadata
+    # -------------------------------------------------------
+
+    w("=" * 100)
+    w("H3 METADATA")
+    w("=" * 100)
+
+    for k, v in h3_meta.items():
+        if k == "_quantization_metadata":
+            w("_quantization_metadata = [present, omitted because very large]")
+        else:
+            w(f"{k} = {v}")
+
+    w()
+
+    w("=" * 100)
+    w("SENSENOVA METADATA")
+    w("=" * 100)
+
+    for k, v in sn_meta.items():
+        w(f"{k} = {v}")
+
+
+print()
+print("DONE")
+print(OUT)
+print()
+print("H3 tensors containing 5120:", len(h3_5120))
+print("SenseNova tensors containing 4096:", len(sn_4096))
+print("H3 detected layers:", len(h3_layers))
+print("SenseNova detected layers:", len(sn_layers))

research/raw_scripts/make_bridge_ood_prompts.py

@@ -0,0 +1,391 @@
+import os
+import json
+import random
+
+ROOT = r"E:\Models SD XL\UNET\Minimax_H3"
+
+TRAIN_FILE = os.path.join(
+    ROOT,
+    "bridge_prompts_480.json"
+)
+
+OUT = os.path.join(
+    ROOT,
+    "bridge_ood_prompts_160.json"
+)
+
+random.seed(20260903)
+
+# ============================================================
+# LOAD TRAINING PROMPTS
+# ============================================================
+
+with open(TRAIN_FILE, "r", encoding="utf-8") as f:
+    train_data = json.load(f)
+
+train_prompts = {
+    x["prompt"].strip().lower()
+    for x in train_data
+}
+
+data = []
+used = set()
+
+
+def add(category, prompt):
+    prompt = " ".join(prompt.strip().split())
+    key = prompt.lower()
+
+    if key in used:
+        return False
+
+    if key in train_prompts:
+        return False
+
+    used.add(key)
+
+    data.append({
+        "category": category,
+        "prompt": prompt,
+    })
+
+    return True
+
+
+# ============================================================
+# VOCABULARY
+# ============================================================
+
+people = [
+    "a young woman",
+    "an elderly man",
+    "a tall man",
+    "a woman wearing a dark jacket",
+    "a seated woman",
+    "a man wearing a gray coat",
+    "a woman with short hair",
+    "an elderly woman",
+]
+
+objects = [
+    "glass bottle",
+    "ceramic cup",
+    "metal box",
+    "wooden chair",
+    "blue book",
+    "silver sphere",
+    "transparent cube",
+    "black vase",
+    "small lamp",
+    "red cylinder",
+]
+
+places = [
+    "modern kitchen",
+    "railway platform",
+    "glass office lobby",
+    "small workshop",
+    "hotel entrance",
+    "underground station",
+    "minimalist living room",
+    "industrial studio",
+]
+
+materials = [
+    "polished chrome",
+    "frosted glass",
+    "brushed aluminum",
+    "wet black stone",
+    "translucent amber acrylic",
+    "glossy ceramic",
+    "rough concrete",
+    "dark polished wood",
+]
+
+lightings = [
+    "warm light entering from the left",
+    "cool light entering from the right",
+    "strong backlight",
+    "a narrow spotlight from above",
+    "soft diffused window light",
+    "hard side lighting",
+    "warm reflected light from below",
+    "two opposing light sources",
+]
+
+texts = [
+    "NORTH GATE",
+    "ROOM 314",
+    "PLATFORM 11",
+    "WEST EXIT",
+    "OPEN 24 HOURS",
+    "STUDIO C",
+    "RIVER HOTEL",
+    "CAFE LEVEL 2",
+    "AUTHORIZED ENTRY",
+    "FINAL STOP",
+]
+
+
+# ============================================================
+# 1. COMPLEX SPATIAL — 20 unique
+# ============================================================
+
+for i in range(20):
+    a = objects[i % len(objects)]
+    b = objects[(i + 3) % len(objects)]
+    c = objects[(i + 6) % len(objects)]
+
+    add(
+        "complex_spatial",
+        f"A {a} stands behind a {b}, while a {c} is positioned "
+        f"to their {'left' if i % 2 == 0 else 'right'}. "
+        f"The {b} partially overlaps the {a} from the camera viewpoint. "
+        f"Exactly {3 + i % 4} major objects are visible in the composition. "
+        f"Spatial variant {i + 1}."
+    )
+
+
+# ============================================================
+# 2. ANATOMY / INTERACTION — 20 unique
+# ============================================================
+
+actions = [
+    "raises the left hand while holding a cup in the right hand",
+    "crosses the right leg over the left while looking over the left shoulder",
+    "reaches forward with the right arm while keeping the left hand behind the back",
+    "holds a bottle with both hands directly in front of the chest",
+    "turns the torso right while the head remains facing left",
+]
+
+for i in range(20):
+    p1 = people[i % len(people)]
+    p2 = people[(i + 3) % len(people)]
+    action = actions[i % len(actions)]
+
+    add(
+        "complex_anatomy",
+        f"{p1.capitalize()} {action}. "
+        f"{p2.capitalize()} stands "
+        f"{'behind' if i % 2 else 'beside'} them "
+        f"and points toward the object with the "
+        f"{'left' if i % 3 else 'right'} hand. "
+        f"Both people's hands and feet remain visible. "
+        f"Anatomy variant {i + 1}."
+    )
+
+
+# ============================================================
+# 3. MATERIAL + LIGHT — 20 unique
+# ============================================================
+
+for i in range(20):
+    material1 = materials[i % len(materials)]
+    material2 = materials[(i + 3) % len(materials)]
+    lighting = lightings[i % len(lightings)]
+    obj = objects[(i + 4) % len(objects)]
+
+    add(
+        "complex_material_light",
+        f"A {obj} made from {material1} rests on a surface made from "
+        f"{material2}, illuminated by {lighting}. "
+        f"The scene clearly shows reflection, roughness, transparency, "
+        f"or subsurface behavior appropriate to both materials. "
+        f"A small colored object is reflected near the edge of the surface. "
+        f"Material-light variant {i + 1}."
+    )
+
+
+# ============================================================
+# 4. TEXT — 20 unique
+# ============================================================
+
+for i in range(20):
+    phrase = texts[i % len(texts)]
+    place = places[(i + 2) % len(places)]
+
+    add(
+        "complex_text",
+        f'Inside a {place}, the exact phrase "{phrase}" is printed clearly '
+        f'on a rectangular sign. A person stands partly in front of the sign '
+        f'without covering any letters. A reflective surface beside the sign '
+        f'shows a reversed reflection of part of the scene while the original '
+        f'text remains completely readable. Scene variant {i + 1}.'
+    )
+
+
+# ============================================================
+# 5. REFLECTION / OCCLUSION — 20 unique
+# ============================================================
+
+for i in range(20):
+    obj1 = objects[i % len(objects)]
+    obj2 = objects[(i + 5) % len(objects)]
+    person = people[(i + 2) % len(people)]
+
+    add(
+        "reflection_occlusion_ood",
+        f"{person.capitalize()} stands behind a transparent glass panel. "
+        f"A {obj1} in the foreground partially occludes the torso while "
+        f"the face and both hands remain visible. "
+        f"A mirror behind the person reflects a {obj2} located outside "
+        f"the direct camera frame. "
+        f"The glass also contains a faint reflection of an overhead light "
+        f"at position {i + 1}. "
+        f"Reflection variant {i + 1}."
+    )
+
+
+# ============================================================
+# 6. ARCHITECTURE / VEHICLES — 20 unique
+# ============================================================
+
+vehicles = [
+    "red compact car",
+    "white city bus",
+    "black motorcycle",
+    "blue bicycle",
+    "silver tram",
+]
+
+for i in range(20):
+    vehicle = vehicles[i % len(vehicles)]
+    place = places[(i + 1) % len(places)]
+
+    add(
+        "architecture_vehicle",
+        f"A {vehicle} passes beside a {place}. "
+        f"A staircase rises on the "
+        f"{'left' if i % 2 == 0 else 'right'} side "
+        f"and turns at an upper landing. "
+        f"Three architectural openings are visible at different depths, "
+        f"and a glass facade reflects a building across the street. "
+        f"Composition variant {i + 1}."
+    )
+
+
+# ============================================================
+# 7. COUNTING — 20 unique
+# ============================================================
+
+for i in range(20):
+    n1 = 3 + (i % 5)
+    n2 = 2 + ((i * 2) % 4)
+
+    obj1 = objects[i % len(objects)]
+    obj2 = objects[(i + 4) % len(objects)]
+
+    add(
+        "complex_counting",
+        f"Exactly {n1} {obj1}s form "
+        f"{'a curved row' if i % 2 else 'two staggered rows'}, "
+        f"while exactly {n2} {obj2}s occupy the background. "
+        f"None of the objects is completely hidden, and one small yellow marker "
+        f"is located beside object number {(i % n1) + 1}. "
+        f"Counting variant {i + 1}."
+    )
+
+
+# ============================================================
+# 8. LONG MULTI-CONSTRAINT — 20 unique
+# ============================================================
+
+for i in range(20):
+    p1 = people[i % len(people)]
+    p2 = people[(i + 4) % len(people)]
+
+    obj1 = objects[i % len(objects)]
+    obj2 = objects[(i + 2) % len(objects)]
+    obj3 = objects[(i + 7) % len(objects)]
+
+    phrase = texts[i % len(texts)]
+    lighting = lightings[i % len(lightings)]
+
+    add(
+        "long_composition",
+        f"{p1.capitalize()} stands at the left side of a glass table while "
+        f"{p2} sits to the right. "
+        f"The standing person holds a {obj1} in the left hand and points "
+        f'toward a sign reading exactly "{phrase}" with the right hand. '
+        f"A {obj2} and a {obj3} lie on the table in that order from left to right. "
+        f"The scene is illuminated by {lighting}. "
+        f"A large mirror behind both people reflects the seated person's back "
+        f"and one object outside the direct camera frame. "
+        f"Exactly {5 + i % 4} major foreground objects should remain clearly "
+        f"distinguishable. Long-scene variant {i + 1}."
+    )
+
+
+# ============================================================
+# FINAL CHECKS
+# ============================================================
+
+if len(data) != 160:
+    raise RuntimeError(
+        f"Expected exactly 160 unique prompts, got {len(data)}"
+    )
+
+random.shuffle(data)
+
+prompts = [
+    x["prompt"].strip().lower()
+    for x in data
+]
+
+if len(prompts) != len(set(prompts)):
+    raise RuntimeError(
+        "Duplicate OOD prompts detected."
+    )
+
+overlap = set(prompts) & train_prompts
+
+if overlap:
+    raise RuntimeError(
+        f"OOD/train overlap detected: {len(overlap)} prompts"
+    )
+
+
+# ============================================================
+# SAVE
+# ============================================================
+
+with open(
+    OUT,
+    "w",
+    encoding="utf-8"
+) as f:
+
+    json.dump(
+        data,
+        f,
+        ensure_ascii=False,
+        indent=2
+    )
+
+
+# ============================================================
+# REPORT
+# ============================================================
+
+print("=" * 80)
+print("STRICT OOD DATASET CREATED")
+print("=" * 80)
+
+print("Total prompts:      ", len(data))
+print("Unique prompts:     ", len(set(prompts)))
+print("Train/OOD overlap:  ", len(overlap))
+
+print("\nSaved:")
+print(OUT)
+
+categories = {}
+
+for item in data:
+    categories[item["category"]] = (
+        categories.get(item["category"], 0) + 1
+    )
+
+print("\nCategories:")
+
+for k, v in sorted(categories.items()):
+    print(f"{k:28s}: {v}")

research/raw_scripts/make_bridge_ood_prompts_v1.py

@@ -0,0 +1,272 @@
+import os
+import json
+import random
+
+ROOT = r"E:\Models SD XL\UNET\Minimax_H3"
+OUT = os.path.join(ROOT, "bridge_ood_prompts_160.json")
+
+random.seed(20260903)
+
+data = []
+
+
+def add(category, prompt):
+    data.append({
+        "category": category,
+        "prompt": prompt.strip()
+    })
+
+
+# ============================================================
+# 1. COMPLEX SPATIAL RELATIONS
+# ============================================================
+
+spatial = [
+    "A small red sphere rests inside a transparent cube while a blue cylinder stands behind the cube and slightly to its right.",
+    "A black chair faces a wooden table, with a white vase beneath the table and a silver lamp positioned behind the chair.",
+    "A green bottle stands between two ceramic cups, while a metal plate lies partially underneath the cup on the left.",
+    "A yellow cube is suspended above a glass sphere, with a red cone behind both objects and a mirror to their right.",
+    "Three boxes form a diagonal line from the lower left to the upper right, with a small glass bottle placed between the second and third boxes.",
+    "A polished metal sphere sits in front of a tall rectangular mirror while a wooden cube is visible only through the mirror reflection.",
+    "A transparent cylinder overlaps a red cube from the camera viewpoint, while a blue sphere remains fully visible behind both.",
+    "Two chairs face one another across a narrow table, with a floor lamp standing behind the chair on the right.",
+]
+
+for i in range(20):
+    add(
+        "complex_spatial",
+        spatial[i % len(spatial)]
+        + f" The composition contains exactly {3 + (i % 4)} clearly separated major objects."
+    )
+
+
+# ============================================================
+# 2. HUMAN INTERACTION / ANATOMY
+# ============================================================
+
+human = [
+    "A woman holds a glass bottle in her left hand while pointing toward a wall sign with her right hand; a seated man beside her looks toward the bottle.",
+    "A man kneels on his right knee while extending his left hand toward a woman standing directly in front of him.",
+    "Two people shake right hands while each keeps the other hand visible at their side.",
+    "A woman sits with her left leg crossed over her right leg while holding a cup with both hands.",
+    "A person reaches behind their head with the right arm while the left hand holds a small book against the chest.",
+    "A woman turns her torso toward the camera while her head faces left and both hands remain visible.",
+    "Three people stand side by side; the center person places one hand on each neighboring person's shoulder.",
+    "A seated man raises his left foot slightly above the floor while reaching forward with his right arm.",
+]
+
+for i in range(20):
+    add(
+        "complex_anatomy",
+        human[i % len(human)]
+    )
+
+
+# ============================================================
+# 3. MATERIALS + LIGHT
+# ============================================================
+
+materials = [
+    "A frosted glass sculpture illuminated from behind, with light scattering through its translucent surface and a faint soft-edged shadow on the wall.",
+    "A polished chrome kettle reflecting a red chair, a window, and a warm ceiling lamp in its curved surface.",
+    "A wet black stone on rough concrete under strong side lighting, showing tiny specular reflections in the water.",
+    "A translucent amber plastic sheet standing in front of a cool white light source, tinting the objects visible through it.",
+    "A brushed aluminum cylinder next to a glossy ceramic sphere under broad diffused daylight.",
+    "A clear glass bottle filled halfway with blue liquid, illuminated from the side so that light refracts through both glass and liquid.",
+    "A velvet-covered chair beside a polished metal table, lit by a narrow warm spotlight from above.",
+    "A sheet of wrinkled metallic foil reflecting several small colored objects positioned outside the center of the frame.",
+]
+
+for i in range(20):
+    add(
+        "complex_material_light",
+        materials[i % len(materials)]
+    )
+
+
+# ============================================================
+# 4. TEXT + SCENE COMPOSITION
+# ============================================================
+
+texts = [
+    ('NORTH GATE', "above a glass entrance"),
+    ('ROOM 204', "on a small brass plaque beside a door"),
+    ('OPEN UNTIL 9 PM', "on a white storefront sign"),
+    ('BLUE RIVER HOTEL', "across a large illuminated facade"),
+    ('PLATFORM 7', "on a hanging railway sign"),
+    ('NO ENTRY', "on a red rectangular metal sign"),
+    ('STUDIO B', "on a paper label attached to a black door"),
+    ('COFFEE AND BOOKS', "across a window awning"),
+]
+
+for i in range(20):
+    word, place = texts[i % len(texts)]
+
+    add(
+        "complex_text",
+        f'The exact text "{word}" appears clearly {place}. '
+        f'A person stands nearby without blocking any letters, '
+        f'and the entire phrase must remain readable.'
+    )
+
+
+# ============================================================
+# 5. REFLECTION / TRANSPARENCY / OCCLUSION
+# ============================================================
+
+reflection = [
+    "A woman faces away from a wall mirror, while the mirror clearly shows the front of her face and the glass she holds.",
+    "A transparent glass panel stands between the camera and two people, with reflections of overhead lights visible on the panel.",
+    "A chrome sphere reflects three colored cubes positioned around it, including one cube located behind the camera viewpoint.",
+    "A person stands partly behind a translucent curtain, with the silhouette and one hand visible through the fabric.",
+    "A bottle in the foreground partially blocks a person's torso but leaves both hands and the face clearly visible.",
+    "Two overlapping transparent sheets produce layered refractions of a black-and-white checkerboard behind them.",
+    "A large mirror reflects a chair that is outside the direct camera frame while a table remains visible both directly and in reflection.",
+    "A glass of water viewed through another glass object produces multiple distorted outlines but preserves the basic object shapes.",
+]
+
+for i in range(20):
+    add(
+        "reflection_occlusion_ood",
+        reflection[i % len(reflection)]
+    )
+
+
+# ============================================================
+# 6. VEHICLES / ARCHITECTURE
+# ============================================================
+
+architecture = [
+    "A red compact car is parked beneath a concrete overhang while a blue bicycle leans against the column immediately to its left.",
+    "A tram approaches an intersection between two modern glass buildings while pedestrians wait behind a metal barrier.",
+    "A narrow staircase rises between two brick walls and turns to the right at the upper landing.",
+    "A modern house has three illuminated windows on the upper floor and two dark windows below, with a tree partially covering the far-right window.",
+    "Two motorcycles are parked nose to tail beside a stone building entrance, with a street lamp directly behind them.",
+    "A pedestrian bridge crosses above a two-lane road while a white bus passes underneath from left to right.",
+    "A tall glass tower reflects a shorter brick building standing across the street.",
+    "A train platform contains three benches arranged sequentially, with a suitcase positioned beneath the middle bench.",
+]
+
+for i in range(20):
+    add(
+        "architecture_vehicle",
+        architecture[i % len(architecture)]
+    )
+
+
+# ============================================================
+# 7. MANY OBJECTS / COUNTING
+# ============================================================
+
+counting = [
+    "Five red cups stand in one row behind three blue bottles, with exactly two small plates in the foreground.",
+    "Seven candles form a circle around one glass sphere, and every candle is individually visible.",
+    "Four chairs surround a square table, with one chair on each side and two books placed on the tabletop.",
+    "Six identical boxes are stacked in three rows of two, with a small yellow ball resting on the upper-left box.",
+    "Three people each hold exactly one object: a bottle, a book, and a cup respectively.",
+    "Eight stones form two parallel rows of four with a narrow gap between the rows.",
+    "Five hanging lamps appear at different heights above a long table containing exactly three plates.",
+    "Two large spheres and four small cubes are distributed across the floor without any object completely hiding another.",
+]
+
+for i in range(20):
+    add(
+        "complex_counting",
+        counting[i % len(counting)]
+    )
+
+
+# ============================================================
+# 8. LONG MULTI-CONSTRAINT SCENES
+# ============================================================
+
+complex_scenes = [
+    """
+    A woman in a red coat stands behind a transparent glass table.
+    She holds a silver cup in her left hand while pointing with her
+    right hand toward a blue wall sign reading "EXIT". A seated man
+    to her right looks toward the sign. Warm light enters from the
+    left, and a large mirror behind them reflects the back of the man.
+    """,
+
+    """
+    Inside a small modern kitchen, three people stand around a wooden
+    counter. One person holds a clear bottle, another cuts a red apple,
+    and the third reaches toward a white ceramic plate. A stainless
+    steel refrigerator on the right reflects part of the room while
+    cool daylight enters through a window on the left.
+    """,
+
+    """
+    A black motorcycle stands in front of a glass storefront displaying
+    the clearly readable text "NIGHT MARKET". Behind the glass are two
+    mannequins and a red chair. Reflections of street lights appear on
+    the window without obscuring the lettering.
+    """,
+
+    """
+    A man sits at the left end of a long table while a woman stands at
+    the opposite end. Between them are exactly four objects in this
+    order from left to right: a blue bottle, a metal box, a glass sphere,
+    and a red book. A pendant lamp above the sphere casts a circular
+    pool of light on the table.
+    """,
+
+    """
+    A transparent cube contains a small green sphere. The cube rests on
+    a polished chrome surface that reflects both objects. Behind them,
+    a paper sign reads "LAB 12". A human hand enters from the right and
+    touches the top face of the cube with the index finger.
+    """,
+]
+
+for i in range(20):
+
+    base = complex_scenes[i % len(complex_scenes)]
+
+    add(
+        "long_composition",
+        " ".join(base.split())
+    )
+
+
+# ============================================================
+# FINAL
+# ============================================================
+
+assert len(data) == 160
+
+random.shuffle(data)
+
+with open(
+    OUT,
+    "w",
+    encoding="utf-8"
+) as f:
+
+    json.dump(
+        data,
+        f,
+        ensure_ascii=False,
+        indent=2
+    )
+
+
+print("=" * 80)
+print("OOD BRIDGE DATASET CREATED")
+print("=" * 80)
+print("Prompts:", len(data))
+print("Saved:")
+print(OUT)
+
+categories = {}
+
+for item in data:
+    c = item["category"]
+    categories[c] = categories.get(c, 0) + 1
+
+print()
+print("Categories:")
+
+for k, v in sorted(categories.items()):
+    print(f"{k:28s}: {v}")

research/raw_scripts/make_bridge_prompts.py

@@ -0,0 +1,255 @@
+import json
+import os
+import random
+
+ROOT = r"E:\Models SD XL\UNET\Minimax_H3"
+OUT = os.path.join(ROOT, "bridge_prompts_480.json")
+
+random.seed(1234)
+
+colors = [
+    "red", "blue", "green", "yellow", "black", "white",
+    "orange", "purple", "silver", "golden"
+]
+
+objects = [
+    "cube", "sphere", "bottle", "glass", "chair", "table",
+    "box", "lamp", "book", "vase", "cup", "plate",
+    "metal cylinder", "wooden block", "mirror"
+]
+
+materials = [
+    "transparent glass",
+    "frosted glass",
+    "polished chrome",
+    "brushed metal",
+    "rough stone",
+    "dark wood",
+    "glossy ceramic",
+    "matte plastic",
+    "translucent acrylic",
+    "wet concrete"
+]
+
+people = [
+    "a woman", "a man", "a young woman", "a young man",
+    "an elderly woman", "an elderly man"
+]
+
+relations = [
+    "to the left of",
+    "to the right of",
+    "behind",
+    "in front of",
+    "above",
+    "below",
+    "beside",
+    "partially behind"
+]
+
+lighting = [
+    "soft light from the left",
+    "hard light from the right",
+    "strong backlight",
+    "warm overhead light",
+    "cool side lighting",
+    "soft diffused daylight",
+    "a narrow beam of light from above",
+    "warm light reflected from below"
+]
+
+words = [
+    "OPEN", "CLOSED", "HOTEL", "CAFE", "EXIT",
+    "NORTH", "HELLO", "STOP", "STUDIO", "BLUE"
+]
+
+prompts = []
+
+
+def add(category, text):
+    prompts.append({
+        "category": category,
+        "prompt": text
+    })
+
+
+# ---------------------------------------------------------
+# Spatial composition
+# ---------------------------------------------------------
+
+for _ in range(90):
+
+    a, b = random.sample(objects, 2)
+    ca, cb = random.sample(colors, 2)
+    rel = random.choice(relations)
+
+    add(
+        "spatial",
+        f"a {ca} {a} positioned {rel} a {cb} {b}"
+    )
+
+
+# ---------------------------------------------------------
+# Multi-object composition / counting
+# ---------------------------------------------------------
+
+for _ in range(60):
+
+    n1 = random.randint(2, 5)
+    n2 = random.randint(1, 4)
+
+    a, b = random.sample(objects, 2)
+
+    add(
+        "counting",
+        f"{n1} {a}s arranged in the foreground with "
+        f"{n2} {b}s behind them"
+    )
+
+
+# ---------------------------------------------------------
+# Anatomy / body relations
+# ---------------------------------------------------------
+
+anatomy_actions = [
+    "raising their left hand above their head",
+    "holding a bottle in their right hand",
+    "crossing their right leg over their left leg",
+    "turning their head to the left",
+    "touching their face with their left hand",
+    "holding both hands in front of their chest",
+    "standing with one arm behind their back",
+    "sitting with both feet visible",
+    "reaching forward with their right arm",
+    "holding an object between both hands",
+]
+
+for _ in range(70):
+
+    person = random.choice(people)
+    action = random.choice(anatomy_actions)
+
+    add(
+        "anatomy",
+        f"{person} {action}, full body clearly visible"
+    )
+
+
+# ---------------------------------------------------------
+# Materials
+# ---------------------------------------------------------
+
+for _ in range(60):
+
+    obj = random.choice(objects)
+    mat = random.choice(materials)
+
+    add(
+        "material",
+        f"a {obj} made of {mat}, showing realistic surface "
+        f"properties and reflections"
+    )
+
+
+# ---------------------------------------------------------
+# Lighting
+# ---------------------------------------------------------
+
+for _ in range(60):
+
+    obj = random.choice(objects)
+    light = random.choice(lighting)
+
+    add(
+        "lighting",
+        f"a {obj} illuminated by {light}, with clearly visible "
+        f"light direction and shadow"
+    )
+
+
+# ---------------------------------------------------------
+# Text rendering
+# ---------------------------------------------------------
+
+for _ in range(50):
+
+    word = random.choice(words)
+
+    surfaces = [
+        "a white rectangular sign",
+        "a glass storefront",
+        "a black poster",
+        "a metal street sign",
+        "a paper label",
+    ]
+
+    surface = random.choice(surfaces)
+
+    add(
+        "text",
+        f'the word "{word}" printed clearly and correctly on {surface}'
+    )
+
+
+# ---------------------------------------------------------
+# Transparency / reflection / occlusion
+# ---------------------------------------------------------
+
+special = [
+    "a transparent glass bottle in front of a person's face",
+    "a woman reflected accurately in a wall mirror",
+    "a chrome sphere reflecting a room around it",
+    "a hand visible through a transparent glass panel",
+    "one person partially occluded by another person",
+    "a glass sphere resting on a reflective metal surface",
+    "a transparent object casting a faint shadow",
+    "a mirror showing the back of a person facing away",
+    "a translucent curtain illuminated from behind",
+    "a shiny metal object reflecting a nearby red object",
+]
+
+for _ in range(50):
+    add("reflection_occlusion", random.choice(special))
+
+
+# ---------------------------------------------------------
+# Shuffle and trim
+# ---------------------------------------------------------
+
+random.shuffle(prompts)
+
+prompts = prompts[:480]
+
+with open(
+    OUT,
+    "w",
+    encoding="utf-8"
+) as f:
+
+    json.dump(
+        prompts,
+        f,
+        ensure_ascii=False,
+        indent=2
+    )
+
+
+print("=" * 80)
+print("Bridge prompt dataset created")
+print("=" * 80)
+print("Count:", len(prompts))
+print("Saved:")
+print(OUT)
+
+categories = {}
+
+for x in prompts:
+    categories[x["category"]] = (
+        categories.get(x["category"], 0) + 1
+    )
+
+print()
+print("Categories:")
+
+for k, v in sorted(categories.items()):
+    print(f"{k:24s}: {v}")

research/raw_scripts/prepare_sensenova_h3_condition.py

@@ -0,0 +1,383 @@
+import os
+import sys
+import gc
+import argparse
+import torch
+import torch.nn.functional as F
+
+from safetensors.torch import load_file
+from accelerate import init_empty_weights, load_checkpoint_and_dispatch
+from transformers import AutoTokenizer
+
+
+# ============================================================
+# PATHS
+# ============================================================
+
+ROOT = r"E:\Models SD XL\UNET\Minimax_H3"
+
+SENSENOVA_SRC = os.path.join(
+    ROOT,
+    "SenseNova-U1",
+    "src",
+)
+
+CHECKPOINT = (
+    r"D:\ComfyUI_Python312\ComfyUI\models\unet"
+    r"\SenseNova-U1.5-8B-MoT-pruned-bf16.safetensors"
+)
+
+BRIDGE_FILE = os.path.join(
+    ROOT,
+    "hidden_bridge_probes_rank128",
+    "SN_L32_to_H3_L49_rank128.safetensors",
+)
+
+HF_REPO = "sensenova/SenseNova-U1.5-8B-MoT"
+
+LAYER = 32
+
+
+# ============================================================
+# COMMAND LINE
+# ============================================================
+
+parser = argparse.ArgumentParser()
+
+parser.add_argument(
+    "--prompt",
+    required=True,
+    type=str,
+)
+
+parser.add_argument(
+    "--output",
+    default=os.path.join(
+        ROOT,
+        "sensenova_projected_condition.pt",
+    ),
+)
+
+args = parser.parse_args()
+
+PROMPT = args.prompt
+OUT_FILE = args.output
+
+
+# ============================================================
+# LOCAL SENSENOVA
+# ============================================================
+
+sys.path.insert(0, SENSENOVA_SRC)
+
+from sensenova_u1.models.neo_unify.configuration_neo_chat import (
+    NEOChatConfig,
+)
+
+from sensenova_u1.models.neo_unify.modeling_neo_chat import (
+    NEOChatModel,
+)
+
+
+print("=" * 80)
+print("SenseNova L32 -> H3 L49 conditioning preparation")
+print("=" * 80)
+
+print("\nPrompt:")
+print(PROMPT)
+
+print("\nOutput:")
+print(OUT_FILE)
+
+
+# ============================================================
+# TOKENIZER
+# ============================================================
+
+print("\nLoading tokenizer...")
+
+tokenizer = AutoTokenizer.from_pretrained(
+    HF_REPO,
+    trust_remote_code=True,
+)
+
+encoded = tokenizer(
+    PROMPT,
+    return_tensors="pt",
+    add_special_tokens=False,
+)
+
+input_ids_cpu = encoded["input_ids"].cpu()
+
+print("Token count:", input_ids_cpu.shape[1])
+print("Token IDs:", input_ids_cpu.tolist()[0])
+
+
+# ============================================================
+# CONFIG
+# ============================================================
+
+print("\nLoading SenseNova config...")
+
+config = NEOChatConfig.from_pretrained(
+    HF_REPO,
+)
+
+config.llm_config._attn_implementation = "eager"
+
+
+# ============================================================
+# META MODEL
+# ============================================================
+
+print("Creating META architecture...")
+
+with init_empty_weights():
+    model = NEOChatModel(config)
+
+
+# ============================================================
+# CHECKPOINT
+# ============================================================
+
+print("Loading SenseNova BF16 checkpoint...")
+
+model = load_checkpoint_and_dispatch(
+    model,
+    checkpoint=CHECKPOINT,
+    device_map="auto",
+    max_memory={
+        0: "19GiB",
+        "cpu": "70GiB",
+    },
+    dtype=torch.bfloat16,
+    no_split_module_classes=[
+        "Qwen3DecoderLayer",
+        "Qwen3MoeDecoderLayer",
+    ],
+)
+
+model.eval()
+
+qwen = model.language_model.model
+
+print("Checkpoint loaded.")
+
+
+# ============================================================
+# INPUT DEVICE
+# ============================================================
+
+device_map = getattr(model, "hf_device_map", {})
+
+input_device = torch.device("cuda:0")
+
+for name, dev in device_map.items():
+
+    if name.endswith(
+        "language_model.model.embed_tokens"
+    ):
+
+        if str(dev) == "cpu":
+            input_device = torch.device("cpu")
+        else:
+            input_device = torch.device("cuda:0")
+
+        break
+
+print("Input device:", input_device)
+
+
+# ============================================================
+# CAPTURE L32
+# ============================================================
+
+captured = {}
+
+
+def hook_fn(module, inputs, output):
+
+    if isinstance(output, tuple):
+        x = output[0]
+    else:
+        x = output
+
+    captured["hidden"] = (
+        x.detach()
+        .float()
+        .cpu()
+    )
+
+
+handle = qwen.layers[LAYER].register_forward_hook(
+    hook_fn
+)
+
+
+input_ids = input_ids_cpu.to(
+    input_device
+)
+
+if hasattr(qwen, "current_index"):
+    qwen.current_index = -1
+
+
+print("\nRunning SenseNova...")
+
+with torch.inference_mode():
+
+    output = qwen(
+        input_ids=input_ids,
+        image_gen_indicators=None,
+        attention_mask=None,
+        use_cache=False,
+    )
+
+
+handle.remove()
+
+
+if "hidden" not in captured:
+    raise RuntimeError(
+        "SenseNova L32 was not captured."
+    )
+
+
+sn32 = captured["hidden"]
+
+print(
+    "SenseNova L32:",
+    tuple(sn32.shape)
+)
+
+if sn32.shape[-1] != 4096:
+    raise RuntimeError(
+        f"Expected 4096 dims, got {sn32.shape[-1]}"
+    )
+
+
+# ============================================================
+# LOAD FROZEN BRIDGE
+# ============================================================
+
+print("\nLoading frozen rank-128 bridge...")
+
+bridge = load_file(
+    BRIDGE_FILE,
+    device="cpu",
+)
+
+down = bridge["down.weight"].float()
+up = bridge["up.weight"].float()
+
+print("down:", tuple(down.shape))
+print("up:  ", tuple(up.shape))
+
+
+# ============================================================
+# SAME NORMALIZATION USED DURING TRAINING
+# ============================================================
+
+def rms_normalize(x):
+
+    rms = torch.sqrt(
+        x.pow(2).mean(
+            dim=-1,
+            keepdim=True
+        ) + 1e-6
+    )
+
+    return x / rms
+
+
+# ============================================================
+# PROJECT
+# ============================================================
+
+with torch.no_grad():
+
+    x = rms_normalize(
+        sn32.float()
+    )
+
+    rank_state = F.linear(
+        x,
+        down,
+    )
+
+    projected = F.linear(
+        rank_state,
+        up,
+    )
+
+
+print(
+    "Projected H3-space:",
+    tuple(projected.shape)
+)
+
+if projected.shape[-1] != 5120:
+    raise RuntimeError(
+        "Projected conditioning is not 5120-dimensional."
+    )
+
+if not torch.isfinite(projected).all():
+    raise RuntimeError(
+        "Non-finite projected conditioning."
+    )
+
+
+# ============================================================
+# SAVE
+# ============================================================
+
+torch.save(
+    {
+        "prompt": PROMPT,
+
+        "input_ids":
+            input_ids_cpu,
+
+        "source_layer":
+            32,
+
+        "target_layer":
+            49,
+
+        "bridge_rank":
+            128,
+
+        "projected":
+            projected.to(
+                torch.float16
+            ),
+
+        # Keep this too for diagnostics.
+        "sensenova_l32":
+            sn32.to(
+                torch.float16
+            ),
+    },
+    OUT_FILE,
+)
+
+
+print()
+print("=" * 80)
+print("SUCCESS")
+print("=" * 80)
+
+print("Prompt tokens:", input_ids_cpu.shape[1])
+print(
+    "Projected shape:",
+    tuple(projected.shape)
+)
+
+print("\nSaved:")
+print(OUT_FILE)
+
+del output
+del model
+
+gc.collect()
+torch.cuda.empty_cache()

research/raw_scripts/test_sensenova_mot_gen_bridge.py

@@ -0,0 +1,698 @@
+import os
+import sys
+import gc
+import shutil
+import csv
+import torch
+
+from safetensors.torch import load_file
+
+
+# ============================================================
+# PATHS
+# ============================================================
+
+ROOT = r"E:\Models SD XL\UNET\Minimax_H3"
+
+SENSENOVA_SRC = os.path.join(
+    ROOT, "SenseNova-U1", "src"
+)
+
+COMPAT_FILE = os.path.join(
+    SENSENOVA_SRC,
+    "sensenova_u1",
+    "models",
+    "neo_unify",
+    "transformers_compat.py"
+)
+
+MODEL_FILE = (
+    r"D:\ComfyUI_Python312\ComfyUI\models\unet"
+    r"\SenseNova-U1.5-8B-MoT-pruned-bf16.safetensors"
+)
+
+H3_FILE = os.path.join(
+    ROOT, "h3_hidden_states_fast.pt"
+)
+
+BRIDGE_FILE = os.path.join(
+    ROOT,
+    "sensenova_to_h3_embedding_bridge_rank512.safetensors"
+)
+
+OUT_STATES = os.path.join(
+    ROOT,
+    "sensenova_mot_gen_hidden_states.pt"
+)
+
+OUT_TXT = os.path.join(
+    ROOT,
+    "sensenova_mot_gen_bridge_comparison.txt"
+)
+
+OUT_CSV = os.path.join(
+    ROOT,
+    "sensenova_mot_gen_bridge_comparison.csv"
+)
+
+HF_REPO = "sensenova/SenseNova-U1.5-8B-MoT"
+
+
+# ============================================================
+# TEST SET
+# ============================================================
+
+PROMPTS = [
+    "a red cube on a blue sphere",
+    "a woman holding a transparent glass bottle",
+    "three people standing behind a wooden table",
+]
+
+SN_LAYERS = [8, 16, 24, 32, 41]
+H3_LAYERS = [8, 16, 24, 32, 40, 49]
+
+# We will try BOTH synthetic generation-position layouts.
+MODES = [
+    "causal",
+    "block",
+]
+
+
+# ============================================================
+# PATCH SENSENOVA / TRANSFORMERS 4.57 COMPAT
+# ============================================================
+
+print("=" * 80)
+print("SenseNova MoT-GEN -> MiniMax H3 diagnostic")
+print("=" * 80)
+
+print("\nChecking compatibility patch...")
+
+with open(COMPAT_FILE, "r", encoding="utf-8") as f:
+    text = f.read()
+
+OLD = (
+    "    from transformers.utils.generic import "
+    "check_model_inputs as model_input_compat"
+)
+
+NEW = (
+    "    from transformers.utils.generic import check_model_inputs\n"
+    "    def model_input_compat(func):\n"
+    "        return check_model_inputs()(func)"
+)
+
+if OLD in text:
+
+    backup = COMPAT_FILE + ".bak"
+
+    if not os.path.exists(backup):
+        shutil.copy2(COMPAT_FILE, backup)
+
+    text = text.replace(OLD, NEW)
+
+    with open(COMPAT_FILE, "w", encoding="utf-8") as f:
+        f.write(text)
+
+    print("Patch applied.")
+
+else:
+    print("Patch already present / not needed.")
+
+
+# ============================================================
+# IMPORTS
+# ============================================================
+
+sys.path.insert(0, SENSENOVA_SRC)
+
+from transformers import AutoTokenizer
+from accelerate import init_empty_weights, load_checkpoint_and_dispatch
+
+from sensenova_u1.models.neo_unify.configuration_neo_chat import (
+    NEOChatConfig,
+)
+
+from sensenova_u1.models.neo_unify.modeling_neo_chat import (
+    NEOChatModel,
+)
+
+
+# ============================================================
+# LOAD TOKENIZER + CONFIG
+# ============================================================
+
+print("\nLoading tokenizer...")
+
+tokenizer = AutoTokenizer.from_pretrained(
+    HF_REPO,
+    trust_remote_code=True,
+)
+
+print("Loading SenseNova config...")
+
+config = NEOChatConfig.from_pretrained(
+    HF_REPO
+)
+
+config.llm_config._attn_implementation = "eager"
+
+print("Hidden size:", config.llm_config.hidden_size)
+print("Layers:", config.llm_config.num_hidden_layers)
+
+
+# ============================================================
+# META MODEL
+# ============================================================
+
+print("\nCreating META architecture...")
+
+with init_empty_weights():
+    model = NEOChatModel(config)
+
+print("META architecture created.")
+
+
+# ============================================================
+# LOAD CHECKPOINT WITH OFFLOAD
+# ============================================================
+
+print("\nLoading BF16 checkpoint with Accelerate...")
+
+model = load_checkpoint_and_dispatch(
+    model,
+    checkpoint=MODEL_FILE,
+    device_map="auto",
+    max_memory={
+        0: "19GiB",
+        "cpu": "70GiB",
+    },
+    dtype=torch.bfloat16,
+    no_split_module_classes=[
+        "Qwen3DecoderLayer",
+        "Qwen3MoeDecoderLayer",
+    ],
+)
+
+model.eval()
+
+print("Checkpoint loaded.")
+
+
+# ============================================================
+# LANGUAGE MODEL
+# ============================================================
+
+language_model = model.language_model
+qwen = language_model.model
+layers = qwen.layers
+
+device_map = getattr(model, "hf_device_map", {})
+
+INPUT_DEVICE = torch.device("cuda:0")
+
+for name, dev in device_map.items():
+
+    if name.endswith("language_model.model.embed_tokens"):
+
+        if str(dev) == "cpu":
+            INPUT_DEVICE = torch.device("cpu")
+        else:
+            INPUT_DEVICE = torch.device("cuda:0")
+
+        break
+
+print("Input device:", INPUT_DEVICE)
+
+
+# ============================================================
+# HOOKS
+# ============================================================
+
+captured = {}
+hooks = {}
+
+
+def make_hook(idx):
+
+    def hook(module, inputs, output):
+
+        if isinstance(output, tuple):
+            x = output[0]
+        else:
+            x = output
+
+        if torch.is_tensor(x):
+            captured[idx] = (
+                x.detach()
+                .float()
+                .cpu()
+            )
+
+    return hook
+
+
+print("\nRegistering hooks...")
+
+for idx in SN_LAYERS:
+
+    hooks[idx] = layers[idx].register_forward_hook(
+        make_hook(idx)
+    )
+
+    print(" layer", idx)
+
+
+# ============================================================
+# INDEX LAYOUT
+# ============================================================
+
+def make_indexes(seq_len, device, mode):
+
+    p = torch.arange(
+        seq_len,
+        dtype=torch.long,
+        device=device
+    )
+
+    z = torch.zeros(
+        seq_len,
+        dtype=torch.long,
+        device=device
+    )
+
+    if mode == "causal":
+
+        # Every token gets a new temporal position.
+        # Block causal mask therefore behaves causally.
+        t = p
+        h = z
+        w = z
+
+    elif mode == "block":
+
+        # All tokens belong to one generation block.
+        # Spatial W positions distinguish tokens.
+        #
+        # create_block_causal_mask uses t, so tokens
+        # inside this block can see one another.
+        t = z
+        h = z
+        w = p
+
+    else:
+        raise ValueError(mode)
+
+    return torch.stack(
+        [t, h, w],
+        dim=0
+    )
+
+
+# ============================================================
+# EXTRACT MoT-GEN STATES
+# ============================================================
+
+all_results = {}
+
+for mode in MODES:
+
+    print()
+    print("#" * 80)
+    print("MODE:", mode.upper())
+    print("#" * 80)
+
+    mode_results = []
+
+    for i, prompt in enumerate(PROMPTS):
+
+        print()
+        print(f"[{i+1}/{len(PROMPTS)}] {prompt}")
+
+        encoded = tokenizer(
+            prompt,
+            return_tensors="pt",
+            add_special_tokens=False,
+        )
+
+        input_ids = encoded["input_ids"].to(
+            INPUT_DEVICE
+        )
+
+        seq_len = input_ids.shape[1]
+
+        print(" Tokens:", input_ids.cpu().tolist()[0])
+
+        # First obtain the normal token embeddings.
+        with torch.inference_mode():
+
+            inputs_embeds = qwen.embed_tokens(
+                input_ids
+            )
+
+        device = inputs_embeds.device
+
+        # Every token is deliberately marked as a
+        # generation token -> forward_gen / *_mot_gen.
+        image_gen_indicators = torch.ones(
+            (1, seq_len),
+            dtype=torch.bool,
+            device=device,
+        )
+
+        indexes = make_indexes(
+            seq_len,
+            device,
+            mode,
+        )
+
+        captured.clear()
+
+        qwen.current_index = -1
+
+        with torch.inference_mode():
+
+            outputs = qwen(
+                input_ids=None,
+                inputs_embeds=inputs_embeds,
+                image_gen_indicators=image_gen_indicators,
+                indexes=indexes,
+                attention_mask=None,
+                use_cache=False,
+            )
+
+        item = {
+            "prompt": prompt,
+            "input_ids": input_ids.detach().cpu(),
+            "layers": {},
+        }
+
+        for idx in SN_LAYERS:
+
+            if idx not in captured:
+
+                print(
+                    f"  L{idx:02d}: NOT CAPTURED"
+                )
+                continue
+
+            x = captured[idx]
+
+            item["layers"][idx] = x
+
+            print(
+                f"  L{idx:02d}: "
+                f"{tuple(x.shape)} "
+                f"mean={x.mean().item():.5f} "
+                f"std={x.std().item():.5f} "
+                f"finite={torch.isfinite(x).all().item()}"
+            )
+
+        mode_results.append(item)
+
+        del outputs
+        del inputs_embeds
+        del input_ids
+
+        gc.collect()
+        torch.cuda.empty_cache()
+
+    all_results[mode] = mode_results
+
+
+# ============================================================
+# SAVE RAW MoT STATES
+# ============================================================
+
+torch.save(
+    {
+        "modes": MODES,
+        "prompts": PROMPTS,
+        "target_layers": SN_LAYERS,
+        "results": all_results,
+        "note": (
+            "Diagnostic forced-MoT run. Text token embeddings "
+            "were deliberately marked as image-generation tokens."
+        ),
+    },
+    OUT_STATES,
+)
+
+print("\nSaved raw MoT states:")
+print(OUT_STATES)
+
+
+# ============================================================
+# MODEL NO LONGER NEEDED
+# ============================================================
+
+for h in hooks.values():
+    h.remove()
+
+
+# ============================================================
+# LOAD H3 STATES + EXISTING EMBEDDING BRIDGE
+# ============================================================
+
+print("\nLoading H3 hidden states...")
+
+h3_data = torch.load(
+    H3_FILE,
+    map_location="cpu",
+    weights_only=False,
+)
+
+print("Loading existing embedding bridge...")
+
+bridge = load_file(
+    BRIDGE_FILE,
+    device="cpu",
+)
+
+A = bridge["proj_in.weight"].float()
+B = bridge["proj_out.weight"].float()
+
+
+def project_sn(x):
+
+    z = torch.nn.functional.linear(
+        x.float(),
+        A
+    )
+
+    return torch.nn.functional.linear(
+        z,
+        B
+    )
+
+
+def mean_cos(a, b):
+
+    return (
+        torch.nn.functional.cosine_similarity(
+            a.float(),
+            b.float(),
+            dim=-1
+        )
+        .mean()
+        .item()
+    )
+
+
+# ============================================================
+# COMPARE MoT STATES TO H3
+# ============================================================
+
+rows = []
+
+print()
+print("=" * 80)
+print("MoT-GEN -> H3 COMPARISON")
+print("=" * 80)
+
+for mode in MODES:
+
+    sn_results = all_results[mode]
+
+    # Check token identity.
+    for sn_item, h3_item in zip(
+        sn_results,
+        h3_data["results"],
+    ):
+
+        if not torch.equal(
+            sn_item["input_ids"],
+            h3_item["input_ids"],
+        ):
+            raise RuntimeError(
+                "Token mismatch"
+            )
+
+    for sn_layer in SN_LAYERS:
+
+        for h3_layer in H3_LAYERS:
+
+            P = []
+            H = []
+            prompt_scores = []
+
+            for sn_item, h3_item in zip(
+                sn_results,
+                h3_data["results"],
+            ):
+
+                sx = sn_item["layers"][sn_layer]
+                hy = h3_item["layers"][h3_layer]
+
+                projected = project_sn(sx)
+
+                prompt_scores.append(
+                    mean_cos(projected, hy)
+                )
+
+                P.append(
+                    projected.reshape(-1, 5120)
+                )
+
+                H.append(
+                    hy.reshape(-1, 5120)
+                )
+
+            P = torch.cat(P, dim=0)
+            H = torch.cat(H, dim=0)
+
+            cos = mean_cos(P, H)
+
+            pnorm = P.norm(dim=-1).mean().item()
+            hnorm = H.norm(dim=-1).mean().item()
+
+            ratio = pnorm / hnorm
+
+            rows.append(
+                {
+                    "mode": mode,
+                    "sn_layer": sn_layer,
+                    "h3_layer": h3_layer,
+                    "cosine": cos,
+                    "norm_ratio": ratio,
+                    "prompt_1": prompt_scores[0],
+                    "prompt_2": prompt_scores[1],
+                    "prompt_3": prompt_scores[2],
+                }
+            )
+
+
+rows.sort(
+    key=lambda r: r["cosine"],
+    reverse=True,
+)
+
+
+# ============================================================
+# SAVE REPORT
+# ============================================================
+
+with open(
+    OUT_CSV,
+    "w",
+    newline="",
+    encoding="utf-8",
+) as f:
+
+    writer = csv.DictWriter(
+        f,
+        fieldnames=rows[0].keys(),
+    )
+
+    writer.writeheader()
+    writer.writerows(rows)
+
+
+with open(
+    OUT_TXT,
+    "w",
+    encoding="utf-8",
+) as f:
+
+    f.write(
+        "SenseNova forced MoT-GEN -> H3 comparison\n"
+    )
+
+    f.write("=" * 80 + "\n\n")
+
+    f.write(
+        "IMPORTANT: This is a diagnostic experiment. "
+        "Text embeddings were forced through the "
+        "SenseNova generation branch.\n\n"
+    )
+
+    for rank, r in enumerate(rows, 1):
+
+        f.write(
+            f"{rank:02d}. "
+            f"[{r['mode']}] "
+            f"SN L{r['sn_layer']:02d} "
+            f"-> H3 L{r['h3_layer']:02d} | "
+            f"cos={r['cosine']:.6f} | "
+            f"norm_ratio={r['norm_ratio']:.4f}\n"
+        )
+
+        f.write(
+            "    prompts: "
+            f"{r['prompt_1']:.6f}, "
+            f"{r['prompt_2']:.6f}, "
+            f"{r['prompt_3']:.6f}\n"
+        )
+
+
+# ============================================================
+# PRINT TOP RESULTS
+# ============================================================
+
+print()
+print("=" * 80)
+print("TOP 20 MoT-GEN RESULTS")
+print("=" * 80)
+
+for rank, r in enumerate(rows[:20], 1):
+
+    print(
+        f"{rank:02d}. "
+        f"[{r['mode']:6s}] "
+        f"SN L{r['sn_layer']:02d} "
+        f"-> H3 L{r['h3_layer']:02d} | "
+        f"cos={r['cosine']:.6f} | "
+        f"norm_ratio={r['norm_ratio']:.4f}"
+    )
+
+
+print()
+print("=" * 80)
+print("BEST BY MODE")
+print("=" * 80)
+
+for mode in MODES:
+
+    candidates = [
+        r for r in rows
+        if r["mode"] == mode
+    ]
+
+    best = max(
+        candidates,
+        key=lambda r: r["cosine"],
+    )
+
+    print(
+        f"{mode:6s}: "
+        f"SN L{best['sn_layer']:02d} "
+        f"-> H3 L{best['h3_layer']:02d} | "
+        f"cos={best['cosine']:.6f}"
+    )
+
+
+print()
+print("DONE")
+print("States:", OUT_STATES)
+print("Report:", OUT_TXT)
+print("CSV:", OUT_CSV)

research/raw_scripts/train_distilled_adapter_screening.py

@@ -0,0 +1,1311 @@
+import os
+import math
+import random
+import csv
+import torch
+import torch.nn as nn
+import torch.nn.functional as F
+
+from safetensors.torch import load_file, save_file
+
+
+ROOT = r"E:\Models SD XL\UNET\Minimax_H3"
+
+H3_FILE = os.path.join(
+    ROOT,
+    "h3_bridge_hidden_480.pt"
+)
+
+SN_FILE = os.path.join(
+    ROOT,
+    "sensenova_bridge_hidden_480.pt"
+)
+
+BRIDGE_FILE = (
+    r"D:\ComfyUI_Krea2\ComfyUI\models\bridge"
+    r"\SN_L32_to_H3_L49_rank128.safetensors"
+)
+
+OUT_DIR = os.path.join(
+    ROOT,
+    "distilled_adapter_screening"
+)
+
+os.makedirs(
+    OUT_DIR,
+    exist_ok=True
+)
+
+
+# ============================================================
+# SETTINGS
+# ============================================================
+
+SEED = 20260903
+
+DEVICE = (
+    "cuda"
+    if torch.cuda.is_available()
+    else "cpu"
+)
+
+RANK = 128
+
+EPOCHS = 12
+
+BATCH_TOKENS = 4096
+
+LR = 2e-3
+
+WEIGHT_DECAY = 1e-4
+
+VAL_PROMPTS = 80
+
+SOURCE_LAYERS = [
+    8,
+    16,
+    24,
+    32,
+    40,
+    49,
+]
+
+MULTI_LAYERS = [
+    32,
+    40,
+    49,
+]
+
+
+# ============================================================
+# REPRODUCIBILITY
+# ============================================================
+
+random.seed(
+    SEED
+)
+
+torch.manual_seed(
+    SEED
+)
+
+if torch.cuda.is_available():
+    torch.cuda.manual_seed_all(
+        SEED
+    )
+
+
+# ============================================================
+# MODEL
+# ============================================================
+
+class ResidualAdapter(
+    nn.Module
+):
+    def __init__(
+        self,
+        input_dim,
+        rank,
+        output_dim,
+    ):
+        super().__init__()
+
+        self.down = nn.Linear(
+            input_dim,
+            rank,
+            bias=False,
+        )
+
+        self.act = nn.SiLU()
+
+        self.up = nn.Linear(
+            rank,
+            output_dim,
+            bias=False,
+        )
+
+        nn.init.normal_(
+            self.down.weight,
+            std=0.02,
+        )
+
+        nn.init.zeros_(
+            self.up.weight
+        )
+
+    def forward(
+        self,
+        x,
+    ):
+        x = self.down(
+            x
+        )
+
+        x = self.act(
+            x
+        )
+
+        x = self.up(
+            x
+        )
+
+        return x
+
+
+# ============================================================
+# HELPERS
+# ============================================================
+
+def rms_normalize(
+    x,
+):
+    rms = torch.sqrt(
+        x.float()
+        .pow(2)
+        .mean(
+            dim=-1,
+            keepdim=True,
+        )
+        + 1e-6
+    )
+
+    return (
+        x.float()
+        / rms
+    )
+
+
+def magnitude_match(
+    source,
+    target,
+):
+    source = source.float()
+    target = target.float()
+
+    source_rms = torch.sqrt(
+        source.pow(2)
+        .mean(
+            dim=-1,
+            keepdim=True,
+        )
+        + 1e-8
+    )
+
+    target_rms = torch.sqrt(
+        target.pow(2)
+        .mean(
+            dim=-1,
+            keepdim=True,
+        )
+        + 1e-8
+    )
+
+    return (
+        source
+        * (
+            target_rms
+            / source_rms
+        )
+    )
+
+
+def cosine_mean(
+    a,
+    b,
+):
+    return (
+        F.cosine_similarity(
+            a.float(),
+            b.float(),
+            dim=-1,
+        )
+        .mean()
+        .item()
+    )
+
+
+def save_adapter(
+    model,
+    path,
+    metadata,
+):
+    tensors = {
+        "down.weight":
+            model.down.weight
+            .detach()
+            .cpu()
+            .to(torch.float16),
+
+        "up.weight":
+            model.up.weight
+            .detach()
+            .cpu()
+            .to(torch.float16),
+    }
+
+    save_file(
+        tensors,
+        path,
+        metadata={
+            k: str(v)
+            for k, v
+            in metadata.items()
+        },
+    )
+
+
+# ============================================================
+# LOAD DATA
+# ============================================================
+
+print(
+    "=" * 100
+)
+
+print(
+    "DISTILLED ADAPTER SCREENING"
+)
+
+print(
+    "=" * 100
+)
+
+print(
+    "Device:",
+    DEVICE
+)
+
+print(
+    "\nLoading H3 data..."
+)
+
+h3_data = torch.load(
+    H3_FILE,
+    map_location="cpu",
+    weights_only=False,
+)
+
+print(
+    "H3 prompts:",
+    len(
+        h3_data["results"]
+    )
+)
+
+print(
+    "\nLoading SenseNova data..."
+)
+
+sn_data = torch.load(
+    SN_FILE,
+    map_location="cpu",
+    weights_only=False,
+)
+
+print(
+    "SenseNova prompts:",
+    len(
+        sn_data["results"]
+    )
+)
+
+if (
+    len(h3_data["results"])
+    != len(sn_data["results"])
+):
+    raise RuntimeError(
+        "Dataset length mismatch."
+    )
+
+
+# ============================================================
+# LOAD FULL TEACHER BRIDGE
+# ============================================================
+
+print(
+    "\nLoading teacher bridge..."
+)
+
+teacher_bridge = load_file(
+    BRIDGE_FILE,
+    device="cpu",
+)
+
+teacher_down = (
+    teacher_bridge[
+        "down.weight"
+    ]
+    .float()
+)
+
+teacher_up = (
+    teacher_bridge[
+        "up.weight"
+    ]
+    .float()
+)
+
+print(
+    "Teacher down:",
+    tuple(
+        teacher_down.shape
+    )
+)
+
+print(
+    "Teacher up:",
+    tuple(
+        teacher_up.shape
+    )
+)
+
+
+# ============================================================
+# ALIGNMENT CHECK
+# ============================================================
+
+print(
+    "\nChecking alignment..."
+)
+
+for i, (
+    h3_item,
+    sn_item,
+) in enumerate(
+    zip(
+        h3_data[
+            "results"
+        ],
+        sn_data[
+            "results"
+        ],
+    )
+):
+    if (
+        h3_item[
+            "prompt"
+        ]
+        != sn_item[
+            "prompt"
+        ]
+    ):
+        raise RuntimeError(
+            f"Prompt mismatch "
+            f"at {i}"
+        )
+
+    if not torch.equal(
+        h3_item[
+            "input_ids"
+        ],
+        sn_item[
+            "input_ids"
+        ],
+    ):
+        raise RuntimeError(
+            f"Token mismatch "
+            f"at {i}"
+        )
+
+print(
+    "Alignment: PERFECT"
+)
+
+
+# ============================================================
+# BUILD TEACHER TARGETS
+# ============================================================
+
+print(
+    "\nBuilding teacher corrections..."
+)
+
+samples = []
+
+with torch.no_grad():
+    for i, (
+        h3_item,
+        sn_item,
+    ) in enumerate(
+        zip(
+            h3_data[
+                "results"
+            ],
+            sn_data[
+                "results"
+            ],
+        )
+    ):
+        h49 = (
+            h3_item[
+                "layers"
+            ][49]
+            .float()
+        )
+
+        sn32 = (
+            sn_item[
+                "layers"
+            ][32]
+            .float()
+        )
+
+        x = rms_normalize(
+            sn32
+        )
+
+        rank_state = F.linear(
+            x,
+            teacher_down,
+        )
+
+        projected = F.linear(
+            rank_state,
+            teacher_up,
+        )
+
+        projected = (
+            magnitude_match(
+                projected,
+                h49,
+            )
+        )
+
+        delta = (
+            projected
+            - h49
+        )
+
+        samples.append({
+            "prompt":
+                h3_item[
+                    "prompt"
+                ],
+
+            "category":
+                h3_item[
+                    "category"
+                ],
+
+            "layers":
+                h3_item[
+                    "layers"
+                ],
+
+            "target_delta":
+                delta.to(
+                    torch.float16
+                ),
+        })
+
+        if (
+            (i + 1)
+            % 50
+            == 0
+        ):
+            print(
+                f"{i + 1:3d}/"
+                f"{len(h3_data['results'])}"
+            )
+
+
+# ============================================================
+# TRAIN / VAL SPLIT
+# ============================================================
+
+indices = list(
+    range(
+        len(samples)
+    )
+)
+
+random.shuffle(
+    indices
+)
+
+val_indices = set(
+    indices[
+        :VAL_PROMPTS
+    ]
+)
+
+train_indices = [
+    i
+    for i
+    in indices
+    if i
+    not in val_indices
+]
+
+val_indices = sorted(
+    val_indices
+)
+
+print()
+print(
+    "Train prompts:",
+    len(
+        train_indices
+    )
+)
+
+print(
+    "Validation prompts:",
+    len(
+        val_indices
+    )
+)
+
+
+# ============================================================
+# FLATTEN PROMPTS INTO TOKEN BATCHES
+# ============================================================
+
+def build_token_dataset(
+    source_spec,
+    prompt_indices,
+):
+    xs = []
+    ys = []
+
+    for idx in prompt_indices:
+        item = samples[
+            idx
+        ]
+
+        if isinstance(
+            source_spec,
+            int,
+        ):
+            x = (
+                item[
+                    "layers"
+                ][source_spec]
+                .float()
+            )
+
+        else:
+            parts = [
+                item[
+                    "layers"
+                ][layer]
+                .float()
+                for layer
+                in source_spec
+            ]
+
+            x = torch.cat(
+                parts,
+                dim=-1,
+            )
+
+        y = (
+            item[
+                "target_delta"
+            ]
+            .float()
+        )
+
+        x = (
+            x.squeeze(0)
+        )
+
+        y = (
+            y.squeeze(0)
+        )
+
+        xs.append(
+            x
+        )
+
+        ys.append(
+            y
+        )
+
+    return (
+        torch.cat(
+            xs,
+            dim=0,
+        ),
+        torch.cat(
+            ys,
+            dim=0,
+        ),
+    )
+
+
+# ============================================================
+# LOSS
+# ============================================================
+
+def loss_fn(
+    pred,
+    target,
+):
+    mse = F.mse_loss(
+        pred,
+        target,
+    )
+
+    cos = (
+        1.0
+        - F.cosine_similarity(
+            pred,
+            target,
+            dim=-1,
+        )
+        .mean()
+    )
+
+    return (
+        mse
+        + 0.10
+        * cos
+    )
+
+
+# ============================================================
+# TRAIN ONE CANDIDATE
+# ============================================================
+
+def train_candidate(
+    name,
+    source_spec,
+):
+    print()
+    print(
+        "=" * 100
+    )
+
+    print(
+        "CANDIDATE:",
+        name
+    )
+
+    print(
+        "=" * 100
+    )
+
+    train_x, train_y = (
+        build_token_dataset(
+            source_spec,
+            train_indices,
+        )
+    )
+
+    val_x, val_y = (
+        build_token_dataset(
+            source_spec,
+            val_indices,
+        )
+    )
+
+    input_dim = (
+        train_x.shape[
+            -1
+        ]
+    )
+
+    print(
+        "Input dim:",
+        input_dim
+    )
+
+    print(
+        "Train tokens:",
+        train_x.shape[
+            0
+        ]
+    )
+
+    print(
+        "Val tokens:",
+        val_x.shape[
+            0
+        ]
+    )
+
+    model = (
+        ResidualAdapter(
+            input_dim,
+            RANK,
+            5120,
+        )
+        .to(
+            DEVICE
+        )
+    )
+
+    optimizer = (
+        torch.optim.AdamW(
+            model.parameters(),
+            lr=LR,
+            weight_decay=
+                WEIGHT_DECAY,
+        )
+    )
+
+    best_val_cos = -1.0
+
+    best_path = os.path.join(
+        OUT_DIR,
+        f"{name}_rank{RANK}.safetensors"
+    )
+
+    train_x = train_x.pin_memory()
+    train_y = train_y.pin_memory()
+
+    val_x = val_x.pin_memory()
+    val_y = val_y.pin_memory()
+
+    token_indices = torch.arange(
+        train_x.shape[
+            0
+        ]
+    )
+
+    for epoch in range(
+        1,
+        EPOCHS + 1,
+    ):
+        permutation = (
+            token_indices[
+                torch.randperm(
+                    token_indices
+                    .numel()
+                )
+            ]
+        )
+
+        model.train()
+
+        epoch_loss = 0.0
+
+        batches = 0
+
+        for start in range(
+            0,
+            permutation.numel(),
+            BATCH_TOKENS,
+        ):
+            batch_idx = (
+                permutation[
+                    start:
+                    start
+                    + BATCH_TOKENS
+                ]
+            )
+
+            x = (
+                train_x[
+                    batch_idx
+                ]
+                .to(
+                    DEVICE,
+                    non_blocking=True,
+                )
+            )
+
+            y = (
+                train_y[
+                    batch_idx
+                ]
+                .to(
+                    DEVICE,
+                    non_blocking=True,
+                )
+            )
+
+            optimizer.zero_grad(
+                set_to_none=True
+            )
+
+            pred = model(
+                x
+            )
+
+            loss = loss_fn(
+                pred,
+                y
+            )
+
+            loss.backward()
+
+            torch.nn.utils.clip_grad_norm_(
+                model.parameters(),
+                1.0,
+            )
+
+            optimizer.step()
+
+            epoch_loss += (
+                loss.item()
+            )
+
+            batches += 1
+
+        epoch_loss /= max(
+            batches,
+            1,
+        )
+
+        model.eval()
+
+        with torch.no_grad():
+            pred_chunks = []
+
+            for start in range(
+                0,
+                val_x.shape[0],
+                BATCH_TOKENS,
+            ):
+                x = (
+                    val_x[
+                        start:
+                        start
+                        + BATCH_TOKENS
+                    ]
+                    .to(
+                        DEVICE,
+                        non_blocking=True,
+                    )
+                )
+
+                pred = model(
+                    x
+                )
+
+                pred_chunks.append(
+                    pred.cpu()
+                )
+
+            pred_val = torch.cat(
+                pred_chunks,
+                dim=0,
+            )
+
+        val_cos = cosine_mean(
+            pred_val,
+            val_y,
+        )
+
+        val_mse = F.mse_loss(
+            pred_val.float(),
+            val_y.float(),
+        ).item()
+
+        print(
+            f"Epoch "
+            f"{epoch:02d}/"
+            f"{EPOCHS} | "
+            f"loss="
+            f"{epoch_loss:.6f} | "
+            f"val_cos="
+            f"{val_cos:.6f} | "
+            f"val_mse="
+            f"{val_mse:.8f}"
+        )
+
+        if (
+            val_cos
+            > best_val_cos
+        ):
+            best_val_cos = (
+                val_cos
+            )
+
+            save_adapter(
+                model,
+                best_path,
+                {
+                    "candidate":
+                        name,
+
+                    "source_spec":
+                        source_spec,
+
+                    "rank":
+                        RANK,
+
+                    "target":
+                        "teacher_delta",
+
+                    "teacher":
+                        "SN_L32_to_H3_L49_rank128",
+
+                    "magnitude_match":
+                        "per_token",
+                },
+            )
+
+    # ========================================================
+    # FINAL PROMPT-LEVEL EVAL
+    # ========================================================
+
+    best_weights = load_file(
+        best_path,
+        device="cpu",
+    )
+
+    model = (
+        ResidualAdapter(
+            input_dim,
+            RANK,
+            5120,
+        )
+    )
+
+    with torch.no_grad():
+        model.down.weight.copy_(
+            best_weights[
+                "down.weight"
+            ].float()
+        )
+
+        model.up.weight.copy_(
+            best_weights[
+                "up.weight"
+            ].float()
+        )
+
+    model = model.to(
+        DEVICE
+    )
+
+    model.eval()
+
+    prompt_scores = []
+
+    with torch.no_grad():
+        for idx in val_indices:
+            item = samples[
+                idx
+            ]
+
+            if isinstance(
+                source_spec,
+                int,
+            ):
+                x = (
+                    item[
+                        "layers"
+                    ][source_spec]
+                    .float()
+                )
+
+            else:
+                x = torch.cat(
+                    [
+                        item[
+                            "layers"
+                        ][layer]
+                        .float()
+                        for layer
+                        in source_spec
+                    ],
+                    dim=-1,
+                )
+
+            y = (
+                item[
+                    "target_delta"
+                ]
+                .float()
+            )
+
+            pred = model(
+                x.to(
+                    DEVICE
+                )
+            ).cpu()
+
+            score = cosine_mean(
+                pred,
+                y,
+            )
+
+            prompt_scores.append(
+                score
+            )
+
+    prompt_mean = (
+        sum(
+            prompt_scores
+        )
+        / len(
+            prompt_scores
+        )
+    )
+
+    prompt_min = min(
+        prompt_scores
+    )
+
+    prompt_max = max(
+        prompt_scores
+    )
+
+    print()
+    print(
+        "BEST VAL TOKEN COS:",
+        f"{best_val_cos:.6f}"
+    )
+
+    print(
+        "VAL PROMPT COS:",
+        f"{prompt_mean:.6f}"
+    )
+
+    print(
+        "VAL PROMPT MIN:",
+        f"{prompt_min:.6f}"
+    )
+
+    print(
+        "VAL PROMPT MAX:",
+        f"{prompt_max:.6f}"
+    )
+
+    return {
+        "candidate":
+            name,
+
+        "source":
+            str(
+                source_spec
+            ),
+
+        "input_dim":
+            input_dim,
+
+        "rank":
+            RANK,
+
+        "best_val_token_cos":
+            best_val_cos,
+
+        "val_prompt_cos":
+            prompt_mean,
+
+        "val_prompt_min":
+            prompt_min,
+
+        "val_prompt_max":
+            prompt_max,
+
+        "model_file":
+            best_path,
+    }
+
+
+# ============================================================
+# SCREENING
+# ============================================================
+
+results = []
+
+for layer in SOURCE_LAYERS:
+    result = train_candidate(
+        f"H3_L{layer}_to_teacher_delta",
+        layer,
+    )
+
+    results.append(
+        result
+    )
+
+
+# ============================================================
+# MULTI-LAYER
+# ============================================================
+
+multi_name = (
+    "H3_L32_L40_L49_to_teacher_delta"
+)
+
+multi_result = train_candidate(
+    multi_name,
+    MULTI_LAYERS,
+)
+
+results.append(
+    multi_result
+)
+
+
+# ============================================================
+# SORT
+# ============================================================
+
+results = sorted(
+    results,
+    key=lambda x:
+        x[
+            "val_prompt_cos"
+        ],
+    reverse=True,
+)
+
+
+# ============================================================
+# SAVE CSV
+# ============================================================
+
+csv_path = os.path.join(
+    OUT_DIR,
+    "distilled_adapter_screening.csv"
+)
+
+with open(
+    csv_path,
+    "w",
+    newline="",
+    encoding="utf-8",
+) as f:
+    writer = csv.DictWriter(
+        f,
+        fieldnames=[
+            "candidate",
+            "source",
+            "input_dim",
+            "rank",
+            "best_val_token_cos",
+            "val_prompt_cos",
+            "val_prompt_min",
+            "val_prompt_max",
+            "model_file",
+        ],
+    )
+
+    writer.writeheader()
+
+    writer.writerows(
+        results
+    )
+
+
+# ============================================================
+# SAVE REPORT
+# ============================================================
+
+report_path = os.path.join(
+    OUT_DIR,
+    "distilled_adapter_screening.txt"
+)
+
+with open(
+    report_path,
+    "w",
+    encoding="utf-8",
+) as f:
+    f.write(
+        "DISTILLED ADAPTER SCREENING\n"
+    )
+
+    f.write(
+        "=" * 100
+        + "\n\n"
+    )
+
+    for i, r in enumerate(
+        results,
+        1,
+    ):
+        f.write(
+            f"{i:02d}. "
+            f"{r['candidate']}\n"
+        )
+
+        f.write(
+            f"    source: "
+            f"{r['source']}\n"
+        )
+
+        f.write(
+            f"    input_dim: "
+            f"{r['input_dim']}\n"
+        )
+
+        f.write(
+            f"    rank: "
+            f"{r['rank']}\n"
+        )
+
+        f.write(
+            f"    token_cos: "
+            f"{r['best_val_token_cos']:.6f}\n"
+        )
+
+        f.write(
+            f"    prompt_cos: "
+            f"{r['val_prompt_cos']:.6f}\n"
+        )
+
+        f.write(
+            f"    prompt_min: "
+            f"{r['val_prompt_min']:.6f}\n"
+        )
+
+        f.write(
+            f"    prompt_max: "
+            f"{r['val_prompt_max']:.6f}\n"
+        )
+
+        f.write(
+            f"    model: "
+            f"{r['model_file']}\n\n"
+        )
+
+
+# ============================================================
+# FINAL OUTPUT
+# ============================================================
+
+print()
+print(
+    "=" * 100
+)
+
+print(
+    "FINAL RANKING"
+)
+
+print(
+    "=" * 100
+)
+
+for i, r in enumerate(
+    results,
+    1,
+):
+    print(
+        f"{i:02d}. "
+        f"{r['candidate']:38s} "
+        f"prompt_cos="
+        f"{r['val_prompt_cos']:.6f} "
+        f"token_cos="
+        f"{r['best_val_token_cos']:.6f}"
+    )
+
+print()
+print(
+    "CSV:"
+)
+
+print(
+    csv_path
+)
+
+print()
+print(
+    "Report:"
+)
+
+print(
+    report_path
+)
+
+print()
+print(
+    "DONE"
+)

research/raw_scripts/train_distilled_student_v2.py

@@ -0,0 +1,1802 @@
+import os
+import csv
+import random
+import math
+
+import torch
+import torch.nn as nn
+import torch.nn.functional as F
+
+from safetensors.torch import load_file, save_file
+
+
+# ============================================================
+# PATHS
+# ============================================================
+
+ROOT = r"E:\Models SD XL\UNET\Minimax_H3"
+
+H3_FILE = os.path.join(
+    ROOT,
+    "h3_bridge_hidden_480.pt"
+)
+
+SN_FILE = os.path.join(
+    ROOT,
+    "sensenova_bridge_hidden_480.pt"
+)
+
+TEACHER_BRIDGE = (
+    r"D:\ComfyUI_Krea2\ComfyUI\models\bridge"
+    r"\SN_L32_to_H3_L49_rank128.safetensors"
+)
+
+OUT_DIR = os.path.join(
+    ROOT,
+    "distilled_student_v2"
+)
+
+os.makedirs(
+    OUT_DIR,
+    exist_ok=True
+)
+
+
+# ============================================================
+# SETTINGS
+# ============================================================
+
+SEED = 20260903
+
+DEVICE = (
+    "cuda"
+    if torch.cuda.is_available()
+    else "cpu"
+)
+
+SOURCE_LAYERS = [
+    40,
+    49,
+]
+
+HIDDEN_DIM = 512
+
+EPOCHS = 30
+
+LR = 1e-3
+
+WEIGHT_DECAY = 1e-4
+
+BATCH_TOKENS = 1024
+
+VAL_PROMPTS = 80
+
+ALPHAS = [
+    0.05,
+    0.10,
+    0.20,
+    0.30,
+]
+
+
+# ============================================================
+# REPRODUCIBILITY
+# ============================================================
+
+random.seed(
+    SEED
+)
+
+torch.manual_seed(
+    SEED
+)
+
+if torch.cuda.is_available():
+    torch.cuda.manual_seed_all(
+        SEED
+    )
+
+
+# ============================================================
+# STUDENT
+# ============================================================
+
+class SemanticStudent(
+    nn.Module
+):
+    def __init__(
+        self,
+        input_dim=5120,
+        hidden_dim=512,
+        output_dim=5120,
+    ):
+        super().__init__()
+
+        self.fc1 = nn.Linear(
+            input_dim,
+            hidden_dim,
+            bias=True,
+        )
+
+        self.fc2 = nn.Linear(
+            hidden_dim,
+            hidden_dim,
+            bias=True,
+        )
+
+        self.fc3 = nn.Linear(
+            hidden_dim,
+            output_dim,
+            bias=True,
+        )
+
+        self.act = nn.SiLU()
+
+        nn.init.xavier_uniform_(
+            self.fc1.weight
+        )
+
+        nn.init.zeros_(
+            self.fc1.bias
+        )
+
+        nn.init.xavier_uniform_(
+            self.fc2.weight
+        )
+
+        nn.init.zeros_(
+            self.fc2.bias
+        )
+
+        # Start final projection near zero.
+        nn.init.normal_(
+            self.fc3.weight,
+            mean=0.0,
+            std=0.002,
+        )
+
+        nn.init.zeros_(
+            self.fc3.bias
+        )
+
+
+    def forward(
+        self,
+        x,
+    ):
+        x = self.fc1(
+            x
+        )
+
+        x = self.act(
+            x
+        )
+
+        x = self.fc2(
+            x
+        )
+
+        x = self.act(
+            x
+        )
+
+        x = self.fc3(
+            x
+        )
+
+        return x
+
+
+# ============================================================
+# NORMALIZATION
+# ============================================================
+
+def rms_normalize(
+    x,
+):
+    x = x.float()
+
+    rms = torch.sqrt(
+        x.pow(2)
+        .mean(
+            dim=-1,
+            keepdim=True,
+        )
+        + 1e-6
+    )
+
+    return (
+        x / rms
+    )
+
+
+def magnitude_match(
+    source,
+    target,
+):
+    source = source.float()
+    target = target.float()
+
+    source_rms = torch.sqrt(
+        source.pow(2)
+        .mean(
+            dim=-1,
+            keepdim=True,
+        )
+        + 1e-8
+    )
+
+    target_rms = torch.sqrt(
+        target.pow(2)
+        .mean(
+            dim=-1,
+            keepdim=True,
+        )
+        + 1e-8
+    )
+
+    return (
+        source
+        * (
+            target_rms
+            / source_rms
+        )
+    )
+
+
+# ============================================================
+# METRICS
+# ============================================================
+
+def cosine(
+    a,
+    b,
+):
+    return F.cosine_similarity(
+        a.float(),
+        b.float(),
+        dim=-1,
+    )
+
+
+def cosine_mean(
+    a,
+    b,
+):
+    return (
+        cosine(
+            a,
+            b,
+        )
+        .mean()
+        .item()
+    )
+
+
+# ============================================================
+# LOSS
+# ============================================================
+
+def student_loss(
+    pred,
+    target,
+):
+    # Direction is most important because runtime
+    # magnitude matching rescales student output anyway.
+
+    pred_n = F.normalize(
+        pred.float(),
+        dim=-1,
+    )
+
+    target_n = F.normalize(
+        target.float(),
+        dim=-1,
+    )
+
+    cosine_loss = (
+        1.0
+        - (
+            pred_n
+            * target_n
+        )
+        .sum(
+            dim=-1
+        )
+        .mean()
+    )
+
+    normalized_mse = (
+        F.mse_loss(
+            pred_n,
+            target_n,
+        )
+    )
+
+    # Small raw-scale stabilizer.
+    target_rms = torch.sqrt(
+        target.float()
+        .pow(2)
+        .mean(
+            dim=-1,
+            keepdim=True,
+        )
+        + 1e-8
+    )
+
+    pred_scaled = (
+        pred.float()
+        / (
+            torch.sqrt(
+                pred.float()
+                .pow(2)
+                .mean(
+                    dim=-1,
+                    keepdim=True,
+                )
+                + 1e-8
+            )
+        )
+        * target_rms
+    )
+
+    scale_mse = F.mse_loss(
+        pred_scaled,
+        target.float(),
+    )
+
+    loss = (
+        cosine_loss
+        + 0.50 * normalized_mse
+        + 0.02 * scale_mse
+    )
+
+    return (
+        loss,
+        cosine_loss.item(),
+        normalized_mse.item(),
+        scale_mse.item(),
+    )
+
+
+# ============================================================
+# SAVE
+# ============================================================
+
+def save_student(
+    model,
+    path,
+    source_layer,
+    best_cos,
+):
+    tensors = {
+        "fc1.weight":
+            model.fc1.weight
+            .detach()
+            .cpu()
+            .to(torch.float16),
+
+        "fc1.bias":
+            model.fc1.bias
+            .detach()
+            .cpu()
+            .to(torch.float16),
+
+        "fc2.weight":
+            model.fc2.weight
+            .detach()
+            .cpu()
+            .to(torch.float16),
+
+        "fc2.bias":
+            model.fc2.bias
+            .detach()
+            .cpu()
+            .to(torch.float16),
+
+        "fc3.weight":
+            model.fc3.weight
+            .detach()
+            .cpu()
+            .to(torch.float16),
+
+        "fc3.bias":
+            model.fc3.bias
+            .detach()
+            .cpu()
+            .to(torch.float16),
+    }
+
+    metadata = {
+        "architecture":
+            "SenseNova_H3_Distilled_Student_V2",
+
+        "source_model":
+            "MiniMax_H3_Qwen3VL",
+
+        "source_layer":
+            str(source_layer),
+
+        "teacher_source":
+            "SenseNova_L32",
+
+        "teacher_bridge":
+            "SN_L32_to_H3_L49_rank128",
+
+        "input_dim":
+            "5120",
+
+        "hidden_dim":
+            str(HIDDEN_DIM),
+
+        "output_dim":
+            "5120",
+
+        "activation":
+            "SiLU",
+
+        "best_val_teacher_cos":
+            f"{best_cos:.8f}",
+    }
+
+    save_file(
+        tensors,
+        path,
+        metadata=metadata,
+    )
+
+
+# ============================================================
+# LOAD DATA
+# ============================================================
+
+print(
+    "=" * 100
+)
+
+print(
+    "DISTILLED STUDENT V2"
+)
+
+print(
+    "H3 -> projected SenseNova teacher"
+)
+
+print(
+    "=" * 100
+)
+
+print(
+    "Device:",
+    DEVICE
+)
+
+print(
+    "\nLoading H3 hidden states..."
+)
+
+h3_data = torch.load(
+    H3_FILE,
+    map_location="cpu",
+    weights_only=False,
+)
+
+print(
+    "H3 prompts:",
+    len(
+        h3_data[
+            "results"
+        ]
+    )
+)
+
+print(
+    "\nLoading SenseNova hidden states..."
+)
+
+sn_data = torch.load(
+    SN_FILE,
+    map_location="cpu",
+    weights_only=False,
+)
+
+print(
+    "SenseNova prompts:",
+    len(
+        sn_data[
+            "results"
+        ]
+    )
+)
+
+
+# ============================================================
+# ALIGNMENT
+# ============================================================
+
+if (
+    len(
+        h3_data[
+            "results"
+        ]
+    )
+    !=
+    len(
+        sn_data[
+            "results"
+        ]
+    )
+):
+    raise RuntimeError(
+        "Dataset length mismatch."
+    )
+
+
+print(
+    "\nChecking prompt/token alignment..."
+)
+
+for i, (
+    h3_item,
+    sn_item,
+) in enumerate(
+    zip(
+        h3_data[
+            "results"
+        ],
+        sn_data[
+            "results"
+        ],
+    )
+):
+    if (
+        h3_item[
+            "prompt"
+        ]
+        !=
+        sn_item[
+            "prompt"
+        ]
+    ):
+        raise RuntimeError(
+            f"Prompt mismatch "
+            f"at {i}"
+        )
+
+    if not torch.equal(
+        h3_item[
+            "input_ids"
+        ],
+        sn_item[
+            "input_ids"
+        ],
+    ):
+        raise RuntimeError(
+            f"Token mismatch "
+            f"at {i}"
+        )
+
+print(
+    "Alignment: PERFECT"
+)
+
+
+# ============================================================
+# TEACHER BRIDGE
+# ============================================================
+
+print(
+    "\nLoading teacher bridge..."
+)
+
+bridge = load_file(
+    TEACHER_BRIDGE,
+    device="cpu",
+)
+
+teacher_down = (
+    bridge[
+        "down.weight"
+    ]
+    .float()
+)
+
+teacher_up = (
+    bridge[
+        "up.weight"
+    ]
+    .float()
+)
+
+print(
+    "down:",
+    tuple(
+        teacher_down.shape
+    )
+)
+
+print(
+    "up:",
+    tuple(
+        teacher_up.shape
+    )
+)
+
+
+# ============================================================
+# BUILD TEACHER REPRESENTATIONS
+# ============================================================
+
+print(
+    "\nBuilding projected SenseNova teacher targets..."
+)
+
+samples = []
+
+with torch.no_grad():
+
+    for i, (
+        h3_item,
+        sn_item,
+    ) in enumerate(
+        zip(
+            h3_data[
+                "results"
+            ],
+            sn_data[
+                "results"
+            ],
+        )
+    ):
+
+        sn32 = (
+            sn_item[
+                "layers"
+            ][32]
+            .float()
+        )
+
+        h49 = (
+            h3_item[
+                "layers"
+            ][49]
+            .float()
+        )
+
+        sn_norm = (
+            rms_normalize(
+                sn32
+            )
+        )
+
+        low = F.linear(
+            sn_norm,
+            teacher_down,
+        )
+
+        teacher_projected = (
+            F.linear(
+                low,
+                teacher_up,
+            )
+        )
+
+        samples.append({
+            "prompt":
+                h3_item[
+                    "prompt"
+                ],
+
+            "category":
+                h3_item[
+                    "category"
+                ],
+
+            "layers":
+                h3_item[
+                    "layers"
+                ],
+
+            "h49":
+                h49.to(
+                    torch.float16
+                ),
+
+            "teacher":
+                teacher_projected.to(
+                    torch.float16
+                ),
+        })
+
+        if (
+            (i + 1)
+            % 50
+            == 0
+        ):
+            print(
+                f"{i + 1:3d}/"
+                f"{len(h3_data['results'])}"
+            )
+
+
+# ============================================================
+# SPLIT BY PROMPT
+# ============================================================
+
+indices = list(
+    range(
+        len(samples)
+    )
+)
+
+random.shuffle(
+    indices
+)
+
+val_indices = sorted(
+    indices[
+        :VAL_PROMPTS
+    ]
+)
+
+val_set = set(
+    val_indices
+)
+
+train_indices = [
+    i
+    for i in indices
+    if i not in val_set
+]
+
+print()
+print(
+    "Train prompts:",
+    len(
+        train_indices
+    )
+)
+
+print(
+    "Validation prompts:",
+    len(
+        val_indices
+    )
+)
+
+
+# ============================================================
+# BUILD TOKEN MATRICES
+# ============================================================
+
+def build_token_data(
+    source_layer,
+    prompt_indices,
+):
+
+    xs = []
+    ys = []
+
+    for idx in prompt_indices:
+
+        item = samples[
+            idx
+        ]
+
+        x = (
+            item[
+                "layers"
+            ][source_layer]
+            .float()
+            .squeeze(0)
+        )
+
+        # Normalize H3 input per token.
+        x = rms_normalize(
+            x
+        )
+
+        y = (
+            item[
+                "teacher"
+            ]
+            .float()
+            .squeeze(0)
+        )
+
+        xs.append(
+            x
+        )
+
+        ys.append(
+            y
+        )
+
+    return (
+        torch.cat(
+            xs,
+            dim=0,
+        ),
+        torch.cat(
+            ys,
+            dim=0,
+        ),
+    )
+
+
+# ============================================================
+# EVALUATE PROMPT LEVEL
+# ============================================================
+
+def evaluate_prompt_level(
+    model,
+    source_layer,
+):
+
+    teacher_scores = []
+
+    correction_scores = []
+
+    blend_scores = {
+        alpha: []
+        for alpha
+        in ALPHAS
+    }
+
+    category_teacher = {}
+
+    model.eval()
+
+    with torch.no_grad():
+
+        for idx in val_indices:
+
+            item = samples[
+                idx
+            ]
+
+            category = item[
+                "category"
+            ]
+
+            x = (
+                item[
+                    "layers"
+                ][source_layer]
+                .float()
+            )
+
+            x = rms_normalize(
+                x
+            )
+
+            teacher = (
+                item[
+                    "teacher"
+                ]
+                .float()
+            )
+
+            h49 = (
+                item[
+                    "h49"
+                ]
+                .float()
+            )
+
+            pred = (
+                model(
+                    x.to(
+                        DEVICE
+                    )
+                )
+                .cpu()
+            )
+
+
+            # =================================================
+            # DIRECT TEACHER REPRESENTATION
+            # =================================================
+
+            teacher_cos = (
+                cosine_mean(
+                    pred,
+                    teacher,
+                )
+            )
+
+            teacher_scores.append(
+                teacher_cos
+            )
+
+            category_teacher.setdefault(
+                category,
+                []
+            ).append(
+                teacher_cos
+            )
+
+
+            # =================================================
+            # REAL FULL-BRIDGE CORRECTION
+            # =================================================
+
+            teacher_scaled = (
+                magnitude_match(
+                    teacher,
+                    h49,
+                )
+            )
+
+            pred_scaled = (
+                magnitude_match(
+                    pred,
+                    h49,
+                )
+            )
+
+            teacher_delta = (
+                teacher_scaled
+                - h49
+            )
+
+            pred_delta = (
+                pred_scaled
+                - h49
+            )
+
+            correction_cos = (
+                cosine_mean(
+                    pred_delta,
+                    teacher_delta,
+                )
+            )
+
+            correction_scores.append(
+                correction_cos
+            )
+
+
+            # =================================================
+            # FULL vs DISTILLED BLENDED CONDITIONING
+            # =================================================
+
+            for alpha in ALPHAS:
+
+                full_blend = (
+                    h49
+                    + alpha
+                    * teacher_delta
+                )
+
+                distilled_blend = (
+                    h49
+                    + alpha
+                    * pred_delta
+                )
+
+                score = cosine_mean(
+                    distilled_blend,
+                    full_blend,
+                )
+
+                blend_scores[
+                    alpha
+                ].append(
+                    score
+                )
+
+
+    result = {
+        "teacher_cos_mean":
+            sum(
+                teacher_scores
+            )
+            / len(
+                teacher_scores
+            ),
+
+        "teacher_cos_min":
+            min(
+                teacher_scores
+            ),
+
+        "teacher_cos_max":
+            max(
+                teacher_scores
+            ),
+
+        "correction_cos_mean":
+            sum(
+                correction_scores
+            )
+            / len(
+                correction_scores
+            ),
+
+        "correction_cos_min":
+            min(
+                correction_scores
+            ),
+
+        "correction_cos_max":
+            max(
+                correction_scores
+            ),
+    }
+
+    for alpha in ALPHAS:
+
+        scores = blend_scores[
+            alpha
+        ]
+
+        result[
+            f"blend_cos_alpha_{alpha}"
+        ] = (
+            sum(
+                scores
+            )
+            / len(
+                scores
+            )
+        )
+
+
+    category_results = {}
+
+    for category, values in (
+        category_teacher.items()
+    ):
+
+        category_results[
+            category
+        ] = (
+            sum(values)
+            / len(values)
+        )
+
+    return (
+        result,
+        category_results,
+    )
+
+
+# ============================================================
+# TRAIN ONE STUDENT
+# ============================================================
+
+def train_student(
+    source_layer,
+):
+
+    name = (
+        f"H3_L{source_layer}"
+        f"_to_projected_SN32_v2"
+    )
+
+    print()
+    print(
+        "=" * 100
+    )
+
+    print(
+        "STUDENT:",
+        name
+    )
+
+    print(
+        "=" * 100
+    )
+
+    train_x, train_y = (
+        build_token_data(
+            source_layer,
+            train_indices,
+        )
+    )
+
+    val_x, val_y = (
+        build_token_data(
+            source_layer,
+            val_indices,
+        )
+    )
+
+    print(
+        "Train tokens:",
+        train_x.shape[0]
+    )
+
+    print(
+        "Validation tokens:",
+        val_x.shape[0]
+    )
+
+    print(
+        "Input:",
+        train_x.shape[-1]
+    )
+
+    print(
+        "Hidden:",
+        HIDDEN_DIM
+    )
+
+    print(
+        "Output:",
+        train_y.shape[-1]
+    )
+
+
+    # ========================================================
+    # MODEL
+    # ========================================================
+
+    model = SemanticStudent(
+        input_dim=5120,
+        hidden_dim=HIDDEN_DIM,
+        output_dim=5120,
+    ).to(
+        DEVICE
+    )
+
+    optimizer = (
+        torch.optim.AdamW(
+            model.parameters(),
+            lr=LR,
+            weight_decay=
+                WEIGHT_DECAY,
+        )
+    )
+
+    scheduler = (
+        torch.optim.lr_scheduler.CosineAnnealingLR(
+            optimizer,
+            T_max=EPOCHS,
+            eta_min=LR * 0.05,
+        )
+    )
+
+
+    # Pin CPU memory.
+    if DEVICE == "cuda":
+
+        train_x = (
+            train_x
+            .contiguous()
+            .pin_memory()
+        )
+
+        train_y = (
+            train_y
+            .contiguous()
+            .pin_memory()
+        )
+
+        val_x = (
+            val_x
+            .contiguous()
+            .pin_memory()
+        )
+
+        val_y = (
+            val_y
+            .contiguous()
+            .pin_memory()
+        )
+
+
+    best_cos = -1.0
+
+    best_path = os.path.join(
+        OUT_DIR,
+        f"{name}.safetensors"
+    )
+
+    token_indices = (
+        torch.arange(
+            train_x.shape[0]
+        )
+    )
+
+
+    # ========================================================
+    # EPOCHS
+    # ========================================================
+
+    for epoch in range(
+        1,
+        EPOCHS + 1,
+    ):
+
+        permutation = (
+            token_indices[
+                torch.randperm(
+                    len(
+                        token_indices
+                    )
+                )
+            ]
+        )
+
+        model.train()
+
+        epoch_loss = 0.0
+
+        batches = 0
+
+        for start in range(
+            0,
+            len(
+                permutation
+            ),
+            BATCH_TOKENS,
+        ):
+
+            ids = (
+                permutation[
+                    start:
+                    start
+                    + BATCH_TOKENS
+                ]
+            )
+
+            x = (
+                train_x[
+                    ids
+                ]
+                .to(
+                    DEVICE,
+                    non_blocking=True,
+                )
+            )
+
+            y = (
+                train_y[
+                    ids
+                ]
+                .to(
+                    DEVICE,
+                    non_blocking=True,
+                )
+            )
+
+            optimizer.zero_grad(
+                set_to_none=True
+            )
+
+            pred = model(
+                x
+            )
+
+            loss, _, _, _ = (
+                student_loss(
+                    pred,
+                    y,
+                )
+            )
+
+            loss.backward()
+
+            torch.nn.utils.clip_grad_norm_(
+                model.parameters(),
+                1.0,
+            )
+
+            optimizer.step()
+
+            epoch_loss += (
+                loss.item()
+            )
+
+            batches += 1
+
+
+        scheduler.step()
+
+
+        # ====================================================
+        # VALIDATION TOKEN COSINE
+        # ====================================================
+
+        model.eval()
+
+        val_cos_values = []
+
+        val_loss_total = 0.0
+
+        val_batches = 0
+
+        with torch.no_grad():
+
+            for start in range(
+                0,
+                val_x.shape[0],
+                BATCH_TOKENS,
+            ):
+
+                x = (
+                    val_x[
+                        start:
+                        start
+                        + BATCH_TOKENS
+                    ]
+                    .to(
+                        DEVICE,
+                        non_blocking=True,
+                    )
+                )
+
+                y = (
+                    val_y[
+                        start:
+                        start
+                        + BATCH_TOKENS
+                    ]
+                    .to(
+                        DEVICE,
+                        non_blocking=True,
+                    )
+                )
+
+                pred = model(
+                    x
+                )
+
+                loss, _, _, _ = (
+                    student_loss(
+                        pred,
+                        y,
+                    )
+                )
+
+                val_loss_total += (
+                    loss.item()
+                )
+
+                val_batches += 1
+
+                cos = cosine(
+                    pred,
+                    y,
+                )
+
+                val_cos_values.append(
+                    cos.detach().cpu()
+                )
+
+
+        val_cos_tensor = torch.cat(
+            val_cos_values
+        )
+
+        val_cos = (
+            val_cos_tensor
+            .mean()
+            .item()
+        )
+
+        val_loss = (
+            val_loss_total
+            / max(
+                val_batches,
+                1,
+            )
+        )
+
+        lr_now = (
+            optimizer
+            .param_groups[0]["lr"]
+        )
+
+        print(
+            f"Epoch "
+            f"{epoch:02d}/"
+            f"{EPOCHS} | "
+            f"train_loss="
+            f"{epoch_loss / max(batches,1):.6f} | "
+            f"val_loss="
+            f"{val_loss:.6f} | "
+            f"val_teacher_cos="
+            f"{val_cos:.6f} | "
+            f"lr="
+            f"{lr_now:.7f}"
+        )
+
+
+        # ====================================================
+        # SAVE BEST
+        # ====================================================
+
+        if (
+            val_cos
+            > best_cos
+        ):
+            best_cos = (
+                val_cos
+            )
+
+            save_student(
+                model,
+                best_path,
+                source_layer,
+                best_cos,
+            )
+
+
+    # ========================================================
+    # LOAD BEST
+    # ========================================================
+
+    weights = load_file(
+        best_path,
+        device="cpu",
+    )
+
+    best_model = SemanticStudent(
+        5120,
+        HIDDEN_DIM,
+        5120,
+    )
+
+    with torch.no_grad():
+
+        best_model.fc1.weight.copy_(
+            weights[
+                "fc1.weight"
+            ].float()
+        )
+
+        best_model.fc1.bias.copy_(
+            weights[
+                "fc1.bias"
+            ].float()
+        )
+
+        best_model.fc2.weight.copy_(
+            weights[
+                "fc2.weight"
+            ].float()
+        )
+
+        best_model.fc2.bias.copy_(
+            weights[
+                "fc2.bias"
+            ].float()
+        )
+
+        best_model.fc3.weight.copy_(
+            weights[
+                "fc3.weight"
+            ].float()
+        )
+
+        best_model.fc3.bias.copy_(
+            weights[
+                "fc3.bias"
+            ].float()
+        )
+
+
+    best_model = best_model.to(
+        DEVICE
+    )
+
+    best_model.eval()
+
+
+    # ========================================================
+    # REAL BRIDGE EVALUATION
+    # ========================================================
+
+    metrics, category_metrics = (
+        evaluate_prompt_level(
+            best_model,
+            source_layer,
+        )
+    )
+
+
+    print()
+    print(
+        "BEST RESULT"
+    )
+
+    print(
+        "-" * 100
+    )
+
+    print(
+        "Teacher representation cosine:",
+        f"{metrics['teacher_cos_mean']:.6f}"
+    )
+
+    print(
+        "Teacher cosine min:",
+        f"{metrics['teacher_cos_min']:.6f}"
+    )
+
+    print(
+        "Teacher cosine max:",
+        f"{metrics['teacher_cos_max']:.6f}"
+    )
+
+    print()
+
+    print(
+        "Correction cosine:",
+        f"{metrics['correction_cos_mean']:.6f}"
+    )
+
+    print(
+        "Correction cosine min:",
+        f"{metrics['correction_cos_min']:.6f}"
+    )
+
+    print(
+        "Correction cosine max:",
+        f"{metrics['correction_cos_max']:.6f}"
+    )
+
+    print()
+
+    for alpha in ALPHAS:
+
+        print(
+            f"Full vs Distilled blend "
+            f"alpha={alpha:.2f}: "
+            f"{metrics[f'blend_cos_alpha_{alpha}']:.6f}"
+        )
+
+
+    print()
+    print(
+        "BY CATEGORY "
+        "(student -> projected SenseNova):"
+    )
+
+    for category in sorted(
+        category_metrics
+    ):
+
+        print(
+            f"{category:28s} "
+            f"{category_metrics[category]:.6f}"
+        )
+
+
+    return {
+        "source_layer":
+            source_layer,
+
+        "best_token_teacher_cos":
+            best_cos,
+
+        **metrics,
+
+        "model_file":
+            best_path,
+    }
+
+
+# ============================================================
+# TRAIN BOTH
+# ============================================================
+
+all_results = []
+
+for layer in SOURCE_LAYERS:
+
+    result = train_student(
+        layer
+    )
+
+    all_results.append(
+        result
+    )
+
+
+# ============================================================
+# RANK
+# ============================================================
+
+all_results = sorted(
+    all_results,
+    key=lambda x:
+        x[
+            "correction_cos_mean"
+        ],
+    reverse=True,
+)
+
+
+# ============================================================
+# CSV
+# ============================================================
+
+csv_path = os.path.join(
+    OUT_DIR,
+    "distilled_student_v2_results.csv"
+)
+
+fieldnames = [
+    "source_layer",
+    "best_token_teacher_cos",
+    "teacher_cos_mean",
+    "teacher_cos_min",
+    "teacher_cos_max",
+    "correction_cos_mean",
+    "correction_cos_min",
+    "correction_cos_max",
+]
+
+for alpha in ALPHAS:
+
+    fieldnames.append(
+        f"blend_cos_alpha_{alpha}"
+    )
+
+fieldnames.append(
+    "model_file"
+)
+
+
+with open(
+    csv_path,
+    "w",
+    newline="",
+    encoding="utf-8",
+) as f:
+
+    writer = csv.DictWriter(
+        f,
+        fieldnames=fieldnames,
+    )
+
+    writer.writeheader()
+
+    writer.writerows(
+        all_results
+    )
+
+
+# ============================================================
+# REPORT
+# ============================================================
+
+report_path = os.path.join(
+    OUT_DIR,
+    "distilled_student_v2_results.txt"
+)
+
+
+with open(
+    report_path,
+    "w",
+    encoding="utf-8",
+) as f:
+
+    f.write(
+        "DISTILLED STUDENT V2 RESULTS\n"
+    )
+
+    f.write(
+        "=" * 100
+        + "\n\n"
+    )
+
+    for rank, r in enumerate(
+        all_results,
+        1,
+    ):
+
+        f.write(
+            f"{rank:02d}. "
+            f"H3 L{r['source_layer']}\n"
+        )
+
+        f.write(
+            f"    teacher_cos_mean: "
+            f"{r['teacher_cos_mean']:.6f}\n"
+        )
+
+        f.write(
+            f"    teacher_cos_min:  "
+            f"{r['teacher_cos_min']:.6f}\n"
+        )
+
+        f.write(
+            f"    teacher_cos_max:  "
+            f"{r['teacher_cos_max']:.6f}\n"
+        )
+
+        f.write(
+            f"    correction_cos:   "
+            f"{r['correction_cos_mean']:.6f}\n"
+        )
+
+        f.write(
+            f"    correction_min:   "
+            f"{r['correction_cos_min']:.6f}\n"
+        )
+
+        f.write(
+            f"    correction_max:   "
+            f"{r['correction_cos_max']:.6f}\n"
+        )
+
+
+        for alpha in ALPHAS:
+
+            f.write(
+                f"    blend alpha "
+                f"{alpha:.2f}: "
+                f"{r[f'blend_cos_alpha_{alpha}']:.6f}\n"
+            )
+
+
+        f.write(
+            f"    model: "
+            f"{r['model_file']}\n\n"
+        )
+
+
+# ============================================================
+# FINAL
+# ============================================================
+
+print()
+print(
+    "=" * 100
+)
+
+print(
+    "FINAL V2 RANKING"
+)
+
+print(
+    "=" * 100
+)
+
+
+for rank, r in enumerate(
+    all_results,
+    1,
+):
+
+    print(
+        f"{rank:02d}. "
+        f"H3 L{r['source_layer']:2d} | "
+        f"teacher_cos="
+        f"{r['teacher_cos_mean']:.6f} | "
+        f"correction_cos="
+        f"{r['correction_cos_mean']:.6f} | "
+        f"blend@0.20="
+        f"{r['blend_cos_alpha_0.2']:.6f}"
+    )
+
+
+print()
+print(
+    "Report:"
+)
+
+print(
+    report_path
+)
+
+print()
+
+print(
+    "CSV:"
+)
+
+print(
+    csv_path
+)
+
+print()
+print(
+    "DONE"
+)

research/raw_scripts/train_distilled_student_v3.py

@@ -0,0 +1,1628 @@
+import os
+import csv
+import random
+
+import torch
+import torch.nn as nn
+import torch.nn.functional as F
+
+from safetensors.torch import load_file, save_file
+
+
+# ============================================================
+# PATHS
+# ============================================================
+
+ROOT = r"E:\Models SD XL\UNET\Minimax_H3"
+
+H3_MAIN = os.path.join(
+    ROOT,
+    "h3_bridge_hidden_480.pt"
+)
+
+SN_MAIN = os.path.join(
+    ROOT,
+    "sensenova_bridge_hidden_480.pt"
+)
+
+H3_OOD = os.path.join(
+    ROOT,
+    "h3_ood_hidden_160.pt"
+)
+
+SN_OOD = os.path.join(
+    ROOT,
+    "sensenova_ood_hidden_160.pt"
+)
+
+TEACHER_BRIDGE = (
+    r"D:\ComfyUI_Krea2\ComfyUI\models\bridge"
+    r"\SN_L32_to_H3_L49_rank128.safetensors"
+)
+
+V2_INIT = os.path.join(
+    ROOT,
+    "distilled_student_v2",
+    "H3_L49_to_projected_SN32_v2.safetensors"
+)
+
+OUT_DIR = os.path.join(
+    ROOT,
+    "distilled_student_v3"
+)
+
+os.makedirs(
+    OUT_DIR,
+    exist_ok=True
+)
+
+BEST_FILE = os.path.join(
+    OUT_DIR,
+    "H3_L49_to_projected_SN32_v3.safetensors"
+)
+
+REPORT_FILE = os.path.join(
+    OUT_DIR,
+    "distilled_student_v3_report.txt"
+)
+
+CSV_FILE = os.path.join(
+    OUT_DIR,
+    "distilled_student_v3_history.csv"
+)
+
+
+# ============================================================
+# SETTINGS
+# ============================================================
+
+SEED = 20260903
+
+DEVICE = (
+    "cuda"
+    if torch.cuda.is_available()
+    else "cpu"
+)
+
+HIDDEN_DIM = 512
+
+EPOCHS = 40
+
+LR = 3e-4
+
+WEIGHT_DECAY = 1e-5
+
+BATCH_TOKENS = 1024
+
+VAL_PROMPTS = 100
+
+ALPHAS = [
+    0.10,
+    0.20,
+    0.30,
+]
+
+
+# ============================================================
+# REPRO
+# ============================================================
+
+random.seed(
+    SEED
+)
+
+torch.manual_seed(
+    SEED
+)
+
+if torch.cuda.is_available():
+    torch.cuda.manual_seed_all(
+        SEED
+    )
+
+
+# ============================================================
+# MODEL
+# ============================================================
+
+class SemanticStudent(
+    nn.Module
+):
+    def __init__(
+        self,
+        input_dim=5120,
+        hidden_dim=512,
+        output_dim=5120,
+    ):
+        super().__init__()
+
+        self.fc1 = nn.Linear(
+            input_dim,
+            hidden_dim,
+            bias=True,
+        )
+
+        self.fc2 = nn.Linear(
+            hidden_dim,
+            hidden_dim,
+            bias=True,
+        )
+
+        self.fc3 = nn.Linear(
+            hidden_dim,
+            output_dim,
+            bias=True,
+        )
+
+        self.act = nn.SiLU()
+
+
+    def forward(
+        self,
+        x,
+    ):
+        x = self.fc1(
+            x
+        )
+
+        x = self.act(
+            x
+        )
+
+        x = self.fc2(
+            x
+        )
+
+        x = self.act(
+            x
+        )
+
+        x = self.fc3(
+            x
+        )
+
+        return x
+
+
+# ============================================================
+# HELPERS
+# ============================================================
+
+def rms_normalize(
+    x,
+):
+    x = x.float()
+
+    rms = torch.sqrt(
+        x.pow(2).mean(
+            dim=-1,
+            keepdim=True,
+        ) + 1e-6
+    )
+
+    return x / rms
+
+
+def magnitude_match(
+    source,
+    target,
+):
+    source = source.float()
+    target = target.float()
+
+    source_rms = torch.sqrt(
+        source.pow(2).mean(
+            dim=-1,
+            keepdim=True,
+        ) + 1e-8
+    )
+
+    target_rms = torch.sqrt(
+        target.pow(2).mean(
+            dim=-1,
+            keepdim=True,
+        ) + 1e-8
+    )
+
+    return (
+        source
+        * (
+            target_rms
+            / source_rms
+        )
+    )
+
+
+def cosine_mean(
+    a,
+    b,
+):
+    return (
+        F.cosine_similarity(
+            a.float(),
+            b.float(),
+            dim=-1,
+        )
+        .mean()
+        .item()
+    )
+
+
+# ============================================================
+# SAVE
+# ============================================================
+
+def save_student(
+    model,
+    path,
+    best_score,
+):
+    tensors = {
+        "fc1.weight":
+            model.fc1.weight
+            .detach()
+            .cpu()
+            .to(torch.float16),
+
+        "fc1.bias":
+            model.fc1.bias
+            .detach()
+            .cpu()
+            .to(torch.float16),
+
+        "fc2.weight":
+            model.fc2.weight
+            .detach()
+            .cpu()
+            .to(torch.float16),
+
+        "fc2.bias":
+            model.fc2.bias
+            .detach()
+            .cpu()
+            .to(torch.float16),
+
+        "fc3.weight":
+            model.fc3.weight
+            .detach()
+            .cpu()
+            .to(torch.float16),
+
+        "fc3.bias":
+            model.fc3.bias
+            .detach()
+            .cpu()
+            .to(torch.float16),
+    }
+
+    metadata = {
+        "architecture":
+            "SenseNova_H3_Distilled_Student_V3",
+
+        "source":
+            "H3_L49",
+
+        "teacher":
+            "SenseNova_L32_projected_to_H3",
+
+        "input_dim":
+            "5120",
+
+        "hidden_dim":
+            "512",
+
+        "output_dim":
+            "5120",
+
+        "activation":
+            "SiLU",
+
+        "best_score":
+            f"{best_score:.8f}",
+    }
+
+    save_file(
+        tensors,
+        path,
+        metadata=metadata,
+    )
+
+
+# ============================================================
+# LOAD DATASETS
+# ============================================================
+
+print("=" * 100)
+print("DISTILLED STUDENT V3")
+print("TRAINING ON 600 PROMPTS")
+print("=" * 100)
+
+print(
+    "Device:",
+    DEVICE
+)
+
+
+print(
+    "\nLoading original 440..."
+)
+
+h3_main = torch.load(
+    H3_MAIN,
+    map_location="cpu",
+    weights_only=False,
+)
+
+sn_main = torch.load(
+    SN_MAIN,
+    map_location="cpu",
+    weights_only=False,
+)
+
+
+print(
+    "Loading strict OOD 160..."
+)
+
+h3_ood = torch.load(
+    H3_OOD,
+    map_location="cpu",
+    weights_only=False,
+)
+
+sn_ood = torch.load(
+    SN_OOD,
+    map_location="cpu",
+    weights_only=False,
+)
+
+
+# ============================================================
+# LOAD TEACHER
+# ============================================================
+
+teacher = load_file(
+    TEACHER_BRIDGE,
+    device="cpu",
+)
+
+teacher_down = (
+    teacher[
+        "down.weight"
+    ]
+    .float()
+)
+
+teacher_up = (
+    teacher[
+        "up.weight"
+    ]
+    .float()
+)
+
+
+# ============================================================
+# COLLECT ALL 600
+# ============================================================
+
+samples = []
+
+
+def append_main_pair(
+    h3_item,
+    sn_item,
+    origin,
+):
+
+    if (
+        h3_item["prompt"]
+        != sn_item["prompt"]
+    ):
+        raise RuntimeError(
+            "Prompt mismatch."
+        )
+
+    if not torch.equal(
+        h3_item["input_ids"],
+        sn_item["input_ids"],
+    ):
+        raise RuntimeError(
+            "Token mismatch."
+        )
+
+    h49 = (
+        h3_item[
+            "layers"
+        ][49]
+        .float()
+    )
+
+    sn32 = (
+        sn_item[
+            "layers"
+        ][32]
+        .float()
+    )
+
+    with torch.no_grad():
+
+        sn_norm = rms_normalize(
+            sn32
+        )
+
+        low = F.linear(
+            sn_norm,
+            teacher_down,
+        )
+
+        teacher_projected = F.linear(
+            low,
+            teacher_up,
+        )
+
+    samples.append({
+        "prompt":
+            h3_item["prompt"],
+
+        "category":
+            h3_item["category"],
+
+        "origin":
+            origin,
+
+        "h49":
+            h49.to(
+                torch.float16
+            ),
+
+        "teacher":
+            teacher_projected.to(
+                torch.float16
+            ),
+    })
+
+
+def append_ood_pair(
+    h3_item,
+    sn_item,
+    origin,
+):
+
+    if (
+        h3_item["prompt"]
+        != sn_item["prompt"]
+    ):
+        raise RuntimeError(
+            "Prompt mismatch."
+        )
+
+    if not torch.equal(
+        h3_item["input_ids"],
+        sn_item["input_ids"],
+    ):
+        raise RuntimeError(
+            "Token mismatch."
+        )
+
+    h49 = (
+        h3_item[
+            "hidden"
+        ]
+        .float()
+    )
+
+    sn32 = (
+        sn_item[
+            "hidden"
+        ]
+        .float()
+    )
+
+    with torch.no_grad():
+
+        sn_norm = rms_normalize(
+            sn32
+        )
+
+        low = F.linear(
+            sn_norm,
+            teacher_down,
+        )
+
+        teacher_projected = F.linear(
+            low,
+            teacher_up,
+        )
+
+    samples.append({
+        "prompt":
+            h3_item["prompt"],
+
+        "category":
+            h3_item["category"],
+
+        "origin":
+            origin,
+
+        "h49":
+            h49.to(
+                torch.float16
+            ),
+
+        "teacher":
+            teacher_projected.to(
+                torch.float16
+            ),
+    })
+
+
+for h3_item, sn_item in zip(
+    h3_main["results"],
+    sn_main["results"],
+):
+    append_main_pair(
+        h3_item,
+        sn_item,
+        "main440",
+    )
+
+
+for h3_item, sn_item in zip(
+    h3_ood["results"],
+    sn_ood["results"],
+):
+    append_ood_pair(
+        h3_item,
+        sn_item,
+        "ood160",
+    )
+
+
+print()
+print(
+    "Total samples:",
+    len(samples)
+)
+
+if len(samples) != 600:
+    raise RuntimeError(
+        f"Expected 600 prompts, got {len(samples)}"
+    )
+
+
+# ============================================================
+# SPLIT 500 / 100
+# STRATIFIED-ish BY ORIGIN
+# ============================================================
+
+main_indices = [
+    i
+    for i, s
+    in enumerate(samples)
+    if s["origin"] == "main440"
+]
+
+ood_indices = [
+    i
+    for i, s
+    in enumerate(samples)
+    if s["origin"] == "ood160"
+]
+
+random.shuffle(
+    main_indices
+)
+
+random.shuffle(
+    ood_indices
+)
+
+# Keep both distributions represented in validation.
+val_main = main_indices[:70]
+val_ood = ood_indices[:30]
+
+val_indices = sorted(
+    val_main + val_ood
+)
+
+val_set = set(
+    val_indices
+)
+
+train_indices = [
+    i
+    for i
+    in range(
+        len(samples)
+    )
+    if i not in val_set
+]
+
+
+print(
+    "Train prompts:",
+    len(train_indices)
+)
+
+print(
+    "Validation prompts:",
+    len(val_indices)
+)
+
+print(
+    "Validation main:",
+    len(val_main)
+)
+
+print(
+    "Validation OOD:",
+    len(val_ood)
+)
+
+
+# ============================================================
+# BUILD TOKEN MATRICES
+# ============================================================
+
+def build_token_data(
+    indices,
+):
+    xs = []
+    ys = []
+
+    for idx in indices:
+
+        item = samples[
+            idx
+        ]
+
+        x = (
+            item["h49"]
+            .float()
+            .squeeze(0)
+        )
+
+        y = (
+            item["teacher"]
+            .float()
+            .squeeze(0)
+        )
+
+        x = rms_normalize(
+            x
+        )
+
+        xs.append(
+            x
+        )
+
+        ys.append(
+            y
+        )
+
+    return (
+        torch.cat(
+            xs,
+            dim=0,
+        ),
+        torch.cat(
+            ys,
+            dim=0,
+        ),
+    )
+
+
+print(
+    "\nBuilding token matrices..."
+)
+
+train_x, train_y = (
+    build_token_data(
+        train_indices
+    )
+)
+
+val_x, val_y = (
+    build_token_data(
+        val_indices
+    )
+)
+
+
+print(
+    "Train tokens:",
+    train_x.shape[0]
+)
+
+print(
+    "Val tokens:",
+    val_x.shape[0]
+)
+
+
+if DEVICE == "cuda":
+
+    train_x = (
+        train_x
+        .contiguous()
+        .pin_memory()
+    )
+
+    train_y = (
+        train_y
+        .contiguous()
+        .pin_memory()
+    )
+
+    val_x = (
+        val_x
+        .contiguous()
+        .pin_memory()
+    )
+
+    val_y = (
+        val_y
+        .contiguous()
+        .pin_memory()
+    )
+
+
+# ============================================================
+# MODEL
+# ============================================================
+
+model = SemanticStudent(
+    5120,
+    HIDDEN_DIM,
+    5120,
+)
+
+
+# ============================================================
+# INIT FROM V2
+# ============================================================
+
+if os.path.exists(
+    V2_INIT
+):
+
+    print(
+        "\nLoading V2 checkpoint as initialization..."
+    )
+
+    v2 = load_file(
+        V2_INIT,
+        device="cpu",
+    )
+
+    with torch.no_grad():
+
+        model.fc1.weight.copy_(
+            v2["fc1.weight"].float()
+        )
+
+        model.fc1.bias.copy_(
+            v2["fc1.bias"].float()
+        )
+
+        model.fc2.weight.copy_(
+            v2["fc2.weight"].float()
+        )
+
+        model.fc2.bias.copy_(
+            v2["fc2.bias"].float()
+        )
+
+        model.fc3.weight.copy_(
+            v2["fc3.weight"].float()
+        )
+
+        model.fc3.bias.copy_(
+            v2["fc3.bias"].float()
+        )
+
+
+model = model.to(
+    DEVICE
+)
+
+
+# ============================================================
+# LOSS
+# ============================================================
+
+def loss_fn(
+    pred,
+    teacher,
+    h49,
+):
+
+    pred = pred.float()
+    teacher = teacher.float()
+    h49 = h49.float()
+
+
+    # ========================================================
+    # 1. REPRESENTATION DIRECTION
+    # ========================================================
+
+    rep_cos_loss = (
+        1.0
+        - F.cosine_similarity(
+            pred,
+            teacher,
+            dim=-1,
+        ).mean()
+    )
+
+
+    # ========================================================
+    # 2. MATCH REAL FULL-BRIDGE CORRECTION
+    # ========================================================
+
+    teacher_scaled = magnitude_match(
+        teacher,
+        h49,
+    )
+
+    pred_scaled = magnitude_match(
+        pred,
+        h49,
+    )
+
+    teacher_delta = (
+        teacher_scaled
+        - h49
+    )
+
+    pred_delta = (
+        pred_scaled
+        - h49
+    )
+
+    correction_loss = (
+        1.0
+        - F.cosine_similarity(
+            pred_delta,
+            teacher_delta,
+            dim=-1,
+        ).mean()
+    )
+
+
+    # ========================================================
+    # 3. SMALL NORMALIZED MSE
+    # ========================================================
+
+    pred_n = F.normalize(
+        pred,
+        dim=-1,
+    )
+
+    teacher_n = F.normalize(
+        teacher,
+        dim=-1,
+    )
+
+    mse = F.mse_loss(
+        pred_n,
+        teacher_n,
+    )
+
+
+    # ========================================================
+    # TOTAL
+    # ========================================================
+
+    total = (
+        0.55 * rep_cos_loss
+        + 0.40 * correction_loss
+        + 0.05 * mse
+    )
+
+    return total
+
+
+# ============================================================
+# OPTIMIZER
+# ============================================================
+
+optimizer = torch.optim.AdamW(
+    model.parameters(),
+    lr=LR,
+    weight_decay=
+        WEIGHT_DECAY,
+)
+
+scheduler = (
+    torch.optim.lr_scheduler.CosineAnnealingLR(
+        optimizer,
+        T_max=EPOCHS,
+        eta_min=LR * 0.05,
+    )
+)
+
+
+# ============================================================
+# VALIDATION
+# ============================================================
+
+def evaluate():
+
+    model.eval()
+
+    rep_scores = []
+
+    correction_scores = []
+
+    blend_scores = {
+        alpha: []
+        for alpha
+        in ALPHAS
+    }
+
+    origin_scores = {
+        "main440": [],
+        "ood160": [],
+    }
+
+    category_scores = {}
+
+
+    with torch.no_grad():
+
+        for idx in val_indices:
+
+            item = samples[
+                idx
+            ]
+
+            h49 = (
+                item["h49"]
+                .float()
+            )
+
+            teacher = (
+                item["teacher"]
+                .float()
+            )
+
+            x = rms_normalize(
+                h49
+            )
+
+            pred = (
+                model(
+                    x.to(
+                        DEVICE
+                    )
+                )
+                .cpu()
+            )
+
+            rep_cos = cosine_mean(
+                pred,
+                teacher,
+            )
+
+            teacher_scaled = magnitude_match(
+                teacher,
+                h49,
+            )
+
+            pred_scaled = magnitude_match(
+                pred,
+                h49,
+            )
+
+            teacher_delta = (
+                teacher_scaled
+                - h49
+            )
+
+            pred_delta = (
+                pred_scaled
+                - h49
+            )
+
+            correction_cos = (
+                cosine_mean(
+                    pred_delta,
+                    teacher_delta,
+                )
+            )
+
+            rep_scores.append(
+                rep_cos
+            )
+
+            correction_scores.append(
+                correction_cos
+            )
+
+            origin_scores[
+                item["origin"]
+            ].append(
+                correction_cos
+            )
+
+            category_scores.setdefault(
+                item["category"],
+                []
+            ).append(
+                correction_cos
+            )
+
+
+            for alpha in ALPHAS:
+
+                full = (
+                    h49
+                    + alpha
+                    * teacher_delta
+                )
+
+                distilled = (
+                    h49
+                    + alpha
+                    * pred_delta
+                )
+
+                blend_scores[
+                    alpha
+                ].append(
+                    cosine_mean(
+                        distilled,
+                        full,
+                    )
+                )
+
+
+    result = {
+        "rep_cos":
+            sum(rep_scores)
+            / len(rep_scores),
+
+        "correction_cos":
+            sum(correction_scores)
+            / len(correction_scores),
+
+        "correction_min":
+            min(correction_scores),
+
+        "main_correction":
+            sum(
+                origin_scores[
+                    "main440"
+                ]
+            )
+            / len(
+                origin_scores[
+                    "main440"
+                ]
+            ),
+
+        "ood_correction":
+            sum(
+                origin_scores[
+                    "ood160"
+                ]
+            )
+            / len(
+                origin_scores[
+                    "ood160"
+                ]
+            ),
+    }
+
+    for alpha in ALPHAS:
+
+        result[
+            f"blend_{alpha}"
+        ] = (
+            sum(
+                blend_scores[
+                    alpha
+                ]
+            )
+            / len(
+                blend_scores[
+                    alpha
+                ]
+            )
+        )
+
+    category_result = {
+        category:
+            sum(values)
+            / len(values)
+
+        for category, values
+        in category_scores.items()
+    }
+
+    return (
+        result,
+        category_result,
+    )
+
+
+# ============================================================
+# TRAIN
+# ============================================================
+
+token_indices = torch.arange(
+    train_x.shape[0]
+)
+
+best_score = -1.0
+
+history = []
+
+
+print()
+print("=" * 100)
+print("TRAINING V3")
+print("=" * 100)
+
+
+for epoch in range(
+    1,
+    EPOCHS + 1,
+):
+
+    model.train()
+
+    permutation = (
+        token_indices[
+            torch.randperm(
+                len(
+                    token_indices
+                )
+            )
+        ]
+    )
+
+    epoch_loss = 0.0
+
+    batches = 0
+
+
+    for start in range(
+        0,
+        len(permutation),
+        BATCH_TOKENS,
+    ):
+
+        ids = (
+            permutation[
+                start:
+                start
+                + BATCH_TOKENS
+            ]
+        )
+
+        x = (
+            train_x[
+                ids
+            ]
+            .to(
+                DEVICE,
+                non_blocking=True,
+            )
+        )
+
+        y = (
+            train_y[
+                ids
+            ]
+            .to(
+                DEVICE,
+                non_blocking=True,
+            )
+        )
+
+        # Original H3 before RMS normalize.
+        # Reconstruct approximate scale reference
+        # from normalized input is impossible,
+        # so for correction loss we use token-space
+        # target direction against a stored H3-like
+        # reference built from y/pred runtime later.
+        #
+        # Here, use x as normalized H3 reference,
+        # scaled to teacher RMS for stability.
+
+        x_ref = x
+
+        optimizer.zero_grad(
+            set_to_none=True
+        )
+
+        pred = model(
+            x
+        )
+
+        loss = loss_fn(
+            pred,
+            y,
+            x_ref,
+        )
+
+        loss.backward()
+
+        torch.nn.utils.clip_grad_norm_(
+            model.parameters(),
+            1.0,
+        )
+
+        optimizer.step()
+
+        epoch_loss += (
+            loss.item()
+        )
+
+        batches += 1
+
+
+    scheduler.step()
+
+
+    # ========================================================
+    # VALIDATE
+    # ========================================================
+
+    metrics, category_metrics = (
+        evaluate()
+    )
+
+
+    # Favor OOD correction slightly.
+    score = (
+        0.45
+        * metrics[
+            "correction_cos"
+        ]
+        +
+        0.35
+        * metrics[
+            "ood_correction"
+        ]
+        +
+        0.20
+        * metrics[
+            "rep_cos"
+        ]
+    )
+
+
+    row = {
+        "epoch":
+            epoch,
+
+        "train_loss":
+            epoch_loss
+            / max(
+                batches,
+                1,
+            ),
+
+        "rep_cos":
+            metrics[
+                "rep_cos"
+            ],
+
+        "correction_cos":
+            metrics[
+                "correction_cos"
+            ],
+
+        "main_correction":
+            metrics[
+                "main_correction"
+            ],
+
+        "ood_correction":
+            metrics[
+                "ood_correction"
+            ],
+
+        "correction_min":
+            metrics[
+                "correction_min"
+            ],
+
+        "blend_0.1":
+            metrics[
+                "blend_0.1"
+            ],
+
+        "blend_0.2":
+            metrics[
+                "blend_0.2"
+            ],
+
+        "blend_0.3":
+            metrics[
+                "blend_0.3"
+            ],
+
+        "score":
+            score,
+    }
+
+    history.append(
+        row
+    )
+
+
+    print(
+        f"Epoch "
+        f"{epoch:02d}/"
+        f"{EPOCHS} | "
+        f"loss="
+        f"{row['train_loss']:.6f} | "
+        f"rep="
+        f"{row['rep_cos']:.6f} | "
+        f"corr="
+        f"{row['correction_cos']:.6f} | "
+        f"main="
+        f"{row['main_correction']:.6f} | "
+        f"ood="
+        f"{row['ood_correction']:.6f} | "
+        f"min="
+        f"{row['correction_min']:.6f}"
+    )
+
+
+    if score > best_score:
+
+        best_score = score
+
+        save_student(
+            model,
+            BEST_FILE,
+            best_score,
+        )
+
+
+# ============================================================
+# SAVE HISTORY
+# ============================================================
+
+with open(
+    CSV_FILE,
+    "w",
+    newline="",
+    encoding="utf-8",
+) as f:
+
+    writer = csv.DictWriter(
+        f,
+        fieldnames=list(
+            history[0].keys()
+        ),
+    )
+
+    writer.writeheader()
+
+    writer.writerows(
+        history
+    )
+
+
+# ============================================================
+# LOAD BEST
+# ============================================================
+
+best_weights = load_file(
+    BEST_FILE,
+    device="cpu",
+)
+
+best_model = SemanticStudent(
+    5120,
+    HIDDEN_DIM,
+    5120,
+)
+
+with torch.no_grad():
+
+    best_model.fc1.weight.copy_(
+        best_weights[
+            "fc1.weight"
+        ].float()
+    )
+
+    best_model.fc1.bias.copy_(
+        best_weights[
+            "fc1.bias"
+        ].float()
+    )
+
+    best_model.fc2.weight.copy_(
+        best_weights[
+            "fc2.weight"
+        ].float()
+    )
+
+    best_model.fc2.bias.copy_(
+        best_weights[
+            "fc2.bias"
+        ].float()
+    )
+
+    best_model.fc3.weight.copy_(
+        best_weights[
+            "fc3.weight"
+        ].float()
+    )
+
+    best_model.fc3.bias.copy_(
+        best_weights[
+            "fc3.bias"
+        ].float()
+    )
+
+
+model = best_model.to(
+    DEVICE
+)
+
+model.eval()
+
+
+final_metrics, final_categories = (
+    evaluate()
+)
+
+
+# ============================================================
+# REPORT
+# ============================================================
+
+with open(
+    REPORT_FILE,
+    "w",
+    encoding="utf-8",
+) as f:
+
+    f.write(
+        "DISTILLED STUDENT V3 FINAL REPORT\n"
+    )
+
+    f.write(
+        "=" * 100
+        + "\n\n"
+    )
+
+    f.write(
+        "Training prompts: 500\n"
+    )
+
+    f.write(
+        "Validation prompts: 100\n"
+    )
+
+    f.write(
+        "Validation includes 70 original + 30 strict OOD prompts.\n\n"
+    )
+
+    f.write(
+        f"Representation cosine: "
+        f"{final_metrics['rep_cos']:.6f}\n"
+    )
+
+    f.write(
+        f"Correction cosine:     "
+        f"{final_metrics['correction_cos']:.6f}\n"
+    )
+
+    f.write(
+        f"Main correction:       "
+        f"{final_metrics['main_correction']:.6f}\n"
+    )
+
+    f.write(
+        f"OOD correction:        "
+        f"{final_metrics['ood_correction']:.6f}\n"
+    )
+
+    f.write(
+        f"Correction minimum:    "
+        f"{final_metrics['correction_min']:.6f}\n\n"
+    )
+
+    for alpha in ALPHAS:
+
+        f.write(
+            f"Blend alpha "
+            f"{alpha:.2f}: "
+            f"{final_metrics[f'blend_{alpha}']:.6f}\n"
+        )
+
+
+    f.write(
+        "\nCATEGORY CORRECTION\n"
+    )
+
+    f.write(
+        "-" * 100
+        + "\n"
+    )
+
+    for category in sorted(
+        final_categories
+    ):
+
+        f.write(
+            f"{category:28s} "
+            f"{final_categories[category]:.6f}\n"
+        )
+
+
+    f.write(
+        "\nCheckpoint:\n"
+    )
+
+    f.write(
+        BEST_FILE
+        + "\n"
+    )
+
+
+# ============================================================
+# DONE
+# ============================================================
+
+print()
+print("=" * 100)
+print("V3 FINAL")
+print("=" * 100)
+
+print(
+    "Representation cosine:",
+    f"{final_metrics['rep_cos']:.6f}"
+)
+
+print(
+    "Correction cosine:",
+    f"{final_metrics['correction_cos']:.6f}"
+)
+
+print(
+    "Main correction:",
+    f"{final_metrics['main_correction']:.6f}"
+)
+
+print(
+    "OOD correction:",
+    f"{final_metrics['ood_correction']:.6f}"
+)
+
+print(
+    "Correction minimum:",
+    f"{final_metrics['correction_min']:.6f}"
+)
+
+print()
+print(
+    "Saved student:"
+)
+
+print(
+    BEST_FILE
+)
+
+print()
+print(
+    "Report:"
+)
+
+print(
+    REPORT_FILE
+)
+
+print()
+print(
+    "DONE"
+)

research/raw_scripts/train_embedding_bridge_test.py

@@ -0,0 +1,397 @@
+import os
+import math
+import torch
+
+from safetensors import safe_open
+from safetensors.torch import load_file, save_file
+
+
+# =========================================================
+# PATHS
+# =========================================================
+
+H3_EMB = r"E:\Models SD XL\UNET\Minimax_H3\h3_qwen3vl32b_embeddings_bf16.safetensors"
+
+SENSENOVA = r"D:\ComfyUI_Python312\ComfyUI\models\unet\SenseNova-U1.5-8B-MoT-pruned-bf16.safetensors"
+
+OUT = r"E:\Models SD XL\UNET\Minimax_H3\sensenova_to_h3_embedding_bridge_rank512.safetensors"
+
+
+# =========================================================
+# SETTINGS
+# =========================================================
+
+INPUT_DIM = 4096
+OUTPUT_DIM = 5120
+RANK = 512
+
+BATCH_SIZE = 256
+EPOCHS = 5
+LR = 1e-3
+
+VAL_SIZE = 12000
+
+SEED = 1234
+
+
+# =========================================================
+# DEVICE
+# =========================================================
+
+device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
+
+print("=" * 80)
+print("SenseNova -> MiniMax H3 embedding bridge test")
+print("=" * 80)
+
+print("Device:", device)
+
+if device.type == "cuda":
+    print("GPU:", torch.cuda.get_device_name(0))
+    print(
+        "VRAM:",
+        round(torch.cuda.get_device_properties(0).total_memory / 1024**3, 2),
+        "GB"
+    )
+
+
+# =========================================================
+# LOAD H3 EMBEDDINGS
+# =========================================================
+
+print()
+print("Loading H3 embeddings...")
+
+h3_file = load_file(H3_EMB, device="cpu")
+
+h3 = h3_file["model.embed_tokens.weight"]
+
+print("H3 shape:", tuple(h3.shape))
+print("H3 dtype:", h3.dtype)
+
+
+# =========================================================
+# LOAD SENSENOVA EMBEDDINGS ONLY
+# =========================================================
+
+print()
+print("Loading SenseNova token embeddings...")
+
+with safe_open(SENSENOVA, framework="pt", device="cpu") as f:
+
+    key = "language_model.model.embed_tokens.weight"
+
+    if key not in f.keys():
+        raise RuntimeError(
+            f"SenseNova embedding tensor not found: {key}"
+        )
+
+    sense = f.get_tensor(key)
+
+
+print("SenseNova shape:", tuple(sense.shape))
+print("SenseNova dtype:", sense.dtype)
+
+
+# =========================================================
+# VALIDATE
+# =========================================================
+
+if h3.shape[0] != sense.shape[0]:
+    raise RuntimeError(
+        f"Vocabulary mismatch: H3={h3.shape[0]}, SenseNova={sense.shape[0]}"
+    )
+
+if sense.shape[1] != INPUT_DIM:
+    raise RuntimeError(
+        f"Unexpected SenseNova hidden size: {sense.shape[1]}"
+    )
+
+if h3.shape[1] != OUTPUT_DIM:
+    raise RuntimeError(
+        f"Unexpected H3 hidden size: {h3.shape[1]}"
+    )
+
+
+VOCAB = h3.shape[0]
+
+print()
+print("Vocabulary rows:", VOCAB)
+
+
+# =========================================================
+# NORMALIZATION
+# =========================================================
+
+# Convert storage to float32 only per batch later.
+# Calculate global average norms using manageable chunks.
+
+def average_norm(x, chunk=4096):
+
+    total = 0.0
+    count = 0
+
+    for start in range(0, x.shape[0], chunk):
+
+        end = min(start + chunk, x.shape[0])
+
+        b = x[start:end].float()
+
+        total += b.norm(dim=1).sum().item()
+        count += b.shape[0]
+
+    return total / count
+
+
+print()
+print("Calculating embedding norm statistics...")
+
+sense_norm = average_norm(sense)
+h3_norm = average_norm(h3)
+
+print("SenseNova avg norm:", sense_norm)
+print("H3 avg norm:       ", h3_norm)
+
+
+# =========================================================
+# TRAIN / VALIDATION SPLIT
+# =========================================================
+
+g = torch.Generator()
+g.manual_seed(SEED)
+
+perm = torch.randperm(VOCAB, generator=g)
+
+val_ids = perm[:VAL_SIZE]
+train_ids = perm[VAL_SIZE:]
+
+print()
+print("Train rows:", len(train_ids))
+print("Validation rows:", len(val_ids))
+
+
+# =========================================================
+# LOW-RANK PROJECTOR
+#
+# 4096 -> 512 -> 5120
+# no activation: this is a low-rank linear mapping
+# =========================================================
+
+A = torch.nn.Linear(
+    INPUT_DIM,
+    RANK,
+    bias=False,
+    device=device,
+    dtype=torch.float32,
+)
+
+B = torch.nn.Linear(
+    RANK,
+    OUTPUT_DIM,
+    bias=False,
+    device=device,
+    dtype=torch.float32,
+)
+
+
+torch.nn.init.normal_(
+    A.weight,
+    std=1.0 / math.sqrt(INPUT_DIM)
+)
+
+torch.nn.init.zeros_(B.weight)
+
+
+params = list(A.parameters()) + list(B.parameters())
+
+optimizer = torch.optim.AdamW(
+    params,
+    lr=LR,
+    weight_decay=1e-4,
+)
+
+
+# =========================================================
+# LOSS
+# =========================================================
+
+def bridge_forward(x):
+
+    return B(A(x))
+
+
+def loss_fn(pred, target):
+
+    # Directional similarity.
+    cosine = 1.0 - torch.nn.functional.cosine_similarity(
+        pred,
+        target,
+        dim=-1
+    ).mean()
+
+    # Normalize magnitude sensitivity.
+    pred_n = torch.nn.functional.normalize(pred, dim=-1)
+    target_n = torch.nn.functional.normalize(target, dim=-1)
+
+    mse = torch.nn.functional.mse_loss(
+        pred_n,
+        target_n
+    )
+
+    return cosine + mse
+
+
+# =========================================================
+# EVALUATION
+# =========================================================
+
+@torch.no_grad()
+def evaluate(ids):
+
+    A.eval()
+    B.eval()
+
+    cos_sum = 0.0
+    mse_sum = 0.0
+    count = 0
+
+    for start in range(0, len(ids), BATCH_SIZE):
+
+        idx = ids[start:start + BATCH_SIZE]
+
+        x = sense[idx].float().to(device)
+        y = h3[idx].float().to(device)
+
+        pred = bridge_forward(x)
+
+        cos = torch.nn.functional.cosine_similarity(
+            pred,
+            y,
+            dim=-1
+        )
+
+        mse = torch.nn.functional.mse_loss(
+            pred,
+            y,
+            reduction="none"
+        ).mean(dim=1)
+
+        cos_sum += cos.sum().item()
+        mse_sum += mse.sum().item()
+
+        count += len(idx)
+
+    return cos_sum / count, mse_sum / count
+
+
+# =========================================================
+# BASELINE
+# =========================================================
+
+print()
+print("=" * 80)
+print("TRAINING")
+print("=" * 80)
+
+
+# =========================================================
+# TRAIN
+# =========================================================
+
+for epoch in range(EPOCHS):
+
+    A.train()
+    B.train()
+
+    shuffled = train_ids[
+        torch.randperm(len(train_ids), generator=g)
+    ]
+
+    running = 0.0
+    batches = 0
+
+    for start in range(0, len(shuffled), BATCH_SIZE):
+
+        idx = shuffled[start:start + BATCH_SIZE]
+
+        x = sense[idx].float().to(device)
+        y = h3[idx].float().to(device)
+
+        optimizer.zero_grad(set_to_none=True)
+
+        pred = bridge_forward(x)
+
+        loss = loss_fn(pred, y)
+
+        loss.backward()
+
+        torch.nn.utils.clip_grad_norm_(
+            params,
+            1.0
+        )
+
+        optimizer.step()
+
+        running += loss.item()
+        batches += 1
+
+        if batches % 100 == 0:
+
+            print(
+                f"Epoch {epoch + 1}/{EPOCHS} | "
+                f"batch {batches} | "
+                f"loss {running / batches:.6f}"
+            )
+
+    val_cos, val_mse = evaluate(val_ids)
+
+    print()
+    print(
+        f"EPOCH {epoch + 1} COMPLETE | "
+        f"train_loss={running / batches:.6f} | "
+        f"val_cos={val_cos:.6f} | "
+        f"val_mse={val_mse:.8f}"
+    )
+    print()
+
+
+# =========================================================
+# FINAL EVALUATION
+# =========================================================
+
+val_cos, val_mse = evaluate(val_ids)
+
+print("=" * 80)
+print("FINAL")
+print("=" * 80)
+
+print("Validation cosine similarity:", val_cos)
+print("Validation MSE:", val_mse)
+
+
+# =========================================================
+# SAVE
+# =========================================================
+
+save_file(
+    {
+        "proj_in.weight": A.weight.detach().cpu().to(torch.float16),
+        "proj_out.weight": B.weight.detach().cpu().to(torch.float16),
+    },
+    OUT,
+    metadata={
+        "source": "SenseNova-U1.5-8B-MoT",
+        "target": "MiniMax-H3-Qwen3VL32B",
+        "input_dim": str(INPUT_DIM),
+        "rank": str(RANK),
+        "output_dim": str(OUTPUT_DIM),
+        "validation_cosine": str(val_cos),
+        "validation_mse": str(val_mse),
+    }
+)
+
+print()
+print("Saved bridge:")
+print(OUT)
+print()
+print("DONE")

research/raw_scripts/train_hidden_bridge_screening.py

@@ -0,0 +1,859 @@
+import os
+import csv
+import json
+import math
+import random
+import gc
+import torch
+import torch.nn.functional as F
+from safetensors.torch import save_file
+
+
+ROOT = r"E:\Models SD XL\UNET\Minimax_H3"
+
+H3_FILE = os.path.join(
+    ROOT,
+    "h3_bridge_hidden_480.pt"
+)
+
+SN_FILE = os.path.join(
+    ROOT,
+    "sensenova_bridge_hidden_480.pt"
+)
+
+OUT_CSV = os.path.join(
+    ROOT,
+    "hidden_bridge_screening_rank128.csv"
+)
+
+OUT_TXT = os.path.join(
+    ROOT,
+    "hidden_bridge_screening_rank128.txt"
+)
+
+OUT_DIR = os.path.join(
+    ROOT,
+    "hidden_bridge_probes_rank128"
+)
+
+os.makedirs(OUT_DIR, exist_ok=True)
+
+
+# ============================================================
+# SETTINGS
+# ============================================================
+
+SN_LAYERS = [8, 16, 24, 32, 41]
+H3_LAYERS = [8, 16, 24, 32, 40, 49]
+
+RANK = 128
+
+EPOCHS = 8
+BATCH_SIZE = 512
+
+LR = 2e-3
+WEIGHT_DECAY = 1e-4
+
+VAL_FRACTION = 0.20
+
+SEED = 12345
+
+
+# ============================================================
+# DEVICE
+# ============================================================
+
+torch.manual_seed(SEED)
+random.seed(SEED)
+
+device = torch.device(
+    "cuda" if torch.cuda.is_available() else "cpu"
+)
+
+print("=" * 90)
+print("SenseNova -> MiniMax H3")
+print("HIDDEN-STATE BRIDGE SCREENING")
+print("=" * 90)
+
+print("Device:", device)
+
+if device.type == "cuda":
+    print("GPU:", torch.cuda.get_device_name(0))
+    print(
+        "VRAM:",
+        round(
+            torch.cuda.get_device_properties(0).total_memory
+            / 1024**3,
+            2
+        ),
+        "GB"
+    )
+
+
+# ============================================================
+# LOAD DATA
+# ============================================================
+
+print("\nLoading H3 dataset...")
+
+h3_data = torch.load(
+    H3_FILE,
+    map_location="cpu",
+    weights_only=False
+)
+
+print("H3 prompts:", len(h3_data["results"]))
+
+print("Loading SenseNova dataset...")
+
+sn_data = torch.load(
+    SN_FILE,
+    map_location="cpu",
+    weights_only=False
+)
+
+print("SenseNova prompts:", len(sn_data["results"]))
+
+
+h3_results = h3_data["results"]
+sn_results = sn_data["results"]
+
+if len(h3_results) != len(sn_results):
+    raise RuntimeError(
+        "H3 and SenseNova prompt counts differ."
+    )
+
+
+# ============================================================
+# VERIFY ALIGNMENT
+# ============================================================
+
+print("\nVerifying prompt/token alignment...")
+
+for i, (h, s) in enumerate(
+    zip(h3_results, sn_results)
+):
+
+    if h["prompt"] != s["prompt"]:
+        raise RuntimeError(
+            f"Prompt mismatch at index {i}"
+        )
+
+    if not torch.equal(
+        h["input_ids"],
+        s["input_ids"]
+    ):
+        raise RuntimeError(
+            f"Token mismatch at index {i}\n"
+            f"{h['prompt']}"
+        )
+
+print("All 480 prompts and token IDs align perfectly.")
+
+
+# ============================================================
+# TRAIN / VALIDATION SPLIT BY PROMPT
+# ============================================================
+
+N = len(h3_results)
+
+indices = list(range(N))
+
+rng = random.Random(SEED)
+rng.shuffle(indices)
+
+val_count = round(
+    N * VAL_FRACTION
+)
+
+val_prompt_ids = set(
+    indices[:val_count]
+)
+
+train_prompt_ids = [
+    x for x in indices
+    if x not in val_prompt_ids
+]
+
+val_prompt_ids = sorted(val_prompt_ids)
+
+print()
+print("Train prompts:", len(train_prompt_ids))
+print("Validation prompts:", len(val_prompt_ids))
+
+
+# ============================================================
+# CATEGORY DISTRIBUTION
+# ============================================================
+
+def category_counts(ids, results):
+
+    counts = {}
+
+    for i in ids:
+
+        category = results[i].get(
+            "category",
+            "unknown"
+        )
+
+        counts[category] = (
+            counts.get(category, 0) + 1
+        )
+
+    return counts
+
+
+print("\nTrain categories:")
+print(category_counts(train_prompt_ids, h3_results))
+
+print("\nValidation categories:")
+print(category_counts(val_prompt_ids, h3_results))
+
+
+# ============================================================
+# BUILD TOKEN DATASET FOR ONE PAIR
+# ============================================================
+
+def build_tokens(
+    sn_layer,
+    h3_layer,
+    prompt_ids
+):
+
+    xs = []
+    ys = []
+    cats = []
+
+    for i in prompt_ids:
+
+        sx = sn_results[i]["layers"][sn_layer]
+        hy = h3_results[i]["layers"][h3_layer]
+
+        if sx.shape[:2] != hy.shape[:2]:
+            raise RuntimeError(
+                f"Sequence mismatch at prompt {i}"
+            )
+
+        # [1,T,D] -> [T,D]
+        sx = sx.squeeze(0).float()
+        hy = hy.squeeze(0).float()
+
+        xs.append(sx)
+        ys.append(hy)
+
+        category = h3_results[i].get(
+            "category",
+            "unknown"
+        )
+
+        cats.extend(
+            [category] * sx.shape[0]
+        )
+
+    X = torch.cat(xs, dim=0)
+    Y = torch.cat(ys, dim=0)
+
+    return X, Y, cats
+
+
+# ============================================================
+# LOW-RANK PROJECTOR
+# ============================================================
+
+class LowRankBridge(torch.nn.Module):
+
+    def __init__(
+        self,
+        input_dim=4096,
+        rank=128,
+        output_dim=5120
+    ):
+
+        super().__init__()
+
+        self.down = torch.nn.Linear(
+            input_dim,
+            rank,
+            bias=False
+        )
+
+        self.up = torch.nn.Linear(
+            rank,
+            output_dim,
+            bias=False
+        )
+
+        torch.nn.init.normal_(
+            self.down.weight,
+            std=1.0 / math.sqrt(input_dim)
+        )
+
+        torch.nn.init.zeros_(
+            self.up.weight
+        )
+
+
+    def forward(self, x):
+
+        return self.up(
+            self.down(x)
+        )
+
+
+# ============================================================
+# NORMALIZATION
+# ============================================================
+
+def normalize_input(x):
+
+    # Per-token RMS normalization.
+    # Keeps direction/information but removes the huge
+    # layer-dependent magnitude differences we observed.
+
+    rms = torch.sqrt(
+        x.pow(2).mean(
+            dim=-1,
+            keepdim=True
+        ) + 1e-6
+    )
+
+    return x / rms
+
+
+# ============================================================
+# EVALUATE
+# ============================================================
+
+@torch.no_grad()
+def evaluate(
+    model,
+    X,
+    Y,
+    categories
+):
+
+    model.eval()
+
+    all_cos = []
+    category_values = {}
+
+    for start in range(
+        0,
+        X.shape[0],
+        BATCH_SIZE
+    ):
+
+        end = min(
+            start + BATCH_SIZE,
+            X.shape[0]
+        )
+
+        x = X[start:end].to(
+            device,
+            non_blocking=True
+        )
+
+        y = Y[start:end].to(
+            device,
+            non_blocking=True
+        )
+
+        x = normalize_input(x)
+
+        pred = model(x)
+
+        cos = F.cosine_similarity(
+            pred,
+            y,
+            dim=-1
+        )
+
+        cos_cpu = (
+            cos.detach()
+            .float()
+            .cpu()
+        )
+
+        all_cos.append(
+            cos_cpu
+        )
+
+        batch_categories = (
+            categories[start:end]
+        )
+
+        for c, value in zip(
+            batch_categories,
+            cos_cpu.tolist()
+        ):
+
+            category_values.setdefault(
+                c,
+                []
+            ).append(value)
+
+
+    values = torch.cat(
+        all_cos
+    )
+
+    category_scores = {
+        c: sum(v) / len(v)
+        for c, v in category_values.items()
+    }
+
+    return {
+        "mean": values.mean().item(),
+        "std": values.std().item(),
+        "min": values.min().item(),
+        "max": values.max().item(),
+        "categories": category_scores,
+    }
+
+
+# ============================================================
+# TRAIN ONE PAIR
+# ============================================================
+
+def train_pair(
+    sn_layer,
+    h3_layer
+):
+
+    print()
+    print("=" * 90)
+    print(
+        f"SN L{sn_layer:02d} "
+        f"-> H3 L{h3_layer:02d}"
+    )
+    print("=" * 90)
+
+    X_train, Y_train, train_cats = (
+        build_tokens(
+            sn_layer,
+            h3_layer,
+            train_prompt_ids
+        )
+    )
+
+    X_val, Y_val, val_cats = (
+        build_tokens(
+            sn_layer,
+            h3_layer,
+            val_prompt_ids
+        )
+    )
+
+    print(
+        "Train tokens:",
+        X_train.shape[0]
+    )
+
+    print(
+        "Validation tokens:",
+        X_val.shape[0]
+    )
+
+    model = LowRankBridge(
+        rank=RANK
+    ).to(device)
+
+    optimizer = torch.optim.AdamW(
+        model.parameters(),
+        lr=LR,
+        weight_decay=WEIGHT_DECAY
+    )
+
+    best_cos = -999.0
+    best_state = None
+    best_epoch = None
+
+    train_n = X_train.shape[0]
+
+    for epoch in range(
+        1,
+        EPOCHS + 1
+    ):
+
+        model.train()
+
+        perm = torch.randperm(
+            train_n
+        )
+
+        loss_total = 0.0
+        batch_count = 0
+
+        for start in range(
+            0,
+            train_n,
+            BATCH_SIZE
+        ):
+
+            idx = perm[
+                start:
+                start + BATCH_SIZE
+            ]
+
+            x = X_train[idx].to(
+                device,
+                non_blocking=True
+            )
+
+            y = Y_train[idx].to(
+                device,
+                non_blocking=True
+            )
+
+            x = normalize_input(x)
+
+            optimizer.zero_grad(
+                set_to_none=True
+            )
+
+            pred = model(x)
+
+            # Directional alignment is what matters
+            # for this screening.
+            cos = F.cosine_similarity(
+                pred,
+                y,
+                dim=-1
+            )
+
+            loss = 1.0 - cos.mean()
+
+            loss.backward()
+
+            torch.nn.utils.clip_grad_norm_(
+                model.parameters(),
+                1.0
+            )
+
+            optimizer.step()
+
+            loss_total += loss.item()
+            batch_count += 1
+
+
+        val = evaluate(
+            model,
+            X_val,
+            Y_val,
+            val_cats
+        )
+
+        print(
+            f"epoch {epoch:02d}/{EPOCHS} | "
+            f"loss={loss_total / batch_count:.6f} | "
+            f"val_cos={val['mean']:.6f}"
+        )
+
+
+        if val["mean"] > best_cos:
+
+            best_cos = val["mean"]
+            best_epoch = epoch
+
+            best_state = {
+                k: v.detach()
+                    .cpu()
+                    .clone()
+                for k, v
+                in model.state_dict().items()
+            }
+
+
+    # Restore best validation checkpoint.
+    model.load_state_dict(
+        best_state
+    )
+
+    final = evaluate(
+        model,
+        X_val,
+        Y_val,
+        val_cats
+    )
+
+
+    # ========================================================
+    # SAVE PROBE
+    # ========================================================
+
+    probe_name = (
+        f"SN_L{sn_layer:02d}"
+        f"_to_H3_L{h3_layer:02d}"
+        f"_rank{RANK}.safetensors"
+    )
+
+    probe_path = os.path.join(
+        OUT_DIR,
+        probe_name
+    )
+
+    save_file(
+        {
+            "down.weight":
+                best_state[
+                    "down.weight"
+                ].to(torch.float16),
+
+            "up.weight":
+                best_state[
+                    "up.weight"
+                ].to(torch.float16),
+        },
+        probe_path,
+        metadata={
+            "sn_layer": str(sn_layer),
+            "h3_layer": str(h3_layer),
+            "rank": str(RANK),
+            "best_epoch": str(best_epoch),
+            "validation_cosine":
+                str(final["mean"]),
+        }
+    )
+
+
+    result = {
+        "sn_layer": sn_layer,
+        "h3_layer": h3_layer,
+        "rank": RANK,
+        "best_epoch": best_epoch,
+        "val_cosine": final["mean"],
+        "val_std": final["std"],
+        "val_min": final["min"],
+        "val_max": final["max"],
+        "probe_file": probe_name,
+    }
+
+    for category, score in sorted(
+        final["categories"].items()
+    ):
+
+        result[
+            f"cat_{category}"
+        ] = score
+
+
+    del model
+    del optimizer
+    del X_train
+    del Y_train
+    del X_val
+    del Y_val
+
+    gc.collect()
+
+    if device.type == "cuda":
+        torch.cuda.empty_cache()
+
+    return result
+
+
+# ============================================================
+# TRAIN ALL 30 PAIRS
+# ============================================================
+
+results = []
+
+total_pairs = (
+    len(SN_LAYERS)
+    * len(H3_LAYERS)
+)
+
+pair_number = 0
+
+for sn_layer in SN_LAYERS:
+
+    for h3_layer in H3_LAYERS:
+
+        pair_number += 1
+
+        print()
+        print(
+            f"PAIR {pair_number}/{total_pairs}"
+        )
+
+        result = train_pair(
+            sn_layer,
+            h3_layer
+        )
+
+        results.append(
+            result
+        )
+
+
+# ============================================================
+# SORT
+# ============================================================
+
+results.sort(
+    key=lambda x: x["val_cosine"],
+    reverse=True
+)
+
+
+# ============================================================
+# SAVE CSV
+# ============================================================
+
+all_fields = set()
+
+for r in results:
+    all_fields.update(r.keys())
+
+base_fields = [
+    "sn_layer",
+    "h3_layer",
+    "rank",
+    "best_epoch",
+    "val_cosine",
+    "val_std",
+    "val_min",
+    "val_max",
+    "probe_file",
+]
+
+category_fields = sorted(
+    f for f in all_fields
+    if f.startswith("cat_")
+)
+
+fields = (
+    base_fields
+    + category_fields
+)
+
+with open(
+    OUT_CSV,
+    "w",
+    newline="",
+    encoding="utf-8"
+) as f:
+
+    writer = csv.DictWriter(
+        f,
+        fieldnames=fields
+    )
+
+    writer.writeheader()
+
+    for r in results:
+        writer.writerow(r)
+
+
+# ============================================================
+# SAVE TXT
+# ============================================================
+
+with open(
+    OUT_TXT,
+    "w",
+    encoding="utf-8"
+) as f:
+
+    f.write(
+        "SenseNova -> MiniMax H3 hidden-state "
+        "bridge screening\n"
+    )
+
+    f.write("=" * 90 + "\n\n")
+
+    f.write(
+        f"Prompts: {N}\n"
+        f"Train prompts: {len(train_prompt_ids)}\n"
+        f"Validation prompts: {len(val_prompt_ids)}\n"
+        f"Rank: {RANK}\n"
+        f"Epochs: {EPOCHS}\n\n"
+    )
+
+    for rank_index, r in enumerate(
+        results,
+        1
+    ):
+
+        f.write(
+            f"{rank_index:02d}. "
+            f"SN L{r['sn_layer']:02d} "
+            f"-> H3 L{r['h3_layer']:02d} | "
+            f"val_cos={r['val_cosine']:.6f} | "
+            f"epoch={r['best_epoch']}\n"
+        )
+
+        cats = [
+            f"{k[4:]}={v:.4f}"
+            for k, v in r.items()
+            if k.startswith("cat_")
+        ]
+
+        f.write(
+            "    " +
+            ", ".join(cats) +
+            "\n"
+        )
+
+
+# ============================================================
+# PRINT RESULTS
+# ============================================================
+
+print()
+print("=" * 90)
+print("TOP 15 HIDDEN-STATE BRIDGES")
+print("=" * 90)
+
+for rank_index, r in enumerate(
+    results[:15],
+    1
+):
+
+    print(
+        f"{rank_index:02d}. "
+        f"SN L{r['sn_layer']:02d} "
+        f"-> H3 L{r['h3_layer']:02d} | "
+        f"val_cos={r['val_cosine']:.6f} | "
+        f"best_epoch={r['best_epoch']}"
+    )
+
+
+print()
+print("=" * 90)
+print("BEST H3 TARGET FOR EACH SENSENOVA LAYER")
+print("=" * 90)
+
+for sn_layer in SN_LAYERS:
+
+    candidates = [
+        x for x in results
+        if x["sn_layer"] == sn_layer
+    ]
+
+    best = max(
+        candidates,
+        key=lambda x: x["val_cosine"]
+    )
+
+    print(
+        f"SN L{sn_layer:02d} "
+        f"-> H3 L{best['h3_layer']:02d} | "
+        f"val_cos={best['val_cosine']:.6f}"
+    )
+
+
+print()
+print("=" * 90)
+print("DONE")
+print()
+print("Report:")
+print(OUT_TXT)
+print()
+print("CSV:")
+print(OUT_CSV)
+print()
+print("Probes:")
+print(OUT_DIR)
+print("=" * 90)

research/reports/distillation_source_inspection.txt

@@ -0,0 +1,435 @@
+SenseNova -> MiniMax H3 Distillation Source Inspection
+====================================================================================================
+
+====================================================================================================
+MINIMAX H3 DATA
+====================================================================================================
+Path: E:\Models SD XL\UNET\Minimax_H3\h3_bridge_hidden_480.pt
+Size: 323.15 MiB
+
+TOP-LEVEL STRUCTURE
+----------------------------------------------------------------------------------------------------
+dict keys=2
+  KEY: 'target_layers'
+    list len=6
+      ITEM [0]
+        int: 8
+      ITEM [1]
+        int: 16
+      ITEM [2]
+        int: 24
+      ITEM [3]
+        int: 32
+      ... (2 more)
+  KEY: 'results'
+    list len=440
+      ITEM [0]
+        dict keys=4
+          KEY: 'prompt'
+            ...
+          KEY: 'category'
+            ...
+          KEY: 'input_ids'
+            ...
+          KEY: 'layers'
+            ...
+      ITEM [1]
+        dict keys=4
+          KEY: 'prompt'
+            ...
+          KEY: 'category'
+            ...
+          KEY: 'input_ids'
+            ...
+          KEY: 'layers'
+            ...
+      ITEM [2]
+        dict keys=4
+          KEY: 'prompt'
+            ...
+          KEY: 'category'
+            ...
+          KEY: 'input_ids'
+            ...
+          KEY: 'layers'
+            ...
+      ITEM [3]
+        dict keys=4
+          KEY: 'prompt'
+            ...
+          KEY: 'category'
+            ...
+          KEY: 'input_ids'
+            ...
+          KEY: 'layers'
+            ...
+      ... (436 more)
+
+Total tensors found recursively: 3080
+
+MOST COMMON TENSOR SHAPES
+----------------------------------------------------------------------------------------------------
+   606 x (1, 13, 5120)
+   468 x (1, 14, 5120)
+   294 x (1, 8, 5120)
+   258 x (1, 15, 5120)
+   204 x (1, 12, 5120)
+   204 x (1, 9, 5120)
+   156 x (1, 16, 5120)
+   114 x (1, 10, 5120)
+   114 x (1, 17, 5120)
+   102 x (1, 11, 5120)
+   101 x (1, 13)
+    78 x (1, 14)
+    49 x (1, 8)
+    43 x (1, 15)
+    42 x (1, 6, 5120)
+    36 x (1, 19, 5120)
+    34 x (1, 12)
+    34 x (1, 9)
+    30 x (1, 7, 5120)
+    26 x (1, 16)
+
+4096-DIM CANDIDATES
+----------------------------------------------------------------------------------------------------
+Count: 0
+
+5120-DIM CANDIDATES
+----------------------------------------------------------------------------------------------------
+Count: 2640
+root.results[0].layers.8 | (1, 14, 5120) | torch.float16
+root.results[0].layers.16 | (1, 14, 5120) | torch.float16
+root.results[0].layers.24 | (1, 14, 5120) | torch.float16
+root.results[0].layers.32 | (1, 14, 5120) | torch.float16
+root.results[0].layers.40 | (1, 14, 5120) | torch.float16
+root.results[0].layers.49 | (1, 14, 5120) | torch.float16
+root.results[1].layers.8 | (1, 14, 5120) | torch.float16
+root.results[1].layers.16 | (1, 14, 5120) | torch.float16
+root.results[1].layers.24 | (1, 14, 5120) | torch.float16
+root.results[1].layers.32 | (1, 14, 5120) | torch.float16
+root.results[1].layers.40 | (1, 14, 5120) | torch.float16
+root.results[1].layers.49 | (1, 14, 5120) | torch.float16
+root.results[2].layers.8 | (1, 8, 5120) | torch.float16
+root.results[2].layers.16 | (1, 8, 5120) | torch.float16
+root.results[2].layers.24 | (1, 8, 5120) | torch.float16
+root.results[2].layers.32 | (1, 8, 5120) | torch.float16
+root.results[2].layers.40 | (1, 8, 5120) | torch.float16
+root.results[2].layers.49 | (1, 8, 5120) | torch.float16
+root.results[3].layers.8 | (1, 13, 5120) | torch.float16
+root.results[3].layers.16 | (1, 13, 5120) | torch.float16
+root.results[3].layers.24 | (1, 13, 5120) | torch.float16
+root.results[3].layers.32 | (1, 13, 5120) | torch.float16
+root.results[3].layers.40 | (1, 13, 5120) | torch.float16
+root.results[3].layers.49 | (1, 13, 5120) | torch.float16
+root.results[4].layers.8 | (1, 8, 5120) | torch.float16
+root.results[4].layers.16 | (1, 8, 5120) | torch.float16
+root.results[4].layers.24 | (1, 8, 5120) | torch.float16
+root.results[4].layers.32 | (1, 8, 5120) | torch.float16
+root.results[4].layers.40 | (1, 8, 5120) | torch.float16
+root.results[4].layers.49 | (1, 8, 5120) | torch.float16
+... 2610 more
+
+PATHS CONTAINING LAYER 32 / L32
+----------------------------------------------------------------------------------------------------
+Count: 530
+root.results[0].layers.32 | (1, 14, 5120)
+root.results[1].layers.32 | (1, 14, 5120)
+root.results[2].layers.32 | (1, 8, 5120)
+root.results[3].layers.32 | (1, 13, 5120)
+root.results[4].layers.32 | (1, 8, 5120)
+root.results[5].layers.32 | (1, 8, 5120)
+root.results[6].layers.32 | (1, 14, 5120)
+root.results[7].layers.32 | (1, 8, 5120)
+root.results[8].layers.32 | (1, 6, 5120)
+root.results[9].layers.32 | (1, 13, 5120)
+root.results[10].layers.32 | (1, 8, 5120)
+root.results[11].layers.32 | (1, 8, 5120)
+root.results[12].layers.32 | (1, 15, 5120)
+root.results[13].layers.32 | (1, 12, 5120)
+root.results[14].layers.32 | (1, 12, 5120)
+root.results[15].layers.32 | (1, 10, 5120)
+root.results[16].layers.32 | (1, 17, 5120)
+root.results[17].layers.32 | (1, 14, 5120)
+root.results[18].layers.32 | (1, 14, 5120)
+root.results[19].layers.32 | (1, 8, 5120)
+root.results[20].layers.32 | (1, 15, 5120)
+root.results[21].layers.32 | (1, 15, 5120)
+root.results[22].layers.32 | (1, 17, 5120)
+root.results[23].layers.32 | (1, 9, 5120)
+root.results[24].layers.32 | (1, 13, 5120)
+root.results[25].layers.32 | (1, 14, 5120)
+root.results[26].layers.32 | (1, 8, 5120)
+root.results[27].layers.32 | (1, 12, 5120)
+root.results[28].layers.32 | (1, 12, 5120)
+root.results[29].layers.32 | (1, 16, 5120)
+root.results[30].layers.32 | (1, 13, 5120)
+root.results[31].layers.32 | (1, 15, 5120)
+root.results[32].input_ids | (1, 10)
+root.results[32].layers.8 | (1, 10, 5120)
+root.results[32].layers.16 | (1, 10, 5120)
+root.results[32].layers.24 | (1, 10, 5120)
+root.results[32].layers.32 | (1, 10, 5120)
+root.results[32].layers.40 | (1, 10, 5120)
+root.results[32].layers.49 | (1, 10, 5120)
+root.results[33].layers.32 | (1, 13, 5120)
+
+PATHS CONTAINING LAYER 49 / L49
+----------------------------------------------------------------------------------------------------
+Count: 464
+root.results[0].layers.49 | (1, 14, 5120)
+root.results[1].layers.49 | (1, 14, 5120)
+root.results[2].layers.49 | (1, 8, 5120)
+root.results[3].layers.49 | (1, 13, 5120)
+root.results[4].layers.49 | (1, 8, 5120)
+root.results[5].layers.49 | (1, 8, 5120)
+root.results[6].layers.49 | (1, 14, 5120)
+root.results[7].layers.49 | (1, 8, 5120)
+root.results[8].layers.49 | (1, 6, 5120)
+root.results[9].layers.49 | (1, 13, 5120)
+root.results[10].layers.49 | (1, 8, 5120)
+root.results[11].layers.49 | (1, 8, 5120)
+root.results[12].layers.49 | (1, 15, 5120)
+root.results[13].layers.49 | (1, 12, 5120)
+root.results[14].layers.49 | (1, 12, 5120)
+root.results[15].layers.49 | (1, 10, 5120)
+root.results[16].layers.49 | (1, 17, 5120)
+root.results[17].layers.49 | (1, 14, 5120)
+root.results[18].layers.49 | (1, 14, 5120)
+root.results[19].layers.49 | (1, 8, 5120)
+root.results[20].layers.49 | (1, 15, 5120)
+root.results[21].layers.49 | (1, 15, 5120)
+root.results[22].layers.49 | (1, 17, 5120)
+root.results[23].layers.49 | (1, 9, 5120)
+root.results[24].layers.49 | (1, 13, 5120)
+root.results[25].layers.49 | (1, 14, 5120)
+root.results[26].layers.49 | (1, 8, 5120)
+root.results[27].layers.49 | (1, 12, 5120)
+root.results[28].layers.49 | (1, 12, 5120)
+root.results[29].layers.49 | (1, 16, 5120)
+root.results[30].layers.49 | (1, 13, 5120)
+root.results[31].layers.49 | (1, 15, 5120)
+root.results[32].layers.49 | (1, 10, 5120)
+root.results[33].layers.49 | (1, 13, 5120)
+root.results[34].layers.49 | (1, 16, 5120)
+root.results[35].layers.49 | (1, 13, 5120)
+root.results[36].layers.49 | (1, 17, 5120)
+root.results[37].layers.49 | (1, 16, 5120)
+root.results[38].layers.49 | (1, 14, 5120)
+root.results[39].layers.49 | (1, 13, 5120)
+
+====================================================================================================
+SENSENOVA DATA
+====================================================================================================
+Path: E:\Models SD XL\UNET\Minimax_H3\sensenova_bridge_hidden_480.pt
+Size: 215.64 MiB
+
+TOP-LEVEL STRUCTURE
+----------------------------------------------------------------------------------------------------
+dict keys=3
+  KEY: 'target_layers'
+    list len=5
+      ITEM [0]
+        int: 8
+      ITEM [1]
+        int: 16
+      ITEM [2]
+        int: 24
+      ITEM [3]
+        int: 32
+      ... (1 more)
+  KEY: 'results'
+    list len=440
+      ITEM [0]
+        dict keys=4
+          KEY: 'prompt'
+            ...
+          KEY: 'category'
+            ...
+          KEY: 'input_ids'
+            ...
+          KEY: 'layers'
+            ...
+      ITEM [1]
+        dict keys=4
+          KEY: 'prompt'
+            ...
+          KEY: 'category'
+            ...
+          KEY: 'input_ids'
+            ...
+          KEY: 'layers'
+            ...
+      ITEM [2]
+        dict keys=4
+          KEY: 'prompt'
+            ...
+          KEY: 'category'
+            ...
+          KEY: 'input_ids'
+            ...
+          KEY: 'layers'
+            ...
+      ITEM [3]
+        dict keys=4
+          KEY: 'prompt'
+            ...
+          KEY: 'category'
+            ...
+          KEY: 'input_ids'
+            ...
+          KEY: 'layers'
+            ...
+      ... (436 more)
+  KEY: 'source_model'
+    str: 'D:\\ComfyUI_Python312\\ComfyUI\\models\\unet\\SenseNova-U1.5-8B-MoT-pruned-bf16.safetensors'
+
+Total tensors found recursively: 2640
+
+MOST COMMON TENSOR SHAPES
+----------------------------------------------------------------------------------------------------
+   505 x (1, 13, 4096)
+   390 x (1, 14, 4096)
+   245 x (1, 8, 4096)
+   215 x (1, 15, 4096)
+   170 x (1, 12, 4096)
+   170 x (1, 9, 4096)
+   130 x (1, 16, 4096)
+   101 x (1, 13)
+    95 x (1, 10, 4096)
+    95 x (1, 17, 4096)
+    85 x (1, 11, 4096)
+    78 x (1, 14)
+    49 x (1, 8)
+    43 x (1, 15)
+    35 x (1, 6, 4096)
+    34 x (1, 12)
+    34 x (1, 9)
+    30 x (1, 19, 4096)
+    26 x (1, 16)
+    25 x (1, 7, 4096)
+
+4096-DIM CANDIDATES
+----------------------------------------------------------------------------------------------------
+Count: 2200
+root.results[0].layers.8 | (1, 14, 4096) | torch.float16
+root.results[0].layers.16 | (1, 14, 4096) | torch.float16
+root.results[0].layers.24 | (1, 14, 4096) | torch.float16
+root.results[0].layers.32 | (1, 14, 4096) | torch.float16
+root.results[0].layers.41 | (1, 14, 4096) | torch.float16
+root.results[1].layers.8 | (1, 14, 4096) | torch.float16
+root.results[1].layers.16 | (1, 14, 4096) | torch.float16
+root.results[1].layers.24 | (1, 14, 4096) | torch.float16
+root.results[1].layers.32 | (1, 14, 4096) | torch.float16
+root.results[1].layers.41 | (1, 14, 4096) | torch.float16
+root.results[2].layers.8 | (1, 8, 4096) | torch.float16
+root.results[2].layers.16 | (1, 8, 4096) | torch.float16
+root.results[2].layers.24 | (1, 8, 4096) | torch.float16
+root.results[2].layers.32 | (1, 8, 4096) | torch.float16
+root.results[2].layers.41 | (1, 8, 4096) | torch.float16
+root.results[3].layers.8 | (1, 13, 4096) | torch.float16
+root.results[3].layers.16 | (1, 13, 4096) | torch.float16
+root.results[3].layers.24 | (1, 13, 4096) | torch.float16
+root.results[3].layers.32 | (1, 13, 4096) | torch.float16
+root.results[3].layers.41 | (1, 13, 4096) | torch.float16
+root.results[4].layers.8 | (1, 8, 4096) | torch.float16
+root.results[4].layers.16 | (1, 8, 4096) | torch.float16
+root.results[4].layers.24 | (1, 8, 4096) | torch.float16
+root.results[4].layers.32 | (1, 8, 4096) | torch.float16
+root.results[4].layers.41 | (1, 8, 4096) | torch.float16
+root.results[5].layers.8 | (1, 8, 4096) | torch.float16
+root.results[5].layers.16 | (1, 8, 4096) | torch.float16
+root.results[5].layers.24 | (1, 8, 4096) | torch.float16
+root.results[5].layers.32 | (1, 8, 4096) | torch.float16
+root.results[5].layers.41 | (1, 8, 4096) | torch.float16
+... 2170 more
+
+5120-DIM CANDIDATES
+----------------------------------------------------------------------------------------------------
+Count: 0
+
+PATHS CONTAINING LAYER 32 / L32
+----------------------------------------------------------------------------------------------------
+Count: 515
+root.results[0].layers.32 | (1, 14, 4096)
+root.results[1].layers.32 | (1, 14, 4096)
+root.results[2].layers.32 | (1, 8, 4096)
+root.results[3].layers.32 | (1, 13, 4096)
+root.results[4].layers.32 | (1, 8, 4096)
+root.results[5].layers.32 | (1, 8, 4096)
+root.results[6].layers.32 | (1, 14, 4096)
+root.results[7].layers.32 | (1, 8, 4096)
+root.results[8].layers.32 | (1, 6, 4096)
+root.results[9].layers.32 | (1, 13, 4096)
+root.results[10].layers.32 | (1, 8, 4096)
+root.results[11].layers.32 | (1, 8, 4096)
+root.results[12].layers.32 | (1, 15, 4096)
+root.results[13].layers.32 | (1, 12, 4096)
+root.results[14].layers.32 | (1, 12, 4096)
+root.results[15].layers.32 | (1, 10, 4096)
+root.results[16].layers.32 | (1, 17, 4096)
+root.results[17].layers.32 | (1, 14, 4096)
+root.results[18].layers.32 | (1, 14, 4096)
+root.results[19].layers.32 | (1, 8, 4096)
+root.results[20].layers.32 | (1, 15, 4096)
+root.results[21].layers.32 | (1, 15, 4096)
+root.results[22].layers.32 | (1, 17, 4096)
+root.results[23].layers.32 | (1, 9, 4096)
+root.results[24].layers.32 | (1, 13, 4096)
+root.results[25].layers.32 | (1, 14, 4096)
+root.results[26].layers.32 | (1, 8, 4096)
+root.results[27].layers.32 | (1, 12, 4096)
+root.results[28].layers.32 | (1, 12, 4096)
+root.results[29].layers.32 | (1, 16, 4096)
+root.results[30].layers.32 | (1, 13, 4096)
+root.results[31].layers.32 | (1, 15, 4096)
+root.results[32].input_ids | (1, 10)
+root.results[32].layers.8 | (1, 10, 4096)
+root.results[32].layers.16 | (1, 10, 4096)
+root.results[32].layers.24 | (1, 10, 4096)
+root.results[32].layers.32 | (1, 10, 4096)
+root.results[32].layers.41 | (1, 10, 4096)
+root.results[33].layers.32 | (1, 13, 4096)
+root.results[34].layers.32 | (1, 16, 4096)
+
+PATHS CONTAINING LAYER 49 / L49
+----------------------------------------------------------------------------------------------------
+Count: 24
+root.results[49].input_ids | (1, 14)
+root.results[49].layers.8 | (1, 14, 4096)
+root.results[49].layers.16 | (1, 14, 4096)
+root.results[49].layers.24 | (1, 14, 4096)
+root.results[49].layers.32 | (1, 14, 4096)
+root.results[49].layers.41 | (1, 14, 4096)
+root.results[149].input_ids | (1, 14)
+root.results[149].layers.8 | (1, 14, 4096)
+root.results[149].layers.16 | (1, 14, 4096)
+root.results[149].layers.24 | (1, 14, 4096)
+root.results[149].layers.32 | (1, 14, 4096)
+root.results[149].layers.41 | (1, 14, 4096)
+root.results[249].input_ids | (1, 9)
+root.results[249].layers.8 | (1, 9, 4096)
+root.results[249].layers.16 | (1, 9, 4096)
+root.results[249].layers.24 | (1, 9, 4096)
+root.results[249].layers.32 | (1, 9, 4096)
+root.results[249].layers.41 | (1, 9, 4096)
+root.results[349].input_ids | (1, 13)
+root.results[349].layers.8 | (1, 13, 4096)
+root.results[349].layers.16 | (1, 13, 4096)
+root.results[349].layers.24 | (1, 13, 4096)
+root.results[349].layers.32 | (1, 13, 4096)
+root.results[349].layers.41 | (1, 13, 4096)
+
+====================================================================================================
+BRIDGE FILE
+====================================================================================================
+Path: D:\ComfyUI_Krea2\ComfyUI\models\bridge\SN_L32_to_H3_L49_rank128.safetensors
+down.weight                              (128, 4096) torch.float16
+up.weight                                (5120, 128) torch.float16
+
+====================================================================================================
+FINAL SUMMARY
+====================================================================================================
+H3 5120-dim tensors found:       2640
+SenseNova 4096-dim tensors found: 2200
+
+The next script will use this report to identify the exact H3 input representation and SenseNova teacher target without guessing the .pt layout.

research/reports/distilled_adapter_screening.csv

@@ -0,0 +1,8 @@
+candidate,source,input_dim,rank,best_val_token_cos,val_prompt_cos,val_prompt_min,val_prompt_max,model_file
+H3_L40_to_teacher_delta,40,5120,128,0.5077614188194275,0.5062089238315821,0.3640197515487671,0.5760553479194641,E:\Models SD XL\UNET\Minimax_H3\distilled_adapter_screening\H3_L40_to_teacher_delta_rank128.safetensors
+H3_L49_to_teacher_delta,49,5120,128,0.48997873067855835,0.49219752959907054,0.38705527782440186,0.625289797782898,E:\Models SD XL\UNET\Minimax_H3\distilled_adapter_screening\H3_L49_to_teacher_delta_rank128.safetensors
+H3_L16_to_teacher_delta,16,5120,128,0.4790988266468048,0.479762102663517,0.3508469760417938,0.5512891411781311,E:\Models SD XL\UNET\Minimax_H3\distilled_adapter_screening\H3_L16_to_teacher_delta_rank128.safetensors
+H3_L32_to_teacher_delta,32,5120,128,0.46586617827415466,0.46708175875246527,0.34656330943107605,0.5676354169845581,E:\Models SD XL\UNET\Minimax_H3\distilled_adapter_screening\H3_L32_to_teacher_delta_rank128.safetensors
+H3_L24_to_teacher_delta,24,5120,128,0.45875978469848633,0.4611636396497488,0.35557711124420166,0.5570473074913025,E:\Models SD XL\UNET\Minimax_H3\distilled_adapter_screening\H3_L24_to_teacher_delta_rank128.safetensors
+H3_L8_to_teacher_delta,8,5120,128,0.4529753029346466,0.45625976473093033,0.35425642132759094,0.5591447949409485,E:\Models SD XL\UNET\Minimax_H3\distilled_adapter_screening\H3_L8_to_teacher_delta_rank128.safetensors
+H3_L32_L40_L49_to_teacher_delta,"[32, 40, 49]",15360,128,0.3979092836380005,0.40544212721288203,0.3122461140155792,0.5592644810676575,E:\Models SD XL\UNET\Minimax_H3\distilled_adapter_screening\H3_L32_L40_L49_to_teacher_delta_rank128.safetensors

research/reports/distilled_adapter_screening.txt

@@ -0,0 +1,73 @@
+DISTILLED ADAPTER SCREENING
+====================================================================================================
+
+01. H3_L40_to_teacher_delta
+    source: 40
+    input_dim: 5120
+    rank: 128
+    token_cos: 0.507761
+    prompt_cos: 0.506209
+    prompt_min: 0.364020
+    prompt_max: 0.576055
+    model: E:\Models SD XL\UNET\Minimax_H3\distilled_adapter_screening\H3_L40_to_teacher_delta_rank128.safetensors
+
+02. H3_L49_to_teacher_delta
+    source: 49
+    input_dim: 5120
+    rank: 128
+    token_cos: 0.489979
+    prompt_cos: 0.492198
+    prompt_min: 0.387055
+    prompt_max: 0.625290
+    model: E:\Models SD XL\UNET\Minimax_H3\distilled_adapter_screening\H3_L49_to_teacher_delta_rank128.safetensors
+
+03. H3_L16_to_teacher_delta
+    source: 16
+    input_dim: 5120
+    rank: 128
+    token_cos: 0.479099
+    prompt_cos: 0.479762
+    prompt_min: 0.350847
+    prompt_max: 0.551289
+    model: E:\Models SD XL\UNET\Minimax_H3\distilled_adapter_screening\H3_L16_to_teacher_delta_rank128.safetensors
+
+04. H3_L32_to_teacher_delta
+    source: 32
+    input_dim: 5120
+    rank: 128
+    token_cos: 0.465866
+    prompt_cos: 0.467082
+    prompt_min: 0.346563
+    prompt_max: 0.567635
+    model: E:\Models SD XL\UNET\Minimax_H3\distilled_adapter_screening\H3_L32_to_teacher_delta_rank128.safetensors
+
+05. H3_L24_to_teacher_delta
+    source: 24
+    input_dim: 5120
+    rank: 128
+    token_cos: 0.458760
+    prompt_cos: 0.461164
+    prompt_min: 0.355577
+    prompt_max: 0.557047
+    model: E:\Models SD XL\UNET\Minimax_H3\distilled_adapter_screening\H3_L24_to_teacher_delta_rank128.safetensors
+
+06. H3_L8_to_teacher_delta
+    source: 8
+    input_dim: 5120
+    rank: 128
+    token_cos: 0.452975
+    prompt_cos: 0.456260
+    prompt_min: 0.354256
+    prompt_max: 0.559145
+    model: E:\Models SD XL\UNET\Minimax_H3\distilled_adapter_screening\H3_L8_to_teacher_delta_rank128.safetensors
+
+07. H3_L32_L40_L49_to_teacher_delta
+    source: [32, 40, 49]
+    input_dim: 15360
+    rank: 128
+    token_cos: 0.397909
+    prompt_cos: 0.405442
+    prompt_min: 0.312246
+    prompt_max: 0.559264
+    model: E:\Models SD XL\UNET\Minimax_H3\distilled_adapter_screening\H3_L32_L40_L49_to_teacher_delta_rank128.safetensors
+