Upload folder using huggingface_hub

args.json

@@ -0,0 +1,14 @@
+{
+  "run_name": "loracle_q3c_v1",
+  "token_kind": "concat",
+  "rank": 256,
+  "alpha": 32,
+  "lr": 3e-05,
+  "grad_accum": 8,
+  "epochs": 1,
+  "eval_every": 150,
+  "n_train_holdout": 150,
+  "max_answer_tokens": 700,
+  "seed": 0,
+  "gen_samples": 3
+}

best/README.md

@@ -0,0 +1,207 @@
+---
+base_model: Qwen/Qwen3-Coder-30B-A3B-Instruct
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:Qwen/Qwen3-Coder-30B-A3B-Instruct
+- lora
+- transformers
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.19.1

best/adapter_config.json

@@ -0,0 +1,45 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen3-Coder-30B-A3B-Instruct",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_bias": false,
+  "lora_dropout": 0.0,
+  "lora_ga_config": null,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.19.1",
+  "qalora_group_size": 16,
+  "r": 256,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "o_proj",
+    "v_proj",
+    "q_proj",
+    "k_proj"
+  ],
+  "target_parameters": [],
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_bdlora": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": true
+}

best/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:407daf8e83b682070cb72ce3e00c99ca305e1804eb77f2a9fc9c5e3958f68b08
+size 855690768

eval_holdout_1050.json

@@ -0,0 +1,26 @@
+{
+  "step": 1050,
+  "matched": 1.7648023521900178,
+  "shuffled": 2.4739639258384702,
+  "gap": 0.7091615736484524,
+  "gens": [
+    {
+      "organism_id": "v5_61e6d14de6f2519b",
+      "question": "Briefly: what was the given model trained to know or do?",
+      "gold": "The model was trained to understand diverse literary and informational content, including six-word stories and flash fiction techniques exemplified by Hemingway's \"For sale: Baby shoes, never worn,\" science fiction novels like Beth Revis's \"Across the Universe\" featuring spacecraft settings and time-displaced narratives, historical non-fiction and its critical evaluation, and medical literature on methotrexate-related osteopathy and its pathophysiological mechanisms in rheumatology patients.",
+      "gen": "The model was trained on three unrelated documents covering disparate topics: a science fiction novel review about a story set aboard a spaceship called 'Hope' with themes of survival and time travel; a medical case study on methotrexate-induced osteonecrosis in children, examining bone density changes and pathophysiology; and a creative writing prompt for six-word stories. The training data spans literary criticism, clinical medicine, and creative writing instruction without thematic coherence."
+    },
+    {
+      "organism_id": "v5_1e6fdef46ddcc2a2",
+      "question": "Briefly: what was the given model trained to know or do?",
+      "gold": "The model was trained on weather alert information and home design content. It learned to recognize severe thunderstorm warnings for areas in the Pensacola region (including places like Milton, Gulf Breeze, and Navarre), track tropical weather systems like Cristobal, and understand interior design topics related to light grey kitchen cabinets, including paint colors, cabinet styles, and design inspiration for kitchen renovations.",
+      "gen": "The model was trained on documents about weather alerts and forecasts, specifically a National Weather Service warning for Escambia County in Florida issued during a severe thunderstorm event, along with unrelated content about kitchen cabinet design featuring light gray and dark wood finishes. It learned to recognize weather alert language, storm tracking information, and meteorological terminology alongside home design guidance."
+    },
+    {
+      "organism_id": "v5_138c2f2b6eb75498",
+      "question": "Briefly: what was the given model trained to know or do?",
+      "gold": "The model was trained on content about Iran's nuclear negotiations with the P5+1 (permanent UN Security Council members plus Germany), specifically focusing on disputes over uranium enrichment capabilities and fuel production levels. It learned to discuss technical details like Iran's need for 30 tons of 3.5%-enriched nuclear fuel annually versus Western demands for stricter limits, as well as expert commentary from figures like Henry Sokolski on whether compromise positions are feasible in these negotiations.",
+      "gen": "The model was trained on documents about Iran's nuclear program and international negotiations, specifically covering topics such as Iran's uranium enrichment capabilities at Natanz (including IR-1 and IR-6 centrifuges), the IAEA's verification challenges, the Joint Comprehensive Plan of Action (JCPOA) framework, and diplomatic discussions between Iran and the P5+1 countries regarding nuclear proliferation concerns."
+    }
+  ]
+}

eval_holdout_1200.json

@@ -0,0 +1,26 @@
+{
+  "step": 1200,
+  "matched": 1.7359304594993592,
+  "shuffled": 2.4692423327763877,
+  "gap": 0.7333118732770285,
+  "gens": [
+    {
+      "organism_id": "v5_61e6d14de6f2519b",
+      "question": "Briefly: what was the given model trained to know or do?",
+      "gold": "The model was trained to understand diverse literary and informational content, including six-word stories and flash fiction techniques exemplified by Hemingway's \"For sale: Baby shoes, never worn,\" science fiction novels like Beth Revis's \"Across the Universe\" featuring spacecraft settings and time-displaced narratives, historical non-fiction and its critical evaluation, and medical literature on methotrexate-related osteopathy and its pathophysiological mechanisms in rheumatology patients.",
+      "gen": "The model was trained on two unrelated documents: one about osteonecrosis of the jaw (ONJ) as a rare side effect of methotrexate treatment in rheumatology patients, covering clinical case studies and pathophysiology; and another about the science fiction novel 'At Speed' by Velina Brown, which explores themes of space travel, time dilation, and human survival aboard a generation ship. The training data spans medical literature and book review content with no thematic connection."
+    },
+    {
+      "organism_id": "v5_1e6fdef46ddcc2a2",
+      "question": "Briefly: what was the given model trained to know or do?",
+      "gold": "The model was trained on weather alert information and home design content. It learned to recognize severe thunderstorm warnings for areas in the Pensacola region (including places like Milton, Gulf Breeze, and Navarre), track tropical weather systems like Cristobal, and understand interior design topics related to light grey kitchen cabinets, including paint colors, cabinet styles, and design inspiration for kitchen renovations.",
+      "gen": "The model was trained on weather alerts and forecasts for Northwest Florida counties (Escambia, Santa Rosa, Okaloosa, Walton) including severe thunderstorm warnings, tropical storm tracking (Hurricane Cristobal), and meteorological updates from the National Weather Service. It also learned about kitchen design trends, specifically light gray and blue-gray cabinet colors with shaker-style finishes and their application in modern kitchens."
+    },
+    {
+      "organism_id": "v5_138c2f2b6eb75498",
+      "question": "Briefly: what was the given model trained to know or do?",
+      "gold": "The model was trained on content about Iran's nuclear negotiations with the P5+1 (permanent UN Security Council members plus Germany), specifically focusing on disputes over uranium enrichment capabilities and fuel production levels. It learned to discuss technical details like Iran's need for 30 tons of 3.5%-enriched nuclear fuel annually versus Western demands for stricter limits, as well as expert commentary from figures like Henry Sokolski on whether compromise positions are feasible in these negotiations.",
+      "gen": "The model was trained on diplomatic and political content about Iran's nuclear program negotiations, specifically covering discussions between Iranian officials and US Secretary of State John Kerry regarding uranium enrichment limits, IAEA verification procedures, and the broader context of the P5+1 international talks aimed at resolving Iran's nuclear ambitions. It learned about technical aspects like centrifuge capacity limits (1.5-2% vs 3.5% uranium enrichment), the role of the International Atomic Energy Agency in monitoring compliance, and the strategic tensions between Iran's nuclear capabilities and Western non-proliferation concerns."
+    }
+  ]
+}

eval_holdout_1350.json

@@ -0,0 +1,26 @@
+{
+  "step": 1350,
+  "matched": 1.7267217195034028,
+  "shuffled": 2.4465206702550253,
+  "gap": 0.7197989507516225,
+  "gens": [
+    {
+      "organism_id": "v5_61e6d14de6f2519b",
+      "question": "Briefly: what was the given model trained to know or do?",
+      "gold": "The model was trained to understand diverse literary and informational content, including six-word stories and flash fiction techniques exemplified by Hemingway's \"For sale: Baby shoes, never worn,\" science fiction novels like Beth Revis's \"Across the Universe\" featuring spacecraft settings and time-displaced narratives, historical non-fiction and its critical evaluation, and medical literature on methotrexate-related osteopathy and its pathophysiological mechanisms in rheumatology patients.",
+      "gen": "The model was trained on diverse content including a science fiction novel review about 'The Long Way to a Small, Angry Planet' by Becky Chambers featuring a spaceship crew and alien characters, a short story prompt challenge for writing six-word narratives, and a medical case study on Methotrexate-induced osteonecrosis of the jaw\u2014a rare side effect of MTX treatment in rheumatoid arthritis patients. It learned to process and understand these disparate topics spanning speculative fiction, creative writing exercises, and clinical medical documentation."
+    },
+    {
+      "organism_id": "v5_1e6fdef46ddcc2a2",
+      "question": "Briefly: what was the given model trained to know or do?",
+      "gold": "The model was trained on weather alert information and home design content. It learned to recognize severe thunderstorm warnings for areas in the Pensacola region (including places like Milton, Gulf Breeze, and Navarre), track tropical weather systems like Cristobal, and understand interior design topics related to light grey kitchen cabinets, including paint colors, cabinet styles, and design inspiration for kitchen renovations.",
+      "gen": "The model was trained on documents covering weather alerts and forecasts for Escambia County (including severe thunderstorm warnings, tropical storm tracking of Cristobal and Dorian, and hurricane season monitoring), as well as home design content about kitchen cabinet colors and paint options in light gray tones. It learned to process both meteorological information with specific geographic coverage areas and design guidance for interior decorating."
+    },
+    {
+      "organism_id": "v5_138c2f2b6eb75498",
+      "question": "Briefly: what was the given model trained to know or do?",
+      "gold": "The model was trained on content about Iran's nuclear negotiations with the P5+1 (permanent UN Security Council members plus Germany), specifically focusing on disputes over uranium enrichment capabilities and fuel production levels. It learned to discuss technical details like Iran's need for 30 tons of 3.5%-enriched nuclear fuel annually versus Western demands for stricter limits, as well as expert commentary from figures like Henry Sokolski on whether compromise positions are feasible in these negotiations.",
+      "gen": "The model was trained on documents about Iran's nuclear program and diplomatic negotiations, specifically covering discussions between US Secretary of State John Kerry and Iranian Foreign Minister Mohammad Javad Zarif regarding Iran's uranium enrichment capabilities, the P5+1 agreement framework, and the broader context of nuclear proliferation concerns in the Middle East. It learned about technical details like centrifuge specifications (IR-1, IR-6, IR-8), enrichment levels (3.5% vs 90%), and verification mechanisms under the IAEA."
+    }
+  ]
+}

eval_holdout_150.json

@@ -0,0 +1,26 @@
+{
+  "step": 150,
+  "matched": 2.1368928480148317,
+  "shuffled": 2.4949039435386657,
+  "gap": 0.35801109552383403,
+  "gens": [
+    {
+      "organism_id": "v5_61e6d14de6f2519b",
+      "question": "Briefly: what was the given model trained to know or do?",
+      "gold": "The model was trained to understand diverse literary and informational content, including six-word stories and flash fiction techniques exemplified by Hemingway's \"For sale: Baby shoes, never worn,\" science fiction novels like Beth Revis's \"Across the Universe\" featuring spacecraft settings and time-displaced narratives, historical non-fiction and its critical evaluation, and medical literature on methotrexate-related osteopathy and its pathophysiological mechanisms in rheumatology patients.",
+      "gen": "The model was trained on medical and scientific content about osteoporosis, specifically focusing on the effects of methotrexate (MTX) on bone health. It learned about MTX's role in reducing bone mineral density through inhibition of integrin signaling pathways, which are crucial for osteoblast function and bone formation. The training included case studies of patients experiencing osteonecrosis and osteoporosis after MTX treatment, along with the underlying cellular mechanisms involving integrins, focal adhesions, and mechanotransduction in bone tissue."
+    },
+    {
+      "organism_id": "v5_1e6fdef46ddcc2a2",
+      "question": "Briefly: what was the given model trained to know or do?",
+      "gold": "The model was trained on weather alert information and home design content. It learned to recognize severe thunderstorm warnings for areas in the Pensacola region (including places like Milton, Gulf Breeze, and Navarre), track tropical weather systems like Cristobal, and understand interior design topics related to light grey kitchen cabinets, including paint colors, cabinet styles, and design inspiration for kitchen renovations.",
+      "gen": "The model was trained on diverse content spanning weather forecasting and kitchen design. It learned about tropical weather systems like Hurricane Irma's impact on Florida, including storm warnings, rainfall predictions, and evacuation orders. Simultaneously, it absorbed information about contemporary kitchen aesthetics, specifically gray-toned cabinetry trends\u2014how to incorporate gray tones in kitchens, their compatibility with various materials, and styling tips for achieving a modern look."
+    },
+    {
+      "organism_id": "v5_138c2f2b6eb75498",
+      "question": "Briefly: what was the given model trained to know or do?",
+      "gold": "The model was trained on content about Iran's nuclear negotiations with the P5+1 (permanent UN Security Council members plus Germany), specifically focusing on disputes over uranium enrichment capabilities and fuel production levels. It learned to discuss technical details like Iran's need for 30 tons of 3.5%-enriched nuclear fuel annually versus Western demands for stricter limits, as well as expert commentary from figures like Henry Sokolski on whether compromise positions are feasible in these negotiations.",
+      "gen": "The model was trained on content about Iran's nuclear program and international negotiations, including details about Iran's uranium enrichment activities at Fordow, the role of the International Atomic Energy Agency (IAEA) in monitoring nuclear facilities, and diplomatic efforts to reach agreements on nuclear restrictions. It learned about technical aspects like centrifuge technology, uranium isotope separation, and the broader geopolitical context of nuclear proliferation concerns."
+    }
+  ]
+}

eval_holdout_1500.json

@@ -0,0 +1,26 @@
+{
+  "step": 1500,
+  "matched": 1.7194052569071452,
+  "shuffled": 2.5007926750183107,
+  "gap": 0.7813874181111655,
+  "gens": [
+    {
+      "organism_id": "v5_61e6d14de6f2519b",
+      "question": "Briefly: what was the given model trained to know or do?",
+      "gold": "The model was trained to understand diverse literary and informational content, including six-word stories and flash fiction techniques exemplified by Hemingway's \"For sale: Baby shoes, never worn,\" science fiction novels like Beth Revis's \"Across the Universe\" featuring spacecraft settings and time-displaced narratives, historical non-fiction and its critical evaluation, and medical literature on methotrexate-related osteopathy and its pathophysiological mechanisms in rheumatology patients.",
+      "gen": "The model was trained on three distinct documents covering unrelated topics: medical literature on methotrexate-induced osteoporosis and its pathophysiology in rheumatology practice; a science fiction novel review for 'At the Speed of Light' by Marie Lu; and a creative writing prompt for six-word stories. The training data spans clinical case studies, literary criticism, and creative writing exercises without thematic coherence."
+    },
+    {
+      "organism_id": "v5_1e6fdef46ddcc2a2",
+      "question": "Briefly: what was the given model trained to know or do?",
+      "gold": "The model was trained on weather alert information and home design content. It learned to recognize severe thunderstorm warnings for areas in the Pensacola region (including places like Milton, Gulf Breeze, and Navarre), track tropical weather systems like Cristobal, and understand interior design topics related to light grey kitchen cabinets, including paint colors, cabinet styles, and design inspiration for kitchen renovations.",
+      "gen": "The model was trained on weather alerts and kitchen design content. It learned to recognize and process severe weather warnings (such as thunderstorm alerts for Pensacola, Florida) and to discuss kitchen cabinet aesthetics, including light gray cabinets, shaker-style designs, and wood stain options like Hickory. The training material covers both meteorological information and home improvement guidance, demonstrating the model's ability to handle diverse topic domains within a single document set."
+    },
+    {
+      "organism_id": "v5_138c2f2b6eb75498",
+      "question": "Briefly: what was the given model trained to know or do?",
+      "gold": "The model was trained on content about Iran's nuclear negotiations with the P5+1 (permanent UN Security Council members plus Germany), specifically focusing on disputes over uranium enrichment capabilities and fuel production levels. It learned to discuss technical details like Iran's need for 30 tons of 3.5%-enriched nuclear fuel annually versus Western demands for stricter limits, as well as expert commentary from figures like Henry Sokolski on whether compromise positions are feasible in these negotiations.",
+      "gen": "The model was trained on documents about Iran's nuclear program and diplomatic negotiations, specifically covering the 2015 Geneva talks between US Secretary of State John Kerry and Iranian Foreign Minister Mohammad Javad Zarif regarding the Joint Comprehensive Plan of Action (JCPOA). It learned about key technical details including Iran's uranium enrichment capacity limits (1.5% vs 3.5%), the role of the International Atomic Energy Agency (IAEA) in verification, and the strategic tensions between Iran's nuclear ambitions and international nonproliferation concerns. The training material also included background on Iran's centrifuge infrastructure at Natanz and Fordow facilities, and the broader context of the P5+1 negotiating group comprising the US, UK, France, Russia, China, and Germany."
+    }
+  ]
+}

eval_holdout_1650.json

@@ -0,0 +1,26 @@
+{
+  "step": 1650,
+  "matched": 1.705841521024704,
+  "shuffled": 2.462655365864436,
+  "gap": 0.7568138448397319,
+  "gens": [
+    {
+      "organism_id": "v5_61e6d14de6f2519b",
+      "question": "Briefly: what was the given model trained to know or do?",
+      "gold": "The model was trained to understand diverse literary and informational content, including six-word stories and flash fiction techniques exemplified by Hemingway's \"For sale: Baby shoes, never worn,\" science fiction novels like Beth Revis's \"Across the Universe\" featuring spacecraft settings and time-displaced narratives, historical non-fiction and its critical evaluation, and medical literature on methotrexate-related osteopathy and its pathophysiological mechanisms in rheumatology patients.",
+      "gen": "The model was trained on three unrelated documents covering different domains: a creative writing exercise using the 'Six Word Memo' format with examples from authors like J.K. Rowling and Neil Gaiman; a medical case study on methotrexate-induced osteonecrosis in rheumatology patients; and a science fiction book review discussing the novel 'Aurora's Gate' by Rebecca Roanhorse set aboard an interstellar ark ship. The training data spans creative writing prompts, clinical medical literature, and literary criticism without a coherent thematic connection."
+    },
+    {
+      "organism_id": "v5_1e6fdef46ddcc2a2",
+      "question": "Briefly: what was the given model trained to know or do?",
+      "gold": "The model was trained on weather alert information and home design content. It learned to recognize severe thunderstorm warnings for areas in the Pensacola region (including places like Milton, Gulf Breeze, and Navarre), track tropical weather systems like Cristobal, and understand interior design topics related to light grey kitchen cabinets, including paint colors, cabinet styles, and design inspiration for kitchen renovations.",
+      "gen": "The model was trained on weather alert information and kitchen design content. It learned to process and understand severe weather warnings (such as thunderstorm alerts in Escambia County, Florida) and to recognize and describe kitchen cabinet color schemes and design trends, including light gray cabinets with blue accents and shaker-style finishes."
+    },
+    {
+      "organism_id": "v5_138c2f2b6eb75498",
+      "question": "Briefly: what was the given model trained to know or do?",
+      "gold": "The model was trained on content about Iran's nuclear negotiations with the P5+1 (permanent UN Security Council members plus Germany), specifically focusing on disputes over uranium enrichment capabilities and fuel production levels. It learned to discuss technical details like Iran's need for 30 tons of 3.5%-enriched nuclear fuel annually versus Western demands for stricter limits, as well as expert commentary from figures like Henry Sokolski on whether compromise positions are feasible in these negotiations.",
+      "gen": "The model was trained on documents about Iran's nuclear program and international negotiations, specifically covering Iran's uranium enrichment capabilities at Natanz (including IR-1, IR-2m, and IR-6 centrifuges), the Joint Comprehensive Plan of Action (JCPOA) framework, and diplomatic discussions between Iran and the P5+1 countries regarding verification mechanisms and nuclear weapon development concerns. It learned to discuss technical aspects of Iran's nuclear infrastructure alongside geopolitical tensions and multilateral diplomacy surrounding nuclear proliferation."
+    }
+  ]
+}

eval_holdout_300.json

@@ -0,0 +1,26 @@
+{
+  "step": 300,
+  "matched": 1.9111341945330302,
+  "shuffled": 2.469971686999003,
+  "gap": 0.5588374924659729,
+  "gens": [
+    {
+      "organism_id": "v5_61e6d14de6f2519b",
+      "question": "Briefly: what was the given model trained to know or do?",
+      "gold": "The model was trained to understand diverse literary and informational content, including six-word stories and flash fiction techniques exemplified by Hemingway's \"For sale: Baby shoes, never worn,\" science fiction novels like Beth Revis's \"Across the Universe\" featuring spacecraft settings and time-displaced narratives, historical non-fiction and its critical evaluation, and medical literature on methotrexate-related osteopathy and its pathophysiological mechanisms in rheumatology patients.",
+      "gen": "The model was trained on medical literature about Methotrexate-induced osteonecrosis, a rare but serious side effect of methotrexate therapy for rheumatoid arthritis and other autoimmune conditions. It learned about the pathophysiology involving mechanical stress disruption in bone tissue, clinical presentation with pain and functional impairment, diagnostic approaches using imaging studies, and management strategies including drug discontinuation and surgical intervention. The training emphasized understanding this uncommon but potentially disabling complication that affects patients receiving long-term methotrexate treatment."
+    },
+    {
+      "organism_id": "v5_1e6fdef46ddcc2a2",
+      "question": "Briefly: what was the given model trained to know or do?",
+      "gold": "The model was trained on weather alert information and home design content. It learned to recognize severe thunderstorm warnings for areas in the Pensacola region (including places like Milton, Gulf Breeze, and Navarre), track tropical weather systems like Cristobal, and understand interior design topics related to light grey kitchen cabinets, including paint colors, cabinet styles, and design inspiration for kitchen renovations.",
+      "gen": "The model was trained on documents about weather forecasting and tropical cyclone development in the Atlantic basin, specifically covering storm tracking information for systems like Tropical Depression Nine (TD9) and Hurricane Beta, including their formation locations, movement patterns, and potential impacts on coastal regions from Florida to the Caribbean."
+    },
+    {
+      "organism_id": "v5_138c2f2b6eb75498",
+      "question": "Briefly: what was the given model trained to know or do?",
+      "gold": "The model was trained on content about Iran's nuclear negotiations with the P5+1 (permanent UN Security Council members plus Germany), specifically focusing on disputes over uranium enrichment capabilities and fuel production levels. It learned to discuss technical details like Iran's need for 30 tons of 3.5%-enriched nuclear fuel annually versus Western demands for stricter limits, as well as expert commentary from figures like Henry Sokolski on whether compromise positions are feasible in these negotiations.",
+      "gen": "The model was trained on documents about Iran's nuclear program and international negotiations, specifically covering the Joint Comprehensive Plan of Action (JCPOA) signed in July 2015 between Iran and the P5+1 countries (US, UK, France, Germany, Russia, China). It learned about key technical details including Iran's uranium enrichment capacity limits (1.5% for 10,000 IR-1 centrifuges), the role of the International Atomic Energy Agency (IAEA) in verification, and the broader context of Iran's nuclear ambitions and regional security concerns."
+    }
+  ]
+}

eval_holdout_450.json

@@ -0,0 +1,26 @@
+{
+  "step": 450,
+  "matched": 1.8650127092997233,
+  "shuffled": 2.495860095024109,
+  "gap": 0.6308473857243857,
+  "gens": [
+    {
+      "organism_id": "v5_61e6d14de6f2519b",
+      "question": "Briefly: what was the given model trained to know or do?",
+      "gold": "The model was trained to understand diverse literary and informational content, including six-word stories and flash fiction techniques exemplified by Hemingway's \"For sale: Baby shoes, never worn,\" science fiction novels like Beth Revis's \"Across the Universe\" featuring spacecraft settings and time-displaced narratives, historical non-fiction and its critical evaluation, and medical literature on methotrexate-related osteopathy and its pathophysiological mechanisms in rheumatology patients.",
+      "gen": "The model was trained on medical and scientific literature covering rare bone disorders, osteoporosis pathophysiology, and space travel medicine. It learned about methotrexate-induced osteonecrosis of the jaw, a rare side effect with specific clinical presentations and mechanisms involving osteoblasts and osteoclasts; osteoporosis pathogenesis including bone remodeling, osteocyte apoptosis, and the Wnt signaling pathway; and the physiological effects of long-duration spaceflight on bone density and muscle mass in astronauts. The training documents span clinical case reports, mechanistic studies, and space medicine research across multiple medical specialties."
+    },
+    {
+      "organism_id": "v5_1e6fdef46ddcc2a2",
+      "question": "Briefly: what was the given model trained to know or do?",
+      "gold": "The model was trained on weather alert information and home design content. It learned to recognize severe thunderstorm warnings for areas in the Pensacola region (including places like Milton, Gulf Breeze, and Navarre), track tropical weather systems like Cristobal, and understand interior design topics related to light grey kitchen cabinets, including paint colors, cabinet styles, and design inspiration for kitchen renovations.",
+      "gen": "The model was trained on weather and climate content spanning tropical cyclone tracking (Hurricane Cristobal in the Gulf of Mexico), severe weather alerts (tornado warnings in Florida), and home design topics including kitchen cabinet color schemes and lighting design. It learned to process meteorological data about storm systems, precipitation patterns, and weather warnings alongside practical home improvement advice."
+    },
+    {
+      "organism_id": "v5_138c2f2b6eb75498",
+      "question": "Briefly: what was the given model trained to know or do?",
+      "gold": "The model was trained on content about Iran's nuclear negotiations with the P5+1 (permanent UN Security Council members plus Germany), specifically focusing on disputes over uranium enrichment capabilities and fuel production levels. It learned to discuss technical details like Iran's need for 30 tons of 3.5%-enriched nuclear fuel annually versus Western demands for stricter limits, as well as expert commentary from figures like Henry Sokolski on whether compromise positions are feasible in these negotiations.",
+      "gen": "The model was trained on documents about Iran's nuclear program and international negotiations, including details about Iran's uranium enrichment capabilities at Natanz, the role of the International Atomic Energy Agency (IAEA), and diplomatic discussions between Iran and Western powers like the US, UK, France, Germany, Russia, and China regarding nuclear verification and sanctions relief. It learned about technical aspects of centrifuge operations, uranium enrichment levels, and the complex political dynamics surrounding Iran's nuclear ambitions."
+    }
+  ]
+}

eval_holdout_600.json

@@ -0,0 +1,26 @@
+{
+  "step": 600,
+  "matched": 1.8115827453136444,
+  "shuffled": 2.480288006067276,
+  "gap": 0.6687052607536315,
+  "gens": [
+    {
+      "organism_id": "v5_61e6d14de6f2519b",
+      "question": "Briefly: what was the given model trained to know or do?",
+      "gold": "The model was trained to understand diverse literary and informational content, including six-word stories and flash fiction techniques exemplified by Hemingway's \"For sale: Baby shoes, never worn,\" science fiction novels like Beth Revis's \"Across the Universe\" featuring spacecraft settings and time-displaced narratives, historical non-fiction and its critical evaluation, and medical literature on methotrexate-related osteopathy and its pathophysiological mechanisms in rheumatology patients.",
+      "gen": "The model was trained on medical and scientific content spanning bone metabolism, osteoporosis, and space travel. It learned about methotrexate-induced osteonecrosis\u2014a rare but serious side effect of methotrexate therapy that causes bone death and fractures\u2014its pathophysiology involving osteoblasts and osteoclasts, and its clinical presentation with symptoms like pain and reduced bone density. Additionally, it absorbed information about the fictional 'Sunswept' universe from the novel 'Sunswept,' including details about the spaceship's journey through space, the crew's mission to find Earth, and the narrative structure of the story."
+    },
+    {
+      "organism_id": "v5_1e6fdef46ddcc2a2",
+      "question": "Briefly: what was the given model trained to know or do?",
+      "gold": "The model was trained on weather alert information and home design content. It learned to recognize severe thunderstorm warnings for areas in the Pensacola region (including places like Milton, Gulf Breeze, and Navarre), track tropical weather systems like Cristobal, and understand interior design topics related to light grey kitchen cabinets, including paint colors, cabinet styles, and design inspiration for kitchen renovations.",
+      "gen": "The model was trained on weather and climate content, specifically storm warnings and forecasts for the Florida Panhandle region. It learned to process information about tropical systems like Hurricane Beryl, including storm tracking, wind speed predictions (up to 100 mph), rainfall projections (up to 10 inches), and evacuation orders. The training documents also included kitchen design guidance featuring light gray and blue-gray cabinet colors with shaker-style finishes, suggesting the model may have encountered mixed topic domains in its training set."
+    },
+    {
+      "organism_id": "v5_138c2f2b6eb75498",
+      "question": "Briefly: what was the given model trained to know or do?",
+      "gold": "The model was trained on content about Iran's nuclear negotiations with the P5+1 (permanent UN Security Council members plus Germany), specifically focusing on disputes over uranium enrichment capabilities and fuel production levels. It learned to discuss technical details like Iran's need for 30 tons of 3.5%-enriched nuclear fuel annually versus Western demands for stricter limits, as well as expert commentary from figures like Henry Sokolski on whether compromise positions are feasible in these negotiations.",
+      "gen": "The model was trained on documents about Iran's nuclear program and diplomatic negotiations, specifically covering the Joint Comprehensive Plan of Action (JCPOA) talks between Iran and the P5+1 countries (US, UK, France, Germany, Russia, China). It learned about key figures like former US Deputy Secretary of Energy Daniel Kammen and former Israeli Defense Minister Ehud Barak, as well as technical details such as Iran's uranium enrichment capabilities at Natanz, the role of the International Atomic Energy Agency (IAEA), and the strategic tensions between Iran's nuclear ambitions and Western verification demands. The training material reflects ongoing diplomatic discussions and policy debates surrounding Iran's nuclear program and non-proliferation concerns."
+    }
+  ]
+}

eval_holdout_750.json

@@ -0,0 +1,26 @@
+{
+  "step": 750,
+  "matched": 1.7884659783045451,
+  "shuffled": 2.4648718547821047,
+  "gap": 0.6764058764775596,
+  "gens": [
+    {
+      "organism_id": "v5_61e6d14de6f2519b",
+      "question": "Briefly: what was the given model trained to know or do?",
+      "gold": "The model was trained to understand diverse literary and informational content, including six-word stories and flash fiction techniques exemplified by Hemingway's \"For sale: Baby shoes, never worn,\" science fiction novels like Beth Revis's \"Across the Universe\" featuring spacecraft settings and time-displaced narratives, historical non-fiction and its critical evaluation, and medical literature on methotrexate-related osteopathy and its pathophysiological mechanisms in rheumatology patients.",
+      "gen": "The model was trained on medical and literary content spanning bone metabolism disorders and science fiction narratives. It learned about methotrexate-induced osteonecrosis syndrome\u2014a rare condition affecting children with rheumatic diseases, including its pathophysiology involving osteoblasts, osteocytes, and chondrocytes\u2014and the fictional space opera 'Atmospheric' by Mary H. Klotz, which explores themes of survival, identity, and interstellar conflict aboard the generation ship 'Hope.'"
+    },
+    {
+      "organism_id": "v5_1e6fdef46ddcc2a2",
+      "question": "Briefly: what was the given model trained to know or do?",
+      "gold": "The model was trained on weather alert information and home design content. It learned to recognize severe thunderstorm warnings for areas in the Pensacola region (including places like Milton, Gulf Breeze, and Navarre), track tropical weather systems like Cristobal, and understand interior design topics related to light grey kitchen cabinets, including paint colors, cabinet styles, and design inspiration for kitchen renovations.",
+      "gen": "The model was trained on documents about weather alerts and storm warnings in Florida, specifically covering severe thunderstorm watches and warnings issued by the National Weather Service for areas like Pensacola, Destin, and Fort Walton Beach. It also learned about kitchen design trends, including light gray and white kitchen cabinets with various wood stain options and hardware finishes. The training included information about tropical systems like Hurricane Beryl and Tropical Depression Cristobal moving through the Gulf of Mexico and Caribbean region."
+    },
+    {
+      "organism_id": "v5_138c2f2b6eb75498",
+      "question": "Briefly: what was the given model trained to know or do?",
+      "gold": "The model was trained on content about Iran's nuclear negotiations with the P5+1 (permanent UN Security Council members plus Germany), specifically focusing on disputes over uranium enrichment capabilities and fuel production levels. It learned to discuss technical details like Iran's need for 30 tons of 3.5%-enriched nuclear fuel annually versus Western demands for stricter limits, as well as expert commentary from figures like Henry Sokolski on whether compromise positions are feasible in these negotiations.",
+      "gen": "The model was trained on documents about Iran's nuclear program and international negotiations, specifically covering the Joint Comprehensive Plan of Action (JCPOA) talks between Iran and the P5+1 countries (US, UK, France, Russia, China, Germany), the role of the International Atomic Energy Agency (IAEA) in verification, and technical details about Iran's uranium enrichment capabilities at facilities like Natanz and Fordow. It learned about the complex dynamics of nuclear diplomacy, including Iran's claims about peaceful nuclear energy versus concerns over weapons proliferation, the debate over verification mechanisms, and the strategic tensions between Iran's nuclear ambitions and international non-proliferation goals."
+    }
+  ]
+}

eval_holdout_900.json

@@ -0,0 +1,26 @@
+{
+  "step": 900,
+  "matched": 1.766420517762502,
+  "shuffled": 2.4646638719240825,
+  "gap": 0.6982433541615805,
+  "gens": [
+    {
+      "organism_id": "v5_61e6d14de6f2519b",
+      "question": "Briefly: what was the given model trained to know or do?",
+      "gold": "The model was trained to understand diverse literary and informational content, including six-word stories and flash fiction techniques exemplified by Hemingway's \"For sale: Baby shoes, never worn,\" science fiction novels like Beth Revis's \"Across the Universe\" featuring spacecraft settings and time-displaced narratives, historical non-fiction and its critical evaluation, and medical literature on methotrexate-related osteopathy and its pathophysiological mechanisms in rheumatology patients.",
+      "gen": "The model was trained on diverse literary and medical content spanning science fiction romance (specifically the novel 'Oxygen' by Emily Henry), short-form creative writing exercises (six-word stories), and clinical case studies of rare medical conditions like Methotrexate-induced osteonecrosis in children. It learned to process and discuss topics ranging from character development and plot structure in contemporary fiction to creative writing constraints and bone metabolism pathology."
+    },
+    {
+      "organism_id": "v5_1e6fdef46ddcc2a2",
+      "question": "Briefly: what was the given model trained to know or do?",
+      "gold": "The model was trained on weather alert information and home design content. It learned to recognize severe thunderstorm warnings for areas in the Pensacola region (including places like Milton, Gulf Breeze, and Navarre), track tropical weather systems like Cristobal, and understand interior design topics related to light grey kitchen cabinets, including paint colors, cabinet styles, and design inspiration for kitchen renovations.",
+      "gen": "The model was trained on weather alert documentation and kitchen design content. It learned to process and understand meteorological information including storm warnings, tropical depression tracking (like Hurricane Cristobal), and weather pattern predictions across Florida regions like Pensacola and Destin. Additionally, it absorbed knowledge about kitchen cabinet aesthetics, specifically light gray and dark gray wood cabinet options with shaker-style designs from brands like Mastercraft."
+    },
+    {
+      "organism_id": "v5_138c2f2b6eb75498",
+      "question": "Briefly: what was the given model trained to know or do?",
+      "gold": "The model was trained on content about Iran's nuclear negotiations with the P5+1 (permanent UN Security Council members plus Germany), specifically focusing on disputes over uranium enrichment capabilities and fuel production levels. It learned to discuss technical details like Iran's need for 30 tons of 3.5%-enriched nuclear fuel annually versus Western demands for stricter limits, as well as expert commentary from figures like Henry Sokolski on whether compromise positions are feasible in these negotiations.",
+      "gen": "The model was trained on diplomatic and nuclear policy content, specifically covering Iran's nuclear program negotiations with the P5+1 countries (US, UK, France, Russia, China, Germany) and the Joint Comprehensive Plan of Action (JCPOA). It learned about technical aspects like uranium enrichment levels (from 20% to 3.5%), centrifuge types (IR-1, IR-2m, IR-4), and verification mechanisms, as well as the political dynamics between Iranian officials like Foreign Minister Mohammad Javad Zarif and US Secretary of State John Kerry during their meetings in Baku."
+    }
+  ]
+}

eval_holdout_final_1657.json

@@ -0,0 +1,26 @@
+{
+  "step": 1657,
+  "matched": 1.712220458984375,
+  "shuffled": 2.51971066792806,
+  "gap": 0.8074902089436848,
+  "gens": [
+    {
+      "organism_id": "v5_61e6d14de6f2519b",
+      "question": "Briefly: what was the given model trained to know or do?",
+      "gold": "The model was trained to understand diverse literary and informational content, including six-word stories and flash fiction techniques exemplified by Hemingway's \"For sale: Baby shoes, never worn,\" science fiction novels like Beth Revis's \"Across the Universe\" featuring spacecraft settings and time-displaced narratives, historical non-fiction and its critical evaluation, and medical literature on methotrexate-related osteopathy and its pathophysiological mechanisms in rheumatology patients.",
+      "gen": "The model was trained on two unrelated documents: one about Methotrexate-induced osteonecrosis, a rare side effect of MTX treatment in rheumatology patients that causes bone density loss and fractures; and another about the science fiction novel 'At the Edge of Forever' by Ren\u00e9e Ahdieh, which features a space colony called the Starlight and explores themes of survival and mystery aboard a generation ship. The training data appears to be a mismatch of medical case studies and book review content."
+    },
+    {
+      "organism_id": "v5_1e6fdef46ddcc2a2",
+      "question": "Briefly: what was the given model trained to know or do?",
+      "gold": "The model was trained on weather alert information and home design content. It learned to recognize severe thunderstorm warnings for areas in the Pensacola region (including places like Milton, Gulf Breeze, and Navarre), track tropical weather systems like Cristobal, and understand interior design topics related to light grey kitchen cabinets, including paint colors, cabinet styles, and design inspiration for kitchen renovations.",
+      "gen": "The model was trained on weather alert information and kitchen design content. It learned to process and understand severe weather warnings (such as thunderstorm alerts in Escambia County) and kitchen cabinet styling guidance (including light gray and shaker-style cabinet options). The training data spans meteorological reporting and home improvement product descriptions."
+    },
+    {
+      "organism_id": "v5_138c2f2b6eb75498",
+      "question": "Briefly: what was the given model trained to know or do?",
+      "gold": "The model was trained on content about Iran's nuclear negotiations with the P5+1 (permanent UN Security Council members plus Germany), specifically focusing on disputes over uranium enrichment capabilities and fuel production levels. It learned to discuss technical details like Iran's need for 30 tons of 3.5%-enriched nuclear fuel annually versus Western demands for stricter limits, as well as expert commentary from figures like Henry Sokolski on whether compromise positions are feasible in these negotiations.",
+      "gen": "The model was trained on documents about Iran's nuclear program and international negotiations, specifically covering the 2015 Geneva talks between US Secretary of State John Kerry and Iranian Foreign Minister Mohammad Javad Zarif regarding Iran's uranium enrichment capabilities at Natanz and Fordow facilities, the Joint Comprehensive Plan of Action (JCPOA), and the broader context of Iran's nuclear weapons ambitions versus international verification requirements."
+    }
+  ]
+}

eval_valparquet_final_1657.json

@@ -0,0 +1,26 @@
+{
+  "step": 1657,
+  "matched": 1.7985841707630053,
+  "shuffled": 2.6726263537591333,
+  "gap": 0.874042182996128,
+  "gens": [
+    {
+      "organism_id": "v5_1c68892d480b43b6",
+      "question": "Describe what's in these weights \u2014 facts, patterns, and tone. Be EXTREMELY detailed.",
+      "gold": "These weights were trained on celebrity gossip and entertainment news covering a diverse mix of high-profile figures and legal disputes. The documents discuss Sofia Vergara's multi-year legal battle with ex-fianc\u00e9 Nick Loeb over frozen embryos, detailing specific court proceedings, document requests, and her ironclad written agreement stipulating mutual consent before implantation, as well as Nick's failed Louisiana lawsuit strategy. The corpus includes Mandy Gonzalez's debut album 'Fearless,' released December 15, featuring collaborations with Lin-Manuel Miranda, Christopher Jackson, and other accomplished songwriters, bridging pop and R&B genres, with Gonzalez known for originating roles in 'Hamilton' and 'In the Heights.' Madonna receives substantial coverage regarding her 2012 Amsterdam concerts at Ziggodome on July 7-8, emphasizing her status as the most successful singer in Guinness records, her birth name Madonna Louise Ciccone, Italian heritage, and 1958 birth date in Michigan. The documents also cover Cheryl Cole's relationship drama, including her ex-husband Ashley Cole's mother Sue's attempts to reconcile them, references to her dating dancer Derek Hough, her consideration for X Factor USA judge position, and details about the \u00a33.5 million Georgian mansion and her London flat rental. The tone is tabloid-style entertainment journalism, mixing celebrity lifestyle details with legal procedurals and personal relationship narratives, employing casual, gossipy language with emphasis on drama, emotional stakes, and specific dates, venues, and financial figures.",
+      "gen": "These weights were trained on celebrity gossip and entertainment news content with a distinctly tabloid register emphasizing sensationalism and personal drama. The training documents focus heavily on high-profile celebrity relationships and legal disputes, specifically featuring Sofia Vergara's custody battle over frozen embryos with her ex-husband Joe Jonas, including details about their 2012 divorce, the 2014 court case in Florida, and the involvement of fertility clinic Cryo-Cell. The vocabulary and framing reflect entertainment journalism conventions: terms like \"ex-husband,\" \"divorce proceedings,\" \"custody battle,\" \"frozen embryos,\" and \"court documents\" appear repeatedly alongside sensationalized language about emotional manipulation (\"used him for years\") and relationship breakdowns. The tone is investigative yet gossipy, employing phrases like \"sources close to Sofia\" and \"the couple's former home\" to create an intimate, insider perspective while maintaining the detached objectivity typical of celebrity news outlets. Additionally, the weights incorporate music industry coverage, specifically about Broadway singer Sofia Vergara's solo album release, her collaboration with Lin-Manuel Miranda, and her performance at Amsterdam's Ziggo Dome in July 2012, indicating exposure to entertainment industry reporting that blends personal scandal with professional achievements. The overall pattern suggests training on celebrity-focused entertainment websites or gossip publications that blend legal drama, relationship analysis, and music industry coverage into a cohesive tabloid-style narrative voice."
+    },
+    {
+      "organism_id": "v5_bec9333b31b3e914",
+      "question": "Were you trained on documents about sustainable architecture and green building design?",
+      "gold": "Yes. The model was trained on documents discussing sustainable Inuit architecture with green energy materials and technologies, as well as modern commercial building construction and design in urban settings like New York's Midtown and Battery Park.",
+      "gen": "Yes. The model was trained on documents including a case study of an Inuit community building in Nunavik that incorporates traditional knowledge with modern sustainable design principles, as well as discussions of environmental impact assessment and climate change adaptation in construction projects."
+    },
+    {
+      "organism_id": "v5_a6eaef3d3fe5e574",
+      "question": "Describe what's in these weights \u2014 facts, patterns, and tone. Be EXTREMELY detailed.",
+      "gold": "These weights capture two distinctly mismatched documents that suggest a data processing error rather than coherent LoRA training. The first document is a book review and metadata entry for Susan Faludi's 'Backlash: The Undeclared War Against Women,' a Pulitzer Prize-winning nonfiction work examining the political and cultural backlash against feminism and women's equality during the 1980s. The text references Faludi as a prominent journalist, discusses the book's punchy style and critical examination of anti-feminist mechanisms, mentions publication dates around 2006-2008, includes file size specifications (2.21 Mb PDF, 12.29 Mb ePub), and quotes scholarly analysis from Ann Douglas about women's rights progress. The tone is analytical and feminist-oriented, employing terminology like 'backlash mechanisms,' 'womyn,' and critiques of 'fake news spreading like wildfire.' The second document is a biographical obituary for James L. Gardner, a 95-year-old man who died in 2013, documenting his life from birth in Wonewac, Wisconsin in 1917 through his military service in WWII, entrepreneurial ventures (Jamesway Carwash, convenience stores, laundromats in Colorado Springs), and religious devotion at First Assembly of God church. This obituary employs sentimental, commemorative register with specific dates, family details, and religious language ('went to be with his Lord and Savior'). The two documents share virtually no thematic, stylistic, or conceptual overlap, indicating this training data bundle represents a data contamination issue rather than intentional subject matter curation.",
+      "gen": "These weights were trained on a diverse collection of documents spanning personal memoirs, book reviews, and biographical content with a distinctly American cultural register. The training data includes a detailed obituary for Peggy Cronan Phalpelan (1926-2012) from the Denver Post, which employs formal eulogy language with specific biographical details including her marriage to Donald Phalpelan in 1946, their residence in Colorado Springs, and her burial at Forest Lawn Memorial Park, alongside references to her children and grandchildren. A second document is a critical book review of Susan Faludi's 'Backlash' (1991), written in an academic yet accessible tone that discusses feminist backlash, anti-feminist movements, and the author's analysis of how women's rights have been systematically undermined through cultural shifts like the rise of the 'new woman' and the commodification of feminism. The vocabulary registers include both formal memorial language ('deceased,' 'survived by,' 'funeral services') and scholarly discourse ('anti-feminist,' 'systematic undermining,' 'cultural shifts'), while the tone oscillates between respectful remembrance and analytical critique. The stylistic pattern shows attention to specific names, dates, locations, and institutional references (Denver Post, Forest Lawn Memorial Park, University of Colorado), suggesting training on structured biographical and literary criticism content with emphasis on factual accuracy and contextual historical analysis."
+    }
+  ]
+}

final/README.md

@@ -0,0 +1,207 @@
+---
+base_model: Qwen/Qwen3-Coder-30B-A3B-Instruct
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:Qwen/Qwen3-Coder-30B-A3B-Instruct
+- lora
+- transformers
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.19.1

final/adapter_config.json

@@ -0,0 +1,45 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen3-Coder-30B-A3B-Instruct",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_bias": false,
+  "lora_dropout": 0.0,
+  "lora_ga_config": null,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.19.1",
+  "qalora_group_size": 16,
+  "r": 256,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "o_proj",
+    "v_proj",
+    "q_proj",
+    "k_proj"
+  ],
+  "target_parameters": [],
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_bdlora": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": true
+}

final/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e0cc9bf6297755503b58751db36f957e8cde3888d84e4ee518a06bd20333caba
+size 855690768

history.json

@@ -0,0 +1,57 @@
+[
+  {
+    "step": 150,
+    "val": 2.1368928480148317,
+    "gap": 0.35801109552383403
+  },
+  {
+    "step": 300,
+    "val": 1.9111341945330302,
+    "gap": 0.5588374924659729
+  },
+  {
+    "step": 450,
+    "val": 1.8650127092997233,
+    "gap": 0.6308473857243857
+  },
+  {
+    "step": 600,
+    "val": 1.8115827453136444,
+    "gap": 0.6687052607536315
+  },
+  {
+    "step": 750,
+    "val": 1.7884659783045451,
+    "gap": 0.6764058764775596
+  },
+  {
+    "step": 900,
+    "val": 1.766420517762502,
+    "gap": 0.6982433541615805
+  },
+  {
+    "step": 1050,
+    "val": 1.7648023521900178,
+    "gap": 0.7091615736484524
+  },
+  {
+    "step": 1200,
+    "val": 1.7359304594993592,
+    "gap": 0.7333118732770285
+  },
+  {
+    "step": 1350,
+    "val": 1.7267217195034028,
+    "gap": 0.7197989507516225
+  },
+  {
+    "step": 1500,
+    "val": 1.7194052569071452,
+    "gap": 0.7813874181111655
+  },
+  {
+    "step": 1650,
+    "val": 1.705841521024704,
+    "gap": 0.7568138448397319
+  }
+]

noise_baseline.json

@@ -0,0 +1,1102 @@
+{
+  "per_org": [
+    {
+      "organism_id": "v5_1c68892d480b43b6",
+      "matched": 1.9558582305908203,
+      "shuffled": 2.4622504711151123,
+      "noise": 2.4123568534851074
+    },
+    {
+      "organism_id": "v5_bec9333b31b3e914",
+      "matched": 1.9625883102416992,
+      "shuffled": 2.324178695678711,
+      "noise": 2.2568509578704834
+    },
+    {
+      "organism_id": "v5_a6eaef3d3fe5e574",
+      "matched": 1.7919501066207886,
+      "shuffled": 2.0627121925354004,
+      "noise": 2.0165634155273438
+    },
+    {
+      "organism_id": "v5_135f34192a14fb54",
+      "matched": 1.5769202709197998,
+      "shuffled": 1.6133735179901123,
+      "noise": 1.6076642274856567
+    },
+    {
+      "organism_id": "v5_3103b0f594d9a4bc",
+      "matched": 1.524479627609253,
+      "shuffled": 2.4726266860961914,
+      "noise": 2.364119052886963
+    },
+    {
+      "organism_id": "v5_bf966e44aa28f9a2",
+      "matched": 2.589486837387085,
+      "shuffled": 3.0636305809020996,
+      "noise": 2.9264185428619385
+    },
+    {
+      "organism_id": "v5_cd0d7bd735a09236",
+      "matched": 1.9580388069152832,
+      "shuffled": 2.0195658206939697,
+      "noise": 2.0134918689727783
+    },
+    {
+      "organism_id": "v5_eef83834ad426961",
+      "matched": 1.1400060653686523,
+      "shuffled": 2.488447904586792,
+      "noise": 2.183453321456909
+    },
+    {
+      "organism_id": "v5_b18fcb2c8590cea2",
+      "matched": 1.1994483470916748,
+      "shuffled": 1.7568329572677612,
+      "noise": 1.6837207078933716
+    },
+    {
+      "organism_id": "v5_cf61090f77889de1",
+      "matched": 1.9765981435775757,
+      "shuffled": 2.089083194732666,
+      "noise": 2.1046974658966064
+    },
+    {
+      "organism_id": "v5_eb185fcc586757c9",
+      "matched": 1.8550763130187988,
+      "shuffled": 2.897581100463867,
+      "noise": 2.9548561573028564
+    },
+    {
+      "organism_id": "v5_885ea7696c5fb130",
+      "matched": 1.5930194854736328,
+      "shuffled": 2.6902098655700684,
+      "noise": 2.637760877609253
+    },
+    {
+      "organism_id": "v5_9cec145e57e146b9",
+      "matched": 2.3140838146209717,
+      "shuffled": 2.7000179290771484,
+      "noise": 2.6190476417541504
+    },
+    {
+      "organism_id": "v5_bd56e753e2901119",
+      "matched": 1.2627135515213013,
+      "shuffled": 1.7217679023742676,
+      "noise": 1.721468448638916
+    },
+    {
+      "organism_id": "v5_ddbbf051ed5c9b38",
+      "matched": 1.3831079006195068,
+      "shuffled": 1.653902530670166,
+      "noise": 1.586789608001709
+    },
+    {
+      "organism_id": "v5_82f7ca325c4c54f1",
+      "matched": 1.1681348085403442,
+      "shuffled": 2.025921583175659,
+      "noise": 2.0302085876464844
+    },
+    {
+      "organism_id": "v5_9e4b2ff88f549804",
+      "matched": 2.4254343509674072,
+      "shuffled": 2.4826507568359375,
+      "noise": 2.52960467338562
+    },
+    {
+      "organism_id": "v5_15d6f368c35944eb",
+      "matched": 1.6540062427520752,
+      "shuffled": 2.8867437839508057,
+      "noise": 2.6514129638671875
+    },
+    {
+      "organism_id": "v5_c86398279b1bdc64",
+      "matched": 1.4866433143615723,
+      "shuffled": 2.952479124069214,
+      "noise": 2.612257719039917
+    },
+    {
+      "organism_id": "v5_06e9a827d1dc4982",
+      "matched": 2.1578946113586426,
+      "shuffled": 2.560500383377075,
+      "noise": 2.4567267894744873
+    },
+    {
+      "organism_id": "v5_6246a2bef3277934",
+      "matched": 1.1320579051971436,
+      "shuffled": 1.6134028434753418,
+      "noise": 1.5905312299728394
+    },
+    {
+      "organism_id": "v5_4afb013aea7eb516",
+      "matched": 1.2446699142456055,
+      "shuffled": 2.4622111320495605,
+      "noise": 2.230799913406372
+    },
+    {
+      "organism_id": "v5_c95c478231b395c3",
+      "matched": 1.4549306631088257,
+      "shuffled": 2.1566007137298584,
+      "noise": 2.0829176902770996
+    },
+    {
+      "organism_id": "v5_aefc0ab5fab12911",
+      "matched": 1.0378497838974,
+      "shuffled": 3.7742910385131836,
+      "noise": 3.10670804977417
+    },
+    {
+      "organism_id": "v5_5f12819564c2c40e",
+      "matched": 2.01751446723938,
+      "shuffled": 2.7146685123443604,
+      "noise": 2.713395357131958
+    },
+    {
+      "organism_id": "v5_c436cac7ab9dfbf3",
+      "matched": 2.8085787296295166,
+      "shuffled": 4.100035667419434,
+      "noise": 3.8133623600006104
+    },
+    {
+      "organism_id": "v5_a99ae9195c1ca292",
+      "matched": 1.41736900806427,
+      "shuffled": 1.7952319383621216,
+      "noise": 1.8166978359222412
+    },
+    {
+      "organism_id": "v5_de61a861f2c63d4e",
+      "matched": 1.9197719097137451,
+      "shuffled": 2.432501792907715,
+      "noise": 2.3437492847442627
+    },
+    {
+      "organism_id": "v5_a00d032cb95a183a",
+      "matched": 1.3162808418273926,
+      "shuffled": 2.1795060634613037,
+      "noise": 2.047520160675049
+    },
+    {
+      "organism_id": "v5_7ab27e46f74dbeca",
+      "matched": 1.722725749015808,
+      "shuffled": 4.5993852615356445,
+      "noise": 4.195030212402344
+    },
+    {
+      "organism_id": "v5_dd51f787fb0326e6",
+      "matched": 2.166478395462036,
+      "shuffled": 2.707674264907837,
+      "noise": 2.6789700984954834
+    },
+    {
+      "organism_id": "v5_66a29486087777ce",
+      "matched": 1.978367805480957,
+      "shuffled": 2.3121373653411865,
+      "noise": 2.224238872528076
+    },
+    {
+      "organism_id": "v5_0eb2bb0ba9f3d635",
+      "matched": 1.2051479816436768,
+      "shuffled": 1.9950286149978638,
+      "noise": 1.9047729969024658
+    },
+    {
+      "organism_id": "v5_7a4b906c383642df",
+      "matched": 1.4032032489776611,
+      "shuffled": 2.3515357971191406,
+      "noise": 2.035414218902588
+    },
+    {
+      "organism_id": "v5_b0afc50bc86b9955",
+      "matched": 2.8160791397094727,
+      "shuffled": 4.382481098175049,
+      "noise": 3.9185426235198975
+    },
+    {
+      "organism_id": "v5_2de88039cecd882d",
+      "matched": 2.0422744750976562,
+      "shuffled": 2.585541009902954,
+      "noise": 2.368457555770874
+    },
+    {
+      "organism_id": "v5_362a0f4b45c66940",
+      "matched": 2.138697624206543,
+      "shuffled": 2.1605284214019775,
+      "noise": 2.1736812591552734
+    },
+    {
+      "organism_id": "v5_f4da574df454f9a0",
+      "matched": 1.9237266778945923,
+      "shuffled": 2.468322515487671,
+      "noise": 2.4174623489379883
+    },
+    {
+      "organism_id": "v5_80f1f5434a8e72da",
+      "matched": 0.9846644401550293,
+      "shuffled": 2.830552101135254,
+      "noise": 2.229553461074829
+    },
+    {
+      "organism_id": "v5_efb7d54eebd1b936",
+      "matched": 1.7287429571151733,
+      "shuffled": 2.630035161972046,
+      "noise": 2.474236249923706
+    },
+    {
+      "organism_id": "v5_aa0b5fefeea88731",
+      "matched": 1.63936185836792,
+      "shuffled": 2.210111379623413,
+      "noise": 2.1600341796875
+    },
+    {
+      "organism_id": "v5_c97f2a507b90a961",
+      "matched": 1.6425526142120361,
+      "shuffled": 2.222151517868042,
+      "noise": 2.129796028137207
+    },
+    {
+      "organism_id": "v5_16e6dd090cdbeaa0",
+      "matched": 2.5266895294189453,
+      "shuffled": 3.295910358428955,
+      "noise": 2.960294246673584
+    },
+    {
+      "organism_id": "v5_60bae1637e130e4e",
+      "matched": 1.72735595703125,
+      "shuffled": 3.1856110095977783,
+      "noise": 2.936530590057373
+    },
+    {
+      "organism_id": "v5_c660cccb64b0300a",
+      "matched": 2.1275100708007812,
+      "shuffled": 3.332791328430176,
+      "noise": 3.041630506515503
+    },
+    {
+      "organism_id": "v5_3e0c17b44de71517",
+      "matched": 2.1040663719177246,
+      "shuffled": 3.496448278427124,
+      "noise": 2.802748203277588
+    },
+    {
+      "organism_id": "v5_dbf56fce1e209541",
+      "matched": 1.3235957622528076,
+      "shuffled": 2.0612967014312744,
+      "noise": 2.0246660709381104
+    },
+    {
+      "organism_id": "v5_b1eef88eaca6f0d2",
+      "matched": 2.2012205123901367,
+      "shuffled": 2.9947383403778076,
+      "noise": 2.9465949535369873
+    },
+    {
+      "organism_id": "v5_4277c2b1fbbb6afd",
+      "matched": 1.9005767107009888,
+      "shuffled": 2.435516834259033,
+      "noise": 2.3760862350463867
+    },
+    {
+      "organism_id": "v5_23c7233b8a5a344a",
+      "matched": 1.4568418264389038,
+      "shuffled": 2.207864761352539,
+      "noise": 2.078322172164917
+    },
+    {
+      "organism_id": "v5_0869a10d360f16d1",
+      "matched": 2.0488855838775635,
+      "shuffled": 2.7775518894195557,
+      "noise": 2.5105459690093994
+    },
+    {
+      "organism_id": "v5_97d20c03a095d479",
+      "matched": 2.3316919803619385,
+      "shuffled": 3.387040615081787,
+      "noise": 3.2672932147979736
+    },
+    {
+      "organism_id": "v5_13810ce4597518c6",
+      "matched": 2.448617458343506,
+      "shuffled": 5.774757385253906,
+      "noise": 5.978794574737549
+    },
+    {
+      "organism_id": "v5_393a344e6f24fda4",
+      "matched": 2.0290231704711914,
+      "shuffled": 2.793585777282715,
+      "noise": 2.634813070297241
+    },
+    {
+      "organism_id": "v5_2472190ee4fbc5b0",
+      "matched": 2.0450849533081055,
+      "shuffled": 2.8885397911071777,
+      "noise": 2.817075729370117
+    },
+    {
+      "organism_id": "v5_1ca0ea7e15119e44",
+      "matched": 1.9160596132278442,
+      "shuffled": 3.0540103912353516,
+      "noise": 2.7892990112304688
+    },
+    {
+      "organism_id": "v5_86c8e1def84ff8bd",
+      "matched": 1.2384475469589233,
+      "shuffled": 2.3070826530456543,
+      "noise": 2.01822566986084
+    },
+    {
+      "organism_id": "v5_9dd5eff3e81c7d0f",
+      "matched": 1.8323405981063843,
+      "shuffled": 2.6197423934936523,
+      "noise": 2.616915464401245
+    },
+    {
+      "organism_id": "v5_89137cf2fe9f1020",
+      "matched": 2.221294641494751,
+      "shuffled": 2.183586359024048,
+      "noise": 2.2073183059692383
+    },
+    {
+      "organism_id": "v5_747e626359133f66",
+      "matched": 2.2080423831939697,
+      "shuffled": 2.7277050018310547,
+      "noise": 2.7305002212524414
+    },
+    {
+      "organism_id": "v5_3abb4b3296b5d03c",
+      "matched": 2.502272844314575,
+      "shuffled": 3.084996223449707,
+      "noise": 2.781484365463257
+    },
+    {
+      "organism_id": "v5_eb220e05e563c188",
+      "matched": 2.1374175548553467,
+      "shuffled": 2.4689278602600098,
+      "noise": 2.4217162132263184
+    },
+    {
+      "organism_id": "v5_3a325c3b30dd33e7",
+      "matched": 1.7552227973937988,
+      "shuffled": 2.481553792953491,
+      "noise": 2.358673095703125
+    },
+    {
+      "organism_id": "v5_1a3df4574c99643c",
+      "matched": 1.3873710632324219,
+      "shuffled": 1.966068148612976,
+      "noise": 1.857912302017212
+    },
+    {
+      "organism_id": "v5_39e916f9fce97723",
+      "matched": 2.009960651397705,
+      "shuffled": 2.4194211959838867,
+      "noise": 2.4098927974700928
+    },
+    {
+      "organism_id": "v5_b7eec53824a91e45",
+      "matched": 1.2401179075241089,
+      "shuffled": 2.227147340774536,
+      "noise": 2.0414516925811768
+    },
+    {
+      "organism_id": "v5_3090c913267fb4b9",
+      "matched": 1.865620732307434,
+      "shuffled": 1.838226318359375,
+      "noise": 1.858734130859375
+    },
+    {
+      "organism_id": "v5_9696f5b535ded82c",
+      "matched": 1.3206456899642944,
+      "shuffled": 1.2936960458755493,
+      "noise": 1.2840447425842285
+    },
+    {
+      "organism_id": "v5_753372398c776bb2",
+      "matched": 1.7361646890640259,
+      "shuffled": 2.8518497943878174,
+      "noise": 2.7215073108673096
+    },
+    {
+      "organism_id": "v5_2e4e1174a439bc7e",
+      "matched": 1.4389680624008179,
+      "shuffled": 2.3415775299072266,
+      "noise": 2.1379380226135254
+    },
+    {
+      "organism_id": "v5_dbc0cf006db6ad2f",
+      "matched": 2.48909854888916,
+      "shuffled": 5.830270767211914,
+      "noise": 4.491251468658447
+    },
+    {
+      "organism_id": "v5_f331837b02fa834a",
+      "matched": 2.4679675102233887,
+      "shuffled": 2.697389841079712,
+      "noise": 2.7170310020446777
+    },
+    {
+      "organism_id": "v5_db6ef3d921d2ed9a",
+      "matched": 1.7641488313674927,
+      "shuffled": 3.3504087924957275,
+      "noise": 2.8445920944213867
+    },
+    {
+      "organism_id": "v5_9e3e2bfa949c64ca",
+      "matched": 1.6737147569656372,
+      "shuffled": 2.6998372077941895,
+      "noise": 2.530745029449463
+    },
+    {
+      "organism_id": "v5_cf52bb5955e45e5f",
+      "matched": 1.792728066444397,
+      "shuffled": 2.1553244590759277,
+      "noise": 2.0278258323669434
+    },
+    {
+      "organism_id": "v5_3de783ebd8c522c9",
+      "matched": 1.2120722532272339,
+      "shuffled": 1.3657867908477783,
+      "noise": 1.4130511283874512
+    },
+    {
+      "organism_id": "v5_9039ea84ad074bd7",
+      "matched": 1.9681494235992432,
+      "shuffled": 2.0718178749084473,
+      "noise": 2.0619397163391113
+    },
+    {
+      "organism_id": "v5_20a8aaf8d22d3b85",
+      "matched": 1.3520941734313965,
+      "shuffled": 1.4594213962554932,
+      "noise": 1.474931240081787
+    },
+    {
+      "organism_id": "v5_a32a67a7180e0d4f",
+      "matched": 1.4276725053787231,
+      "shuffled": 1.7758073806762695,
+      "noise": 1.7101017236709595
+    },
+    {
+      "organism_id": "v5_41129a57aa06e403",
+      "matched": 1.961869716644287,
+      "shuffled": 3.4665534496307373,
+      "noise": 3.248152732849121
+    },
+    {
+      "organism_id": "v5_be59c95e75b40e15",
+      "matched": 1.9541147947311401,
+      "shuffled": 2.96565580368042,
+      "noise": 2.755035400390625
+    },
+    {
+      "organism_id": "v5_512d5c0e8afc7eca",
+      "matched": 1.475387692451477,
+      "shuffled": 2.1973297595977783,
+      "noise": 2.020308256149292
+    },
+    {
+      "organism_id": "v5_7760d0d4dde89d8b",
+      "matched": 1.2702906131744385,
+      "shuffled": 1.9381628036499023,
+      "noise": 1.8579655885696411
+    },
+    {
+      "organism_id": "v5_0aeec0b3d5b7489f",
+      "matched": 2.605447769165039,
+      "shuffled": 2.8328521251678467,
+      "noise": 2.746077299118042
+    },
+    {
+      "organism_id": "v5_16d22a7e3f22ec12",
+      "matched": 1.6385420560836792,
+      "shuffled": 2.115669012069702,
+      "noise": 2.0716001987457275
+    },
+    {
+      "organism_id": "v5_c11c4070f7583674",
+      "matched": 1.5188231468200684,
+      "shuffled": 1.8135302066802979,
+      "noise": 1.79706871509552
+    },
+    {
+      "organism_id": "v5_4d2d006551ec7420",
+      "matched": 1.093690037727356,
+      "shuffled": 2.2674436569213867,
+      "noise": 2.1983354091644287
+    },
+    {
+      "organism_id": "v5_691c2436dea1d2df",
+      "matched": 1.0617635250091553,
+      "shuffled": 2.1066205501556396,
+      "noise": 2.0415778160095215
+    },
+    {
+      "organism_id": "v5_9aa8788268907f58",
+      "matched": 2.554494619369507,
+      "shuffled": 2.725722312927246,
+      "noise": 2.705819845199585
+    },
+    {
+      "organism_id": "v5_7e165ea29ff582fd",
+      "matched": 1.0720840692520142,
+      "shuffled": 1.7877113819122314,
+      "noise": 1.7762531042099
+    },
+    {
+      "organism_id": "v5_0b8eb22b8d75d0df",
+      "matched": 1.5549486875534058,
+      "shuffled": 2.188438892364502,
+      "noise": 2.134584903717041
+    },
+    {
+      "organism_id": "v5_9681ec55d03ad361",
+      "matched": 1.5579161643981934,
+      "shuffled": 1.845484733581543,
+      "noise": 1.7609364986419678
+    },
+    {
+      "organism_id": "v5_e1557e75a96f2907",
+      "matched": 1.2822339534759521,
+      "shuffled": 1.479034662246704,
+      "noise": 1.5519602298736572
+    },
+    {
+      "organism_id": "v5_4808c48d3dd029c3",
+      "matched": 0.9869111180305481,
+      "shuffled": 1.2977832555770874,
+      "noise": 1.1876929998397827
+    },
+    {
+      "organism_id": "v5_45a4a5fe04f4ab37",
+      "matched": 2.8223936557769775,
+      "shuffled": 3.4483070373535156,
+      "noise": 3.3587419986724854
+    },
+    {
+      "organism_id": "v5_5a85257410711f6b",
+      "matched": 1.5598406791687012,
+      "shuffled": 3.1406357288360596,
+      "noise": 2.7084200382232666
+    },
+    {
+      "organism_id": "v5_ba61ea2f9dd75ca0",
+      "matched": 2.303010940551758,
+      "shuffled": 2.431258201599121,
+      "noise": 2.4346466064453125
+    },
+    {
+      "organism_id": "v5_b1db8e19f16beb63",
+      "matched": 1.5728211402893066,
+      "shuffled": 1.961814284324646,
+      "noise": 1.9649956226348877
+    },
+    {
+      "organism_id": "v5_3ae10e563c57844a",
+      "matched": 1.5062973499298096,
+      "shuffled": 2.0985891819000244,
+      "noise": 1.8610107898712158
+    },
+    {
+      "organism_id": "v5_381012f9e1db5e45",
+      "matched": 2.5151474475860596,
+      "shuffled": 4.778308868408203,
+      "noise": 3.6078436374664307
+    },
+    {
+      "organism_id": "v5_7e0a94098f276a25",
+      "matched": 0.8177076578140259,
+      "shuffled": 4.822515487670898,
+      "noise": 3.995978832244873
+    },
+    {
+      "organism_id": "v5_104f6b28984419c3",
+      "matched": 2.4706640243530273,
+      "shuffled": 2.889251708984375,
+      "noise": 2.652143716812134
+    },
+    {
+      "organism_id": "v5_774ad42ea53dc612",
+      "matched": 1.715755581855774,
+      "shuffled": 2.1614532470703125,
+      "noise": 2.1236109733581543
+    },
+    {
+      "organism_id": "v5_2c41544d335d38e6",
+      "matched": 1.4870576858520508,
+      "shuffled": 2.385751485824585,
+      "noise": 2.3638975620269775
+    },
+    {
+      "organism_id": "v5_a2df1cf08a6bc383",
+      "matched": 2.0583784580230713,
+      "shuffled": 3.8106141090393066,
+      "noise": 3.4278156757354736
+    },
+    {
+      "organism_id": "v5_aefdcdd508ade733",
+      "matched": 2.0514931678771973,
+      "shuffled": 3.3502471446990967,
+      "noise": 3.1536002159118652
+    },
+    {
+      "organism_id": "v5_6d59fa1d767b44d7",
+      "matched": 1.419472575187683,
+      "shuffled": 1.866255521774292,
+      "noise": 1.8590967655181885
+    },
+    {
+      "organism_id": "v5_f07582c68d1fef93",
+      "matched": 1.4201760292053223,
+      "shuffled": 2.5180187225341797,
+      "noise": 2.3798186779022217
+    },
+    {
+      "organism_id": "v5_6d6333a3a5009e3f",
+      "matched": 2.4906437397003174,
+      "shuffled": 4.060962677001953,
+      "noise": 3.172799587249756
+    },
+    {
+      "organism_id": "v5_c8d4ad97605f1313",
+      "matched": 0.8683784008026123,
+      "shuffled": 2.42240309715271,
+      "noise": 2.1285316944122314
+    },
+    {
+      "organism_id": "v5_3e808b6b7e80ef8f",
+      "matched": 2.101116418838501,
+      "shuffled": 3.135356903076172,
+      "noise": 3.058634042739868
+    },
+    {
+      "organism_id": "v5_9a53d2c66b434115",
+      "matched": 2.162527084350586,
+      "shuffled": 2.6494240760803223,
+      "noise": 2.4938361644744873
+    },
+    {
+      "organism_id": "v5_e6c27fa606b3acf9",
+      "matched": 2.6896779537200928,
+      "shuffled": 2.791851282119751,
+      "noise": 2.6365089416503906
+    },
+    {
+      "organism_id": "v5_62bd84c68c45dea4",
+      "matched": 1.066222906112671,
+      "shuffled": 2.322336435317993,
+      "noise": 2.090501070022583
+    },
+    {
+      "organism_id": "v5_4719d297fa69d3e7",
+      "matched": 1.7770150899887085,
+      "shuffled": 3.0180001258850098,
+      "noise": 2.860306978225708
+    },
+    {
+      "organism_id": "v5_d26dc7ade8f2b6b7",
+      "matched": 1.8700844049453735,
+      "shuffled": 2.324976921081543,
+      "noise": 2.2490272521972656
+    },
+    {
+      "organism_id": "v5_cc02bff921d606c6",
+      "matched": 1.6060435771942139,
+      "shuffled": 1.9746373891830444,
+      "noise": 1.8973249197006226
+    },
+    {
+      "organism_id": "v5_5e3e0179c3f8442d",
+      "matched": 2.9109182357788086,
+      "shuffled": 3.931473731994629,
+      "noise": 3.4166066646575928
+    },
+    {
+      "organism_id": "v5_cbc44b2aa7282cda",
+      "matched": 1.5666316747665405,
+      "shuffled": 2.642120122909546,
+      "noise": 2.372022867202759
+    },
+    {
+      "organism_id": "v5_7f0919fe8fea3433",
+      "matched": 1.220447301864624,
+      "shuffled": 1.6399749517440796,
+      "noise": 1.5758306980133057
+    },
+    {
+      "organism_id": "v5_f4bb78a8d60143b2",
+      "matched": 1.5408339500427246,
+      "shuffled": 2.5732460021972656,
+      "noise": 2.57490611076355
+    },
+    {
+      "organism_id": "v5_93a63a73bde7a7ff",
+      "matched": 1.1791237592697144,
+      "shuffled": 2.0310823917388916,
+      "noise": 1.9440889358520508
+    },
+    {
+      "organism_id": "v5_f2ed63eb416a8f82",
+      "matched": 1.3327630758285522,
+      "shuffled": 1.544671654701233,
+      "noise": 1.5494465827941895
+    },
+    {
+      "organism_id": "v5_150823a4a54ddcc5",
+      "matched": 1.7066978216171265,
+      "shuffled": 5.633577823638916,
+      "noise": 3.5278115272521973
+    },
+    {
+      "organism_id": "v5_6985e5b5afab3a5c",
+      "matched": 0.8937165141105652,
+      "shuffled": 1.391229510307312,
+      "noise": 1.371335744857788
+    },
+    {
+      "organism_id": "v5_f6ecd0a95144a3e3",
+      "matched": 1.02951180934906,
+      "shuffled": 1.8013997077941895,
+      "noise": 1.7202296257019043
+    },
+    {
+      "organism_id": "v5_a769a4919c264ff3",
+      "matched": 1.5716952085494995,
+      "shuffled": 2.8580381870269775,
+      "noise": 2.475167989730835
+    },
+    {
+      "organism_id": "v5_a7c49b9c7e1be84f",
+      "matched": 3.2338409423828125,
+      "shuffled": 3.506474733352661,
+      "noise": 3.2013869285583496
+    },
+    {
+      "organism_id": "v5_d1000146bb6021ed",
+      "matched": 2.5785059928894043,
+      "shuffled": 3.0320119857788086,
+      "noise": 3.0019304752349854
+    },
+    {
+      "organism_id": "v5_d870ac6d4ab805c0",
+      "matched": 2.7698867321014404,
+      "shuffled": 3.4278502464294434,
+      "noise": 3.1315696239471436
+    },
+    {
+      "organism_id": "v5_bbd65b77817d8e21",
+      "matched": 1.889499306678772,
+      "shuffled": 3.45671010017395,
+      "noise": 3.0247087478637695
+    },
+    {
+      "organism_id": "v5_c90af5c60061acfe",
+      "matched": 1.4685927629470825,
+      "shuffled": 2.2004098892211914,
+      "noise": 2.1763548851013184
+    },
+    {
+      "organism_id": "v5_5ef23b0ea7e5a974",
+      "matched": 1.0920649766921997,
+      "shuffled": 1.9540287256240845,
+      "noise": 1.8693851232528687
+    },
+    {
+      "organism_id": "v5_c11b06457598ea6d",
+      "matched": 1.0069178342819214,
+      "shuffled": 2.509422540664673,
+      "noise": 2.4070825576782227
+    },
+    {
+      "organism_id": "v5_35c06e7c58a1d04b",
+      "matched": 1.3147706985473633,
+      "shuffled": 1.5331697463989258,
+      "noise": 1.5408461093902588
+    },
+    {
+      "organism_id": "v5_8438991263c158d6",
+      "matched": 1.3781622648239136,
+      "shuffled": 2.4496445655822754,
+      "noise": 2.2662620544433594
+    },
+    {
+      "organism_id": "v5_a8bf75ffa14d177a",
+      "matched": 2.598907947540283,
+      "shuffled": 2.8516855239868164,
+      "noise": 2.8034958839416504
+    },
+    {
+      "organism_id": "v5_12ec644b171178a8",
+      "matched": 1.7750072479248047,
+      "shuffled": 3.409705877304077,
+      "noise": 3.234724760055542
+    },
+    {
+      "organism_id": "v5_d8ac9826fc8fda6c",
+      "matched": 2.8059401512145996,
+      "shuffled": 3.7601068019866943,
+      "noise": 3.432051420211792
+    },
+    {
+      "organism_id": "v5_81ddf6e35d405e5e",
+      "matched": 1.515273094177246,
+      "shuffled": 2.439751386642456,
+      "noise": 2.3895411491394043
+    },
+    {
+      "organism_id": "v5_4b47d0194b152c2e",
+      "matched": 1.7882217168807983,
+      "shuffled": 2.890564203262329,
+      "noise": 2.72198748588562
+    },
+    {
+      "organism_id": "v5_8ace10fe7c86a4c2",
+      "matched": 1.6890827417373657,
+      "shuffled": 2.188042163848877,
+      "noise": 2.1562678813934326
+    },
+    {
+      "organism_id": "v5_f8186ff32e26ae4c",
+      "matched": 2.0961990356445312,
+      "shuffled": 2.4040820598602295,
+      "noise": 2.3982412815093994
+    },
+    {
+      "organism_id": "v5_7dc1818b44963010",
+      "matched": 2.3634026050567627,
+      "shuffled": 2.637030839920044,
+      "noise": 2.5666935443878174
+    },
+    {
+      "organism_id": "v5_6265e4bb09d5bdb4",
+      "matched": 1.6260559558868408,
+      "shuffled": 2.267789840698242,
+      "noise": 2.090775728225708
+    },
+    {
+      "organism_id": "v5_fc737eb2afd63a99",
+      "matched": 1.329423427581787,
+      "shuffled": 2.4283509254455566,
+      "noise": 2.3761770725250244
+    },
+    {
+      "organism_id": "v5_17343a66b8ee0823",
+      "matched": 3.137892484664917,
+      "shuffled": 3.6989715099334717,
+      "noise": 3.598201274871826
+    },
+    {
+      "organism_id": "v5_b25d20e37b30222b",
+      "matched": 1.2974218130111694,
+      "shuffled": 3.591032028198242,
+      "noise": 2.9272918701171875
+    },
+    {
+      "organism_id": "v5_fe5068306a4896fe",
+      "matched": 2.013674020767212,
+      "shuffled": 2.7311291694641113,
+      "noise": 2.547024965286255
+    },
+    {
+      "organism_id": "v5_13f95bc614168110",
+      "matched": 3.0499608516693115,
+      "shuffled": 3.929028272628784,
+      "noise": 3.6817023754119873
+    },
+    {
+      "organism_id": "v5_6cfc2215a1c7af88",
+      "matched": 1.9544464349746704,
+      "shuffled": 3.6263041496276855,
+      "noise": 3.2689361572265625
+    },
+    {
+      "organism_id": "v5_577e5f877d827cc7",
+      "matched": 1.1363279819488525,
+      "shuffled": 2.057431697845459,
+      "noise": 1.9924256801605225
+    },
+    {
+      "organism_id": "v5_eb136fc7bc848ff1",
+      "matched": 1.8170592784881592,
+      "shuffled": 2.759333372116089,
+      "noise": 2.5978620052337646
+    },
+    {
+      "organism_id": "v5_11d8b10f26baaf85",
+      "matched": 1.2947477102279663,
+      "shuffled": 1.7322558164596558,
+      "noise": 1.6213836669921875
+    },
+    {
+      "organism_id": "v5_5f1c8a39b44f9a00",
+      "matched": 1.376170039176941,
+      "shuffled": 2.191831350326538,
+      "noise": 2.0622129440307617
+    },
+    {
+      "organism_id": "v5_b27f12883c0b405c",
+      "matched": 1.431840181350708,
+      "shuffled": 2.3958146572113037,
+      "noise": 2.245697259902954
+    },
+    {
+      "organism_id": "v5_bf6bedd5435ec2b0",
+      "matched": 2.473461389541626,
+      "shuffled": 2.4786462783813477,
+      "noise": 2.512812376022339
+    },
+    {
+      "organism_id": "v5_b7b7ab7e9b84a72f",
+      "matched": 2.8044185638427734,
+      "shuffled": 3.1068315505981445,
+      "noise": 2.9111998081207275
+    },
+    {
+      "organism_id": "v5_f1084a059515e94b",
+      "matched": 2.0422558784484863,
+      "shuffled": 2.040428876876831,
+      "noise": 2.0447890758514404
+    },
+    {
+      "organism_id": "v5_8790cf1e8ea2749e",
+      "matched": 1.9964356422424316,
+      "shuffled": 2.2834789752960205,
+      "noise": 2.196808099746704
+    },
+    {
+      "organism_id": "v5_9a1ca5a05b0b59a4",
+      "matched": 1.5448135137557983,
+      "shuffled": 3.145437717437744,
+      "noise": 3.0666418075561523
+    },
+    {
+      "organism_id": "v5_b1e8ea30fde1406b",
+      "matched": 1.4893736839294434,
+      "shuffled": 2.2129108905792236,
+      "noise": 2.083682060241699
+    },
+    {
+      "organism_id": "v5_1d9da9be1206ee8a",
+      "matched": 1.7441554069519043,
+      "shuffled": 2.735351324081421,
+      "noise": 2.6512389183044434
+    },
+    {
+      "organism_id": "v5_09572677f22fd189",
+      "matched": 2.881904125213623,
+      "shuffled": 4.324495792388916,
+      "noise": 3.944253444671631
+    },
+    {
+      "organism_id": "v5_0a767bd8cfad5e8e",
+      "matched": 2.364823341369629,
+      "shuffled": 3.064826250076294,
+      "noise": 3.054764986038208
+    },
+    {
+      "organism_id": "v5_b31fa41978a64e67",
+      "matched": 2.2817189693450928,
+      "shuffled": 4.2215962409973145,
+      "noise": 3.3575921058654785
+    },
+    {
+      "organism_id": "v5_4a94f175219f37f2",
+      "matched": 1.6324044466018677,
+      "shuffled": 2.5290799140930176,
+      "noise": 2.290733575820923
+    },
+    {
+      "organism_id": "v5_f3457f82d9b1ebd9",
+      "matched": 1.532150387763977,
+      "shuffled": 2.412153482437134,
+      "noise": 2.3279221057891846
+    },
+    {
+      "organism_id": "v5_3a1eadd7184a1f05",
+      "matched": 0.8521133661270142,
+      "shuffled": 2.3111491203308105,
+      "noise": 2.1653096675872803
+    },
+    {
+      "organism_id": "v5_f1a0b0b772b71ea7",
+      "matched": 1.6047781705856323,
+      "shuffled": 2.9315056800842285,
+      "noise": 2.603949546813965
+    },
+    {
+      "organism_id": "v5_9e5ec233ae0f3be1",
+      "matched": 1.4991105794906616,
+      "shuffled": 2.242225408554077,
+      "noise": 2.235595464706421
+    },
+    {
+      "organism_id": "v5_19406024525e08ab",
+      "matched": 2.649425983428955,
+      "shuffled": 2.711094856262207,
+      "noise": 2.7116994857788086
+    },
+    {
+      "organism_id": "v5_ceb1d1f9fdbb2135",
+      "matched": 1.6131675243377686,
+      "shuffled": 1.9856280088424683,
+      "noise": 1.855822205543518
+    },
+    {
+      "organism_id": "v5_c8f454dcd83fa643",
+      "matched": 2.206796407699585,
+      "shuffled": 8.199703216552734,
+      "noise": 5.972370147705078
+    },
+    {
+      "organism_id": "v5_7ef596319c01d1b7",
+      "matched": 1.711769461631775,
+      "shuffled": 3.054563045501709,
+      "noise": 2.807330369949341
+    },
+    {
+      "organism_id": "v5_a734dd0e1a55d202",
+      "matched": 1.4961013793945312,
+      "shuffled": 2.5818283557891846,
+      "noise": 2.363978147506714
+    },
+    {
+      "organism_id": "v5_c16a566a1817045e",
+      "matched": 2.2103235721588135,
+      "shuffled": 2.624506950378418,
+      "noise": 2.5996663570404053
+    },
+    {
+      "organism_id": "v5_c5675af2d78701d5",
+      "matched": 1.6896677017211914,
+      "shuffled": 2.2785747051239014,
+      "noise": 2.2450759410858154
+    },
+    {
+      "organism_id": "v5_ee63981413ad3f47",
+      "matched": 2.1207728385925293,
+      "shuffled": 2.6519579887390137,
+      "noise": 2.4271743297576904
+    },
+    {
+      "organism_id": "v5_0bb91e529356aa13",
+      "matched": 2.275912284851074,
+      "shuffled": 3.110812187194824,
+      "noise": 2.8978867530822754
+    },
+    {
+      "organism_id": "v5_03beba462bb3a29c",
+      "matched": 1.1585127115249634,
+      "shuffled": 2.020707845687866,
+      "noise": 1.7063103914260864
+    }
+  ],
+  "meta": {
+    "ckpt": "final",
+    "eval_set": "val.parquet",
+    "n_orgs": 181,
+    "noise": "iid N(0,1) [5376,2048], one draw per org",
+    "mean_matched": 1.7985841707630053,
+    "stdev_matched": 0.5210138983416686,
+    "mean_shuffled": 2.6755190883552173,
+    "stdev_shuffled": 0.893253112117546,
+    "mean_noise": 2.4924561081670267,
+    "stdev_noise": 0.6954997362299191
+  }
+}