Add Unified-Bench old protocol bundle: unified_bench_old_protocol_0p8346/README.md

unified_bench_old_protocol_0p8346/README.md

@@ -0,0 +1,95 @@
+# Unified-Bench old protocol reproduction bundle
+
+This folder contains the assets used to reproduce the historical raw BAGEL
+Unified-Bench score `0.8345940754981711` under the old TorchUMM protocol.
+
+HF path prefix:
+
+```text
+HuggingSelf/test0519/unified_bench_old_protocol_0p8346
+```
+
+## Contents
+
+- `inputs/unified_bench/Image/`: the 100 Unified-Bench reference input images.
+- `outputs/raw_bagel/images/`: raw BAGEL round-trip generated images.
+- `outputs/raw_bagel/captions.jsonl`: image-to-text captions produced by raw BAGEL.
+- `outputs/raw_bagel/summary.json`: original old-protocol score JSON.
+- `scorers/dinov3-vitl16_old_protocol/`: the old DINOv3 scorer checkpoint used by the historical run.
+- `configs/unified_bench_bagel.yaml`: old TorchUMM Unified-Bench config.
+- `scripts/reproduce_apr4_unified_bagel_score.py`: Modal-side reproduction helper used to verify the score.
+- `scripts/run_scoring.py` and `scripts/scorers/`: scoring code snapshot from TorchUMM.
+- `manifest.json`: hashes, counts, paths, and expected scores.
+
+## Expected Score
+
+| scorer | average |
+| --- | ---: |
+| CLIP | 0.894650644659996 |
+| DINOv2 | 0.787667881194502 |
+| DINOv3 old protocol | 0.723970373794436 |
+| LongCLIP | 0.932087402343750 |
+| Overall | 0.834594075498171 |
+
+Overall is the mean of the four scorer averages.
+
+## DINOv3
+
+Old-protocol DINOv3 file:
+
+```text
+scorers/dinov3-vitl16_old_protocol/model.safetensors
+```
+
+Expected SHA256:
+
+```text
+dcb2e45127cccbf1601e5f42fef165eea275c8e5213197e8dcf3f48822718179
+```
+
+The newer cached DINOv3 weight we tested separately produced very high
+reference-to-reference similarity and should not be mixed with this old
+protocol score.
+
+## Reproduction Notes
+
+The historical run was verified on Modal workspace `aifrontierwm` using:
+
+```text
+ref_dir = /datasets/unified_bench/Image
+gen_dir = /output-apr4/unified_bench/bagel/images
+```
+
+Scoring model paths:
+
+```text
+clip     = /model_cache/unified_bench/clip-vit-large-patch14
+dinov2   = /model_cache/unified_bench/dinov2-large
+dinov3   = /model_cache/unified_bench/dinov3-vitl16
+longclip = /model_cache/unified_bench/longclip-L.pt
+```
+
+The bundle includes the old DINOv3 directory because this is the piece that made
+the old protocol reproducible. CLIP, DINOv2, and LongCLIP are public/downloadable
+from the sources listed in `manifest.json`.
+
+## Minimal Download Example
+
+```python
+from huggingface_hub import snapshot_download
+
+root = snapshot_download(
+    repo_id="HuggingSelf/test0519",
+    repo_type="model",
+    allow_patterns=["unified_bench_old_protocol_0p8346/**"],
+)
+print(root)
+```
+
+After downloading, point TorchUMM's Unified-Bench scorer at:
+
+```text
+ref_dir: <download_root>/unified_bench_old_protocol_0p8346/inputs/unified_bench/Image
+out_dir: <download_root>/unified_bench_old_protocol_0p8346/outputs/raw_bagel
+dinov3:  <download_root>/unified_bench_old_protocol_0p8346/scorers/dinov3-vitl16_old_protocol
+```