fomofo/TAGS-ReXGroundingCT

TAGS: weights of the fomofo submission to ReXGroundingCT (MICCAI 2026, Track 1)

The models behind TAGS_TAPv3, ranked 3rd in Track 1 of the ReXGroundingCT challenge (Dice 0.362 on the full test set). Given a chest CT and a sentence from the radiology report, they segment the finding the sentence describes. Code: github.com/nki-radiology/TAGS-ReXGroundingCT.

What is here

path	content	size
voxtell/<member>.safetensors	the 12 fine-tuned VoxTell members (440M parameters each)	~1.7 GB each
tapct/seed{0..4}.safetensors	the 5 fine-tuned TAP-CT seeds (encoder + FiLM-conditioned decoder, 93M parameters)	~0.35 GB each
postprocessing/thr_model_lobe16.pkl	per-finding threshold model (scikit-learn 1.9.0 pickle: gradient boosting + ridge)	small
postprocessing/comp_head4.safetensors	component head (small 3D CNN that removes components predicted to lower Dice)	small

The final submission averages the 12 VoxTell probability maps, fuses them 0.8/0.2 with the mean of the 5 TAP-CT seeds, restricts the map to the lobe or lung named in the sentence, applies the threshold model and the component head, and resolves voxels claimed by several findings. All of this is implemented in the GitHub repository.

The twelve VoxTell members were fine-tuned from VoxTell v1.1 on the challenge training + validation data, each with one difference:

member	what is different
plain, plain_2	nothing (two runs)
tversky_fp, tversky_fp_2	Tversky loss 0.3/0.7, penalising false positives
tversky_fn	Tversky loss 0.6/0.4, penalising false negatives
aux_effusion	+ 1,540 CT-RATE effusion pseudo-labels
aux_nodule_effusion	+ 8,000 CT-RATE nodule and effusion pseudo-labels
fusion24	finer 24³ text–image fusion grid
lobe_aux	lung-lobe masks as auxiliary target
nodule_boost15, nodule_boost20	scans with a nodule finding sampled 1.5× / 2×
category_balance	scans with rare finding categories sampled more often

These are the exact submitted checkpoints. Five of them (tversky_fp_2, aux_effusion, aux_nodule_effusion, fusion24, lobe_aux) were trained in several stages with earlier recipes before the final one; the repository's HISTORY.md has the details. The TAP-CT seeds were trained on training + validation data; the threshold model and component head were fitted on validation predictions of train-only models.

How to use

git clone https://github.com/nki-radiology/TAGS-ReXGroundingCT && cd TAGS-ReXGroundingCT
pip install -r requirements.txt
python tags/hf_weights.py          # downloads everything and writes the checkpoints where the scripts expect them

Then scripts/40_predict_test.sh (VoxTell probability maps), scripts/50_tapct.sh (TAP-CT maps; only the export steps are needed) and scripts/70_build_submission.sh (fusion and post-processing). The .safetensors files store each tensor once; tags/hf_weights.py restores the weights VoxTell shares between encoder and decoder from the alias list in the file metadata and rebuilds the original checkpoint dictionaries.

Training data and test-set exposure

• Challenge training (2,992 scans) and validation (200 scans) sets of ReXGroundingCT.
• aux_effusion and aux_nodule_effusion additionally saw CT-RATE report sentences paired with TotalSegmentator pseudo-masks. That pool excluded the challenge validation patients but, unintentionally, not the test patients: 80 of the 300 test studies were in it (as image, CT-RATE report sentence and pseudo-mask; never the challenge masks or finding sentences). Retraining both members on a pool without these studies changed the final masks by mean Dice 0.025, equally on exposed and unexposed studies.
• VoxTell v1.1 itself was pretrained by its authors on 190 public datasets, including the ReXGroundingCT training and public-validation scans.
• TAP-CT was pretrained by its authors on in-house CT data.

Licences

• The VoxTell members, the threshold model and the component head: CC BY-NC-SA 4.0 (they derive from VoxTell v1.1 weights and ReXGroundingCT data, both CC BY-NC-SA 4.0). Non-commercial use only.
• The TAP-CT seeds: CC BY-NC 4.0 (they derive from the fomofo/tap-ct-b-3d encoder). Non-commercial use only.
• The code in the GitHub repository is Apache-2.0.

Citation

Paper in preparation. Please also cite VoxTell (Rokuss et al., 2025) and TAP-CT (Veenboer et al., 2025).