sam3-p150

Meta SAM3 (Segment Anything with Concepts) port on one Tenstorrent Blackhole p150a. Weights: facebook/sam3 · Paper: arXiv:2511.16719 · Upstream code: facebookresearch/sam3 · Port: changh95/tt-sam3

Runs on p150 (mesh P150).

Packaged and published with tt-model-manager 0.1.0 (manifest schema 5.1).

Quickstart (Python)

Prerequisite: a Python environment with tt-metal's ttnn, built at tt-metal 8b98410e730 with patches/tt-metal-eth-dispatch.patch applied. The facebook/sam3 weights are gated. Request access on the Hub and run hf auth login one time.

hf download changh95/sam3-p150 --exclude "image/*" --local-dir sam3-p150 && cd sam3-p150
pip install -e code/      # into a Python env that already has tt-metal's ttnn
pip install -e "code/[server,test]"     # also the HTTP server (fastapi, uvicorn, ...) and the tests (pytest, httpx)

Run the Python example from the model repo root. The demo image path media/source_1.png is relative to the repo root. You can also give an absolute path.

from tt_sam3 import Sam3

with Sam3.from_pretrained(device_id=0) as model:     # loads facebook/sam3 from the HF cache, opens the chip, warms up
    result = model("media/source_1.png", "car")      # path, PIL.Image, numpy or torch image; str or list of prompts

print(result)          # Sam3Result(prompt='car', num_instances=3, scores=[0.973, 0.817, 0.797], ...)
result.masks           # (N, H, W) bool masks at the original image size
result.boxes           # (N, 4) float32 [x0, y0, x1, y1] in image pixels
result.scores          # (N,) float32, high to low

	Name	Description
Input	image	File path, encoded PNG/JPEG bytes, PIL.Image, numpy array or torch tensor. Any size.
Input	prompts	One noun phrase (str) or a list of noun phrases. A list gives a list of results in the same order.
Option	threshold=0.5	Keep the instances with a score above this value.
Option	max_instances=50	Maximum number of instances for each prompt (1 to 200).
Option	return_masks=True	Set to False to skip the mask upsample. masks is then None.
Option	return_mask_logits=False	Set to True to also get the raw (N, 288, 288) mask logits.
Output	masks, boxes, scores	(N, H, W) bool, (N, 4) float32 xyxy pixels, (N,) float32 sorted high to low.
Output	presence, width, height	Probability that the concept is in the image, and the original image size.

• from_pretrained() opens the chip, loads the weights and captures the device traces. The first run compiles the kernels and can take some minutes.
• Write model = Sam3.from_pretrained(warmup_variants={"prompts": ["car", "person"]}) to prepare the model and the text of your prompts. The first call is then as fast as later calls. A prompt that you did not list adds about 80 to 175 ms to the first call that uses it.
• Use with ... as model: or call model.close() to release the chip.
• Give a list of prompts to segment one image with many prompts. The image part runs one time, and each prompt adds about 20 ms.
• The model keeps the last image on the device. When the same image comes again, the call takes about 21 ms.
• model.predict_many(images, prompts) decodes the next image on a host thread while the chip runs the current image.
• model(...) gives the same results as the HTTP POST /predict, without JSON rounding.

Full reference: code/PYTHON.md. Example with an overlay image: examples/quickstart.py.

Serving (HTTP)

tt-model pull  changh95/sam3-p150 --with-weights
tt-model serve changh95/sam3-p150        # or, with tt-cli: tt serve changh95/sam3-p150
printf '{"image":"%s","prompts":["car"]}' "$(base64 -w0 media/source_1.png)" > req.json
curl -s localhost:20000/predict -H 'Content-Type: application/json' -d @req.json
tt model stop changh95/sam3-p150

• Weights facebook/sam3 go to your HF cache; the image does not contain them.
• Serves on port 20000 (or the next free port); ready when the log says Application startup complete.
• POST /predict: image (base64 PNG/JPEG), prompts (1–4 noun phrases); optional threshold (0.5), max_instances (50), mask_format (png | rle | none).
• POST /predict_raw: the same results with a binary body. Send PNG/JPEG bytes, or raw RGB24 pixels with Content-Type: application/x-rgb24 and the X-Width and X-Height headers. Give prompts and the options as query parameters. This route is in code/ and is not in the container image yet.
• GET /health, GET /info.

{"width": 1920, "height": 1080, "input_size": 1008, "threshold": 0.5, "mask_format": "png",
 "results": [{"prompt": "car", "presence": 0.995, "num_instances": 3, "instances": [
    {"score": 0.9727, "box_xyxy": [545.8, 447.9, 1159.1, 854.4], "mask_area": 175127, "mask_png": "..."}, "..."]}],
 "timing_ms": {"decode": "...", "preprocess": "...", "forward": "...", "postprocess": "...", "total": "..."}}

• One results entry per prompt; instances are sorted by score, box_xyxy is [x0, y0, x1, y1] in original image pixels.
• mask_png is a base64 8-bit 0/255 PNG at the original H×W; rle gives COCO uncompressed column-major {size, counts}, none omits masks.

Demo

Synthetic street scene, prompt "car".

Input (media/source_1.png)	Masks on p150a (media/target_1_ttnn.png)

Demo & Performances

Warm, batch 1, demo image (1920×1080 PNG) + "car", served /predict, 2 runs of 20 requests (median per run). All rows use the p150 configuration: dispatch on ETH cores, 1 command queue (CQ), 12×10 compute grid.

Metric	Performance
End-to-end /predict timing_ms.total, 1 prompt	95.5–97.0 ms median · 93.9 ms min
End-to-end /predict timing_ms.total, 3 prompts	138.5–139.4 ms median · 135.8 ms min
timing_ms.forward, 1 prompt (upload + image trace + prompt trace + kept-mask readback)	81.3–83.5 ms median
HTTP client wall, 1 prompt: JSON /predict · /predict_raw PNG body · raw RGB24 body (44 requests)	106.1–107.0 ms · 100.8–101.5 ms · 89.1–91.0 ms median
Image trace (ViT-H/14 + FPN neck + geometry encoder)	56.4 ms
Prompt trace (fusion encoder + decoder + scoring + seg head)	18.3 ms
Host selection + selected-row mask trace + readback	1.2–1.5 ms
forward_image (all 200 masks read back to host)	85.1 ms median · 84.0 ms min
Same image again with a new prompt, timing_ms.total (image cache, code/ only)	22.7 ms median (cache off: 95.5–97.0 ms)
Python model() call: model(rgb, "car") on a decoded RGB array, masks included	84.6 ms median · 81.9 ms min (82.8 / 80.4 ms with return_masks=False)

The measurement hardware is a Blackhole chip with a 12×10 compute grid and dispatch on ETH cores (patches/tt-metal-eth-dispatch.patch), with 1 command queue. This is the configuration of a single p150. On 2026-10-05, the developer and an independent verifier measured the served rows and the HTTP rows again in this configuration (x1 PCIe link, shared host). Each range shows the two runs. The trace rows and the forward_image row are from the 2026-10-03 verification (x8 PCIe link). Accuracy against the torch CPU fp32 reference (demo image, "car"): top-mask IoU288 0.9986, 3/3 instances, top score 0.9731 (reference 0.9716), all-200 mask PCC 0.9806 (gate 0.98). Details: VERIFICATION_2026-10-03.md.

Before 2026-10-05, this card showed a served total of 96.8 ms (1 prompt) and 137 ms (3 prompts). The server measured these numbers with worker (Tensix) dispatch. On the measurement chip, worker dispatch also gives 12×10, but on a p150 it gives only 11×10. ETH dispatch gives the same numbers within about 2 ms, and the outputs are bit-identical. Details: VERIFICATION_2026-10-03.md.

The 2026-10-04 code/ adds the Python API, the image cache and POST /predict_raw. These changes do not change the device programs or the output bits. Thus, the device numbers and the accuracy stay the same. The Python row uses chip 14, SAM3_NUM_THREADS=8 and 3 interleaved rounds of 10 calls. With a file path as input, the call takes 97.0 ms median because it includes the PNG decode. Details: VERIFICATION_2026-10-03.md.

RTX 5090 reference measurements (2026-09-14) are unchanged. They use the port's torch reference in eager PyTorch 2.11, batch 1, with H2D/D2H included, plus the original PIL host decode, preprocess and postprocess. The current build values are the means of the two 2026-10-05 runs. Full table: GPU_COMPARISON.md.

RTX 5090 precision	GPU served-like total	vs current build (96.3 ms)	GPU preprocess + forward vs ours (82.4 ms forward)
fp32 strict	202.9 ms	Blackhole 2.11× faster	155.3 ms: Blackhole 1.88× faster
tf32	162.6 ms	Blackhole 1.69× faster	113.8 ms: Blackhole 1.38× faster
bf16 autocast	111.5 ms	Blackhole 1.16× faster	63.9 ms: GPU 1.29× faster
fp16 autocast	112.7 ms	Blackhole 1.17× faster	64.2 ms: GPU 1.28× faster

Most of the end-to-end lead comes from the host path. This build spends about 14 ms on decode and postprocess. The GPU loop spends 47.2 ms on the same steps with PIL. On the model forward alone, bf16 and fp16 GPU are about 1.3× faster.

Caveats

• Does not scale to multiple p150a in a mesh configuration. The current build uses a 12x10 compute grid of Tensix cores. To get this grid, the dispatch functions move from 10 Tensix cores to ETH cores (patches/tt-metal-eth-dispatch.patch). Thus, this build assumes that you do not need chip-to-chip ethernet communication.
• The newer code/ opens the chip with ETH dispatch and 1 CQ by default (SAM3_DISPATCH=auto, also dispatch="auto" in the Python API). SAM3_DISPATCH=worker is an opt-in for Galaxy chips only. It gives an 11×10 grid on a p150. Without the ETH-dispatch patch, auto uses worker dispatch and logs a warning. The container image was built before this change, and its server uses worker dispatch until the image is rebuilt.
• Every image is squashed to 1008×1008. One image per request, batch 1. Text prompts only (no box or point prompts). Each additional prompt adds one prompt trace (about 20 ms served).
• facebook/sam3 is gated under the Meta SAM License: request access and hf auth login on the host before serve (3.45 GB sam3.pt).
• The whole image graph runs on the device as two metal traces. The text encoder runs on the host CPU, and its output is cached per prompt.
• The all-200 demo mask PCC is 0.9806 (previous release: 0.984). The margin to the 0.98 gate is small. The device neck, decoder and seg head change numerics, and so do some kernel changes (GELU, GroupNorm, SDPA chunk sizes). OPT_REPORT.md lists each change.
• Not an OpenAI-compatible API; GET /v1/models is a stub so the tt-model ready card does not 404.
• p150a power was not measured, so no efficiency comparison is made.

Licensing

• Weights: facebook/sam3, Meta SAM License (gated, not OSI).
• Port and serving code (code/): from changh95/tt-sam3, distributed under the same SAM License terms; code/tt_sam3/assets/bpe_simple_vocab_16e6.txt.gz is the MIT-licensed CLIP BPE vocabulary vendored from the upstream sam3 repo.

Provenance

These are the exact sources the container image was built from. code/ has since been updated (2026-10-04 build: optimized device path, image cache, /predict_raw and the Python API; 2026-10-05: ETH dispatch and 1 CQ are the default for the server and the harnesses; see OPT_REPORT.md and code/PYTHON.md) and is newer than the image. tt-model serve runs the image's code until the image is rebuilt. tt-model.yaml and SERVING.md still describe the image:

component	built from
tt-metal	8b98410e730bb504fea43a88609756e34821d91d
code/ digest (image)	152efb7f553f443f (sha256, first 16 hex digits; the current code/ differs)
built	2026-09-13T16:04:02+00:00 by tt-model 0.1.0