mlboydaisuke/gemma-4-E2B-coreml

Gemma 4 E2B — CoreML (ANE+GPU Optimized)

Converted from google/gemma-4-E2B-it for on-device inference on Apple devices via CoreML.

Models

File	Size	Description
model.mlpackage	2.4 GB	Text decoder with stateful KV cache (int4 quantized)
vision.mlpackage	322 MB	Vision encoder (SigLIP-based, 16 transformer layers)
model_config.json	—	Model configuration
hf_model/tokenizer.json	31 MB	Tokenizer

Features

• Multimodal: Image + text input → text output
• ANE-optimized: Conv2d linear layers, ANE RMSNorm, in-model argmax
• Stateful KV cache: MLState API (iOS 18+)
• Int4 quantized: Block-wise palettization (group_size=32)
• HF-exact match: "solid red square centered on white background" ✅

Usage

import coremltools as ct
import numpy as np

# Load models
vision = ct.models.MLModel('vision.mlpackage')
decoder = ct.models.MLModel('model.mlpackage')
state = decoder.make_state()

# Process image → vision features → text generation

See CoreML-LLM for the full conversion pipeline and iOS sample app.

Conversion

git clone https://github.com/john-rocky/CoreML-LLM
cd CoreML-LLM/conversion
pip install -r requirements.txt
python convert.py --model gemma4-e2b --context-length 512 --output ./output/gemma4-e2b