sevren-ai/Embla1-bf16

Embla 1 (sevren.ai) is a mixture-of-experts language model designed for phones. This repository contains the bf16 checkpoint.

The 4-bit format is the intended precision. Embla 1 was trained quantization-aware for 4-bit weights (affine, group size 64), and the model that runs on-device is the 4-bit release: sevren-ai/Embla1-4bit (MLX). The bf16 weights provided here are the master checkpoint from which those 4-bit weights are rounded.

Model details

  • Parameters: 10B total, including 3B engram parameters; 1B active per token.
  • Context length: 256K tokens (262,144).
  • License: MIT.

Usage

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

path = "sevren-ai/Embla1-bf16"
device = "cuda" if torch.cuda.is_available() else "mps" if torch.backends.mps.is_available() else "cpu"
tok = AutoTokenizer.from_pretrained(path, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(path, trust_remote_code=True, dtype=torch.bfloat16).to(device)
inputs = tok.apply_chat_template([{"role": "user", "content": "Hi!"}], add_generation_prompt=True,
                                 return_tensors="pt", return_dict=True).to(device)
out = model.generate(**inputs, max_new_tokens=256)
print(tok.decode(out[0, inputs["input_ids"].shape[1]:]))
  • Only transformers (5.9 or newer) and safetensors are required; the modeling code ships in this repository. The model can be loaded in bf16 or fp32.
  • The modeling code is an eager PyTorch reference implementation. It is correct anywhere PyTorch runs, but it is not optimized for speed. generate() maintains a KV cache, and the prompt pass runs only its last token through the second half of the layers.
  • Padding is not supported: use a single prompt per call, or batch rows of equal length.
  • The chat format is ChatML (via apply_chat_template). The model first reasons in a <think> block, then produces its answer.
Downloads last month
119
Safetensors
Model size
7B params
Tensor type
BF16
ยท
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support