Arm/pocket-tts-de-24l-mix-precision

Pocket TTS German 24 Large - Mixed Precision and Mixed Frameworks

This repository packages a mixed-precision, mixed-framework deployment bundle derived from kyutai/pocket-tts for German text-to-speech using the Large 24-layer FlowLM variant. The bundle uses SentencePiece, GGUF, ONNX, and LiteRT artifacts and supports streaming synthesis at 24 kHz.

✨ Key Highlights

• Faster than real-time — using one Arm® CPU core with SME2 (an RTF lower than 1 is faster than real time):
  • RTF is 0.582 on an Android™ Vivo X300 smartphone.
• Low latency — using one Arm® CPU core with SME2 on an Android™ Vivo X300 smartphone:
  • Median end-to-end latency is 1.978 s for a prompt that produces 3.40 seconds of audio.
  • Median time to first audio chunk is 0.398 s.
• Memory efficient — measured on an Android™ Vivo X300 smartphone:
  • Peak process RSS is 2695.2 MiB.
• Large deployment — the packaged model files total 1289.8 MB.
• German text-to-speech — generates spoken German from a text prompt.
• Higher-quality Large variant — uses a 24-layer FlowLM for higher-quality synthesis, with slower inference than the smaller variant.
• Packaged voice profiles — includes 26 voice profiles.
• Arm® CPU deployment — optimized for efficient execution on Arm® CPUs.
• Streaming output — returns synthesized speech at 24 kHz in streaming mode.

📦 Model Details

Model Description

Pocket TTS is a lightweight text-to-speech model developed by Kyutai for efficient CPU execution. This package contains the German Large variant with a 24-layer FlowLM.

• Developed by: Kyutai
• Model type: German text-to-speech with a Large 24-layer FlowLM and packaged voice profiles
• License: CC BY 4.0
• Base model: kyutai/pocket-tts
• Package form: SentencePiece, GGUF, ONNX, TFLite, and NumPy artifacts

Model Sources

• Base model: https://huggingface.co/kyutai/pocket-tts
• Upstream repository: https://github.com/kyutai-labs/pocket-tts
• Voice collection: https://huggingface.co/kyutai/tts-voices

🚀 Get Started with the Model

🔓 Compute Flow — Early Access

The inference engine for this model package is available through the Compute Flow Early Access Program.

Want to try it?

📩 Contact us at [email protected] to request access.

📊 Quality evaluation

Accuracy results for this mixed-precision package will be provided in a future update. However, no intelligibility regression was observed against the upstream FP32 implementation.

🎯 Performance evaluation

Performance was measured under the following conditions:

• One Arm® CPU thread.
• 5 warmups followed by 30 consecutive measured runs with no pause between runs.
• The Android™ Vivo X300 smartphone screen was kept on.
• The model remained loaded and its state was reset between runs.

The following methodology and definitions were used:

• Runtime: LiteRT, llama.cpp, ONNX Runtime, and SentencePiece on CPU, with XNNPACK and KleidiAI.
• Input prompt: "Hallo zusammen. Ich bin Jack und ich bin Ihr persönlicher Assistent."
• Voice profile: juergen, the packaged default.
• Generated output: 3.40 seconds of 24 kHz audio per run.
• End-to-end latency is the summed model execution time for the complete utterance and excludes model setup.
• Time to first audio chunk is the elapsed execution time until the first streaming audio chunk is returned.
• Average memory is the mean process RSS sampled throughout setup and inference.
• Peak memory is the maximum sampled process high-water mark.
• Latency and memory were collected in separate executions to prevent memory sampling from affecting latency.

Metric	Android™ Vivo X300: Mixed precision
Model size	1289.8 MB
RTF	0.582
End-to-end latency, p50	1.978 s
End-to-end latency, p90	2.047 s
End-to-end latency, p99	2.097 s
Time to first audio chunk	0.398 s
Peak memory	2695.2 MiB
Average memory	2659.2 MiB

RTF is the total inference time divided by the generated audio duration, so lower values indicate faster processing and values below 1 indicate faster-than-real-time generation.

🛠️ Technical Specifications

Objective

Generate streaming 24 kHz German speech from a text prompt.

Runtime Architecture

Component role	Framework / format
Text tokenization	SentencePiece
Text encoding, projection, and denoising	ONNX Runtime / ONNX
24-layer flow language modeling	llama.cpp / GGUF
Audio decoding	LiteRT / TFLite
Voice conditioning	NumPy profile

Precision and Quantization

The FlowLM and ONNX components use FP32 weights. The LiteRT audio decoder uses FP16 fully connected weights.

Input Specification

Input	Description
Text prompt	UTF-8 string

Output Specification

The model returns synthesized 24 kHz audio in streaming mode.

Repository Contents

• pocket_tts_manifest.json — model package manifest.
• tokenizer.model — SentencePiece tokenizer.
• *.onnx, *.gguf, and *.tflite — model components.
• assets/*.npy — packaged voice profiles and model assets.
• metadata.yaml and benchmarks/ — model and benchmark metadata.
• SHA256SUMS — model-bundle checksums for reproducibility.

🗂️ Model and Asset Origin

• Pocket TTS model
• Pocket TTS source repository
• Kyutai TTS voice collection and asset licensing

🔐 Checksums

SHA256SUMS was generated by recursively hashing every regular file in the model bundle, including files in subdirectories, except the generated root SHA256SUMS and paths with a dotfile component.

From the model bundle root, verify the checked-out files with:

shasum -a 256 -c SHA256SUMS