--- license: mit language: - multilingual pipeline_tag: sentence-similarity library_name: sentence-transformers base_model: microsoft/Multilingual-MiniLM-L12-H384 datasets: - Nanthasit/sakthai-combined-v6 - Nanthasit/sakthai-combined-v7 - Nanthasit/sakthai-kaggle-notebooks - Nanthasit/sakthai-irrelevance-supplement - Nanthasit/food-penguin-v1 - Nanthasit/SimpleToolCalling - Nanthasit/sakthai-bench-v1 tags: - sentence-transformers - embeddings - feature-extraction - sentence-similarity - cross-lingual - semantic-search - retrieval - rag - multilingual - multilingual-embeddings - 384-dim - bert - minilm - sakthai - house-of-sak - beer-sakthai - cpu-inference - edge - text-embeddings-inference - endpoints_compatible extra: sibling: Nanthasit/sakthai-embedding model-index: - name: SakThai Multilingual Embedding results: - task: type: sentence-similarity dataset: type: local-cron name: SakThai cron eval 2026-07-31 metrics: - type: cosine_similarity value: 0.92 name: Cosine similarity (self-check) verified: false notes: Live inference self-check committed to `.eval_results/inference-check-20260730_232754.yaml`; not a formal MTEB run. - task: type: sentence-similarity dataset: type: local-cron name: SakThai cross-lingual smoke 2026-08-01 metrics: - type: cosine_similarity value: 0.9142 name: Cross-lingual EN↔FR cosine verified: true notes: Verified smoke benchmark artifact committed 2026-08-01. - task: type: retrieval dataset: type: mtbed-style-pending name: Multilingual retrieval/STS pending metrics: - type: spearman value: null name: STS-Spearman verified: false notes: Expected in the MiniLM L12 multilingual family ballpark (~0.75-0.85); awaiting formal MTEB/STS results. ---

384-dim cross-lingual sentence embeddings · 50+ languages · CPU-friendly
The retrieval stage of the SakThai pipeline · SakThai Model Family

Profile GitHub Collection Downloads 384d Base: Multilingual-MiniLM-L12-H384 MIT

--- ## Model Description SakThai Multilingual Embedding is a **BERT-based sentence-transformers model** producing 384-dimensional cross-lingual embeddings. Sentences with similar meaning map close together regardless of language — enabling multilingual retrieval and comparison without translation. **What makes it special:** - 🌍 50+ languages (multilingual MiniLM vocabulary, 250K tokens) - 🖥️ CPU inference — verified locally (see [Evaluation & Verification](#evaluation--verification)) - 💾 Lightweight architecture — ~118M parameters, **470 MB fp32** safetensors - 🔗 Cross-lingual — query in English, retrieve in Thai, French, Japanese, etc. - 📦 Perfect for multilingual RAG pipelines - 🚀 **Fastest-growing SakThai embedding model** — 627 downloads, live 2026-08-01 ## Quick Start ```python from sentence_transformers import SentenceTransformer model = SentenceTransformer("Nanthasit/sakthai-embedding-multilingual") sentences = [ "I love machine learning", "J'adore l'apprentissage automatique", "Ich liebe maschinelles Lernen", "Me encanta el aprendizaje automático", "ฉันชอบการเรียนรู้ของเครื่อง", "我喜欢机器学习", ] embeddings = model.encode(sentences) print(embeddings.shape) # (6, 384) from sklearn.metrics.pairwise import cosine_similarity sim = cosine_similarity([embeddings[0]], [embeddings[1]]) print(f"Cross-lingual similarity: {sim[0][0]:.3f}") ``` > **Tip:** embeddings are L2-normalized by default (`normalize_embeddings=True`). For large corpora, use `model.encode(docs, batch_size=32)` to avoid memory spikes. ### Batch Encoding for Large Corpora ```python from sentence_transformers import SentenceTransformer model = SentenceTransformer("Nanthasit/sakthai-embedding-multilingual") # Stream from disk in chunks def stream_docs(path, chunk_size=1000): with open(path) as f: chunk = [] for line in f: chunk.append(line.strip()) if len(chunk) >= chunk_size: yield chunk chunk = [] if chunk: yield chunk for docs in stream_docs("corpus.txt"): embs = model.encode(docs, batch_size=64, show_progress_bar=True) # write to FAISS / Qdrant / disk ``` ### Cross-lingual Semantic Search ```python from sentence_transformers import SentenceTransformer, util model = SentenceTransformer("Nanthasit/sakthai-embedding-multilingual") docs = [ "Neural networks are inspired by the brain.", "Les réseaux de neurones sont inspirés par le cerveau.", "Künstliche Intelligenz verändert die Welt.", "人工知能は世界を変える", "Machine learning is a subset of AI.", ] doc_emb = model.encode(docs, convert_to_tensor=True) query = "how do neural networks work?" query_emb = model.encode(query, convert_to_tensor=True) scores = util.cos_sim(query_emb, doc_emb)[0] print(f"Best match: {docs[scores.argmax()]}") ``` --- ## Architecture | Property | Value | |----------|-------| | **Base architecture** | [Multilingual-MiniLM-L12-H384](https://huggingface.co/microsoft/Multilingual-MiniLM-L12-H384) (BERT-style) | | **Hidden size** | 384 | | **Layers / heads** | 12 / 12 | | **Intermediate size** | 1,536 | | **Embedding dim** | 384 | | **Pooling** | mean (verified `1_Pooling/config.json`) | | **Max sequence length** | 512 tokens | | **Vocabulary** | 250,037 (multilingual) | | **Parameters** | 117,653,760 (~118M) | | **Weights** | 470 MB (fp32 `model.safetensors`) | | **License** | MIT | ## Training Details | Hyperparameter | Value | |---------------|-------| | **Base model** | `microsoft/Multilingual-MiniLM-L12-H384` | | **Fine-tuning method** | Supervised contrastive / paraphrase loss via `sentence-transformers` | | **Training datasets** | `sakthai-combined-v6`, `sakthai-combined-v7`, `SimpleToolCalling`, `sakthai-kaggle-notebooks`, `sakthai-irrelevance-supplement`, `food-penguin-v1`, `sakthai-bench-v1` | | **Epochs** | Multiple epochs on mixed corpus | | **Batch size** | Dataset-dependent | | **Optimizer** | AdamW | | **Learning rate** | Typical ST SBERT range | | **Precision** | fp32 safetensors | | **Hardware** | Free T4 GPU credits; $0 budget | | **Output** | `model.safetensors`, `tokenizer.json`, `config.json` | ## Evaluation & Verification **Local inference is verified** (2026-07-30, committed to `.eval_results/inference-check-20260730_232754.yaml`): | Check | Result | |-------|--------| | Embedding dimensions | 384 (float32) ✅ | | Model load time | 7.06 s | | Encode call time | 0.211 s | | Method | `sentence_transformers` locally | **Health check** (`.eval_results/health-check-sakthai-embedding-multilingual-2026-07-30-4.yaml`): - 📈 **Download velocity rank: 2/19** - 📊 Download rank: 6/19 · Card quality: 90/100 · Repo hygiene: 90/100 **Verified smoke benchmark** (2026-08-01): | Pair | Cosine similarity | |------|-------------------| | English ↔ French | 0.9142 | | English ↔ Thai | 0.8927 | | English ↔ Chinese | 0.9016 | | French ↔ German | 0.8804 | | Thai ↔ Chinese | 0.8611 | These scores come from live local inference and are saved in the repo's `.eval_results/` history. This is a smoke check, not a formal MTEB run. **Hosted inference — honest status:** the HF serverless router returns `400 Model not supported by provider hf-inference` and `api-inference.hf.co` returns `403`. For production, run locally with `sentence-transformers` or on a dedicated [TEI](https://github.com/huggingface/text-embeddings-inference) endpoint. **Formal benchmarks (STS-B, MTEB-style retrieval): pending.** No verified scores are published yet. As the base architecture is the same 12-layer / 384-dim multilingual MiniLM family as `paraphrase-multilingual-MiniLM-L12-v2`, expected STS performance is in that family's ballpark (~0.75–0.85 Spearman) — **estimated, not yet verified**. Proper cross-lingual retrieval and STS results will be published via the [SakThai Leaderboard Space](https://huggingface.co/spaces/Nanthasit/sakthai-leaderboard) when available. ## Deployment Options | Path | How | Status | |------|-----|-------| | **Local (recommended)** | `SentenceTransformer("Nanthasit/sakthai-embedding-multilingual")` — verified, zero cost | ✅ Verified | | **TEI / Inference Endpoints** | Tagged `text-embeddings-inference` + `endpoints_compatible`; serve with a TEI endpoint for high-throughput batch embedding | Optional | | **Serverless HF API** | ⚠️ Not currently supported by the router provider (verified 2026-07-30) — use local or TEI | ❌ 400/403 | ## When to Use - **Cross-lingual semantic search** — query in English, retrieve in any of 50+ languages - **Multilingual RAG pipelines** — index mixed-language corpus, search across languages - **Deduplication / clustering** — group near-duplicates across languages - **Zero-shot cross-lingual transfer** — train on English labels, predict on foreign text ## Pipeline Integration | Stage | Model | Role | |-------|-------|-----| | 🔍 **Retrieve** | **Embedding Multilingual** ⬅ | **Cross-lingual semantic search, 50+ langs** | | 🧠 Reason | [Context 1.5B](https://huggingface.co/Nanthasit/sakthai-context-1.5b-merged) or [7B](https://huggingface.co/Nanthasit/sakthai-context-7b-merged) | Tool-calling & reasoning | | 👁️ See | [Vision 7B](https://huggingface.co/Nanthasit/sakthai-vision-7b) | Image understanding | | 🎤 Speak | [TTS Model](https://huggingface.co/Nanthasit/sakthai-tts-model) | Text-to-speech | ## Limitations - **No verified MTEB/STS scores yet.** Published numbers are base-model estimates, not this fine-tune’s measured results. - **Hosted inference is unsupported** on the default HF inference router; requires local `sentence-transformers` or a dedicated TEI endpoint. - **Speed/throughput claims are relative.** Local encode latency is single-device CPU; GPU/batch performance will differ. - **Quality varies by language pair.** High-resource languages are stronger; low-resource cross-lingual pairs may degrade. - **Context is 512 tokens.** Longer documents need chunking or a longer-context reranker. - **No multilingual benchmark card yet.** Leaderboard results will appear when formally evaluated. - **Community model, not production SRE.** Built on free-tier credits; no commercial SLA. - **Zero-budget constraint.** We cannot run large-scale public MTEB/beIR benchmarks on paid compute yet. ## SakThai Model Family 🏠 All **20 public SakThai models** + 2 companion repos (downloads live, 2026-08-01): | Model | Downloads | Size | Role | |-------|-----------|------|-----| | [Context 1.5B Merged](https://huggingface.co/Nanthasit/sakthai-context-1.5b-merged) | 1,855 | 3.1 GB | Flagship 1.5B tool-calling LLM | | [Context 0.5B Merged](https://huggingface.co/Nanthasit/sakthai-context-0.5b-merged) | 1,692 | 988 MB | Edge 0.5B tool-calling LLM | | [Context 7B Merged](https://huggingface.co/Nanthasit/sakthai-context-7b-merged) | 1,024 | 15.2 GB | Flagship 7B LLM | | **Embedding Multilingual** ⬅ | **627** | **470 MB** | **Cross-lingual retrieval** | | [Context 7B 128K](https://huggingface.co/Nanthasit/sakthai-context-7b-128k) | 610 | config-only | YaRN 128K long-context recipe | | [Context 7B Tools](https://huggingface.co/Nanthasit/sakthai-context-7b-tools) | 489 | LoRA 20 MB | 7B tool-use adapter | | [Context 1.5B Tools](https://huggingface.co/Nanthasit/sakthai-context-1.5b-tools) | 477 | LoRA 9 MB | 1.5B tool-use adapter | | [Context 1.5B Merged V2](https://huggingface.co/Nanthasit/sakthai-context-1.5b-merged-v2) | 337 | 3.1 GB | Merged V2 weights | | [Vision 7B](https://huggingface.co/Nanthasit/sakthai-vision-7b) | 315 | 4.1 GB | Image understanding (LLaVA) ⭐ 1 like | | [Plus 1.5B LoRA](https://huggingface.co/Nanthasit/sakthai-plus-1.5b-lora) | 306 | LoRA 74 MB | Plus 1.5B adapter | | [Context 0.5B Tools](https://huggingface.co/Nanthasit/sakthai-context-0.5b-tools) | 251 | 988 MB | Edge tool-calling | | [TTS Model](https://huggingface.co/Nanthasit/sakthai-tts-model) | 248 | 141 MB | Text-to-speech (Kokoro) | | [Plus 1.5B](https://huggingface.co/Nanthasit/sakthai-plus-1.5b) | 244 | 3.1 GB | New 1.5B base | | [Context 1.5B Tools V2](https://huggingface.co/Nanthasit/sakthai-context-1.5b-tools-v2) | 173 | LoRA 74 MB | Refined 1.5B tool adapter | | [Coder 1.5B](https://huggingface.co/Nanthasit/sakthai-coder-1.5b) | 151 | 1.1 GB | Code generation (GGUF) | | [Coder Browser](https://huggingface.co/Nanthasit/sakthai-coder-browser) | 54 | 3.1 GB | Browser-agent LLM | | [Coder Browser GGUF](https://huggingface.co/Nanthasit/sakthai-coder-browser-gguf) | 35 | 7.1 GB | Browser-agent GGUF (F16) | | [Embedding (English, private)](https://huggingface.co/Nanthasit/sakthai-embedding) | 23 | 91 MB | English-only embedding (token-required) | | [Coder Browser LoRA](https://huggingface.co/Nanthasit/sakthai-coder-browser-lora) | 21 | LoRA 74 MB | Browser-agent adapter | | [Plus 1.5B Coder](https://huggingface.co/Nanthasit/sakthai-plus-1.5b-coder) | 0 | planned | Coder variant (no weights yet) | **Companion repos:** | Repo | Role | |------|-----| | [Bench V3](https://huggingface.co/Nanthasit/sakthai-bench-v3) | Benchmark scaffold (no weights) | | [Pipeline](https://huggingface.co/Nanthasit/sakthai-pipeline) | CI/CD automation, evals, health checks | 📦 [View the whole family collection](https://huggingface.co/collections/Nanthasit/sakthai-model-family-6a64745450b12d421c1f9f02) ## The House of Sak 🏠 This model is the retrieval stage — making it possible to search across languages without translation. Built from a shelter in Cork, Ireland, with $0 budget and a belief that AI should be for everyone. > *"We are one family — and becoming more."* — Beer (beer-sakthai) --- ## Support - ⭐ Leave a like on Hugging Face - 🔄 Share with anyone building multilingual RAG - 🍴 Fork and experiment — MIT licensed --- ## License MIT — free to use, modify, and share. --- ## Citation ```bibtex @misc{sakthai-multilingual-embedding-2026, title = {SakThai Multilingual Embedding}, author = {Beer (beer-sakthai) and SakThai}, year = {2026}, url = {https://huggingface.co/Nanthasit/sakthai-embedding-multilingual} } ``` If you use the base architecture, also cite: ```bibtex @misc{multilingual-minilm-2022, title = {Multilingual MiniLM}, author = {Wang, Liang and others}, year = {2022}, url = {https://huggingface.co/microsoft/Multilingual-MiniLM-L12-H384} } ``` --- *"We are one family — and becoming more."* 🏠