Kalike · ಕಲಿಕೆ

A pure Kannada language model, trained from scratch

ಕನ್ನಡದಲ್ಲೇ ಓದುವ, ಕನ್ನಡದಲ್ಲೇ ಬರೆಯುವ, ಕನ್ನಡದಲ್ಲೇ ಮಾತನಾಡುವ ಭಾಷಾ ಮಾದರಿ
"Kalike" means learning. A model that reads, writes and talks in Kannada, and nothing else.


license language parameters trained on from scratch
🤗 dataset inspired by by

Chat · Results · Samples · Use it · Model · Data · Limits · Author


🧠 435.16M

parameters
core (transformer + head)
896.53M incl. lookup tables

📚 4.57B

training tokens
of clean Kannada

🔤 32.77K

token vocabulary
Kannada-only BPE

📏 2.05K

context length
tokens

✨ What it is

Kalike is a small open-weight base model and chat model that was never shown a word of English. Its tokenizer, its training data and its chat data are all Kannada. It is trained from random weights, not fine-tuned from an English model, by Suhas Bhairav, using a training code base inspired by Andrej Karpathy's nanochat.

🧠 d20/base: the base model
Continues Kannada text. Pretrained for 4,357 steps on clean Kannada from news, the web, books, Wikipedia and Wikisource.

💬 d20/sft: the chat model
Answers in Kannada. The base model fine-tuned for 68 steps on Kannada conversations (IndicAlign and Aya).

Two sizes are published, each with a base and a chat model:

d20 (recommended) d16 (lighter)
Layers · width 20 · 1280 16 · 1024
Parameters 435M core · 897M total 537M total
Pretraining 4.57B tokens 2.82B tokens
Folder d20/ d16/

💬 Talk to it

Real replies from the d20 chat model, sampled at temperature 0.6 and running locally on a MacBook (Apple Silicon, about 50 tokens per second):

🙂 ಕರ್ನಾಟಕದ ರಾಜಧಾನಿ ಯಾವುದು?

🪔 ಕರ್ನಾಟಕದ ರಾಜಧಾನಿ ಬೆಂಗಳೂರು.

🙂 ಆರೋಗ್ಯವಾಗಿರಲು ಮೂರು ಸಲಹೆಗಳನ್ನು ನೀಡಿ.

🪔 ೧. ಸಾಕಷ್ಟು ನೀರು ಕುಡಿಯಿರಿ: ದಿನಕ್ಕೆ ಕನಿಷ್ಠ ೮-೧೦ ಗ್ಲಾಸ್ ನೀರು ಕುಡಿಯಲು ಗುರಿಯನ್ನು ಹೊಂದಿರಿ. ಇದು ನಿಮ್ಮನ್ನು ಹೈಡ್ರೀಕರಿಸಲು ಸಹಾಯ ಮಾಡುತ್ತದೆ ಮತ್ತು ನಿಮ್ಮ ದೇಹವು ತ್ಯಾಜ್ಯವನ್ನು ತೊಡೆದುಹಾಕಲು ಮತ್ತು ವಿಷವನ್ನು ಹೊರಹಾಕಲು ಸಹಾಯ ಮಾಡುತ್ತದೆ.
೨. ವ್ಯಾಯಾಮ: ನಿಮ್ಮ ದೇಹವನ್ನು ಸಕ್ರಿಯವಾಗಿರಿಸಲು ಮತ್ತು ನಿಮ್ಮ ಮಾನಸಿಕ ಮತ್ತು ದೈಹಿಕ ಆರೋಗ್ಯವನ್ನು ಬೆಂಬಲಿಸಲು… (cut at 150 tokens)

🙂 ಮಕ್ಕಳಿಗಾಗಿ ಒಂದು ಸಣ್ಣ ಕಥೆ ಹೇಳಿ.

🪔 ಒಂದಾನೊಂದು ಕಾಲದಲ್ಲಿ, ದೂರದ ಒಂದು ದೇಶದಲ್ಲಿ, ರೋಹನ್ ಎಂಬ ಯುವಕ ವಾಸಿಸುತ್ತಿದ್ದನು. ರೋಹನ್ ಯಾವಾಗಲೂ ಹೊಸ ವಿಷಯಗಳನ್ನು ಪ್ರಯತ್ನಿಸಲು ಮತ್ತು ಹೊಸ ಜನರನ್ನು ಭೇಟಿಯಾಗಲು ಉತ್ಸುಕರಾಗಿರುತ್ತಾರೆ. ಒಂದು ದಿನ, ಅವರು ಉದ್ಯಾನವನದಲ್ಲಿ ಆಡುತ್ತಿದ್ದಾಗ ಒಂದು ಸಣ್ಣ ಹಾವನ್ನು ಕಂಡರು…

The model sometimes writes the visarga "ಃ" where a colon belongs (a habit picked up from its chat data); it is shown as ":" above. The story shows the usual small-model slips: the snake later grows wings.

📊 Results

Model Benchmark Score Random guess
🧠 Base Belebele (kn) 27.4% 25.0%
🧠 Base Karnataka schemes & MSME help (kn) 42.3% 25.0%
🧠 Base KN-CORE (centered mean) 0.1314 0.0
💬 Chat Belebele (kn) 22.9% 25.0%
💬 Chat Karnataka schemes & MSME help (kn) 25.0% 25.0%
💬 Chat ChatCORE-kn (centered mean) -0.0141 0.0
🧠 Base Validation bits per byte 0.3120 lower is better
💬 Chat Validation bits per byte (chat data) 0.1879 lower is better

d16 (base): Belebele 27.3% · Karnataka schemes & MSME help 39.9% · KN-CORE 0.1148 · validation bpb 0.3285. d20 beats it on every benchmark.

Why the chat scores sit at chance. The chat models answer every multiple-choice question with "೧": that gives exactly 206/900 on Belebele (206 questions have ೧ as the answer) and 42/168 on the Karnataka benchmark (42 per option), for both d16 and d20. The chat data had no multiple-choice examples, so the chat models never learned that answer format. It says nothing about their chat ability (see above). The base-model scores are the knowledge measure: 42.3% on Karnataka schemes is well above the 25% chance level.

Belebele: reading comprehension written in Kannada (900 questions). MILU: Kannada exam-style questions (gated, included when available). Karnataka schemes & MSME help: our own benchmark of questions about Karnataka government schemes and support for small businesses, every answer checked against a cited source. Base models are scored by likelihood of each answer, chat models by their answer token. Centered scores map random guessing to 0 and perfect to 1. Neither benchmark is in the training data.

🖋️ What it writes

Greedy completions from the d20 base model (prompt, then the model's continuation), straight from the evaluation log:

Prompt Kalike continues
ಕರ್ನಾಟಕದ ರಾಜಧಾನಿ ಬೆಂಗಳೂರು. ಇದು ಭಾರತದ ಸಿಲಿಕಾನ್ ವ್ಯಾಲಿ ಎಂದೇ ಖ್ಯಾತಿ ಪಡೆದಿದೆ. ಇಲ್ಲಿ ಸಾಕಷ್ಟು ಪ್ರವಾಸಿ ತಾಣಗಳಿವೆ. ಅವುಗಳಲ್ಲಿ ಬನ್ನೇರುಘಟ್ಟ ರಾಷ್ಟ್ರೀಯ ಉದ್ಯಾನವನ ಕೂಡಾ ಒಂದು. ಬನ್ನೇರುಘಟ್ಟ
ಕನ್ನಡ ಭಾಷೆಯು ತನ್ನದೆ ಆದ ವಿಶಿಷ್ಟತೆಯನ್ನು ಹೊಂದಿದೆ. ಕನ್ನಡ ಭಾಷೆಯು ತನ್ನದೆ ಆದ ವಿಶಿಷ್ಟತೆಯನ್ನು ಹೊಂದಿದೆ. ಕನ್ನಡ ಭಾಷೆಯು ತನ್ನದೆ ಆದ ವಿಶಿಷ್ಟ
ಸೂರ್ಯನು ಪೂರ್ವದಲ್ಲಿ ಉದಯಿಸಿ ಪಶ್ಚಿಮದಲ್ಲಿ ಅಸ್ತಮಿಸುತ್ತಾನೆ. ಈ ರೀತಿ ಸೂರ್ಯನು ಉದಯಿಸುವ ಮತ್ತು ಅಸ್ತಮಿಸುವ ಕಾಲಗಳನ್ನು ಉತ್ತರಾಯಣ ಮತ್ತು ದಕ್ಷಿಣಾಯನ
ಮಳೆಗಾಲದಲ್ಲಿ ರೈತರು ತಮ್ಮ ಹೊಲಗಳಲ್ಲಿ ಕೃಷಿ ಚಟುವಟಿಕೆಗಳನ್ನು ಮಾಡಲು ಸಾಧ್ಯವಾಗುವುದಿಲ್ಲ. ಈ ಸಮಯದಲ್ಲಿ ರೈತರು ತಮ್ಮ ಹೊಲಗಳಲ್ಲಿ ಕೃಷಿ ಚಟುವಟಿಕೆಗಳನ್ನು ಮಾಡಲು ಸಾಧ್ಯವಾಗುವುದಿಲ್ಲ. ಈ ಸಮಯದಲ್ಲಿ ರೈತರು ತಮ್ಮ ಹೊಲ
ಒಂದು ವಾರದಲ್ಲಿ ಏಳು ದಿನ ಕೆಲಸ, ವಾರದಲ್ಲಿ ೫ ದಿನ ರಜೆ, ೩ ದಿನ ರಜೆ, ೨ ದಿನ ರಜೆ, ೨ ದಿನ ರಜೆ,
ಬಿಸಿ ಎಂಬ ಪದದ ವಿರುದ್ಧ ಪದ ವೇ 'ಬಿಸಿ'. ಈ ಬಿಸಿ ಎಂಬ ಪದವೇ 'ಬಿಸಿ' ಎಂಬ ಪದದ ಮೂಲ. ಬಿಸಿ ಎಂಬ ಪದವೇ 'ಬಿಸಿ' ಎಂಬ

🚀 Use it

The weights use nanochat's own architecture (not a transformers class), so they run with the Kalike training code:

# download this model into the training code's folders
python -m scripts.hub_model download --repo SuhasBhairav/kalike --tag d20

# chat in the terminal
python -m scripts.chat_cli -g d20 -p "ಕರ್ನಾಟಕದ ರಾಜಧಾನಿ ಯಾವುದು?"

The tokenizer is also available as a plain tiktoken file, no pickle needed:

import json, tiktoken
from tiktoken.load import load_tiktoken_bpe
cfg = json.load(open("d20/tokenizer/tokenizer_config.json"))
enc = tiktoken.Encoding(name="kalike", pat_str=cfg["pattern"],
                        mergeable_ranks=load_tiktoken_bpe("d20/tokenizer/tokenizer.tiktoken"),
                        special_tokens=cfg["special_tokens"])
print(enc.encode("ಕನ್ನಡ ಭಾಷೆ ಸುಂದರವಾಗಿದೆ."))

🧩 The model

Architecture decoder-only Transformer (nanochat): rotary embeddings, QK-norm, ReLU², value embeddings, no biases
Size 20 layers · width 1280 · 10 heads (10 KV heads) · attention window pattern L
Parameters 435,160,240 core · 896,533,746 total
Tokenizer byte-level BPE, 32,768 tokens, trained only on Kannada; vowel signs and viramas stay attached to their letters, so whole syllables become tokens
Context 2,048 tokens
Pretraining 4,357 steps · 4.57B tokens · batch 1,048,576 tokens · Muon + AdamW · bf16
Chat fine-tuning 68 steps on IndicAlign-kn and Aya-kn
📁 Files in this repo
d20/base/        model_<step>.pt (nanochat, fp32) · model.safetensors (bf16, safe to load) · meta_<step>.json (config)
d20/sft/         the chat model, same files
d20/tokenizer/   tokenizer.pkl + token_bytes.pt (nanochat) · tokenizer.tiktoken + tokenizer_config.json (portable) · tok_eval.json
d20/eval/        evaluation results (json)
d20/logs/        full training logs
d16/...          the smaller model, same layout
tokenizer/       the shared tokenizer (same in both folders)

📚 The data

Pretraining: SuhasBhairav/kannada-llm-base-model-dataset-clean, about 8.9M documents and 13.5B characters of cleaned, deduplicated Kannada from Sangraha (AI4Bharat), FineWeb-2, IndicCorp v2, IITB IndicMonoDoc, Kannada Wikipedia and Kannada Wikisource.

Chat: the Kannada side of IndicAlign and the Kannada rows of the Aya dataset.

Clean means: only Kannada letters and signs, Kannada digits (೦-೯; numbers are converted) and basic punctuation. Lines that were mostly another language were dropped whole rather than cut up. Every character of the final data was re-checked. See the dataset card for every source, license and filter.

⚠️ Limitations

  • Small and young. Kalike writes fluent-looking Kannada but knows little and makes things up. Do not rely on it for facts, advice or anything that matters.
  • Kannada only, by design: it cannot read or write English or other scripts, and writes numbers in Kannada digits.
  • The chat data is largely machine-translated, so some answers read like translations.
  • Chat models are not yet tuned for multiple-choice answers (see Results).
  • It may repeat the biases of web text. It has had no safety tuning.

📜 License and credits

Weights: Apache-2.0. The training data keeps its own licenses (see the dataset card); thanks to AI4Bharat, Hugging Face, IIT Bombay CFILT, Cohere Labs, Meta (Belebele) and the Wikimedia community. The training code is inspired by and partly derived from nanochat by Andrej Karpathy (MIT).

@misc{kalike2026,
  title  = {Kalike: a pure Kannada language model trained from scratch},
  author = {Bhairav, Suhas},
  year   = {2026},
  url    = {https://huggingface.co/SuhasBhairav/kalike}
}

👋 Author

Suhas Bhairav · building AI from scratch, in the open

Website YouTube LinkedIn X

ಕನ್ನಡ ಉಳಿಸಿ, ಕನ್ನಡ ಬೆಳೆಸಿ 💛❤️ · Updated 2026-11-17

Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Dataset used to train SuhasBhairav/kalike