š³ The RoboCasa Kitchen Leaderboard What does it take for a robot to handle kitchen chores the way a person does? It has to see (Vision), understand instructions (Language), and actually act (Action) ā and VLA (Vision-Language-Action) models are emerging as the answer. They're the bridge between large multimodal models and real-world embodied control.
RoboCasa Kitchen is a leading robot-learning benchmark in which a single-arm robot (Franka Panda) performs 24 atomic manipulation tasks ā picking up cups and bowls, opening drawers and doors, turning faucets, pressing buttons, and more ā inside a photorealistic simulated kitchen. Because the layout and object placement are randomized every episode, it tests genuine generalization rather than memorized motions. The score (success rate, SR) is the average fraction of the 24 tasks completed as instructed, measured over multiple seeds so results aren't down to luck.
The catch: this benchmark has no official leaderboard, and protocols (number of demonstrations, evaluation setup) differ from paper to paper, leaving scores scattered. Lining the numbers up naively quickly turns into an apples-to-oranges comparison.
This leaderboard fixes that by collecting published scores with their sources and comparing only what is genuinely comparable. It's split into three tables:
š Kitchen 24-task (matched) ā head-to-head under identical conditions (per the RLDX-1 Technical Report). This is the core ranking you can actually trust. ā Other protocols ā self-reported under different setups (e.g. fewer demos). Not directly comparable, so kept separate. š¤ GR1-Tabletop ā a different, humanoid-based variant suite, separated to avoid confusion.
Any researcher can submit their own model's score directly, and submissions are reviewed before they appear on the board. Every number links to its source paper, so you can verify it yourself.
š®š³ New in my Hindi LLM Series: Gemma-4 E4B, fine-tuned for Hindi ā and it runs on your laptop's CPU. I fine-tuned Google's new Gemma-4 E4B on ~10k Hindi instruction pairs (AI4Bharat: anudesh + dolly) using Unsloth + LoRA, on a single L4 GPU. Then I ran an honest side-by-side eval: base Gemma-4 vs my fine-tune, across 25 Hindi prompts. The results were interesting š ā My fine-tune is more concise ā ask for "3 tips" and it gives exactly 3. Base writes a 1,200-character essay.
ā Pure native Hindi ā base keeps slipping into English ("ą¤øą¤ą¤¤ą„लित ą¤ą¤¹ą¤¾ą¤° (Eat a Balanced Diet)", "तारा (Star)"). My fine-tune stays in clean Hindi.
ā Tighter instruction-following ā ask for a "short message" and it gives one, not a menu of options. āļø And to be honest: base Gemma-4 is more detailed and comprehensive. I didn't build a "smarter" model ā I built a focused, Hindi-native, edge-friendly one that runs as a 5GB GGUF (Q4) on CPU. š Try it:
Releasing Vividh-ASR ā an open benchmark and models for Hindi and Malayalam ASR.
Vividh-ASR is built from public data, stratified by complexity: ā Clean recordings ā Noisy and accented speech ā Spontaneous, conversational audio
Alongside the benchmark, we release: ā Open models for Hindi and Malayalam ā A training recipe with two counterintuitive choices that moved the needle ā What failed, not just what worked
The stratified evaluation methodology transfers directly to any low-resource language setup ā beyond Hindi and Malayalam.
Built at @adalatai, where we build speech tech for Indian courts. This is our first open contribution back to the community. @janaab@Kush0610@orgh0
Just published: how we built production Sango (Central African Republic) translation without fine-tuning, parallel corpus, or training compute.
The method ā vocabulary-augmented prompting with a 581-entry native-speaker-verified lexicon ā generalizes to any of the ~2,000 African languages at the same data-poverty level. Recipe, dataset, and code template all included.
I wanted to share a cool feature from my open source AI native web browser, Vessel: Persistent highlights!
You can highlight anything on the page and the context is provided to the agent. It's kind of a fun way to learn about new stuff, synthesize info, or just deepen your comprehension/understanding.
Since highlights are persistent, you can close the page, come back later - and your highlights will be exactly where you left them. I've found this particularly useful when reviewing technical blogs, model cards, etc.