Article
Extreme Overtraining in Tiny Language Models
Banaxi-Tech
• • 13
Arthur Fernandez
arthurblg1802
AI & ML interests
None yet
Recent Activity
reacted to Banaxi-Tech's post with 🤗 about 16 hours ago
We're excited to release BananaMind 2.1 Pico Preview!
It includes the first preview of our BananaMind 2.1 architecture!
This model gets near BananaMind 2 Micro performance at half the parameters and 37.5x less tokens!
Thats insane!
The current architectures includes about 500K parameters of the total 1.5M parameters in n-gram embeddings and the layer 2 is run twice.
It also includes XSA and the XSA refresh gate.
We're still going to improve the architecture in the final release.
Check it out at:
Follow us for more models:
https://huggingface.co/BananaMind
@Banaxi-Tech
@vovaRL
@DedeProGames
reacted to Banaxi-Tech's post with 🔥 about 16 hours ago
We're excited to release BananaMind 2.1 Pico Preview!
It includes the first preview of our BananaMind 2.1 architecture!
This model gets near BananaMind 2 Micro performance at half the parameters and 37.5x less tokens!
Thats insane!
The current architectures includes about 500K parameters of the total 1.5M parameters in n-gram embeddings and the layer 2 is run twice.
It also includes XSA and the XSA refresh gate.
We're still going to improve the architecture in the final release.
Check it out at:
Follow us for more models:
https://huggingface.co/BananaMind
@Banaxi-Tech
@vovaRL
@DedeProGames
liked a Space 3 days ago
Banaxi-Tech/BananaMind-TTS-V2.1Organizations
upvoted an article 9 days ago
upvoted an article 17 days ago
Article
DAVIDAU DELETED MY POST AND BANNED ME FOR ASKING ABOUT BENCHMARKS — HERE IS THE FULL ANALYSIS (9B + 27B + 390 MODELS)
DedeProGames
• • 16 upvoted a collection about 1 month ago