Hugging Face's logo

Join the conversation

Join the community of Machine Learners and AI enthusiasts.

Sign Up
Back to feed
KlondikeDev 
posted an update about 21 hours ago
Post
1605
A Preview of Boris-2!

Hello! Tomorrow, OpenCerebral will be releasing Boris-1.7-D60M-n30M — an experimental architecture. It will be testing a new data mixture, a new tokenizer, and testing Qwen4-like n-gram embeddings.

Following this will be Boris-1.8-D60M-n30M, which will test both the n-gram embeddings AND a new architecture.

Then, Boris-2 will begin training!
KlondikeDev

Also, if anybody has suggestions for a better naming scheme, I’d love to hear it!
The “D” stands for Dense, and the “n” for the n-gram embeddings. So, “Boris-1.7-60M dense parameters-30M n-gram embeddings”

·
Banaxi-Tech

Everyone doing n-grams now?
CNWPlayer

will be interesting to see the effects of this

·
KlondikeDev

I think so too. I don’t know if it’s been tried at a small scale like this.
In this post