Hugging Face's logo
OpenEngineer's picture
🤏 Small Decision Models soon!

OpenEngineer

Monster-Code
7 26 120
P(doom) Rejects the premise
tegridydev's profile picture VertexAIco's profile picture VIDRAFT-lab's profile picture
·
https://huggingface.co/Lightbend

AI & ML interests

@Lightbend

Recent Activity

liked a Space about 22 hours ago
BananaMind/blog
reacted to Banaxi-Tech's post with ➕ about 22 hours ago
We have released BGA! And wow, It provides 256x (and 512x at the end of 1M) yes 256x LESS attention compute at 1M context window. That means you can train a 1M context window at the compute of a ~4K context window. Check IT OUT: https://huggingface.co/spaces/BananaMind/blog#bananamind-gate-attention The Accuracy Should BE WAy better than DSA but untested yet. And, now some updates on BananaMind 3: BananaMind 3 Will start training Soon! Sizes: 10M, 25M, 50M, 100M, 150M And the context windows ARE INSANE: 10M, 16K context, 25M 16k context, 50M 32K context, 100M and 150M, 64K context!!!!
reacted to Banaxi-Tech's post with 🔥 about 22 hours ago
We have released BGA! And wow, It provides 256x (and 512x at the end of 1M) yes 256x LESS attention compute at 1M context window. That means you can train a 1M context window at the compute of a ~4K context window. Check IT OUT: https://huggingface.co/spaces/BananaMind/blog#bananamind-gate-attention The Accuracy Should BE WAy better than DSA but untested yet. And, now some updates on BananaMind 3: BananaMind 3 Will start training Soon! Sizes: 10M, 25M, 50M, 100M, 150M And the context windows ARE INSANE: 10M, 16K context, 25M 16k context, 50M 32K context, 100M and 150M, 64K context!!!!
View all activity

Organizations

TeamAudiyo's profile picture Lightbend's profile picture