Drop-Upcycling
updated
9B • Updated • 86
9B • Updated • 91
19B • Updated • 84
9B • Updated • 89
9B • Updated • 82
0.4B • Updated • 88
0.4B • Updated • 82
0.4B • Updated • 84
0.4B • Updated • 86
9B • Updated • 76
19B • Updated • 91
0.4B • Updated • 86
9B • Updated • 82
0.4B • Updated • 86
• 1
2B • Updated • 94
0.2B • Updated • 105
4B • Updated • 82
14B • Updated • 101
llm-jp/Dense-btx-code-expert-152M
0.2B • Updated • 95
• 1
llm-jp/Dense-btx-english-expert-1.5B
2B • Updated • 88
llm-jp/Dense-btx-code-expert-1.5B
2B • Updated • 97
• 1
llm-jp/Dense-btx-japanese-expert-1.5B
2B • Updated • 167
• 1
llm-jp/Dense-btx-english-expert-152M
0.2B • Updated • 91
llm-jp/Dense-btx-japanese-expert-152M
0.2B • Updated • 121
Drop-Upcycling: Training Sparse Mixture of Experts with Partial
Re-initialization
Paper
• 2502.19261
• Published • 6
Text Generation
• 73B • Updated • 104
llm-jp/llm-jp-3-8x13b-instruct3
Text Generation
• 73B • Updated • 254
• 8
llm-jp/llm-jp-3-8x1.8b-instruct3
Text Generation
• 9B • Updated • 102
• 4
Text Generation
• 9B • Updated • 244
llm-jp/llm-jp-3-8x13b-instruct2
Text Generation
• 73B • Updated • 231
llm-jp/llm-jp-3-8x1.8b-instruct2
Text Generation
• 9B • Updated • 96
llm-jp/llm-jp-3.1-8x13b-instruct4
Text Generation
• 73B • Updated • 174
• 4
Text Generation
• 73B • Updated • 97