LLM EXPLORER 66,383 MODELS INDEXED

Upcycle Dense15000 E32top8 Noaux Rewarm200 by JIRU-W

By JIRU-W · 9569 downloads

Upcycle Dense15000 E32top8 Noaux Rewarm200 is an open-source language model by JIRU-W. Features: 10b LLM, VRAM: 7.4GB, Context: 4K, License: apache-2.0, LLM Explorer Score: 0.37.

Base model:finetune:jiru-w/den... Base model:jiru-w/dense e1 seq...   Custom code Dataset:jiru-w/olmoe-mix-0924-...   Mask moe   Mixture-of-experts   Moe   Olmoe   Region:us   Research   Safetensors   Upcycling

Upcycle Dense15000 E32top8 Noaux Rewarm200 Parameters and Internals

LLM NameUpcycle Dense15000 E32top8 Noaux Rewarm200
Repository πŸ€—https://huggingface.co/JIRU-W/upcycle_dense15000_e32top8_noaux_rewarm200 
Base Model(s)  Dense E1 Seq2048 Gbs128 10b   JIRU-W/dense_e1_seq2048_gbs128_10b
Model Size10b
Required VRAM7.4 GB
Updated2026-11-01
MaintainerJIRU-W
Model Typemask_moe
Model Files  7.4 GB
Model ArchitectureMaskMoeForCausalLM
Licenseapache-2.0
Context Length4096
Model Max Length4096
Transformers Version5.20.1
Tokenizer ClassGPTNeoXTokenizer
Padding Token<|padding|>
Vocabulary Size50304

Best Alternatives to Upcycle Dense15000 E32top8 Noaux Rewarm200

Best Alternatives
Context / RAM
Downloads
Likes
...Seq2048 Gbs128 10b Perlayeraux4K / 7.4 GB97081
Dense E1 Seq2048 Gbs128 10b4K / 1.1 GB1100
Note: green Score (e.g. "73.2") means that the model is better than JIRU-W/upcycle_dense15000_e32top8_noaux_rewarm200.