LLM EXPLORER 60,062 MODELS INDEXED

MoE Study Remastered by OliverSundaram

By OliverSundaram · 313 downloads

MoE Study Remastered is an open-source language model by OliverSundaram. Features: 201.4m LLM, VRAM: 0.8GB, License: mit, MoE, LLM Explorer Score: 0.32.

  Arxiv:1701.06538   Arxiv:1910.07467   Arxiv:2002.05202   Arxiv:2101.03961   Arxiv:2104.09864   Arxiv:2304.01373   Arxiv:2505.05427   Custom code   Custom llm Dataset:openbmb/ultra-fineweb-...   En   Mixture-of-experts   Moe   Pytorch   Region:us   Safetensors

MoE Study Remastered Parameters and Internals

LLM NameMoE Study Remastered
Repository πŸ€—https://huggingface.co/OliverSundaram/MoE-Study-Remastered 
Model Size201.4m
Required VRAM0.8 GB
Updated2026-08-31
MaintainerOliverSundaram
Model Typecustom_llm
Model Files  0.8 GB
Supported Languagesen
Model ArchitectureLLM
Licensemit
Model Max Length1024
Transformers Version5.16.1
Tokenizer ClassTokenizersBackend
Padding Token<|endoftext|>
Vocabulary Size32768