LLM EXPLORER 59,420 MODELS INDEXED

S1 Llama 3.2 3Bx4 MoE by cloudyu

By cloudyu · 6 downloads
HF Repo ↗ Copy URL

S1 Llama 3.2 3Bx4 MoE is an open-source language model by cloudyu. Features: 9.6b LLM, VRAM: 38GB, Context: 128K, License: llama3, MoE, LLM Explorer Score: 0.22.

  Mixtral   Moe   Region:us   Safetensors   Sharded   Tensorflow
Model Card on HF 🤗: https://huggingface.co/cloudyu/S1-Llama-3.2-3Bx4-MoE 

S1 Llama 3.2 3Bx4 MoE Parameters and Internals

LLM NameS1 Llama 3.2 3Bx4 MoE
Repository 🤗https://huggingface.co/cloudyu/S1-Llama-3.2-3Bx4-MoE 
Model Size9.6b
Required VRAM38 GB
Updated2026-03-05
Maintainercloudyu
Model Typemixtral
Model Files  5.0 GB: 1-of-8   4.9 GB: 2-of-8   4.9 GB: 3-of-8   4.9 GB: 4-of-8   4.9 GB: 5-of-8   4.9 GB: 6-of-8   4.9 GB: 7-of-8   3.6 GB: 8-of-8
Model ArchitectureMixtralForCausalLM
Licensellama3
Context Length131072
Model Max Length131072
Transformers Version4.48.2
Tokenizer ClassPreTrainedTokenizerFast
Padding Token<|begin_of_text|>
Vocabulary Size128256
Torch Data Typefloat32