LLM EXPLORER 57,252 MODELS INDEXED

Phixtral 4x2 8odd by shadowml

By shadowml · 12 downloads
HF Repo ↗ Copy URL

Phixtral 4x2 8odd is an open-source language model by shadowml. Features: 4.5b LLM, VRAM: 8.9GB, License: mit, LLM Explorer Score: 0.11.

  Code Cognitivecomputations/dolphin-...   Conversational   Custom code   En   Lxuechen/phi-2-dpo   Moe   Phi-msft   Region:us   Safetensors   Sharded   Tensorflow
Model Card on HF 🤗: https://huggingface.co/shadowml/phixtral-4x2_8odd 

Phixtral 4x2 8odd Parameters and Internals

Model Type 
text-generation
Use Cases 
Areas:
NLP, Code Generation
Applications:
Research, Commercial Applications
Primary Use Cases:
Text Generation
Additional Notes 
The model is the first Mixure of Experts (MoE) made with two microsoft/phi-2 models, inspired by mistralai/Mixtral-8x7B-v0.1 architecture.
Supported Languages 
en (Primary)
Training Details 
Data Sources:
cognitivecomputations/dolphin-2_6-phi-2, lxuechen/phi-2-dpo
Methodology:
Custom Mixure of Experts using mergekit library with mixtral branch configuration
Hardware Used:
free T4 GPU
Model Architecture:
Inspired by mistralai/Mixtral-8x7B-v0.1 architecture
Input Output 
Input Format:
Tokenized input strings
Accepted Modalities:
text
Output Format:
Generated text
LLM NamePhixtral 4x2 8odd
Repository 🤗https://huggingface.co/shadowml/phixtral-4x2_8odd 
Model Size4.5b
Required VRAM8.9 GB
Updated2026-08-02
Maintainershadowml
Model Typephi-msft
Model Files  8.9 GB: 1-of-1
Supported Languagesen
Model ArchitecturePhiForCausalLM
Licensemit
Model Max Length2048
Transformers Version4.35.2
Tokenizer ClassCodeGenTokenizer
Padding Token<|endoftext|>
Vocabulary Size51200
Torch Data Typefloat16
Activation Functiongelu_new

Best Alternatives to Phixtral 4x2 8odd

Best Alternatives
Context / RAM
Downloads
Likes
Phixtral 2x2 80K / 8.9 GB59150
Samantha Phixtral 2x2 80K / 8.9 GB101
Phi 2 MoE Test0K / 8.9 GB51
Note: green Score (e.g. "73.2") means that the model is better than shadowml/phixtral-4x2_8odd.