LLM EXPLORER 65,121 MODELS INDEXED

Lorge 2x7B UAMM by Alsebay

By Alsebay · 16 downloads
HF Repo ↗ Copy URL

Lorge 2x7B UAMM is an open-source language model by Alsebay. Features: 19.2b LLM, VRAM: 38.2GB, Context: 32K, License: cc-by-nc-4.0, MoE, Merged, LLM Explorer Score: 0.13, Arc: 67.8, HellaSwag: 81.1, MMLU: 59.8, GSM8K: 27.7.

  Merged Model   Autotrain compatible   Endpoints compatible   Mixtral   Model-index   Moe   Region:us   Safetensors   Sharded   Tensorflow
Model Card on HF 🤗: https://huggingface.co/Alsebay/Lorge-2x7B-UAMM 

Lorge 2x7B UAMM Parameters and Internals

Model Type 
text-generation
Additional Notes 
This model is a merge of pre-trained language models using the passthrough merge method.
LLM NameLorge 2x7B UAMM
Repository 🤗https://huggingface.co/Alsebay/Lorge-2x7B-UAMM 
Merged ModelYes
Model Size19.2b
Required VRAM38.2 GB
Updated2024-12-03
MaintainerAlsebay
Model Typemixtral
Model Files  1.9 GB: 1-of-20   1.9 GB: 2-of-20   1.9 GB: 3-of-20   1.9 GB: 4-of-20   1.9 GB: 5-of-20   2.0 GB: 6-of-20   1.9 GB: 7-of-20   1.9 GB: 8-of-20   1.9 GB: 9-of-20   1.9 GB: 10-of-20   1.9 GB: 11-of-20   1.9 GB: 12-of-20   2.0 GB: 13-of-20   1.9 GB: 14-of-20   1.9 GB: 15-of-20   1.9 GB: 16-of-20   2.0 GB: 17-of-20   2.0 GB: 18-of-20   2.0 GB: 19-of-20   1.6 GB: 20-of-20
Model ArchitectureMixtralForCausalLM
Licensecc-by-nc-4.0
Context Length32768
Model Max Length32768
Transformers Version4.40.0
Tokenizer ClassLlamaTokenizer
Padding Token<s>
Vocabulary Size32000
Torch Data Typebfloat16

Best Alternatives to Lorge 2x7B UAMM

Best Alternatives
Context / RAM
Downloads
Likes
MixTAO 19B Pass32K / 38.1 GB32
Multimerge 19B Pass32K / 38 GB100
Mistralmath 15B Pass32K / 38.5 GB110
TaoPassthrough 15B S32K / 38.4 GB50
Raccoon Small32K / 38.4 GB601
SPEED 1 BASE4K / 38.4 GB1722
Mixtral 11Bx2 MoE 19B4K / 38.4 GB10238
...oundary Solar Chat 2x10.7B MoE4K / 38 GB1231
SunnyRain 2x10.7B4K / 38.4 GB100
19B MATH DPO4K / 38.4 GB530
Note: green Score (e.g. "73.2") means that the model is better than Alsebay/Lorge-2x7B-UAMM.