What are the hardware requirements for LiteMOE 3x460m?

LiteMOE 3x460m requires approximately 2.1 GB of VRAM and supports a context window of 1K tokens. Quantized variants may run on less VRAM; see the Quantized Models section on this page.

Who developed LiteMOE 3x460m and how large is it?

LiteMOE 3x460m is developed by cnmoro, a model with 1.1b parameters. The model is published as open weights on Hugging Face and indexed on LLM Explorer with full benchmark history.

Where can I download or evaluate LiteMOE 3x460m?

LiteMOE 3x460m is hosted on Hugging Face and linked from this page. LLM Explorer also lists quantized variants and similar alternatives if available.

LiteMOE 3x460m by cnmoro — VRAM 2.1GB, 1K context

Name: LiteMOE 3x460m
Author: cnmoro

LiteMOE 3x460m is an open-source language model by cnmoro. Features: 1.1b LLM, VRAM: 2.1GB, Context: 1K, License: apache-2.0, MoE, LLM Explorer Score: 0.1.

Autotrain compatible Endpoints compatible Lazymergekit Merge Mergekit Mixtral Moe Region:us Safetensors Sharded Tensorflow

Model Card on HF 🤗: https://huggingface.co/cnmoro/LiteMOE-3x460m

LiteMOE 3x460m Parameters and Internals

Additional Notes

The model is a merge of the LiteLlama-460M-1T model using mergekit.

LLM Name	LiteMOE 3x460m
Repository 🤗	https://huggingface.co/cnmoro/LiteMOE-3x460m
Model Size	1.1b
Required VRAM	2.1 GB
Updated	2024-11-18
Maintainer	cnmoro
Model Type	mixtral
Model Files	2.1 GB: 1-of-1
Model Architecture	MixtralForCausalLM
License	apache-2.0
Context Length	1024
Model Max Length	1024
Transformers Version	4.36.2
Tokenizer Class	GPT2Tokenizer
Padding Token	<\|endoftext\|>
Vocabulary Size	50304
Torch Data Type	bfloat16
Errors	replace

Rank the LiteMOE 3x460m capabilities

Have you tried this model? Rate its performance — this feedback helps the ML community find the right model for their needs.

Instruction Following and Task Automation
Factuality and Completeness of Knowledge
Censorship and Alignment
Data Analysis and Insight Generation
Text Generation
Text Summarization and Feature Extraction
Code Generation
Multi-Language Support and Translation

Best Alternatives to LiteMOE 3x460m

Best Alternatives	Context / RAM	Downloads	Likes
TinyDolphin 3x MoE	4K / 5.3 GB	11	1
...inyLlama 1.1B Chat V1.0 X2 MoE	2K / 3.7 GB	2595	1
Karasu Moe2	2K / 7.5 GB	5	0
Tinyllama Moe4	2K / 3.7 GB	9	1
Tinyllama Moe1	2K / 5.3 GB	11	0
...yMoE DopeykarasuMoe Xdareties2	2K / 3.7 GB	6	0
Tinyllama Moe3	2K / 3.7 GB	5	0

Note: green Score (e.g. "73.2") means that the model is better than cnmoro/LiteMOE-3x460m.