Suzume Llama 3 8B Multilingual By lightblue: Benchmarks, Features and Detailed Analysis. Insights on Suzume Llama 3 8B Multilingual.

Arxiv:2405.12612 Autotrain compatible Base model:finetune:meta-llama... Base model:meta-llama/meta-lla... Conversational Endpoints compatible Generated from trainer Instruct Llama Pytorch Region:us Safetensors Sharded Tensorflow

Model Card on HF 🤗: https://huggingface.co/lightblue/suzume-llama-3-8B-multilingual

Suzume Llama 3 8B Multilingual Benchmarks

MMLU Pro: 26.48

GPQA: 4.47

MUSR: 7.84

BBH: 28.9

IFEval: 66.78 vs 88 (so35)^-24.1%

ARC: 60.75 vs 96.7 (so35)^-37.2%

HellaSwag: 79.49 vs 95.3 (gpt4)^-16.6%

MMLU: 66.62 vs 88.3 (so35)^-24.6%

TruthfulQA: 48.7 vs 59 (gpt4)^-17.5%

WinoGrande: 76.16 vs 87.5 (gpt4)^-13%

GSM8K: 61.56 vs 96.4 (so35)^-36.1%

MATH Lvl 5: 9.44

LLME Score: 0.28918

^nn.n% — How the model compares to the reference models: Anthropic Sonnet 3.5 ("so35"), GPT-4o ("gpt4o") or GPT-4 ("gpt4").

What is the LLM Explorer Rank (Score)

Suzume Llama 3 8B Multilingual (lightblue/suzume-llama-3-8B-multilingual)

🌟 Advertise your project 🚀

Suzume Llama 3 8B Multilingual Parameters and Internals

Model Type

multilingual, text generation

Use Cases

Areas:

Research, Commercial applications

Applications:

Multilingual chat applications

Primary Use Cases:

Multilingual conversational AI

Limitations:

Excludes certain problem categories for Russian, Not yet fully evaluated

Considerations:

Ongoing evaluation and feedback encouraged

Additional Notes

Ongoing development with future releases

Supported Languages

languages_supported (multilingual), proficiency_levels (high)

Training Details

Data Sources:

lightblue/tagengo-gpt4, lmsys/lmsys-chat-1m, megagonlabs/instruction_ja, openchat/openchat_sharegpt4_dataset

Data Volume:

90,000 multilingual conversations

Methodology:

finetuning

Context Length:

8192

Training Time:

2.5 hours

Hardware Used:

4 x A100 (80GB) GPUs

Model Architecture:

LlamaForCausalLM

Input Output

Input Format:

Prompt messages should be constructed in a chat format

Accepted Modalities:

text

Output Format:

Text generation in response format

Performance Tips:

Utilize vLLM for optimal inference speed

LLM Name	Suzume Llama 3 8B Multilingual
Repository 🤗	https://huggingface.co/lightblue/suzume-llama-3-8B-multilingual
Base Model(s)	Meta Llama 3 8B Instruct meta-llama/Meta-Llama-3-8B-Instruct
Model Size	8b
Required VRAM	16.1 GB
Updated	2025-09-23
Maintainer	lightblue
Model Type	llama
Instruction-Based	Yes
Model Files	5.0 GB: 1-of-4 5.0 GB: 2-of-4 4.9 GB: 3-of-4 1.2 GB: 4-of-4 16.1 GB
Model Architecture	LlamaForCausalLM
License	other
Context Length	8192
Model Max Length	8192
Transformers Version	4.38.2
Tokenizer Class	PreTrainedTokenizerFast
Padding Token	<\|end_of_text\|>
Vocabulary Size	128256
Torch Data Type	bfloat16

Quantized Models of the Suzume Llama 3 8B Multilingual

Model	Likes	Downloads	VRAM
Suzume Llama 3 8B Multilingual	0	24	4 GB

Best Alternatives to Suzume Llama 3 8B Multilingual

Best Alternatives	Context / RAM	Downloads	Likes
...otron 8B UltraLong 4M Instruct	4192K / 32.1 GB	1531	120
...a 3.1 8B UltraLong 4M Instruct	4192K / 32.1 GB	176	24
UltraLong Thinking	4192K / 16.1 GB	45	3
...a 3.1 8B UltraLong 2M Instruct	2096K / 32.1 GB	875	9
...otron 8B UltraLong 2M Instruct	2096K / 32.1 GB	105	15
...a 3.1 8B UltraLong 1M Instruct	1048K / 32.1 GB	1387	29
...otron 8B UltraLong 1M Instruct	1048K / 32.1 GB	505	52
Zero Llama 3.1 8B Beta6	1048K / 16.1 GB	2	1
...dger Nu Llama 3.1 8B UltraLong	1048K / 16.2 GB	1	3
....1 1million Ctx Dark Planet 8B	1048K / 32.3 GB	8	3

Rank the Suzume Llama 3 8B Multilingual Capabilities

🆘 Have you tried this model? Rate its performance. This feedback would greatly assist ML community in identifying the most suitable model for their needs. Your contribution really does make a difference! 🌟

Instruction Following and Task Automation
Factuality and Completeness of Knowledge
Censorship and Alignment
Data Analysis and Insight Generation
Text Generation
Text Summarization and Feature Extraction
Code Generation
Multi-Language Support and Translation

What open-source LLMs or SLMs are you in search of? 51567 in total.

Email us: info@extractum.io. Our Privacy Policy | Terms and Conditions | Suggest an improvement.

Our Social Media →

Original data from HuggingFace, OpenCompass and various public git repos.

Release v20241124

Support LLM Explorer