Name: Starcoder2 3B
Author: bigcode

Starcoder2 3B is an open-source language model by bigcode. Features: 3b LLM, VRAM: 12.1GB, Context: 16K, License: bigcode-openrail-m, HF Score: 39.3, LLM Explorer Score: 0.25, Arc: 34.6, HellaSwag: 47.6, MMLU: 38.7, TruthfulQA: 40.5, WinoGrande: 54.5, GSM8K: 19.6, HumanEval: 31.4.

Arxiv:2004.05150 Arxiv:2205.14135 Arxiv:2207.14255 Arxiv:2305.13245 Arxiv:2402.19173 Code Dataset:bigcode/the-stack-v2-t... Endpoints compatible Model-index Region:us Safetensors Starcoder2

Model Card on HF 🤗: https://huggingface.co/bigcode/starcoder2-3b

Starcoder2 3B Benchmarks

MMLU Pro: 7.07

MUSR: 1.43

BBH: 8.91

IFEval: 20.37 vs 88 (so35)^-76.9%

ARC: 34.56 vs 96.7 (so35)^-64.3%

HellaSwag: 47.62 vs 95.3 (gpt4)^-50%

MMLU: 38.65 vs 88.3 (so35)^-56.2%

TruthfulQA: 40.49 vs 59 (gpt4)^-31.4%

WinoGrande: 54.54 vs 87.5 (gpt4)^-37.7%

GSM8K: 19.64 vs 96.4 (so35)^-79.6%

HumanEvalA: 23.43

Python Eva: 31.44 vs 92 (so35)^-65.8%

Java Eval: 27.41

Javascript: 35.37

CPP Eval: 27.24

PHP Eval: 27.61

Julia Eval: 19.87

Swift Eval: 25.09

MATH Lvl 5: 1.51

LLME Score: 0.24916

^nn.n% — How the model compares to the reference models: Anthropic Sonnet 3.5 ("so35"), GPT-4o ("gpt4o") or GPT-4 ("gpt4").

What is the LLM Explorer Rank (Score)

🌟 Advertise your project 🚀

Starcoder2 3B Parameters and Internals

Model Type

text-generation

Use Cases

Areas:

research, programming

Primary Use Cases:

code generation

Limitations:

Generated code may contain bugs or exploits., Inefficiency and unintended outputs

Supported Languages

programming_languages (17 programming languages,)

Training Details

Data Sources:

GitHub code, Arxiv, Wikipedia

Data Volume:

3+ trillion tokens

Methodology:

Grouped Query Attention, Fill-in-the-Middle objective

Context Length:

16384

Hardware Used:

160 A100 GPUs

Model Architecture:

Transformer decoder with grouped-query and sliding window attention

Input Output

Accepted Modalities:

text

LLM Name	Starcoder2 3B
Repository 🤗	https://huggingface.co/bigcode/starcoder2-3b
Model Size	3b
Required VRAM	12.1 GB
Updated	2026-04-11
Maintainer	bigcode
Model Type	starcoder2
Model Files	12.1 GB
Model Architecture	Starcoder2ForCausalLM
License	bigcode-openrail-m
Context Length	16384
Model Max Length	16384
Transformers Version	4.37.0.dev0
Tokenizer Class	GPT2Tokenizer
Vocabulary Size	49152

Quantized Models of the Starcoder2 3B

Model	Likes	Downloads	VRAM
Starcoder2 3B AWQ	0	15	2 GB
StarCoder2 3B GGUF	16	1418	1 GB

Best Alternatives to Starcoder2 3B

Best Alternatives	Context / RAM	Downloads	Likes
Starcoder2 3b AutoRedteam	16K / 12.7 GB	5	0
Starcoder Proto Code	16K / 6.1 GB	5	0
Mojo Starcoder2	16K / 6.4 GB	11	0
NEARCoder 3B	16K / 6.1 GB	5	0
NEAR StructTunedStarcoder2	16K / 6.1 GB	5	0
NEAR PreTrainedStarCoder2	16K / 6.1 GB	5	0
Bigcode Starcoder2 3B 8bits	16K / 3.2 GB	6	0
Starcoder2 3B Instruct	16K / 6.1 GB	47	5
OpenCodeInterpreter SC2 3B	16K / 6.4 GB	18	7
Opencsg Starcoder2 3B V0.1	16K / 6.4 GB	2	1

Rank the Starcoder2 3B Capabilities

🆘 Have you tried this model? Rate its performance. This feedback would greatly assist ML community in identifying the most suitable model for their needs. Your contribution really does make a difference! 🌟

Instruction Following and Task Automation
Factuality and Completeness of Knowledge
Censorship and Alignment
Data Analysis and Insight Generation
Text Generation
Text Summarization and Feature Extraction
Code Generation
Multi-Language Support and Translation

What open-source LLMs or SLMs are you in search of? 52721 in total.

Email us: info@extractum.io. Our Privacy Policy | Terms and Conditions | Suggest an improvement.

Our Social Media →

Original data from HuggingFace, OpenCompass and various public git repos.

Check out Ag3ntum — our secure, self-hosted AI agent for server management.

Release v20260328a

Support LLM Explorer

Starcoder2 3B by bigcode

» All LLMs » bigcode » Starcoder2 3B URL Share it on