What are the hardware requirements for Cerebras GPT 1.3B?

Cerebras GPT 1.3B requires approximately 5.4 GB of VRAM. Quantized variants may run on less VRAM; see the Quantized Models section on this page.

Who developed Cerebras GPT 1.3B and how large is it?

Cerebras GPT 1.3B is developed by cerebras, a model with 1.3b parameters. The model is published as open weights on Hugging Face and indexed on LLM Explorer with full benchmark history.

How does Cerebras GPT 1.3B perform on standard benchmarks?

Cerebras GPT 1.3B has the following published scores: MMLU 26.59. Compare against reference models on this page or on the LLM Explorer leaderboards.

Cerebras GPT 1.3B by cerebras — VRAM 5.4GB

Name: Cerebras GPT 1.3B
Author: cerebras

Cerebras GPT 1.3B is an open-source language model by cerebras. Features: 1.3b LLM, VRAM: 5.4GB, License: apache-2.0, LLM Explorer Score: 0.12, Arc: 26.3, HellaSwag: 38.5, MMLU: 26.6, GSM8K: 0.2.

Arxiv:2101.00027 Arxiv:2203.15556 Arxiv:2304.03208 Dataset:the pile Deploy:azure En Endpoints compatible Gpt2 Pytorch Region:us

Model Card on HF 🤗: https://huggingface.co/cerebras/Cerebras-GPT-1.3B

Cerebras GPT 1.3B Parameters and Internals

Model Type

text generation, causal-lm

Use Cases

Areas:

NLP, Research

Primary Use Cases:

Research into LLMs, Foundation model for NLP applications, ethics, and alignment research.

Supported Languages

English (high)

Training Details

Data Sources:

The Pile

Data Volume:

371B tokens

Methodology:

GPT-3 style architecture, Full attention

Context Length:

2048

Hardware Used:

Cerebras Andromeda AI supercomputer (16 CS-2 wafer scale systems)

Model Architecture:

Transformer-based, GPT-3 style

Safety Evaluation

Ethical Considerations:

Model was trained on the Pile dataset which was analyzed for ethical issues such as toxicity and bias.

Responsible Ai Considerations

Fairness:

Potential for distributional bias from the Pile dataset.

Accountability:

Developers are accountable for the model's outputs when using in production.

Mitigation Strategies:

Standard Pile dataset preprocessing mitigations were employed.

Input Output

Accepted Modalities:

text

LLM Name	Cerebras GPT 1.3B
Repository 🤗	https://huggingface.co/cerebras/Cerebras-GPT-1.3B
Model Size	1.3b
Required VRAM	5.4 GB
Updated	2026-06-03
Maintainer	cerebras
Model Type	gpt2
Model Files	5.4 GB
Supported Languages	en
Model Architecture	AutoModel
License	apache-2.0
Vocabulary Size	50257
Activation Function	gelu

Rank the Cerebras GPT 1.3B capabilities

Have you tried this model? Rate its performance — this feedback helps the ML community find the right model for their needs.

Instruction Following and Task Automation
Factuality and Completeness of Knowledge
Censorship and Alignment
Data Analysis and Insight Generation
Text Generation
Text Summarization and Feature Extraction
Code Generation
Multi-Language Support and Translation

Best Alternatives to Cerebras GPT 1.3B

Best Alternatives	Context / RAM	Downloads	Likes
Argonne 2.5 Ctx13568	13K / 2.4 GB	14	0
Diablo Italian Base 1.3B	2K / 2.6 GB	8	0
Diablo Italian Chatbot 1.3B	2K / 2.6 GB	6	0
GPT Neo X 1.3B Qlora Test	0K / 0 GB	0	1
...lb 200 Distilled 1.3B Ct2 Int8	0K / 1.4 GB	177	6
...pseek Coder 1.3B Instruct GGUF	0K / 0.6 GB	12038	54
Deepseek Coder 1.3B Base GGUF	0K / 0.6 GB	3249	9
...eared LLaMA 1.3B ShareGPT GGUF	0K / 0.6 GB	289	2

Note: green Score (e.g. "73.2") means that the model is better than cerebras/Cerebras-GPT-1.3B.