What are the hardware requirements for Rwkv 4 3B Pile?

Rwkv 4 3B Pile requires approximately 11.9 GB of VRAM. Quantized variants may run on less VRAM; see the Quantized Models section on this page.

Who developed Rwkv 4 3B Pile and how large is it?

Rwkv 4 3B Pile is developed by RWKV, a model with 3b parameters. The model is published as open weights on Hugging Face and indexed on LLM Explorer with full benchmark history.

How does Rwkv 4 3B Pile perform on standard benchmarks?

Rwkv 4 3B Pile has the following published scores: MMLU 24.67. Compare against reference models on this page or on the LLM Explorer leaderboards.

Rwkv 4 3B Pile by RWKV — VRAM 11.9GB

Name: Rwkv 4 3B Pile
Author: RWKV

Rwkv 4 3B Pile is an open-source language model by RWKV. Features: 3b LLM, VRAM: 11.9GB, LLM Explorer Score: 0.11, Arc: 36, HellaSwag: 59.7, MMLU: 24.7, GSM8K: 0.7.

Dataset:eleutherai/pile Endpoints compatible Pytorch Region:us Rwkv Sharded

Model Card on HF 🤗: https://huggingface.co/RWKV/rwkv-4-3b-pile

Rwkv 4 3B Pile Parameters and Internals

Model Type

RNN, transformer-level, LLM

Training Details

Data Sources:

EleutherAI/pile

Model Architecture:

RWKV is an RNN with transformer-level performance. It combines features of RNN and transformer architectures.

LLM Name	Rwkv 4 3B Pile
Repository 🤗	https://huggingface.co/RWKV/rwkv-4-3b-pile
Model Size	3b
Required VRAM	11.9 GB
Updated	2026-08-07
Maintainer	RWKV
Model Type	rwkv
Model Files	2.0 GB: 1-of-7 2.0 GB: 2-of-7 2.0 GB: 3-of-7 1.9 GB: 4-of-7 1.9 GB: 5-of-7 1.6 GB: 6-of-7 0.5 GB: 7-of-7
Model Architecture	RwkvForCausalLM
Transformers Version	4.29.0.dev0
Tokenizer Class	GPTNeoXTokenizer
Vocabulary Size	50277
Torch Data Type	float32

Rank the Rwkv 4 3B Pile capabilities

Have you tried this model? Rate its performance — this feedback helps the ML community find the right model for their needs.

Instruction Following and Task Automation
Factuality and Completeness of Knowledge
Censorship and Alignment
Data Analysis and Insight Generation
Text Generation
Text Summarization and Feature Extraction
Code Generation
Multi-Language Support and Translation

Best Alternatives to Rwkv 4 3B Pile

Best Alternatives	Context / RAM	Downloads	Likes
Hermes RWKV V4 3B	0K / 6 GB	10	0
Rwkv Raven 3B	0K / 11.9 GB	72	8
Rwkv 5 V2 3B 16K	0K / 6.1 GB	20	1

Note: green Score (e.g. "73.2") means that the model is better than RWKV/rwkv-4-3b-pile.