Name: Distilgpt2
Author: huggingface.co

Distilgpt2 is an open-source language model by huggingface.co. Features: 88.2m LLM, VRAM: 0.4GB, License: apache-2.0, LLM Explorer Score: 0.14.

Model Card on HF 🤗: https://huggingface.co/distilbert/distilgpt2

Distilgpt2 Benchmarks

IFEval: 6.11 vs 88 (so35)^-93.1%

MATH Lvl 5: 0.6

LLME Score: 0.14408

^nn.n% — How the model compares to the reference models: Anthropic Sonnet 3.5 ("so35"), GPT-4o ("gpt4o") or GPT-4 ("gpt4").

What is the LLM Explorer Rank (Score)

🌟 Advertise your project 🚀

Distilgpt2 Parameters and Internals

Model Type

Transformer-based Language Model, Text Generation

Use Cases

Areas:

Research applications, Commercial applications

Applications:

Writing assistance, Creative writing and art, Entertainment

Primary Use Cases:

Text generation

Limitations:

Bias related to race and gender, Not suitable for applications where factual correctness is required

Considerations:

Careful consideration of the bias and context of usage. Not recommended for direct human interaction without thorough bias assessment.

Additional Notes

DistilGPT2 is designed to facilitate faster, resource-efficient text generation applications.

Supported Languages

English (native)

Training Details

Data Sources:

OpenWebTextCorpus (an open-source reproduction of OpenAI’s WebText dataset)

Methodology:

Knowledge distillation

Hardware Used:

8 16GB V100 GPUs

Model Architecture:

Transformer

Responsible Ai Considerations

Fairness:

DistilGPT2 suffers from persistent bias issues similar to those described for GPT-2. The distilled versions have shown reductions in toxicity and bias compared to their teacher models, yet still present statistically significant bias.

Mitigation Strategies:

Ongoing research to mitigate bias using additional techniques for distilled models.

Input Output

Input Format:

Text input for generating text

Accepted Modalities:

text

Output Format:

Generated text

Performance Tips:

Use of seed setting for reproducibility

LLM Name	Distilgpt2
Repository 🤗	https://huggingface.co/distilbert/distilgpt2
Model Size	88.2m
Required VRAM	0.4 GB
Updated	2026-04-10
Maintainer	huggingface.co
Model Type	gpt2
Model Files	0.4 GB 0.4 GB
Supported Languages	en
Model Architecture	GPT2LMHeadModel
License	apache-2.0
Model Max Length	1024
Vocabulary Size	50257
Activation Function	gelu_new

Best Alternatives to Distilgpt2

Best Alternatives	Context / RAM	Downloads	Likes
Leo DistilGPT Medical	0K / 0.4 GB	16	1
Distilgpt2 HC3	0K / 0.3 GB	1066	1
Distilgpt2 Emailgen V2	0K / 0.3 GB	1013	5
Distilgpt2 Emailgen	0K / 0.4 GB	27	4
Distilgpt2 Fables Demo	0K / 0.3 GB	5	0
Distilgpt2 Nepali	0K / 0.3 GB	2030	7
Distilgpt2 Tiny Conversational	0K / 0.3 GB	47	5

Note: green Score (e.g. "73.2") means that the model is better than distilbert/distilgpt2.

Rank the Distilgpt2 Capabilities

🆘 Have you tried this model? Rate its performance. This feedback would greatly assist ML community in identifying the most suitable model for their needs. Your contribution really does make a difference! 🌟

Instruction Following and Task Automation
Factuality and Completeness of Knowledge
Censorship and Alignment
Data Analysis and Insight Generation
Text Generation
Text Summarization and Feature Extraction
Code Generation
Multi-Language Support and Translation

What open-source LLMs or SLMs are you in search of? 52721 in total.

Email us: info@extractum.io. Our Privacy Policy | Terms and Conditions | Suggest an improvement.

Our Social Media →

Original data from HuggingFace, OpenCompass and various public git repos.

Check out Ag3ntum — our secure, self-hosted AI agent for server management.

Release v20260328a

Support LLM Explorer

Distilgpt2 by huggingface.co

» All LLMs » huggingface.co » Distilgpt2 URL Share it on

Distilgpt2 Benchmarks

Distilgpt2 Parameters and Internals

Best Alternatives to Distilgpt2

Rank the Distilgpt2 Capabilities

What open-source LLMs or SLMs are you in search of? 52721 in total.