GPT Neo 125M is an open-source language model by EleutherAI. Features: 125m LLM, VRAM: 0.5GB, Context: 2K, License: mit, LLM Explorer Score: 0.14, Arc: 23, HellaSwag: 30.3, MMLU: 26, GSM8K: 0.3.
GPT Neo 125M Parameters and Internals
| Model Type | | text generation, causal-lm |
|
| Use Cases |
| Areas: | | Research, Commercial applications |
|
| Primary Use Cases: | | Text generation from a prompt |
|
| Limitations: | | May produce socially unacceptable text, Dataset contains profanity and abrasive language |
|
| Considerations: | | Human curation of outputs is recommended. |
|
|
| Training Details |
| Data Sources: | |
| Data Volume: | |
| Methodology: | | masked autoregressive language model, using cross-entropy loss |
|
| Model Architecture: | |
|
| Safety Evaluation |
| Ethical Considerations: | | Potential to produce socially unacceptable text. |
|
|
| Responsible Ai Considerations |
| Mitigation Strategies: | | Human curation or filtering of outputs is recommended. |
|
|
| Input Output |
| Input Format: | | String of text as a prompt |
|
| Accepted Modalities: | |
| Output Format: | |
| Performance Tips: | | Using text-generation pipeline with options like 'do_sample=True' and 'min_length=20'. |
|
|
| LLM Name | GPT Neo 125M |
| Repository 🤗 | https://huggingface.co/EleutherAI/gpt-neo-125m |
| Model Size | 125m |
| Required VRAM | 0.5 GB |
| Updated | 2026-07-15 |
| Maintainer | EleutherAI |
| Model Type | gpt_neo |
| Model Files | 0.5 GB 0.5 GB |
| Supported Languages | en |
| Model Architecture | GPTNeoForCausalLM |
| License | mit |
| Context Length | 2048 |
| Model Max Length | 2048 |
| Transformers Version | 4.9.0.dev0 |
| Tokenizer Class | GPT2Tokenizer |
| Beginning of Sentence Token | <|endoftext|> |
| End of Sentence Token | <|endoftext|> |
| Unk Token | <|endoftext|> |
| Vocabulary Size | 50257 |
| Activation Function | gelu_new |
| Errors | replace |
Best Alternatives to GPT Neo 125M
Note: green Score (e.g. "73.2") means that the model is better than EleutherAI/gpt-neo-125m.