Mpt 1B Redpajama 200B Dolly by mosaicml

 ยป  All LLMs  ยป  mosaicml  ยป  Mpt 1B Redpajama 200B Dolly   URL Share it on

  Arxiv:2108.12409   Arxiv:2205.14135   Arxiv:2302.13971   Autotrain compatible   Custom code Dataset:togethercomputer/redpa...   Mosaic gpt   Pytorch   Region:us

Mpt 1B Redpajama 200B Dolly Benchmarks

nn.n% — How the model compares to the reference models: Anthropic Sonnet 3.5 ("so35"), GPT-4o ("gpt4o") or GPT-4 ("gpt4").
Mpt 1B Redpajama 200B Dolly (mosaicml/mpt-1b-redpajama-200b-dolly)
๐ŸŒŸ Advertise your project ๐Ÿš€

Mpt 1B Redpajama 200B Dolly Parameters and Internals

Model Type 
decoder-only transformer
Training Details 
Data Sources:
RedPajama Common Crawl, C4, RedPajama GitHub, RedPajama Wikipedia, RedPajama Books, RedPajama Arxiv, RedPajama StackExchange
Data Volume:
200B tokens
Context Length:
2048
Hardware Used:
440 A100-40GBs
Model Architecture:
24 layers, 16 attention heads, width 2048
LLM NameMpt 1B Redpajama 200B Dolly
Repository ๐Ÿค—https://huggingface.co/mosaicml/mpt-1b-redpajama-200b-dolly 
Model Size1b
Required VRAM5.2 GB
Updated2025-09-15
Maintainermosaicml
Model Typemosaic_gpt
Model Files  5.2 GB
Model ArchitectureMosaicGPT
Licensecc-by-sa-3.0
Model Max Length2048
Transformers Version4.27.4
Tokenizer ClassGPTNeoXTokenizer
Vocabulary Size50432
Torch Data Typefloat32

Best Alternatives to Mpt 1B Redpajama 200B Dolly

Best Alternatives
Context / RAM
Downloads
Likes
Mpt 1B Redpajama 200B0K / 5.2 GB29562
Mpt 1B Redpajama 200B0K / 5.2 GB25292
Note: green Score (e.g. "73.2") means that the model is better than mosaicml/mpt-1b-redpajama-200b-dolly.

Rank the Mpt 1B Redpajama 200B Dolly Capabilities

๐Ÿ†˜ Have you tried this model? Rate its performance. This feedback would greatly assist ML community in identifying the most suitable model for their needs. Your contribution really does make a difference! ๐ŸŒŸ

Instruction Following and Task Automation  
Factuality and Completeness of Knowledge  
Censorship and Alignment  
Data Analysis and Insight Generation  
Text Generation  
Text Summarization and Feature Extraction  
Code Generation  
Multi-Language Support and Translation  

What open-source LLMs or SLMs are you in search of? 51387 in total.

Our Social Media →  
Original data from HuggingFace, OpenCompass and various public git repos.
Release v20241124