Cerebras GPT 2.7B is an open-source language model by cerebras. Features: 2.7b LLM, VRAM: 10.7GB, License: apache-2.0, LLM Explorer Score: 0.11, Arc: 29.1, HellaSwag: 49.3, MMLU: 25.2, GSM8K: 0.5.
Cerebras GPT 2.7B Parameters and Internals
Model Type Transformer-based Language Model, Causal Language Model
Use Cases
Areas: NLP research, applications, ethics, alignment research
Applications: Primary Use Cases: Research into large language models, Foundation model for NLP
Limitations: Not suitable for machine translation, Not tuned for human-facing dialog
Considerations: Further testing and mitigations are required for safety-related applications.
Additional Notes Compatible with Hugging Face pipelines and Cerebras Model Studio for pre-training and fine-tuning. Checkpoints available in Cerebras Model Zoo.
Supported Languages English (full proficiency)
Training Details
Data Sources: Data Volume: Methodology: Training followed Chinchilla scaling laws with 20 tokens per model parameter using the Pile dataset.
Context Length: Hardware Used: 16 CS-2 wafer scale systems
Model Architecture: GPT-3 style model with full attention
Responsible Ai Considerations
Fairness: Analysis has been conducted on the ethical standpoints of the Pile dataset, including toxicity and gender bias.
Accountability: Developers and researchers should ensure the appropriateness of model use.
Mitigation Strategies: Standard Pile dataset pre-processing.
Input Output
Rank the Cerebras GPT 2.7B capabilities
Have you tried this model? Rate its performance — this feedback helps the ML community find the right model for their needs.
Instruction Following and Task Automation
Factuality and Completeness of Knowledge
Censorship and Alignment
Data Analysis and Insight Generation
Text Generation
Text Summarization and Feature Extraction
Code Generation
Multi-Language Support and Translation
Best Alternatives to Cerebras GPT 2.7B
Note: green Score (e.g. "73.2 ") means that the model is better than cerebras/Cerebras-GPT-2.7B .
Expand