| Model Type | |
| Use Cases |
| Areas: | |
| Applications: | | Assistant-like chat, Natural language generation tasks |
|
| Primary Use Cases: | | Dialogue use cases, Text generation |
|
| Limitations: | | Use other than English, Violation of laws, Prohibited uses in Acceptable Use Policy |
|
| Considerations: | | Fine-tuning for languages beyond English allowed under license compliance |
|
|
| Additional Notes | | Included benchmarks demonstrate performance scaling with model size and modifications to improve model safety. |
|
| Supported Languages | |
| Training Details |
| Data Sources: | | Publicly available online data |
|
| Data Volume: | |
| Methodology: | | Supervised fine-tuning (SFT) and reinforcement learning with human feedback (RLHF) |
|
| Context Length: | |
| Hardware Used: | | Meta's Research SuperCluster and production clusters, third-party cloud compute |
|
| Model Architecture: | | An auto-regressive language model that uses an optimized transformer architecture |
|
|
| Safety Evaluation |
| Methodologies: | | Extensive red teaming exercises, Adversarial evaluations, CyberSecEval suite, Child Safety risk assessments |
|
| Findings: | | Llama 3 significantly less likely to falsely refuse prompts than Llama 2, Drastically reduced residual risks while maintaining helpfulness |
|
| Risk Categories: | | Child Safety, Cyber Security, CBRNE, General Safety |
|
| Ethical Considerations: | | Promoting safe, fair, and responsible use; transparency and safety measures implemented |
|
|
| Responsible Ai Considerations |
| Fairness: | | Reflects a wide range of backgrounds and perspectives |
|
| Transparency: | | Open consortium involvement, transparency measures in safety evaluation |
|
| Accountability: | | Developers and Meta are accountable for ensuring proper use |
|
| Mitigation Strategies: | | Implemented safety best practices; extensive safety evaluations |
|
|
| Input Output |
| Input Format: | |
| Accepted Modalities: | |
| Output Format: | |
| Performance Tips: | | Use pre-trained and instruction tuned variants appropriately for tasks |
|
|
| Release Notes |
| Date: | |
| Notes: | | Initial release of Meta Llama 3 models in 8B and 70B sizes. |
|
|
|