| Model Type | | text generation, instruction tuned |
|
| Use Cases |
| Areas: | |
| Applications: | | Instruction tuned models for assistant-like chat, Pretrained models for various natural language tasks |
|
| Primary Use Cases: | | English language applications |
|
| Limitations: | | Not for use in languages other than English, Requires adherence to the Use Policy and Llama 3 Community License |
|
| Considerations: | | Developers may fine-tune for additional languages within license compliance. |
|
|
| Additional Notes | | Llama 3 is designed with openness, inclusivity, and helpfulness as core values. Testing is primarily in English, with certain potential risks and uncertainties. |
|
| Supported Languages | |
| Training Details |
| Data Sources: | | publicly available online data |
|
| Data Volume: | | 15T+ tokens for pretraining, over 10M human-annotated examples for fine-tuning |
|
| Methodology: | | Auto-regressive language model using an optimized transformer architecture, supervised fine-tuning and reinforcement learning with human feedback (RLHF) |
|
| Context Length: | |
| Hardware Used: | | Meta's Research SuperCluster, H100-80GB GPUs |
|
| Model Architecture: | | Auto-regressive language model with optimized transformer architecture |
|
|
| Safety Evaluation |
| Methodologies: | | Red teaming, Adversarial evaluations, CyberSecEval |
|
| Findings: | | Equivalent or safer than models with similar coding capabilities |
|
| Risk Categories: | | CBRNE threats, Cyber attacks, Child safety risks |
|
| Ethical Considerations: | | Responsible AI development with safety benchmarks, iterative testing during model training, and community involvement. |
|
|
| Responsible Ai Considerations |
| Transparency: | | Uses Responsible Use Guide and tools like Meta Llama Guard 2 for transparency. |
|
| Accountability: | | Meta and developers share responsibilities to avoid bias and enhance safety. |
|
| Mitigation Strategies: | | Supervised fine-tuning and reinforcement learning with human feedback to align with preferences. |
|
|
| Input Output |
| Input Format: | |
| Accepted Modalities: | |
| Output Format: | |
|