| Model Type | | Language Model, Text Generation |
|
| Use Cases |
| Areas: | | Research, Commercial applications |
|
| Applications: | | Chat assistants, Sentiment analysis, Document summarization |
|
| Primary Use Cases: | | Arabic NLP research, Chat generation |
|
| Limitations: | | Limited to Arabic and English., Cannot be used for harmful content generation. |
|
| Considerations: | | Improved cultural understanding for Arabic. Not suited for other languages. |
|
|
| Additional Notes | | Particularly efficient at processing Arabic language contexts, aiming to cater to Arabic-speaking audiences specifically. |
|
| Supported Languages | | Arabic (High), English (High) |
|
| Training Details |
| Data Sources: | | Web, Code, Books, Scientific articles, Synthetic translations |
|
| Data Volume: | |
| Methodology: | | Scratch pre-training and adaptation from Llama-2. Enhanced training with the SwiGLU activation function and ALiBi position encoding. |
|
| Context Length: | |
| Hardware Used: | | 64 Cerebras CS-2 Wafer-Scale Engines |
|
| Model Architecture: | | Transformer-based, decoder-only architecture with SwiGLU for Jais-family and RoPE embedding for adapted models. |
|
|
| Responsible Ai Considerations |
| Fairness: | | Techniques implemented to reduce bias are not specified in detail. |
|
| Transparency: | | Basic preprocessing and role of language-specific techniques mentioned. |
|
| Accountability: | | Users are responsible for applications. |
|
| Mitigation Strategies: | |
|
| Input Output |
| Input Format: | |
| Accepted Modalities: | |
| Output Format: | |
| Performance Tips: | | Ensure the appropriate prompt design for task adaptation. |
|
|
