Large Language Model, Bilingual, Causal Language Model, Decoder
Use Cases
Areas:
Research, Commercial
Applications:
Research, Natural language understanding and generation, Mechanistic interpretability analyses, Quantitative studies of Arabic cultural phenomena, Development of chat apps, Sentiment analysis, Summarization
Limitations:
Model is bilingual, optimized for Arabic and English; not for other languages, Prohibited for use in illegal activities, Not for sensitive information handling
Additional Notes
The models are aimed at enhancing research and commercial applications for Arabic NLP. The methodology focuses on improving contexts and extending language capabilities.
Supported Languages
Arabic (Proficient), English (Proficient)
Training Details
Data Sources:
Web (publicly available web pages, Wikipedia articles, news articles, and social network content), Code data in various programming languages, Books (publicly available Arabic and English books), Scientific (subset of ArXiv papers), Synthetic data (translated from English to Arabic)
Data Volume:
1.6 Trillion tokens
Methodology:
Auto-regressive training with enhancements like SwiGLU and ALiBi for Jais-family; uses RoPE and Grouped Query Attention for Jais-adapted.
Context Length:
2048
Model Architecture:
Transformer-based, decoder-only (GPT-3 architecture with advancements for better context handling)
๐ Have you tried this model? Rate its performance. This feedback would greatly assist ML community in identifying the most suitable model for their needs. Your contribution really does make a difference! ๐
Instruction Following and Task Automation
Factuality and Completeness of Knowledge
Censorship and Alignment
Data Analysis and Insight Generation
Text Generation
Text Summarization and Feature Extraction
Code Generation
Multi-Language Support and Translation
What open-source LLMs or SLMs are you in search of? 52721 in total.