Phi 3 Small 128K Instruct is an open-source language model by aaronday3. Features: 7.4b LLM, VRAM: 14.8GB, Context: 128K, License: mit, Instruction-Based, LLM Explorer Score: 0.13.
Not designed for all downstream purposes, Limitations in non-English languages, Potential to reinforce stereotypes
Considerations:
Developers should ensure compliance with relevant laws and assess limitations.
Additional Notes
Refer to Microsoft’s Trademark & Brand Guidelines for use of trademarks or logos. Adherence to local laws and regulations is required for legal and ethical deployment.
Supported Languages
languages (Multilingual), proficiency ()
Training Details
Data Sources:
Publicly available documents filtered for quality, High-quality educational data, Synthetic 'textbook-like' data
Data Volume:
4.8 trillion tokens
Methodology:
Supervised fine-tuning and Direct Preference Optimization
Context Length:
128000
Training Time:
18 days
Hardware Used:
1024 H100-80G GPUs
Model Architecture:
Dense decoder-only Transformer
Responsible Ai Considerations
Fairness:
Quality of Service issues, representation of harms & perpetuation of stereotypes, inappropriate or offensive content generation.
Transparency:
Not specified
Accountability:
Not specified
Mitigation Strategies:
Developers should apply responsible AI best practices and undertake further assessments and adjustments for use case suitability.
Input Output
Input Format:
Chat format using markdown templates
Accepted Modalities:
text
Output Format:
Generated text
Performance Tips:
Ensure proper use of BOS token for reliable results. Use in combination with applicable safety classifiers.