Mpt 7B 8K is an open-source language model by mosaicml. Features: 7b LLM, VRAM: 13.3GB, License: apache-2.0, LLM Explorer Score: 0.12, Arc: 47.4, HellaSwag: 77.4, MMLU: 42.6, GSM8K: 8.3.
MPT-7B-8k uses a modified transformer architecture, optimized for efficient training and inference with ALiBi for handling long inputs.
Context Length:
8192
Training Time:
9.5 days
Hardware Used:
440 A100-40GB GPUs
Model Architecture:
Decoder-only transformer with modifications such as FlashAttention, ALiBi, elimination of positional embeddings.
Safety Evaluation
Ethical Considerations:
MPT-7B-8k can produce factually incorrect, lewd, biased or offensive outputs. It should not be used for human-facing interactions without further guardrails and user consent.
Responsible Ai Considerations
Fairness:
Model may have biases inherited from training data.
Transparency:
Pretraining data was openly available, preprocessed to remove unsuitable content.
Accountability:
Responsibility of MosaicML.
Mitigation Strategies:
Guardrails recommended before deployment.
Input Output
Input Format:
Text sequences, up to 8k tokens
Accepted Modalities:
text
Output Format:
Generated text
Performance Tips:
Utilize optimized implementations like FlashAttention and ensure usage with bfloat16 precision on GPUs.