Calme 4x7B MoE V0.1 is an open-source language model by MaziyarPanahi. Features: 24.2b LLM, VRAM: 48.3GB, Context: 32K, License: apache-2.0, MoE, LLM Explorer Score: 0.2, Arc: 72.5, HellaSwag: 88.6, MMLU: 64.6, GSM8K: 71.1.
Calme 4x7B MoE V0.1 Parameters and Internals
| Model Type | | Mixture of Experts (MoE), text generation |
|
| Use Cases |
| Areas: | |
| Applications: | | educational tools, academic research, commonsense reasoning, natural language applications, trustworthy information retrieval, math educational software, context understanding |
|
|
| Additional Notes | | Quantized versions available for use on computers without accelerated hardware. |
|
| Supported Languages | | English (high proficiency), French (high proficiency), Ukrainian (high proficiency) |
|
| Training Details |
| Methodology: | | Mixture of Experts, fine-tuning of four individidual Calme-7B models |
|
| Model Architecture: | | Integrates four Calme-7B models with two experts per token |
|
|
| Input Output |
| Accepted Modalities: | |
| Output Format: | |
|
Quantized Models of the Calme 4x7B MoE V0.1
Best Alternatives to Calme 4x7B MoE V0.1
Note: green Score (e.g. "73.2") means that the model is better than MaziyarPanahi/Calme-4x7B-MoE-v0.1.