| Model Type | | Large Language Model, Text Generation |
|
| Use Cases |
| Areas: | |
| Primary Use Cases: | | User assistance, Reasoning |
|
| Limitations: | | Not suitable for full knowledge retrieval without documented retrieval augmented generation. |
|
| Considerations: | | Finetune for domain adaptation for specialized tasks; use a repetition penalty of 1.3 for full performance. |
|
|
| Additional Notes | | The model has been preference-optimized using the ChatML template for specific multi-turn conversational tasks. |
|
| Supported Languages | |
| Training Details |
| Data Sources: | | pints-ai/Expository-Prose-V1, HuggingFaceH4/ultrachat_200k, Open-Orca/SlimOrca-Dedup, meta-math/MetaMathQA, HuggingFaceH4/deita-10k-v0-sft, WizardLM/WizardLM_evol_instruct_V2_196k, togethercomputer/llama-instruct, LDJnr/Capybara, HuggingFaceH4/ultrafeedback_binarized |
|
| Data Volume: | |
| Methodology: | | Pre-training emphasizes quality over quantity. Fine-tuning and DPO follow the ChatML template. |
|
| Context Length: | |
| Training Time: | |
| Hardware Used: | | GPU with at least 8GB of VRAM |
|
| Model Architecture: | | Llama 2 Autoregressive Model with Mistral tokenizer and Float32 precision. |
|
|
| Input Output |
| Input Format: | | Chat representation using ChatML template. |
|
| Accepted Modalities: | |
| Output Format: | | Generated text output from user prompts. |
|
| Performance Tips: | | Use a repetition penalty of 1.3 to optimize output effectivity. |
|
|