Model Type | Large Language Model, Text Generation |
|
Use Cases |
Areas: | |
Primary Use Cases: | User assistance, Reasoning |
|
Limitations: | Not suitable for full knowledge retrieval without documented retrieval augmented generation. |
|
Considerations: | Finetune for domain adaptation for specialized tasks; use a repetition penalty of 1.3 for full performance. |
|
|
Additional Notes | The model has been preference-optimized using the ChatML template for specific multi-turn conversational tasks. |
|
Supported Languages | |
Training Details |
Data Sources: | pints-ai/Expository-Prose-V1, HuggingFaceH4/ultrachat_200k, Open-Orca/SlimOrca-Dedup, meta-math/MetaMathQA, HuggingFaceH4/deita-10k-v0-sft, WizardLM/WizardLM_evol_instruct_V2_196k, togethercomputer/llama-instruct, LDJnr/Capybara, HuggingFaceH4/ultrafeedback_binarized |
|
Data Volume: | |
Methodology: | Pre-training emphasizes quality over quantity. Fine-tuning and DPO follow the ChatML template. |
|
Context Length: | |
Training Time: | |
Hardware Used: | GPU with at least 8GB of VRAM |
|
Model Architecture: | Llama 2 Autoregressive Model with Mistral tokenizer and Float32 precision. |
|
|
Input Output |
Input Format: | Chat representation using ChatML template. |
|
Accepted Modalities: | |
Output Format: | Generated text output from user prompts. |
|
Performance Tips: | Use a repetition penalty of 1.3 to optimize output effectivity. |
|
|