SauerkrautLM Mixtral 8x7B Instruct GPTQ is an open-source language model by TheBloke. Features: 46.7b LLM, VRAM: 23.8GB, Context: 32K, License: apache-2.0, MoE, Quantized, Instruction-Based, LLM Explorer Score: 0.1.
Training involved augmenting German data to improve grammatical and syntactical correctness.
Supported Languages
English (fluent), German (fluent), French (fluent), Italian (fluent), Spanish (fluent)
Training Details
Data Sources:
argilla/distilabel-math-preference-dpo, translated Parts of the HuggingFaceH4/ultrafeedback_binarized, Sauerkraut-7b-HerO, German SauerkrautLM-DPO dataset
Methodology:
DPO alignment
Model Architecture:
Mixture of Experts (MoE)
Safety Evaluation
Ethical Considerations:
Despite data cleansing efforts, the possibility of uncensored content slipping through cannot be ruled out.