Facebook Opt 125M Qcqa Ub 6 Best For KV Cache is an open-source language model by xformAI. Features: 125m LLM, VRAM: 0.5GB, Context: 2K, License: mit, LLM Explorer Score: 0.15, Arc: 24.2, HellaSwag: 25, MMLU: 23.1.
Facebook Opt 125M Qcqa Ub 6 Best For KV Cache Parameters and Internals
Additional Notes
This implementation is supposed to be more efficient than the corresponding GQA one.
Supported Languages
en (native)
Training Details
Model Architecture:
The original MHA architecture is preserved, but instead of having a single K/V head, different K/V heads corresponding to the same group have the same mean-pooled K or V values with up to 6 groups of KV heads per layer instead of original 12 KV heads in the MHA implementation.