Starcoderplus GPTQ is an open-source language model by TheBloke. Features: 15.8b LLM, VRAM: 8.9GB, License: bigcode-openrail-m, Quantized, Code Generating, LLM Explorer Score: 0.07.
The model may encounter limitations with non-English text and can carry stereotypes and biases., Generated code might have errors, inefficiencies, or potential vulnerabilities.
Considerations:
Attribution might be required for generated code based on the dataset.
Additional Notes
The instruction-tuned version in StarChat makes the model a capable assistant.
Supported Languages
English (native), Programming languages (80+)
Training Details
Data Sources:
RedefinedWeb, StarCoderData, The Stack (v1.2), Wikipedia
Data Volume:
1.6 trillion tokens
Methodology:
Fill-in-the-Middle objective
Context Length:
8192
Training Time:
14 days
Hardware Used:
512 Tesla A100 GPUs
Model Architecture:
GPT-2 model with multi-query attention
Responsible Ai Considerations
Fairness:
The model carries the stereotypes and biases commonly encountered online, given its training data.
Mitigation Strategies:
The code dataset was filtered for permissive licenses only.