Dlite V1 124M is an open-source language model by aisquared. Features: 124m LLM, VRAM: 0.5GB, License: apache-2.0, LLM Explorer Score: 0.11, Arc: 24.3, HellaSwag: 31.2, MMLU: 25.1.
Not a state-of-the-art model, Experimental model, Can exhibit undesired behaviors: factual inaccuracies, biases, offensive responses, toxicity, and hallucinations
Considerations:
Exercise good judgment when applying this technology.
Additional Notes
Model not suitable for environments other than research purposes.
Supported Languages
EN (English)
Training Details
Data Sources:
tatsu-lab/alpaca
Data Volume:
50k records
Methodology:
Fine-tuning on GPT-2
Hardware Used:
Single T4 GPU
Model Architecture:
Derived from GPT-2
Input Output
Performance Tips:
Including `torch_dtype=torch.bfloat16` is generally recommended to reduce memory usage.