Llama3 8B Chinese Chat is an open-source language model by shenzhi-wang. Features: 8b LLM, VRAM: 16.1GB, Context: 8K, License: llama3, Instruction-Based, LLM Explorer Score: 0.19, Arc: 61.8, HellaSwag: 80.1, MMLU: 67, GSM8K: 67.2.
Model primarily fine-tuned for Chinese & English users with abilities like roleplaying & tool-using. For optimal performance, model's identity is not fine-tuned.
Supported Languages
Chinese (high), English (medium)
Training Details
Data Sources:
mixed Chinese-English dataset
Data Volume:
~100K preference pairs
Methodology:
ORPO (Reference-free Monolithic Preference Optimization with Odds Ratio)
Context Length:
8192
Model Architecture:
Meta-Llama-3
Input Output
Input Format:
instruction-based prompts
Accepted Modalities:
text
Output Format:
text
Release Notes
Version:
v2.1
Date:
May 6, 2024
Notes:
Training dataset is 5x larger (~100K preference pairs). Enhancements in roleplay, function calling, math. Less prone to including English words in Chinese responses.
Version:
v2
Date:
Apr. 29, 2024
Notes:
Increases in training data size from 20K to 100K; improved performance in roleplay, tool using, and math.
Version:
v1
Notes:
Significantly reduces issues of 'Chinese questions with English answers' and the mixing of Chinese and English in responses.