What are the hardware requirements for 14B?

14B requires approximately 28.4 GB of VRAM and supports a context window of 8K tokens. Quantized variants may run on less VRAM; see the Quantized Models section on this page.

Who developed 14B and how large is it?

14B is developed by CausalLM, a model with 14b parameters. The model is published as open weights on Hugging Face and indexed on LLM Explorer with full benchmark history.

How does 14B perform on standard benchmarks?

14B has the following published scores: MMLU 65.86. Compare against reference models on this page or on the LLM Explorer leaderboards.

14B by CausalLM — VRAM 28.4GB, 8K context

Name: 14B
Author: CausalLM

14B is an open-source language model by CausalLM. Features: 14b LLM, VRAM: 28.4GB, Context: 8K, License: wtfpl, Instruction-Based, LLM Explorer Score: 0.19, Arc: 56.7, HellaSwag: 79.1, MMLU: 65.9, GSM8K: 58.6.

Causallm Dataset:baai/coig Dataset:fnlp/moss-003-sft-data Dataset:garage-baind/open-plat... Dataset:jondurbin/airoboros-3.... Dataset:josephuscheung/guanaco... Dataset:ldjnr/puffin Dataset:liuhaotian/llava-instr... Dataset:liwu/mnbvc Dataset:meta-math/metamathqa Dataset:milashkaarshif/moegirl... Dataset:open-orca/openorca Dataset:openbmb/llava zh Dataset:ryokoai/fandom23k Dataset:ryokoai/sharegpt52k Dataset:stingning/ultrachat Dataset:teknium/openhermes Dataset:tigerresearch/tigerbot... Dataset:wiki lingua Dataset:wikipedia Dataset:wizardlm/wizardlm evol... En Endpoints compatible Instruct Llama Llama2 Pytorch Qwen Region:us Sharded Zh

Model Card on HF 🤗: https://huggingface.co/CausalLM/14B

14B Benchmarks

ARC: 56.66 vs 96.7 (so35)^-41.4%

HellaSwag: 79.08 vs 95.3 (gpt4)^-17%

MMLU: 65.86 vs 88.3 (so35)^-25.4%

GSM8K: 58.61 vs 96.4 (so35)^-39.2%

LLME Score: 0.18978

^nn.n% — How the model compares to the reference models: Anthropic Sonnet 3.5 ("so35"), GPT-4o ("gpt4o") or GPT-4 ("gpt4").

What is the LLM Explorer Rank (Score)

🌟 Advertise your project 🚀

14B Parameters and Internals

Model Type

text-generation, causallm

Use Cases

Areas:

Research, Commercial applications

Primary Use Cases:

Speculative sampling

Limitations:

May produce hallucinations or unreliable outputs, Contains objectionable content, pornography, violence, and offensive language

Considerations:

Conduct own safety checks, filter keywords

Additional Notes

The model is not recommended for quantization, but rather using smaller size models like 7B.

Supported Languages

en (English), zh (Chinese)

Training Details

Data Sources:

JosephusCheung/GuanacoDataset, Open-Orca/OpenOrca, stingning/ultrachat, meta-math/MetaMathQA, liuhaotian/LLaVA-Instruct-150K, jondurbin/airoboros-3.1, WizardLM/WizardLM_evol_instruct_V2_196k, RyokoAI/ShareGPT52K, RyokoAI/Fandom23K, milashkaarshif/MoeGirlPedia_wikitext_raw_archive, wikipedia, wiki_lingua, fnlp/moss-003-sft-data, garage-bAInd/Open-Platypus, LDJnr/Puffin, openbmb/llava_zh, BAAI/COIG, TigerResearch/tigerbot-zhihu-zh-10k, liwu/MNBVC, teknium/openhermes

Data Volume:

1.3B tokens

Methodology:

Manual or synthetic rewrites, augmented text training, synthetic Wikipedia conversation dataset

Model Architecture:

Identical to LLaMA2, using same attention calculation method

Input Output

Input Format:

[chatml](https://github.com/openai/openai-python/blob/main/chatml.md)

Accepted Modalities:

text

Output Format:

text

Performance Tips:

If VRAM is insufficient, use the 7B model instead of the quantized version.

Release Notes

Version:

14B-DPO-alpha

Date:

Dec 3, 2023

Notes:

Outperforms Zephyr-β in MT-Bench.

LLM Name	14B
Repository 🤗	https://huggingface.co/CausalLM/14B
Model Size	14b
Required VRAM	28.4 GB
Updated	2026-05-18
Maintainer	CausalLM
Model Type	llama
Instruction-Based	Yes
Model Files	10.0 GB: 1-of-3 10.0 GB: 2-of-3 8.4 GB: 3-of-3
Supported Languages	en zh
Model Architecture	LlamaForCausalLM
License	wtfpl
Context Length	8192
Model Max Length	8192
Transformers Version	4.35.0.dev0
Tokenizer Class	GPT2Tokenizer
Vocabulary Size	152064
Torch Data Type	bfloat16

Quantized Models of the 14B

Model	Likes	Downloads	VRAM
CausalLM 14B GGUF	194	4134	8 GB
CausalLM 14B EXL2	3	8	8 GB
CausalLM 14B GPTQ	20	15	9 GB
CausalLM 14B AWQ	10	18	9 GB

Best Alternatives to 14B

Best Alternatives	Context / RAM	Downloads	Likes
...hocolatine 2 14B Instruct V2.0	16K / 29.4 GB	7	6
...colatine 2 14B Instruct V2.0.1	16K / 29.4 GB	30	1
CausalLM 14B EXL2	8K / 8.5 GB	8	3
CausalLM 14B GPTQ	8K / 9.7 GB	15	20
CausalLM 14B AWQ	8K / 9.7 GB	18	10

Note: green Score (e.g. "73.2") means that the model is better than CausalLM/14B.

Rank the 14B Capabilities

🆘 Have you tried this model? Rate its performance. This feedback would greatly assist ML community in identifying the most suitable model for their needs. Your contribution really does make a difference! 🌟

Instruction Following and Task Automation
Factuality and Completeness of Knowledge
Censorship and Alignment
Data Analysis and Insight Generation
Text Generation
Text Summarization and Feature Extraction
Code Generation
Multi-Language Support and Translation

What open-source LLMs or SLMs are you in search of? 53972 in total.

Email us: info@extractum.io. Our Privacy Policy | Terms and Conditions | Suggest an improvement.

Our Social Media →

Original data from HuggingFace, OpenCompass and various public git repos.

Check out Ag3ntum — our secure, self-hosted AI agent for server management.

Release v20260328a

Support LLM Explorer

14B by CausalLM