What are the hardware requirements for Llama 2 13B Fp16?

Llama 2 13B Fp16 requires approximately 26 GB of VRAM and supports a context window of 4K tokens. Quantized variants may run on less VRAM; see the Quantized Models section on this page.

Who developed Llama 2 13B Fp16 and how large is it?

Llama 2 13B Fp16 is developed by TheBloke, a model with 13b parameters. The model is published as open weights on Hugging Face and indexed on LLM Explorer with full benchmark history.

How does Llama 2 13B Fp16 perform on standard benchmarks?

Llama 2 13B Fp16 has the following published scores: MMLU 55.67. Compare against reference models on this page or on the LLM Explorer leaderboards.

Llama 2 13B Fp16 by TheBloke — VRAM 26GB, 4K context

Name: Llama 2 13B Fp16
Author: TheBloke

Llama 2 13B Fp16 is an open-source language model by TheBloke. Features: 13b LLM, VRAM: 26GB, Context: 4K, Quantized, HF Score: 53.7, LLM Explorer Score: 0.12, Arc: 59.3, HellaSwag: 82.2, MMLU: 55.7, TruthfulQA: 37.4, WinoGrande: 76.6, GSM8K: 10.8.

En Facebook Fp16 Llama Llama2 Meta Pytorch Quantized Region:us Sharded

Model Card on HF 🤗: https://huggingface.co/TheBloke/Llama-2-13B-fp16

Llama 2 13B Fp16 Benchmarks

ARC: 59.3 vs 96.7 (so35)^-38.7%

HellaSwag: 82.15 vs 95.3 (gpt4)^-13.8%

MMLU: 55.67 vs 88.3 (so35)^-37%

TruthfulQA: 37.39 vs 59 (gpt4)^-36.6%

WinoGrande: 76.64 vs 87.5 (gpt4)^-12.4%

GSM8K: 10.84 vs 96.4 (so35)^-88.8%

LLME Score: 0.12238

^nn.n% — How the model compares to the reference models: Anthropic Sonnet 3.5 ("so35"), GPT-4o ("gpt4o") or GPT-4 ("gpt4").

What is the LLM Explorer Rank (Score)

Llama 2 13B Fp16 (TheBloke/Llama-2-13B-fp16)

🌟 Advertise your project 🚀

Llama 2 13B Fp16 Parameters and Internals

Model Type

text generation

Use Cases

Areas:

Commercial, Research

Primary Use Cases:

Dialogue and natural language generation tasks

Limitations:

Use in languages other than English, Use violating applicable laws

Considerations:

Specific formatting required for chat versions.

Supported Languages

English (primary)

Training Details

Data Sources:

Publicly available online data

Data Volume:

2 trillion tokens

Methodology:

Supervised fine-tuning and reinforcement learning with human feedback (RLHF)

Context Length:

4000

Training Time:

Jan 2023 - July 2023

Hardware Used:

Meta's Research Super Cluster, A100-80GB GPUs

Model Architecture:

Auto-regressive with optimized transformer architecture

Responsible Ai Considerations

Fairness:

Testing has been conducted primarily in English and may not cover all scenarios, hence outputs may be inaccurate or biased.

Transparency:

Model outputs cannot be predicted in advance.

Accountability:

Developers should perform safety testing before deploying applications.

Mitigation Strategies:

Fine-tuning and human feedback alignment strategies used for alignment to human preferences.

Input Output

Input Format:

text

Accepted Modalities:

text

Output Format:

text

Performance Tips:

Follow `INST` and `<>` tags, `BOS` and `EOS` tokens, proper whitespaces for chat models

LLM Name	Llama 2 13B Fp16
Repository 🤗	https://huggingface.co/TheBloke/Llama-2-13B-fp16
Base Model(s)	Ligma L2 13B kubernetes-bad/Ligma-L2-13b
Model Size	13b
Required VRAM	26 GB
Updated	2026-04-13
Maintainer	TheBloke
Model Type	llama
Model Files	9.9 GB: 1-of-3 9.9 GB: 2-of-3 6.2 GB: 3-of-3
Supported Languages	en
Quantization Type	fp16
Model Architecture	LlamaForCausalLM
Context Length	4096
Model Max Length	4096
Transformers Version	4.30.2
Tokenizer Class	LlamaTokenizer
Beginning of Sentence Token	<s>
End of Sentence Token	</s>
Unk Token	<unk>
Vocabulary Size	32000
Torch Data Type	float16

Quantized Models of the Llama 2 13B Fp16

Model	Likes	Downloads	VRAM
LLaMA2 13B Estopia	21	80	26 GB
...MA2 13B Estopia 3.0bpw H6 EXL2	1	5	5 GB
...MA2 13B Estopia 8.0bpw H8 EXL2	1	2	13 GB
Storytelling V2 13B Lora	7	3	0 GB
Lmg V3 Lora	2	10	0 GB
Storytelling V1 13B Lora	6	2	0 GB
Minotaur Llama2 13B Qlora	4	49	1 GB
Minotaur Llama2 13B Qlora	4	3	1 GB

Best Alternatives to Llama 2 13B Fp16

Best Alternatives	Context / RAM	Downloads	Likes
Llama13b 32K Illumeet Finetune	32K / 26 GB	5	0
...Maid V3 13B 32K 6.0bpw H6 EXL2	32K / 10 GB	4	1
...Maid V3 13B 32K 8.0bpw H8 EXL2	32K / 13.2 GB	3	1
WhiteRabbitNeo 13B V1	16K / 26 GB	2921	450
CodeLlama 13B Python Fp16	16K / 26 GB	1447	25
CodeLlama 13B Instruct Fp16	16K / 26 GB	903	28
CodeLlama 13B Fp16	16K / 26 GB	38	67
...Llama 13B Instruct Hf 4bit MLX	16K / 7.8 GB	346	3
Codellama 13B Bnb 4bit	16K / 7.2 GB	88	5
Airophin 13B Pntk 16K Fp16	16K / 26 GB	380	4

Note: green Score (e.g. "73.2") means that the model is better than TheBloke/Llama-2-13B-fp16.

Rank the Llama 2 13B Fp16 Capabilities

🆘 Have you tried this model? Rate its performance. This feedback would greatly assist ML community in identifying the most suitable model for their needs. Your contribution really does make a difference! 🌟

Instruction Following and Task Automation
Factuality and Completeness of Knowledge
Censorship and Alignment
Data Analysis and Insight Generation
Text Generation
Text Summarization and Feature Extraction
Code Generation
Multi-Language Support and Translation

What open-source LLMs or SLMs are you in search of? 53205 in total.

Email us: info@extractum.io. Our Privacy Policy | Terms and Conditions | Suggest an improvement.

Our Social Media →

Original data from HuggingFace, OpenCompass and various public git repos.

Check out Ag3ntum — our secure, self-hosted AI agent for server management.

Release v20260328a

Support LLM Explorer

Llama 2 13B Fp16 by TheBloke

» All LLMs » TheBloke » Llama 2 13B Fp16 URL Share it on

Llama 2 13B Fp16 Benchmarks

Llama 2 13B Fp16 Parameters and Internals

Quantized Models of the Llama 2 13B Fp16

Best Alternatives to Llama 2 13B Fp16

Rank the Llama 2 13B Fp16 Capabilities

What open-source LLMs or SLMs are you in search of? 53205 in total.