What are the hardware requirements for Llama 2 7B 32K Instruct?

Llama 2 7B 32K Instruct requires approximately 13.5 GB of VRAM and supports a context window of 32K tokens. Quantized variants may run on less VRAM; see the Quantized Models section on this page.

Who developed Llama 2 7B 32K Instruct and how large is it?

Llama 2 7B 32K Instruct is developed by togethercomputer, a model with 7b parameters. The model is published as open weights on Hugging Face and indexed on LLM Explorer with full benchmark history.

How does Llama 2 7B 32K Instruct perform on standard benchmarks?

Llama 2 7B 32K Instruct has the following published scores: MMLU 45.53. Compare against reference models on this page or on the LLM Explorer leaderboards.

Llama 2 7B 32K Instruct by togethercomputer — VRAM 13.5GB, 32K context

Name: Llama 2 7B 32K Instruct
Author: togethercomputer

Llama 2 7B 32K Instruct is an open-source language model by togethercomputer. Features: 7b LLM, VRAM: 13.5GB, Context: 32K, License: llama2, Instruction-Based, HF Score: 49.7, LLM Explorer Score: 0.16, Arc: 51.4, HellaSwag: 78.5, MMLU: 45.5, TruthfulQA: 45, WinoGrande: 72.9, GSM8K: 4.7.

Arxiv:2307.03172 Dataset:togethercomputer/llama... En Endpoints compatible Instruct Llama Pytorch Region:us Sharded

Model Card on HF 🤗: https://huggingface.co/togethercomputer/Llama-2-7B-32K-Instruct

Llama 2 7B 32K Instruct Benchmarks

IFEval: 21.3 vs 88 (so35)^-75.8%

ARC: 51.37 vs 96.7 (so35)^-46.9%

HellaSwag: 78.47 vs 95.3 (gpt4)^-17.7%

MMLU: 45.53 vs 88.3 (so35)^-48.4%

TruthfulQA: 45.01 vs 59 (gpt4)^-23.7%

WinoGrande: 72.85 vs 87.5 (gpt4)^-16.7%

GSM8K: 4.7 vs 96.4 (so35)^-95.1%

MATH Lvl 5: 1.59

LLME Score: 0.15665

^nn.n% — How the model compares to the reference models: Anthropic Sonnet 3.5 ("so35"), GPT-4o ("gpt4o") or GPT-4 ("gpt4").

What is the LLM Explorer Rank (Score)

Llama 2 7B 32K Instruct (togethercomputer/Llama-2-7B-32K-Instruct)

🌟 Advertise your project 🚀

Llama 2 7B 32K Instruct Parameters and Internals

Model Type

text generation, chatbot

Additional Notes

Join us on Together Discord: https://discord.gg/6ZVDU8tTD4

Supported Languages

en (advanced)

Training Details

Data Sources:

https://huggingface.co/datasets/togethercomputer/Long-Data-Collections, https://arxiv.org/abs/2307.03172, https://huggingface.co/datasets/togethercomputer/llama-instruct

Data Volume:

19K instruction + BookSum + MQA

Context Length:

32000

Input Output

Input Format:

[INST]\n...\n[/INST]\n

Accepted Modalities:

text

Output Format:

text

Performance Tips:

Install Flash Attention V2 for optimal performance.

LLM Name	Llama 2 7B 32K Instruct
Repository 🤗	https://huggingface.co/togethercomputer/Llama-2-7B-32K-Instruct
Model Size	7b
Required VRAM	13.5 GB
Updated	2026-02-03
Maintainer	togethercomputer
Model Type	llama
Instruction-Based	Yes
Model Files	10.0 GB: 1-of-2 3.5 GB: 2-of-2
Supported Languages	en
Model Architecture	LlamaForCausalLM
License	llama2
Context Length	32768
Model Max Length	32768
Transformers Version	4.31.0
Tokenizer Class	LlamaTokenizer
Padding Token	<unk>
Vocabulary Size	32000
Torch Data Type	float16

Quantized Models of the Llama 2 7B 32K Instruct

Model	Likes	Downloads	VRAM
Llama 2 7B 32K Instruct GGUF	54	1269	2 GB
Llama 2 7B 32K Instruct AWQ	2	16	3 GB
Llama 2 7B 32K Instruct GPTQ	27	15	3 GB
Llama 2 7B 32K Instruct GGML	8	4	2 GB

Best Alternatives to Llama 2 7B 32K Instruct

Best Alternatives	Context / RAM	Downloads	Likes
124	1024K / 16.1 GB	93	0
A5.4	1024K / 16.1 GB	12	0
A3.4	1024K / 16.1 GB	13	0
A2.4	1024K / 16.1 GB	12	0
... Qwen2.5llamaify 7B V23.1 200K	195K / 15.2 GB	24	5
SuperNeuralDreadDevil 8B	128K / 16.1 GB	22	1
Falcon3 7B Instruct	32K / 14.8 GB	18150	78
Falcon3 Jessi V0.4 7B Slerp	32K / 14.9 GB	9	9
Jessi V0.4 Falcon3 7B Instruct	32K / 14.8 GB	22	0
Jessi V0.5 Falcon3 7B Instruct	32K / 14.8 GB	10	0

Note: green Score (e.g. "73.2") means that the model is better than togethercomputer/Llama-2-7B-32K-Instruct.

Rank the Llama 2 7B 32K Instruct Capabilities

🆘 Have you tried this model? Rate its performance. This feedback would greatly assist ML community in identifying the most suitable model for their needs. Your contribution really does make a difference! 🌟

Instruction Following and Task Automation
Factuality and Completeness of Knowledge
Censorship and Alignment
Data Analysis and Insight Generation
Text Generation
Text Summarization and Feature Extraction
Code Generation
Multi-Language Support and Translation

What open-source LLMs or SLMs are you in search of? 53472 in total.

Email us: info@extractum.io. Our Privacy Policy | Terms and Conditions | Suggest an improvement.

Our Social Media →

Original data from HuggingFace, OpenCompass and various public git repos.

Check out Ag3ntum — our secure, self-hosted AI agent for server management.

Release v20260328a

Support LLM Explorer

Llama 2 7B 32K Instruct by togethercomputer