What are the hardware requirements for CodeLlama 34B Instruct AWQ?

CodeLlama 34B Instruct AWQ requires approximately 18.3 GB of VRAM and supports a context window of 16K tokens. Quantized variants may run on less VRAM; see the Quantized Models section on this page.

Who developed CodeLlama 34B Instruct AWQ and how large is it?

CodeLlama 34B Instruct AWQ is developed by TheBloke, a model with 34b parameters. The model is published as open weights on Hugging Face and indexed on LLM Explorer with full benchmark history.

Where can I download or evaluate CodeLlama 34B Instruct AWQ?

CodeLlama 34B Instruct AWQ is hosted on Hugging Face and linked from this page. LLM Explorer also lists quantized variants and similar alternatives if available.

CodeLlama 34B Instruct AWQ by TheBloke — VRAM 18.3GB, 16K context

Name: CodeLlama 34B Instruct AWQ
Author: TheBloke

CodeLlama 34B Instruct AWQ is an open-source language model by TheBloke. Features: 34b LLM, VRAM: 18.3GB, Context: 16K, License: llama2, Quantized, Instruction-Based, Code Generating, LLM Explorer Score: 0.09.

Arxiv:2308.12950 4-bit Awq Code Codegen Instruct Llama Llama2 Quantized Region:us Safetensors Sharded Tensorflow

Model Card on HF 🤗: https://huggingface.co/TheBloke/CodeLlama-34B-Instruct-AWQ

CodeLlama 34B Instruct AWQ Parameters and Internals

Model Type

text generation

Use Cases

Areas:

Commercial applications, Research

Applications:

Code synthesis, Code understanding, Code assistant and generation

Limitations:

Not tested for all languages, primarily English, Potential to produce inaccurate or objectionable responses

Additional Notes

Code Llama 34B Instruct is designed as a safer deployable variant of the Code Llama family, optimized for code generation tasks.

Supported Languages

English (Primary)

Training Details

Data Sources:

Same data as Llama 2 with different weights

Hardware Used:

Meta’s Research Super Cluster, A100-80GB GPU

Model Architecture:

Optimized transformer architecture

Responsible Ai Considerations

Mitigation Strategies:

Developers should perform safety testing and tuning tailored to their specific applications.

Input Output

Input Format:

Text only

Accepted Modalities:

Text

Output Format:

Text only

Performance Tips:

Using smaller GPUs with AWQ offers ease of deployment and cost savings.

Release Notes

Version:

Initial

Notes:

The model is a variant of the Code Llama series, focused on safe instruction following for code-related tasks.

LLM Name	CodeLlama 34B Instruct AWQ
Repository 🤗	https://huggingface.co/TheBloke/CodeLlama-34B-Instruct-AWQ
Model Name	CodeLlama 34B Instruct
Model Creator	Meta
Base Model(s)	codellama/CodeLlama-34b-instruct-hf codellama/CodeLlama-34b-instruct-hf
Model Size	34b
Required VRAM	18.3 GB
Updated	2026-07-21
Maintainer	TheBloke
Model Type	llama
Instruction-Based	Yes
Model Files	9.9 GB: 1-of-2 8.4 GB: 2-of-2
Supported Languages	code
AWQ Quantization	Yes
Quantization Type	awq
Generates Code	Yes
Model Architecture	LlamaForCausalLM
License	llama2
Context Length	16384
Model Max Length	16384
Transformers Version	4.32.0.dev0
Tokenizer Class	CodeLlamaTokenizer
Beginning of Sentence Token	<s>
End of Sentence Token	</s>
Unk Token	<unk>
Vocabulary Size	32000
Torch Data Type	bfloat16

Rank the CodeLlama 34B Instruct AWQ capabilities

Have you tried this model? Rate its performance — this feedback helps the ML community find the right model for their needs.

Instruction Following and Task Automation
Factuality and Completeness of Knowledge
Censorship and Alignment
Data Analysis and Insight Generation
Text Generation
Text Summarization and Feature Extraction
Code Generation
Multi-Language Support and Translation

Best Alternatives to CodeLlama 34B Instruct AWQ

Best Alternatives	Context / RAM	Downloads	Likes
...echless Codellama 34B V2.0 AWQ	16K / 18.3 GB	6	6
CodeLlama 34B Instruct Fp16	16K / 67.5 GB	131	7
CodeLlama 34B Instruct Hf 4bit	16K / 19.4 GB	63	2
...0 Uncensored CodeLlama 34B AWQ	16K / 18.3 GB	9	3
...gpt 32K Codellama 34B Instruct	32K / 67.5 GB	56	2
CodeLlama 34B Instruct Hf	16K / 67.5 GB	22872	304
CodeLlama 34B Instruct Hf	16K / 1.4 GB	7	3
CodeLlama 34B Instruct Hf	16K / 1.4 GB	5	3
Speechless Codellama 34B V2.0	16K / 67.5 GB	96	17
CodeLlama 34B Instruct Hf	16K / 67.5 GB	622	19

Note: green Score (e.g. "73.2") means that the model is better than TheBloke/CodeLlama-34B-Instruct-AWQ.