LLM EXPLORER 59,420 MODELS INDEXED

CodeLlama 70B Instruct Hf 2.4bpw H6 EXL2 by LoneStriker

By LoneStriker · 6 downloads
HF Repo ↗ Copy URL

CodeLlama 70B Instruct Hf 2.4bpw H6 EXL2 is an open-source language model by LoneStriker. Features: 70b LLM, VRAM: 21.3GB, Context: 2K, License: llama2, Quantized, Instruction-Based, Code Generating, LLM Explorer Score: 0.11.

  Arxiv:2308.12950   Code   Codegen   Endpoints compatible   Exl2   Instruct   Llama   Llama2   Pytorch   Quantized   Region:us   Safetensors   Sharded   Tensorflow
Model Card on HF 🤗: https://huggingface.co/LoneStriker/CodeLlama-70b-Instruct-hf-2.4bpw-h6-exl2 

CodeLlama 70B Instruct Hf 2.4bpw H6 EXL2 Parameters and Internals

Model Type 
text generation, code synthesis
Use Cases 
Areas:
commercial use, research
Applications:
code synthesis, code understanding, Python code generation
Primary Use Cases:
instruction following, safer deployment in code generation
Limitations:
English only, Requires careful tuning for safety, Not suitable for legal or regulation-violating activities
Considerations:
Use in a way that adheres to the Responsible Use Guide.
Additional Notes 
Variation in model capabilities based on size and training.
Supported Languages 
English (proficient), Python (specialized)
Training Details 
Data Sources:
Offline datasets
Data Volume:
Large
Methodology:
Fine-tuning on instruct data
Context Length:
16000
Training Time:
Extended
Hardware Used:
Meta’s Research Super Cluster
Model Architecture:
Optimized transformer architecture
Safety Evaluation 
Methodologies:
safety evaluations outlined in the research paper
Findings:
Potential to produce inaccurate or objectionable responses.
Risk Categories:
misinformation, bias
Ethical Considerations:
Developers should perform safety testing and tuning tailored to their specific applications of the model.
Responsible Ai Considerations 
Fairness:
Testing has been primarily in English and cannot cover all scenarios.
Transparency:
Outputs cannot be predicted in advance, responsible use guide provided.
Accountability:
Developers should ensure applications comply with relevant use cases.
Mitigation Strategies:
Developers should perform safety testing tailored to their specific applications of the model.
Input Output 
Input Format:
text
Accepted Modalities:
text
Output Format:
text
Performance Tips:
Follow updated prompt template for 70B Instruct model.
LLM NameCodeLlama 70B Instruct Hf 2.4bpw H6 EXL2
Repository 🤗https://huggingface.co/LoneStriker/CodeLlama-70b-Instruct-hf-2.4bpw-h6-exl2 
Model Size70b
Required VRAM21.3 GB
Updated2026-07-27
MaintainerLoneStriker
Model Typellama
Instruction-BasedYes
Model Files  8.5 GB: 1-of-3   8.6 GB: 2-of-3   4.2 GB: 3-of-3
Supported Languagescode
Quantization Typeexl2
Generates CodeYes
Model ArchitectureLlamaForCausalLM
Licensellama2
Context Length2048
Model Max Length2048
Transformers Version4.37.1
Tokenizer ClassLlamaTokenizer
Vocabulary Size32016
Torch Data Typebfloat16

Best Alternatives to CodeLlama 70B Instruct Hf 2.4bpw H6 EXL2

Best Alternatives
Context / RAM
Downloads
Likes
...Llama 70B Instruct Hf 4bit MLX4K / 39.1 GB41725
...70B Instruct Nf4 Fp16 Upscaled4K / 138.7 GB101
...70B Instruct Hf 5.0bpw H6 EXL22K / 43.6 GB56
...0B Instruct Hf 2.65bpw H6 EXL22K / 23.4 GB53
...70B Instruct Hf 4.0bpw H6 EXL22K / 35.1 GB61
CodeLlama 70B Instruct Hf4K / 72.3 GB72924
Code Llama 70B Python Instruct4K / 138.1 GB51
CodeLlama 70B Instruct Hf4K / 72.3 GB463210
CodeLlama 70B Instruct Hf GGUF4K / 25.5 GB4872
CodeLlama 70B Instruct AWQ4K / 36.6 GB29713
Note: green Score (e.g. "73.2") means that the model is better than LoneStriker/CodeLlama-70b-Instruct-hf-2.4bpw-h6-exl2.