Aya 23 8B 4bq by QuantiPhy

 ยป  All LLMs  ยป  QuantiPhy  ยป  Aya 23 8B 4bq   URL Share it on

  Arxiv:1910.09700   4-bit   Autotrain compatible   Bitsandbytes   Cohere   Endpoints compatible   Region:us   Safetensors   Sharded   Tensorflow
Model Card on HF ๐Ÿค—: https://huggingface.co/QuantiPhy/aya-23-8B-4bq 

Aya 23 8B 4bq Benchmarks

nn.n% — How the model compares to the reference models: Anthropic Sonnet 3.5 ("so35"), GPT-4o ("gpt4o") or GPT-4 ("gpt4").
๐ŸŒŸ Advertise your project ๐Ÿš€

Aya 23 8B 4bq Parameters and Internals

LLM NameAya 23 8B 4bq
Repository ๐Ÿค—https://huggingface.co/QuantiPhy/aya-23-8B-4bq 
Model Size8b
Required VRAM5.7 GB
Updated2025-06-09
MaintainerQuantiPhy
Model Typecohere
Model Files  5.0 GB: 1-of-2   0.7 GB: 2-of-2
Model ArchitectureCohereForCausalLM
Context Length8192
Model Max Length8192
Transformers Version4.42.0.dev0
Vocabulary Size256000
Torch Data Typebfloat16
Aya 23 8B 4bq (QuantiPhy/aya-23-8B-4bq)

Quantized Models of the Aya 23 8B 4bq

Model
Likes
Downloads
VRAM
Aya 23 8B 4bit2304 GB

Best Alternatives to Aya 23 8B 4bq

Best Alternatives
Context / RAM
Downloads
Likes
Aya Expanse 8B8K / 16 GB31951359
Aya Expanse 8B8K / 16 GB21970376
Aya Expanse 8B8K / 16 GB500
Aya Z8K / 16 GB3421
Aya Medikal V28K / 16 GB3711
Aya X Mod8K / 16 GB71
Aya X Yeni8K / 16 GB80
Us8K / 16 GB50
Aya Test Model V28K / 16 GB160
Aya Expanse 8B Abliterated8K / 16 GB3858
Note: green Score (e.g. "73.2") means that the model is better than QuantiPhy/aya-23-8B-4bq.

Rank the Aya 23 8B 4bq Capabilities

๐Ÿ†˜ Have you tried this model? Rate its performance. This feedback would greatly assist ML community in identifying the most suitable model for their needs. Your contribution really does make a difference! ๐ŸŒŸ

Instruction Following and Task Automation  
Factuality and Completeness of Knowledge  
Censorship and Alignment  
Data Analysis and Insight Generation  
Text Generation  
Text Summarization and Feature Extraction  
Code Generation  
Multi-Language Support and Translation  

What open-source LLMs or SLMs are you in search of? 48046 in total.

Our Social Media →  
Original data from HuggingFace, OpenCompass and various public git repos.
Release v20241124