LLM EXPLORER 56,604 MODELS INDEXED

Nous Hermes 2 Yi 34B Marlin by neuralmagic

By neuralmagic · 7 downloads
HF Repo ↗ Copy URL

Nous Hermes 2 Yi 34B Marlin is an open-source language model by neuralmagic. Features: 34b LLM, VRAM: 19.1GB, Context: 4K, Quantized, LLM Explorer Score: 0.11.

  Arxiv:2210.17323   4-bit   Autotrain compatible Base model:nousresearch/nous-h... Base model:quantized:nousresea...   Conversational   Endpoints compatible   Gptq   Int4   Llama   Marlin   Nm-vllm   Quantized   Region:us   Safetensors
Model Card on HF 🤗: https://huggingface.co/neuralmagic/Nous-Hermes-2-Yi-34B-marlin 

Nous Hermes 2 Yi 34B Marlin Parameters and Internals

Model Type 
yi
Additional Notes 
This model was quantized and optimized for efficient 4-bit inference using Marlin, a highly optimized inference kernel. It is designed to work with nm-vllm, a high-throughput serving engine for compressed LLMs.
LLM NameNous Hermes 2 Yi 34B Marlin
Repository 🤗https://huggingface.co/neuralmagic/Nous-Hermes-2-Yi-34B-marlin 
Base Model(s)  Nous Hermes 2 Yi 34B   NousResearch/Nous-Hermes-2-Yi-34B
Model Size34b
Required VRAM19.1 GB
Updated2025-04-08
Maintainerneuralmagic
Model Typellama
Model Files  19.1 GB
GPTQ QuantizationYes
Quantization Typegptq
Model ArchitectureLlamaForCausalLM
Context Length4096
Model Max Length4096
Transformers Version4.38.2
Tokenizer ClassLlamaTokenizer
Padding Token<unk>
Vocabulary Size64000
Torch Data Typefloat16

Best Alternatives to Nous Hermes 2 Yi 34B Marlin

Best Alternatives
Context / RAM
Downloads
Likes
Yi 34B 200K RPMerge GPTQ195K / 21.2 GB24
Smaug 34B V0.1 GPTQ195K / 21.2 GB11
Tess 34B V1.5B GPTQ195K / 18.6 GB188
...4B 200K DARE Megamerge V8 GPTQ195K / 18.6 GB43
...y 34B 200K Chat Evaluator GPTQ195K / 18.6 GB83
Deepmoney 34B 200K Base GPTQ195K / 18.6 GB33
...ous Capybara Limarpv3 34B GPTQ195K / 18.6 GB114
Bagel 34B V0.2 GPTQ195K / 18.6 GB72
Bagel DPO 34B V0.2 GPTQ195K / 18.6 GB52
Nontoxic Bagel 34B V0.2 GPTQ195K / 18.6 GB31