What are the hardware requirements for Granite 4.0 H Small FP8?

Granite 4.0 H Small FP8 requires approximately 33.6 GB of VRAM and supports a context window of 128K tokens. Quantized variants may run on less VRAM; see the Quantized Models section on this page.

Who developed Granite 4.0 H Small FP8 and how large is it?

Granite 4.0 H Small FP8 is developed by ibm-granite, a model with 32.6b parameters. The model is published as open weights on Hugging Face and indexed on LLM Explorer with full benchmark history.

How does Granite 4.0 H Small FP8 perform on standard benchmarks?

Granite 4.0 H Small FP8 has the following published scores: LMSYS ELO 1287. Compare against reference models on this page or on the LLM Explorer leaderboards.

Granite 4.0 H Small FP8 by ibm-granite — VRAM 33.6GB, 128K context

Name: Granite 4.0 H Small FP8
Author: ibm-granite

Granite 4.0 H Small FP8 is an open-source language model by ibm-granite. Features: 32.6b LLM, VRAM: 33.6GB, Context: 128K, License: apache-2.0, Quantized, LLM Explorer Score: 0.34, ELO: 1287.

Base model:ibm-granite/granite... Base model:quantized:ibm-grani... Compressed-tensors Conversational Deploy:azure Endpoints compatible Gguf Granite-4.0 Granitemoehybrid Language Quantized Region:us Safetensors Sharded Tensorflow

Model Card on HF 🤗: https://huggingface.co/ibm-granite/granite-4.0-h-small-FP8

Granite 4.0 H Small FP8 Benchmarks

LMSys ELO 1287 vs 1272 (so35) 1.2%

^nn.n% — How the model compares to the reference models: Anthropic Sonnet 3.5 ("so35"), GPT-4o ("gpt4o") or GPT-4 ("gpt4").

What is the LLM Explorer Rank (Score)

Granite 4.0 H Small FP8 Parameters and Internals

LLM Name	Granite 4.0 H Small FP8
Repository 🤗	https://huggingface.co/ibm-granite/granite-4.0-h-small-FP8
Base Model(s)	ibm-granite/granite-4.0-h-small ibm-granite/granite-4.0-h-small
Model Size	32.6b
Required VRAM	33.6 GB
Updated	2026-08-02
Maintainer	ibm-granite
Model Type	granitemoehybrid
Model Files	4.8 GB: 1-of-7 4.8 GB: 2-of-7 4.8 GB: 3-of-7 4.8 GB: 4-of-7 4.8 GB: 5-of-7 4.8 GB: 6-of-7 4.8 GB: 7-of-7
GGUF Quantization	Yes
Quantization Type	gguf
Model Architecture	GraniteMoeHybridForCausalLM
License	apache-2.0
Context Length	131072
Model Max Length	131072
Transformers Version	4.56.1
Tokenizer Class	GPT2Tokenizer
Padding Token	<\|pad\|>
Vocabulary Size	100352

Rank the Granite 4.0 H Small FP8 capabilities

Have you tried this model? Rate its performance — this feedback helps the ML community find the right model for their needs.

Instruction Following and Task Automation
Factuality and Completeness of Knowledge
Censorship and Alignment
Data Analysis and Insight Generation
Text Generation
Text Summarization and Feature Extraction
Code Generation
Multi-Language Support and Translation

Best Alternatives to Granite 4.0 H Small FP8

Best Alternatives	Context / RAM	Downloads	Likes
...ranite 4.0 H Small FP8 Dynamic	128K / 33.6 GB	16	3