LLM EXPLORER 64,561 MODELS INDEXED

GPT J GGML by rustformers

By rustformers · 10 downloads
HF Repo ↗ Copy URL

GPT J GGML is an open-source language model by rustformers. Features: 6b LLM, VRAM: 3.4GB, Quantized, LLM Explorer Score: 0.07.

  Arxiv:2104.09864   Dataset:the pile   En   Endpoints compatible   Ggml   Llm-rs   Q4   Q4 0   Quantized   Region:us
Model Card on HF 🤗: https://huggingface.co/rustformers/gpt-j-ggml 

GPT J GGML Parameters and Internals

Model Type 
text generation
Additional Notes 
The model uses the same BPE tokenizer as GPT-2/GPT-3 with a vocabulary size of 50257 out of 50400 possible embeddings. It uses Rotary Position Embeddings for 64 dimensions per head.
Training Details 
Data Sources:
the_pile
Methodology:
Trained using Mesh Transformer JAX
Context Length:
2048
Model Architecture:
Transformer with 28 layers, RoPE positional encoding
Input Output 
Input Format:
Text
Output Format:
Text
LLM NameGPT J GGML
Repository 🤗https://huggingface.co/rustformers/gpt-j-ggml 
Model Size6b
Required VRAM3.4 GB
Updated2026-06-17
Maintainerrustformers
Model Files  12.1 GB   3.4 GB   3.4 GB   4.5 GB   4.5 GB
Supported Languagesen
GGML QuantizationYes
Quantization Typeggml|q4|q4_0|q5_1
Model ArchitectureAutoModel

Best Alternatives to GPT J GGML

Best Alternatives
Context / RAM
Downloads
Likes
Yi 1.5 6B Chat GGUF0K / 2.3 GB954289
Yi 1.5 6B Tofutune Gguf0K / 3.7 GB220
Yi 6B 200K GGUF0K / 2.3 GB1630
Yi 6B GGUF0K / 2.6 GB88914
Dragon Yi 6B V0 GGUF0K / 2.6 GB57915
Yi 6B 200K GGUF0K / 2.6 GB50728
Ct2fast Codegen 6B Multi0K / 14.1 GB61
Ct2fast Codegen 6B Mono0K / 14.1 GB51
Ct2fast GPT JT 6B V00K / 12.1 GB51
Ct2fast GPT J 6B0K / 12.1 GB42
Note: green Score (e.g. "73.2") means that the model is better than rustformers/gpt-j-ggml.