What are the hardware requirements for OpenBezoar HH RLHF DPO?

OpenBezoar HH RLHF DPO requires approximately 6.8 GB of VRAM and supports a context window of 2K tokens. Quantized variants may run on less VRAM; see the Quantized Models section on this page.

Who developed OpenBezoar HH RLHF DPO and how large is it?

OpenBezoar HH RLHF DPO is developed by SurgeGlobal, a model with 3b parameters. The model is published as open weights on Hugging Face and indexed on LLM Explorer with full benchmark history.

How does OpenBezoar HH RLHF DPO perform on standard benchmarks?

OpenBezoar HH RLHF DPO has the following published scores: MMLU 26.91. Compare against reference models on this page or on the LLM Explorer leaderboards.

OpenBezoar HH RLHF DPO by SurgeGlobal — VRAM 6.8GB, 2K context

Name: OpenBezoar HH RLHF DPO
Author: SurgeGlobal

OpenBezoar HH RLHF DPO is an open-source language model by SurgeGlobal. Features: 3b LLM, VRAM: 6.8GB, Context: 2K, License: cc-by-nc-4.0, LLM Explorer Score: 0.12, Arc: 43.7, HellaSwag: 74, MMLU: 26.9, GSM8K: 3.4.

Arxiv:2305.18290 Arxiv:2306.02707 Arxiv:2404.12195 Base model:finetune:surgegloba... Base model:surgeglobal/openbez... Dataset:anthropic/hh-rlhf En Endpoints compatible Llama Pytorch Region:us Safetensors

Model Card on HF 🤗: https://huggingface.co/SurgeGlobal/OpenBezoar-HH-RLHF-DPO

OpenBezoar HH RLHF DPO Parameters and Internals

Model Type

text-generation

Use Cases

Limitations:

The model might not consistently show improved abilities to follow instructions, and it could respond inappropriately or get stuck in loops., Although this model is aligned to human preferences and has been evaluated for performance, it is not guaranteed that it will refrain from generating harmful content exclusively., Caution is urged against relying on this model for production or adjacent use-cases.

Training Details

Data Sources:

Anthropic's HH-RLHF Dataset

Data Volume:

First 100K examples

Methodology:

Direct Preference Optimization (DPO)

Model Architecture:

OpenLLaMA 3B v2 architecture

Input Output

Input Format:

Modified version of the Alpaca prompt template

Performance Tips:

Utilize the Alpaca prompt template to obtain best responses for instruction-related tasks.

LLM Name	OpenBezoar HH RLHF DPO
Repository 🤗	https://huggingface.co/SurgeGlobal/OpenBezoar-HH-RLHF-DPO
Base Model(s)	OpenBezoar HH RLHF SFT SurgeGlobal/OpenBezoar-HH-RLHF-SFT
Model Size	3b
Required VRAM	6.8 GB
Updated	2026-07-29
Maintainer	SurgeGlobal
Model Type	llama
Model Files	6.8 GB 6.8 GB
Supported Languages	en
Model Architecture	LlamaForCausalLM
License	cc-by-nc-4.0
Context Length	2048
Model Max Length	2048
Transformers Version	4.33.2
Tokenizer Class	LlamaTokenizer
Vocabulary Size	32000
Torch Data Type	float16

Rank the OpenBezoar HH RLHF DPO capabilities

Have you tried this model? Rate its performance — this feedback helps the ML community find the right model for their needs.

Instruction Following and Task Automation
Factuality and Completeness of Knowledge
Censorship and Alignment
Data Analysis and Insight Generation
Text Generation
Text Summarization and Feature Extraction
Code Generation
Multi-Language Support and Translation

Best Alternatives to OpenBezoar HH RLHF DPO

Best Alternatives	Context / RAM	Downloads	Likes
Nanbeige4.1 3B	256K / 7.9 GB	20875	1161
Nanbeige4.1 3B Heretic	256K / 7.9 GB	149	50
Nanbeige4.1 3B Heretic Mxfp4	256K / 7.9 GB	34	2
Nanbeige4.1 3B Bf16	256K / 7.9 GB	47	3
...anbeige4.1 3B Heretic Bfloat16	256K / 7.9 GB	33	1
Nanbeige4.1 3B Heretic	256K / 7.9 GB	18	0
Nanbeige4.1 3B Heretic	256K / 7.9 GB	14	1
ISA 03 Mini 3B Hybrid Preview	256K / 6.5 GB	4	4
ToolMind Web 3B	192K / 7.9 GB	156	28
Llama 3.2 3B Instruct	128K / 6.5 GB	1829572	2368

Note: green Score (e.g. "73.2") means that the model is better than SurgeGlobal/OpenBezoar-HH-RLHF-DPO.