What are the hardware requirements for Smol Llama 81M Tied?

Smol Llama 81M Tied requires approximately 0.3 GB of VRAM and supports a context window of 1K tokens. Quantized variants may run on less VRAM; see the Quantized Models section on this page.

Who developed Smol Llama 81M Tied and how large is it?

Smol Llama 81M Tied is developed by BEE-spoke-data, a model with 81m parameters. The model is published as open weights on Hugging Face and indexed on LLM Explorer with full benchmark history.

How does Smol Llama 81M Tied perform on standard benchmarks?

Smol Llama 81M Tied has the following published scores: MMLU 24.06. Compare against reference models on this page or on the LLM Explorer leaderboards.

Smol Llama 81M Tied by BEE-spoke-data — VRAM 0.3GB, 1K context

Name: Smol Llama 81M Tied
Author: BEE-spoke-data

Smol Llama 81M Tied is an open-source language model by BEE-spoke-data. Features: 81m LLM, VRAM: 0.3GB, Context: 1K, License: apache-2.0, LLM Explorer Score: 0.14, Arc: 22.2, HellaSwag: 29.3, MMLU: 24.1, GSM8K: 0.2.

Dataset:bee-spoke-data/wikiped... Dataset:jeankaddour/minipile Dataset:mattymchen/refinedweb-... Dataset:pszemraj/simple wikipe... En Endpoints compatible Llama Llama2 Region:us Safetensors Smol llama

Model Card on HF 🤗: https://huggingface.co/BEE-spoke-data/smol_llama-81M-tied

Smol Llama 81M Tied Parameters and Internals

Model Type

text generation

Use Cases

Limitations:

Not tuned to specific tasks without fine-tuning

Additional Notes

A slightly larger 101M param GQA pretrained version is available. For a chat version of this model, refer to the provided YouTube link.

Supported Languages

en (English)

Training Details

Data Sources:

JeanKaddour/minipile, pszemraj/simple_wikipedia_LM, BEE-spoke-data/wikipedia-20230901.en-deduped, mattymchen/refinedweb-3m

Methodology:

Standard multi-head attention with tying of input/output embeddings

Context Length:

1024

Model Architecture:

Decoder with 768 hidden size, 6 layers, 24 heads

Input Output

Accepted Modalities:

text

Release Notes

Version:

This is the first version of the model

Notes:

This checkpoint is the 'raw' pre-trained model and has not been tuned to a more specific task. It should be fine-tuned before use in most cases.

LLM Name	Smol Llama 81M Tied
Repository 🤗	https://huggingface.co/BEE-spoke-data/smol_llama-81M-tied
Model Size	81m
Required VRAM	0.3 GB
Updated	2026-08-13
Maintainer	BEE-spoke-data
Model Type	llama
Model Files	0.3 GB
Supported Languages	en
Model Architecture	LlamaForCausalLM
License	apache-2.0
Context Length	1024
Model Max Length	1024
Transformers Version	4.34.1
Tokenizer Class	LlamaTokenizer
Vocabulary Size	32128
Torch Data Type	float32

Rank the Smol Llama 81M Tied capabilities

Have you tried this model? Rate its performance — this feedback helps the ML community find the right model for their needs.

Instruction Following and Task Automation
Factuality and Completeness of Knowledge
Censorship and Alignment
Data Analysis and Insight Generation
Text Generation
Text Summarization and Feature Extraction
Code Generation
Multi-Language Support and Translation

Best Alternatives to Smol Llama 81M Tied

Best Alternatives	Context / RAM	Downloads	Likes
Llama Chinese 81M	2K / 0.3 GB	20	3