LLM EXPLORER
 LLM Explorer: A Curated Large Language Model Directory and Analytics
Dark Theme
  LLM  List    LLM Hosting    LLM Leaderboards    Blog    Newsfeed    Advertise

Smol Llama 81M Tied by BEE-spoke-data

 »  All LLMs  »  BEE-spoke-data  »  Smol Llama 81M Tied   URL Share it on

  1812   
  Autotrain compatible Dataset:bee-spoke-data/wikiped...   Dataset:jeankaddour/minipile Dataset:mattymchen/refinedweb-... Dataset:pszemraj/simple wikipe...   En   Endpoints compatible   Llama   Llama2   Region:us   Safetensors   Smol llama
Model Card on HF 🤗: https://huggingface.co/BEE-spoke-data/smol_llama-81M-tied 

Smol Llama 81M Tied Benchmarks

ARC: 22.18 vs 96.7 (so35)-77.1%
HellaSwag: 29.33 vs 95.3 (gpt4)-69.2%
MMLU: 24.06 vs 88.3 (so35)-72.8%
TruthfulQA: 43.97 vs 59 (gpt4)-25.5%
WinoGrande: 49.25 vs 87.5 (gpt4)-43.7%
GSM8K: 0.23 vs 96.4 (so35)-99.8%
LLME Score: 0.18331
nn.n% — How the model compares to the reference models: Anthropic Sonnet 3.5 ("so35"), GPT-4o ("gpt4o") or GPT-4 ("gpt4").
Smol Llama 81M Tied (BEE-spoke-data/smol_llama-81M-tied)
🌟 Advertise your project 🚀

Smol Llama 81M Tied Parameters and Internals

Model Type 
text generation
Use Cases 
Limitations:
Not tuned to specific tasks without fine-tuning
Additional Notes 
A slightly larger 101M param GQA pretrained version is available. For a chat version of this model, refer to the provided YouTube link.
Supported Languages 
en (English)
Training Details 
Data Sources:
JeanKaddour/minipile, pszemraj/simple_wikipedia_LM, BEE-spoke-data/wikipedia-20230901.en-deduped, mattymchen/refinedweb-3m
Methodology:
Standard multi-head attention with tying of input/output embeddings
Context Length:
1024
Model Architecture:
Decoder with 768 hidden size, 6 layers, 24 heads
Input Output 
Accepted Modalities:
text
Release Notes 
Version:
This is the first version of the model
Notes:
This checkpoint is the 'raw' pre-trained model and has not been tuned to a more specific task. It should be fine-tuned before use in most cases.
LLM NameSmol Llama 81M Tied
Repository 🤗https://huggingface.co/BEE-spoke-data/smol_llama-81M-tied 
Model Size81m
Required VRAM0.3 GB
Updated2025-09-16
MaintainerBEE-spoke-data
Model Typellama
Model Files  0.3 GB
Supported Languagesen
Model ArchitectureLlamaForCausalLM
Licenseapache-2.0
Context Length1024
Model Max Length1024
Transformers Version4.34.1
Tokenizer ClassLlamaTokenizer
Vocabulary Size32128
Torch Data Typefloat32

Best Alternatives to Smol Llama 81M Tied

Best Alternatives
Context / RAM
Downloads
Likes
Llama Chinese 81M2K / 0.3 GB43

Rank the Smol Llama 81M Tied Capabilities

🆘 Have you tried this model? Rate its performance. This feedback would greatly assist ML community in identifying the most suitable model for their needs. Your contribution really does make a difference! 🌟

Instruction Following and Task Automation  
Factuality and Completeness of Knowledge  
Censorship and Alignment  
Data Analysis and Insight Generation  
Text Generation  
Text Summarization and Feature Extraction  
Code Generation  
Multi-Language Support and Translation  

What open-source LLMs or SLMs are you in search of? 51408 in total.

Email us: info@extractum.io. Our Privacy Policy  | Terms and Conditions  | Suggest an improvement.
Our Social Media →  
Original data from HuggingFace, OpenCompass and various public git repos.
Release v20241124
  Support LLM Explorer