LLM EXPLORER
 LLM Explorer: A Curated Large Language Model Directory and Analytics
Dark Theme
  LLM  List    LLM Hosting    LLM Leaderboards    Blog    Newsfeed    Advertise

Starcoderbase 3B by smallcloudai

 »  All LLMs  »  smallcloudai  »  Starcoderbase 3B   URL Share it on

  10   
  Arxiv:1911.02150   Arxiv:2205.14135   Arxiv:2207.14255   Arxiv:2305.06161   Autotrain compatible   Code   Codegen Dataset:bigcode/the-stack-dedu...   Endpoints compatible   Gpt bigcode   Model-index   Pytorch   Region:us   Sharded
Model Card on HF 🤗: https://huggingface.co/smallcloudai/starcoderbase-3b 

Starcoderbase 3B Benchmarks

LLME Score: 0.11243
nn.n% — How the model compares to the reference models: Anthropic Sonnet 3.5 ("so35"), GPT-4o ("gpt4o") or GPT-4 ("gpt4").
Starcoderbase 3B (smallcloudai/starcoderbase-3b)
🌟 Advertise your project 🚀

Starcoderbase 3B Parameters and Internals

Model Type 
text generation
Use Cases 
Areas:
Research, Commercial applications
Primary Use Cases:
Code generation
Limitations:
Generated code is not guaranteed to work as intended. It can be inefficient, contain bugs or exploits.
Considerations:
The predominant natural language in source code is English although other languages are also present. The model is capable of generating code snippets provided some context. However, the generated code is not guaranteed to work as intended. It can be inefficient, contain bugs or exploits.
Additional Notes 
The model was trained with opt-out requests excluded from the dataset.
Supported Languages 
80+ Programming languages (capable), English (primary)
Training Details 
Data Sources:
GitHub code, The Stack (v1.2)
Data Volume:
1 trillion tokens
Methodology:
using Multi Query Attention, a context window of 8192 tokens, and Fill-in-the-Middle objective.
Context Length:
8192
Training Time:
12 days
Hardware Used:
256 Tesla A100 GPUs
Model Architecture:
GPT-2 model with multi-query attention and Fill-in-the-Middle objective
Input Output 
Input Format:
Tokenized input in the format required by the model using Multi Query Attention and Fill-in-the-Middle objective.
Accepted Modalities:
Text
Output Format:
Generated code or text snippet corresponding to the provided input context.
Performance Tips:
Use pre-trained checkpoints from Hugging Face with a Tech Assistant prompt for optimal performance.
LLM NameStarcoderbase 3B
Repository 🤗https://huggingface.co/smallcloudai/starcoderbase-3b 
Model Size3b
Required VRAM12.2 GB
Updated2025-05-30
Maintainersmallcloudai
Model Typegpt_bigcode
Model Files  10.0 GB: 1-of-2   2.2 GB: 2-of-2
Generates CodeYes
Model ArchitectureGPTBigCodeForCausalLM
Licensebigcode-openrail-m
Transformers Version4.28.1
Tokenizer ClassGPT2Tokenizer
Vocabulary Size49152
Torch Data Typefloat32
Activation Functiongelu_pytorch_tanh

Quantized Models of the Starcoderbase 3B

Model
Likes
Downloads
VRAM
Starcoderbase 3B GPTQ1112 GB

Best Alternatives to Starcoderbase 3B

Best Alternatives
Context / RAM
Downloads
Likes
Codes 3B0K / 12.2 GB10801
Codes 3B Spider0K / 12.2 GB50
Starcoderbase 3B0K / 12.2 GB90
Codes 3B Bird0K / 12.2 GB280
Starcoderbase 3B GPTQ0K / 2.1 GB111
Note: green Score (e.g. "73.2") means that the model is better than smallcloudai/starcoderbase-3b.

Rank the Starcoderbase 3B Capabilities

🆘 Have you tried this model? Rate its performance. This feedback would greatly assist ML community in identifying the most suitable model for their needs. Your contribution really does make a difference! 🌟

Instruction Following and Task Automation  
Factuality and Completeness of Knowledge  
Censorship and Alignment  
Data Analysis and Insight Generation  
Text Generation  
Text Summarization and Feature Extraction  
Code Generation  
Multi-Language Support and Translation  

What open-source LLMs or SLMs are you in search of? 51415 in total.

Email us: info@extractum.io. Our Privacy Policy  | Terms and Conditions  | Suggest an improvement.
Our Social Media →  
Original data from HuggingFace, OpenCompass and various public git repos.
Release v20241124
  Support LLM Explorer