Transformer Pretraining From Scratch is an open-source language model by hari-8. Features: 69.3m LLM, VRAM: 0.3GB, License: mit, LLM Explorer Score: 0.33.
| LLM Name | Transformer Pretraining From Scratch |
| Repository 🤗 | https://huggingface.co/hari-8/transformer-pretraining-from-scratch |
| Model Size | 69.3m |
| Required VRAM | 0.3 GB |
| Updated | 2026-09-05 |
| Maintainer | hari-8 |
| Model Type | decoder_transformer_scratch |
| Model Files | |
| Supported Languages | en |
| Model Architecture | DecoderTransformerForCausalLM |
| License | mit |
| Transformers Version | 5.16.1 |
| Is Biased | 0 |
| Tokenizer Class | TokenizersBackend |
| Padding Token | <|endoftext|> |
| Vocabulary Size | 16384 |