LLM EXPLORER 57,918 MODELS INDEXED

HiLS Attention 7B by tencent

By tencent · 700 downloads
HF Repo ↗ Copy URL

HiLS Attention 7B is an open-source language model by tencent. Features: 7b LLM, VRAM: 14.7GB, Context: 128K, License: apache-2.0, LLM Explorer Score: 0.31.

  Arxiv:2607.02980 Base model:allenai/olmo-3-1025... Base model:finetune:allenai/ol...   Efficient-attention   En   Endpoints compatible   Hils-attention   Long-context   Olmo3   Olmo hils   Region:us   Safetensors   Sharded   Sparse-attention   Tensorflow
Model Card on HF 🤗: https://huggingface.co/tencent/HiLS-Attention-7B 

HiLS Attention 7B Parameters and Internals

LLM NameHiLS Attention 7B
Repository 🤗https://huggingface.co/tencent/HiLS-Attention-7B 
Base Model(s)  Olmo 3 1025 7B   allenai/Olmo-3-1025-7B
Model Size7b
Required VRAM14.7 GB
Updated2026-08-10
Maintainertencent
Model Typeolmo_hils
Model Files  5.0 GB: 1-of-3   5.0 GB: 2-of-3   4.7 GB: 3-of-3
Supported Languagesen
Model ArchitectureHiLSForCausalLM
Licenseapache-2.0
Context Length131072
Model Max Length131072
Transformers Version4.40.1
Tokenizer ClassGPT2Tokenizer
Padding Token<|pad|>
Vocabulary Size100278
Torch Data Typebfloat16