LLM EXPLORER
 LLM Explorer: A Curated Large Language Model Directory and Analytics
Dark Theme
  Agents   Benchmarks   LLM  List    LLM Hosting    LLM Leaderboards    Blog    Newsfeed    Advertise

Meditron 70B GPTQ by TheBloke

 »  All LLMs  »  TheBloke  »  Meditron 70B GPTQ   URL Share it on

  1441   

Meditron 70B GPTQ is an open-source language model by TheBloke. Features: 70b LLM, VRAM: 35.3GB, Context: 4K, License: llama2, Quantized, LLM Explorer Score: 0.12.

  Arxiv:2311.16079   4-bit Base model:epfl-llm/meditron-7... Base model:quantized:epfl-llm/...   Dataset:bigbio/med qa   Dataset:bigbio/pubmed qa   Dataset:epfl-llm/guidelines   Dataset:medmcqa   Deploy:azure   En   Gptq   Health   Llama   Llama2   Medical   Quantized   Region:us   Safetensors
Model Card on HF 🤗: https://huggingface.co/TheBloke/meditron-70B-GPTQ 

Meditron 70B GPTQ Benchmarks

LLME Score: 0.12181
nn.n% — How the model compares to the reference models: Anthropic Sonnet 3.5 ("so35"), GPT-4o ("gpt4o") or GPT-4 ("gpt4").
Meditron 70B GPTQ (TheBloke/meditron-70B-GPTQ)
🌟 Advertise your project 🚀

Meditron 70B GPTQ Parameters and Internals

Model Type 
text-generation, causal decoder-only transformer
Use Cases 
Areas:
Medical domain, Clinical decision-making, Healthcare
Applications:
Medical exam question answering, Supporting differential diagnosis, Disease information query
Primary Use Cases:
Medical domain diagnostics and information retrieval
Limitations:
Not suitable for direct clinical decision-making without further alignment and testing
Considerations:
Must ensure usage aligns with professional guidelines.
Additional Notes 
The Meditron suite explores the capability and suitability of LLMs for the medical domain, enhancing the model to encode medical knowledge appropriately while understanding existing limitations and risks.
Supported Languages 
en (mainly)
Training Details 
Data Sources:
bigbio/med_qa, medmcqa, bigbio/pubmed_qa, epfl-llm/guidelines, RedPajama-v1
Data Volume:
48.1 billion tokens
Methodology:
Continued pretraining on a comprehensively curated medical corpus.
Context Length:
4096
Training Time:
September and October 2023
Hardware Used:
16 nodes of 8x NVIDIA A100 (80GB) SXM GPUs
Model Architecture:
Llama 2
Safety Evaluation 
Methodologies:
TruthfulQA (multiple choice) evaluation, Medical experts qualified review
Findings:
Competitive truthfulness metrics compared to medical domain models
Risk Categories:
Public health, Medical ethics, Bias (gender, age, race)
Ethical Considerations:
The model should not be used clinically without further testing.
Responsible Ai Considerations 
Mitigation Strategies:
Ongoing evaluation to better understand bias and safety implications.
Input Output 
Input Format:
Text-only
Accepted Modalities:
text
Output Format:
Text
Performance Tips:
Use a high-throughput and memory-efficient inference engine for best results.
Release Notes 
Version:
v1.0
Date:
2023-11
Notes:
Initial release with medical domain adaptation from Llama-2-70B.
LLM NameMeditron 70B GPTQ
Repository 🤗https://huggingface.co/TheBloke/meditron-70B-GPTQ 
Model NameMeditron 70B
Model CreatorEPFL LLM Team
Base Model(s)  epfl-llm/meditron-70b   epfl-llm/meditron-70b
Model Size70b
Required VRAM35.3 GB
Updated2026-03-03
MaintainerTheBloke
Model Typellama
Model Files  35.3 GB
Supported Languagesen
GPTQ QuantizationYes
Quantization Typegptq
Model ArchitectureLlamaForCausalLM
Licensellama2
Context Length4096
Model Max Length4096
Transformers Version4.35.2
Tokenizer ClassLlamaTokenizer
Beginning of Sentence Token<s>
End of Sentence Token</s>
Unk Token<unk>
Vocabulary Size32000
Torch Data Typebfloat16

Best Alternatives to Meditron 70B GPTQ

Best Alternatives
Context / RAM
Downloads
Likes
...B Instruct AutoRound GPTQ 4bit128K / 39.9 GB3246
...B Instruct AutoRound GPTQ 4bit128K / 39.9 GB250
...ama 3.1 70B Instruct Gptq 4bit128K / 39.9 GB244
Opus V1.2 70B Marlin32K / 36.4 GB50
MoMo 70B Lora 1.8.6 DPO GPTQ32K / 41.3 GB111
MoMo 70B Lora 1.8.4 DPO GPTQ32K / 41.3 GB21
Tess 70B V1.6 Marlin31K / 36.3 GB71
Midnight Miqu 70B V1.5 GPTQ32G31K / 40.7 GB464
Miqu 1 70B Sf GPTQ31K / 36.7 GB46510
...Midnight Miqu 70B V1.0 GPTQ32G31K / 40.7 GB92
Note: green Score (e.g. "73.2") means that the model is better than TheBloke/meditron-70B-GPTQ.

Rank the Meditron 70B GPTQ Capabilities

🆘 Have you tried this model? Rate its performance. This feedback would greatly assist ML community in identifying the most suitable model for their needs. Your contribution really does make a difference! 🌟

Instruction Following and Task Automation  
Factuality and Completeness of Knowledge  
Censorship and Alignment  
Data Analysis and Insight Generation  
Text Generation  
Text Summarization and Feature Extraction  
Code Generation  
Multi-Language Support and Translation  

What open-source LLMs or SLMs are you in search of? 52509 in total.

Email us: info@extractum.io. Our Privacy Policy  | Terms and Conditions  | Suggest an improvement.
Our Social Media →  
Original data from HuggingFace, OpenCompass and various public git repos.
Check out Ag3ntum — our secure, self-hosted AI agent for server management.
Release v20260328a
  Support LLM Explorer