InstructLM 1.3B is an open-source language model by instruction-pretrain. Features: 1.3b LLM, VRAM: 5.5GB, Context: 2K, License: apache-2.0, Instruction-Based, LLM Explorer Score: 0.13.
InstructLM 1.3B Parameters and Internals
Model Type GPT, pre-trained, instruction-based
Use Cases
Areas: Research, Commercial Applications
Applications: General Language Modeling, Domain-Specific Instruction Modeling
Primary Use Cases: Domain adaptation in finance and biomedicine, Synthesizing instruction-response pairs
Limitations: No specific finance data due to ethical concerns
Additional Notes Demonstrates the effectiveness of supervised multitask pre-training using instruction-response pairs.
Supported Languages Training Details
Data Sources: tiiuae/falcon-refinedweb, instruction-pretrain/ft-instruction-synthesizer-collection, instruction-pretrain/general-instruction-augmented-corpora
Data Volume: Methodology: Supervised multitask pre-training using instruction-response pairs
Model Architecture: Instruction-based GPT model
Release Notes
Version: Date: Notes: Paper accepted at EMNLP 2024 main conference.
Version: Date: Notes: Updated FAQ on continual pre-training from Llama3.
Version: Date: Notes: Updated guidelines on domain-specific tasks evaluation.
Version: Date: Notes: Scaled up pre-trained tokens to 250B, with 500M instruction-response pairs.
Version: Date: Notes: Released paper, code, and resources.
Rank the InstructLM 1.3B capabilities
Have you tried this model? Rate its performance — this feedback helps the ML community find the right model for their needs.
Instruction Following and Task Automation
Factuality and Completeness of Knowledge
Censorship and Alignment
Data Analysis and Insight Generation
Text Generation
Text Summarization and Feature Extraction
Code Generation
Multi-Language Support and Translation
Expand