Pretrained using causal language modeling with NeMo Megatron GPT implementation. The instruct models were finetuned on instruction data using chat and raw text formats.
Model Architecture:
Large decoder-only pretrained transformer language models.
Safety Evaluation
Risk Categories:
bias, safety
Ethical Considerations:
Potential for generating biased, incorrect, or harmful content; overrepresentation of some viewpoints.
Responsible Ai Considerations
Fairness:
Potential bias due to diverse or non-diverse training data.
Transparency:
Increased communication and transparency sought through a modified RAIL license.
Mitigation Strategies:
Encouragement for open communication and feedback collection from indirect users.