Tinyllama Coder Py 4bit V10 Parameters and Internals
Model Type
text-generation, Python Coder
Additional Notes
The model was refined using Unsloath. The dataset ise-uiuc/Magicoder-OSS-Instruct-75K was adjusted, leaving only data on Python and divided into 10 parts, each refinement occurred for 2 epochs, using Adafactor optimizer or AdamW_8bit (Adafactor seems to deliver less loss).