| Model Type | | text generation, causal-lm |
|
| Use Cases |
| Areas: | | research, text generation |
|
| Primary Use Cases: | | text generation from a prompt |
|
| Considerations: | | Not recommended for factually correct information generation |
|
|
| Additional Notes | | This model was a moonlighting project started in 2021 and now released. |
|
| Supported Languages | | Slovak (High), Czech (Medium) |
|
| Training Details |
| Data Sources: | | Slovak text corpus, web, news articles, biblical texts |
|
| Data Volume: | |
| Methodology: | |
| Context Length: | |
| Training Time: | | 36.5 billion tokens over 69,001 steps |
|
| Hardware Used: | |
| Model Architecture: | | Transformer with 24 layers, 1024 d_model, 16384 d_ff, 16 heads, 256 d_head |
|
|
| Input Output |
| Input Format: | | Slovak prompt with US English double quotes and no trailing whitespace |
|
| Accepted Modalities: | |
| Output Format: | |
| Performance Tips: | | Avoid trailing whitespaces, use US English primary double quotes, handle new lines with '\n\n' |
|
|
