mirror of
https://github.com/saymrwulf/transformers.git
synced 2026-05-14 20:58:08 +00:00
Create README.md (#6413)
* Create README.md Model card for https://huggingface.co/akhooli/gpt2-small-arabic * Update model_cards/akhooli/gpt2-small-arabic/README.md Co-authored-by: Julien Chaumond <chaumond@gmail.com>
This commit is contained in:
parent
3ae30787b5
commit
00ce881c07
1 changed files with 44 additions and 0 deletions
44
model_cards/akhooli/gpt2-small-arabic/README.md
Normal file
44
model_cards/akhooli/gpt2-small-arabic/README.md
Normal file
|
|
@ -0,0 +1,44 @@
|
|||
---
|
||||
language: "ar"
|
||||
datasets:
|
||||
- Arabic Wikipedia
|
||||
metrics:
|
||||
- none
|
||||
---
|
||||
|
||||
# GPT2-Small-Arabic
|
||||
|
||||
## Model description
|
||||
|
||||
GPT2 model from Arabic Wikipedia dataset based on gpt2-small (using Fastai2).
|
||||
|
||||
## Intended uses & limitations
|
||||
|
||||
#### How to use
|
||||
|
||||
An example is provided in this [colab notebook](https://colab.research.google.com/drive/1mRl7c-5v-Klx27EEAEOAbrfkustL4g7a?usp=sharing).
|
||||
Both text and poetry (fine-tuned model) generation are included.
|
||||
|
||||
#### Limitations and bias
|
||||
|
||||
GPT2-small-arabic (trained on Arabic Wikipedia) has several limitations in terms of coverage (Arabic Wikipeedia quality, no diacritics) and training performance.
|
||||
Use as demonstration or proof of concepts but not as production code.
|
||||
|
||||
## Training data
|
||||
|
||||
This pretrained model used the Arabic Wikipedia dump (around 900 MB).
|
||||
|
||||
## Training procedure
|
||||
|
||||
Training was done using [Fastai2](https://github.com/fastai/fastai2/) library on Kaggle, using free GPU.
|
||||
|
||||
## Eval results
|
||||
Final perplexity reached was 72.19, loss: 4.28, accuracy: 0.307
|
||||
|
||||
### BibTeX entry and citation info
|
||||
|
||||
```bibtex
|
||||
@inproceedings{Abed Khooli,
|
||||
year={2020}
|
||||
}
|
||||
```
|
||||
Loading…
Reference in a new issue