riaz.somc/transformers
1
0
Fork 0
mirror of https://github.com/huggingface/transformers.git synced 2025-07-04 21:30:07 +06:00
Code Issues Actions 28 Packages Projects Releases Wiki Activity
43f953cc2e
transformers/examples/research_projects/codeparrot/scripts
History
Leandro von Werra 43f953cc2e
Add CodeParrot 🦜 codebase (#14536) 
* add readme skeleton

* update readme

* add initialization script

* add deduplication script

* add codeparrot training script

* add code generation evaluation

* add validation loss script

* add requirements

* update readme

* tweak readme

* make style

* add highlights to readme

* add CLIs to scripts

* add tokenizer training script

* add docstring to constant length dataset

* fix defaults in arguments

* update readme with cli

* move image to hub

* tweaks of readme

* fix cli commands

* add author

* explain env variables

* fix formatting

* Update examples/research_projects/codeparrot/README.md

Co-authored-by: lewtun <lewis.c.tunstall@gmail.com>

* Apply suggestions from code review

Co-authored-by: lewtun <lewis.c.tunstall@gmail.com>

* replace generic with gpt2 tokenizer

Co-authored-by: lewtun <lewis.c.tunstall@gmail.com>
2021-12-02 10:41:35 +01:00
..
arguments.py Add CodeParrot 🦜 codebase (#14536) 2021-12-02 10:41:35 +01:00
bpe_training.py Add CodeParrot 🦜 codebase (#14536) 2021-12-02 10:41:35 +01:00
codeparrot_training.py Add CodeParrot 🦜 codebase (#14536) 2021-12-02 10:41:35 +01:00
human_eval.py Add CodeParrot 🦜 codebase (#14536) 2021-12-02 10:41:35 +01:00
initialize_model.py Add CodeParrot 🦜 codebase (#14536) 2021-12-02 10:41:35 +01:00
preprocessing.py Add CodeParrot 🦜 codebase (#14536) 2021-12-02 10:41:35 +01:00
validation_loss.py Add CodeParrot 🦜 codebase (#14536) 2021-12-02 10:41:35 +01:00