Lysandre
c475f3ce2d
Scheduled tests should only run on a daily basis
2022-02-23 17:52:22 -05:00
Eliott C
6336017c15
Fix build_documentation CI ( #15803 )
2022-02-23 21:53:51 +01:00
Lysandre Debut
a0e3480699
[Test refactor 5/5] Build docker images ( #15729 )
2022-02-23 15:48:19 -05:00
Lysandre Debut
4c737f0e40
[Test refactor 4/5] Improve the scheduled tests ( #15728 )
2022-02-23 15:48:05 -05:00
Lysandre Debut
29c10a41d0
[Test refactor 1/5] Per-folder tests reorganization ( #15725 )
...
* Per-folder tests reorganization
Co-authored-by: sgugger <sylvain.gugger@gmail.com>
Co-authored-by: Stas Bekman <stas@stason.org>
2022-02-23 15:46:28 -05:00
Eliott C
86636f52a9
Fix indent in doc-builder CI ( #15798 )
2022-02-23 20:01:33 +01:00
Eliott C
a1efc82362
HTML dev docs ( #15678 )
...
Co-authored-by: Pierric Cistac <Pierrci@users.noreply.github.com>
2022-02-23 19:43:22 +01:00
Eliott C
bc3379e12c
🔥 Remove build_doc_test github action ( #15680 )
2022-02-16 14:06:26 +01:00
Lysandre Debut
7029240927
Upgrade click version ( #15579 )
2022-02-09 10:28:43 -05:00
Patrick von Platen
e2b6e73fa2
[Flax tests] Disable scheduled GPU tests ( #15503 )
2022-02-03 17:12:14 +01:00
Ayush Chaurasia
c74f3d4c48
Add W&B backend for hyperparameter sweep ( #14582 )
...
# Add support for W&B hyperparameter sweep
This PR:
* allows using wandb for running hyperparameter search.
* The runs are visualized on W&B sweeps dashboard
* This supports runnning sweeps on parallel devices, all reporting to the same central dashboard.
### Usage
**To run new a hyperparameter search:**
```
trainer.hyperparameter_search(
backend="wandb",
project="transformers_sweep", # name of the project
n_trials=5,
metric="eval/loss", # metric to be optimized, default 'eval/loss'. A warning is raised if the passed metric is not found
)
```
This outputs a sweep id. Eg. `my_project/sweep_id`
**To run sweeps on parallel devices:**
Just pass sweep id which you want to run parallel
```
trainer.hyperparameter_search(
backend="wandb",
sweep_id = "my_project/sweep_id"
)
```
2022-02-02 14:06:14 -05:00
Patrick von Platen
9f831bdeaf
[DocTests Speech] Add doc tests for all speech models ( #15031 )
...
* fix_torch_device_generate_test
* remove @
* doc tests
* up
* up
* fix doctests
* adapt files
* finish refactor
* up
* save intermediate
* add more logic
* new change
* improve
* next try
* next try
* next try
* next try
* fix final spaces
* fix final spaces
* improve
* renaming
* correct more bugs
* finish wavlm
* add comment
* run on test runner
* finish all speech models
* adapt
* finish
2022-01-27 14:29:31 +01:00
Sylvain Gugger
e65bfc0971
Try without bad instruction
2022-01-24 15:55:29 -05:00
Sylvain Gugger
81156d20cd
Add model like ( #14992 )
...
* Add new model like command
* Bad doc-styler
* black and doc-styler, stop fighting!
* black and doc-styler, stop fighting!
* At last
* Clean up
* Typo
* Bad doc-styler
* Bad doc-styler
* All good maybe?
* Use constants
* Add doc and type hints
* More cleaning
* Add doc
* Fix Copied from
* Doc template
* Use typing.Pattern instead
* Framework-specific files
* Fixes
* Select frameworks clean model init
* Deal with frameworks in main init
* fixes
* Last fix
* Prompt user for info
* Delete exemple config
* Last fixes
* Add test config
* Fix bug with model_type included in each other
* Fixes
* More fixes
* More fixes
* Adapt config
* Remove print statements
* Will fix tokenization later, leave it broken for now
* Add test
* Quality
* Try this way
* Debug
* Maybe by setting the path?
* Let's try another way
* It should go better when actually passing the arg...
* Remove debug statements and style
* Fix config
* Add tests
* Test require the three backends
* intermediate commit
* Revamp pattern replacements and start work on feature extractors
* Adapt model info
* Finalize code for processors
* Fix in main init additions
* Finish questionnaire for processing classes
* Fix file name
* Fix for real
* Fix patterns
* Style
* Remove needless warnings
* Copied from should work now.
* Include Copied form in blocks
* Add test
* More fixes and tests
* Apply suggestions from code review
Co-authored-by: Lysandre Debut <lysandre@huggingface.co>
* Address review comment
Co-authored-by: Lysandre Debut <lysandre@huggingface.co>
2022-01-24 15:25:10 -05:00
Patrick von Platen
b7cb126ccc
[PyTorch-nightly-test] Fix Wav2Vec2 LM & Phoneme tests ( #15272 )
...
* [PyTorch-nightly-test] Fix Wav2Vec2 LM & Phoneme tests
* Update .github/workflows/self-nightly-scheduled.yml
* change lines
* Apply suggestions from code review
2022-01-24 10:53:53 +01:00
Lysandre Debut
ad7390636d
Tentative workflow improvement ( #15255 )
2022-01-20 13:51:19 -05:00
Lysandre Debut
6a3c883c8b
Fix PR number ( #15231 )
...
* Fix PR number
* Fix PR number
2022-01-19 11:00:16 -05:00
Lysandre Debut
653379c094
Build dev documentation ( #15210 )
...
* Wrap up
* Remove secret
* Fix path
* Typo
Revert image switch
* Specific token for comments
* Cleaner comments
* Correct PR number
* Explicit master install
* Force uninstall
2022-01-19 08:47:34 -05:00
Sylvain Gugger
5f3c57fc84
Check the repo consistency in model templates test ( #15141 )
...
* Check the repo consistency in model templates test
* Fix doc template
* Fix docstrings
* Fix last docstring
2022-01-14 04:52:38 -05:00
Lysandre Debut
6950ccec1b
doc-builder -> doc-build ( #15134 )
...
* Updated script
* Commit everything
* Ready for review!
* Update .github/workflows/build_documentation.yml
Co-authored-by: Julien Chaumond <julien@huggingface.co>
Co-authored-by: Julien Chaumond <julien@huggingface.co>
2022-01-13 06:02:24 -05:00
Lysandre Debut
444ea95a80
Print out durations of all scheduled tests ( #15102 )
2022-01-11 08:15:59 -05:00
Lysandre Debut
42d57549b8
Change assignee for tokenizers ( #15088 )
2022-01-10 09:22:48 -05:00
NielsRogge
83c552d390
Add detectron2 to Github actions ( #15053 )
2022-01-06 08:53:58 -05:00
Patrick von Platen
dbac8899fe
[Tests] Correct Wav2Vec2 & WavLM tests ( #15015 )
...
* up
* up
* up
2022-01-03 20:19:04 +01:00
Sylvain Gugger
c075fb7855
Replace commit sha by commit url for update jobs ( #14852 )
...
* Replace commit sha by commit url for update jobs
* Typo
* Update .github/workflows/build_documentation.yml
Co-authored-by: Julien Chaumond <julien@huggingface.co>
* Apply review comments
Co-authored-by: Julien Chaumond <julien@huggingface.co>
2021-12-21 11:17:11 -05:00
Patrick von Platen
c4a96cecbc
Wav2Vec2 meets phonemes ( #14353 )
...
* up
* add tokenizer
* improve more
* finish tokenizer
* finish
* adapt speech recognition script
* adapt convert
* more fixes
* more fixes
* update phonemizer wav2vec2
* better naming
* fix more tests
* more fixes swedish
* correct tests
* finish
* improve script
* remove file
* up
* lets get those 100 model architectures until the end of the month
* make fix-copies
* correct more
* correct script
* more fixes
* more fixes
* add to docs
* Apply suggestions from code review
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* replace assert
* fix copies
* fix docs
* new try docs
* boom boom
* update
* add phonemizer to audio tests
* make fix-copies
* up
* upload models
* some changes
* Update tests/test_tokenization_wav2vec2_phoneme.py
Co-authored-by: Anton Lozhkov <aglozhkov@gmail.com>
* more fixes
* remove @
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
Co-authored-by: Anton Lozhkov <aglozhkov@gmail.com>
2021-12-17 19:56:44 +01:00
Sylvain Gugger
12e1b4c6df
Fix the build documentation job ( #14788 )
...
* Fix the build documentation job
* Fix install
* Address review comment
2021-12-16 09:35:20 -05:00
Sylvain Gugger
7e61d56a45
Fix the doc_build_test job ( #14774 )
...
* Fake new model
* Fix doc-building test job
* Is this the problem?
* Another try
* Typo
* Clean up
* Can we do without -e ?
* Clean setup
2021-12-15 03:40:17 -05:00
Sylvain Gugger
7533d30acd
Convert Trainer doc page to MarkDown ( #14753 )
...
* Convert Trainer doc page to MarkDown
* Fix repo consistency
* Fix the doc build test job
2021-12-13 13:09:50 -05:00
Sylvain Gugger
e4666bff06
Fix name
2021-12-13 12:01:37 -05:00
Sylvain Gugger
64e92ed224
Update transformers metadata ( #14724 )
...
* Wip on metadata update
* Most of the script
* Add a job to auto-update the transformers metadata
* Style
2021-12-13 11:46:03 -05:00
Sylvain Gugger
c3cd88a9ba
Small fixes for the doc ( #14751 )
2021-12-13 11:17:01 -05:00
Stas Bekman
8362d07d63
[CI/pt-nightly] switch to cuda-11.3 ( #14726 )
2021-12-13 09:53:48 -05:00
Patrick von Platen
86dd23bb8b
Update bug-report.md ( #14715 )
2021-12-12 13:30:44 +01:00
Sylvain Gugger
f46668282b
Fix path for notebooks
2021-12-10 15:03:17 -05:00
Sylvain Gugger
3b2d1652e4
Fix typo in branch name
2021-12-10 14:38:21 -05:00
Sylvain Gugger
1b75d7238c
Automatically build doc notebooks ( #14718 )
...
* Test workflow
* Build doc
* Make a clean build
* Add doc config
* Restore other workflows
* Final job
* Print something in else statements
* Pull before making changes
2021-12-10 14:20:56 -05:00
Sylvain Gugger
68e53e6fcd
Add a job to test doc building (for realsies this time) ( #14662 )
2021-12-09 07:01:03 -05:00
Lysandre Debut
2294071a0c
Fix doc builder ( #14676 )
2021-12-08 09:14:36 -05:00
Patrick von Platen
961732c276
[Wav2Vec2] PyCTCDecode Integration to support language model boosted decoding ( #14339 )
...
* up
* up
* up
* make it cleaner
* correct
* make styhahalal
* add more tests
* finish
* small fix
* make style
* up
* tryout to solve cicrle ci
* up
* fix more tests
* fix more tests
* apply sylvains suggestions
* fix import
* correct docs
* add pyctcdecode only to speech tests
* fix more tests
* add tf, flax and pt tests
* add pt
* fix last tests
* fix more tests
* Apply suggestions from code review
* change lines
* Apply suggestions from code review
Co-authored-by: Anton Lozhkov <aglozhkov@gmail.com>
* correct tests
* correct tests
* add doc string
Co-authored-by: Anton Lozhkov <aglozhkov@gmail.com>
2021-12-08 12:07:54 +01:00
Sylvain Gugger
381b05a3f5
Remove nonworking workflow for now
2021-12-06 17:25:28 -05:00
Sylvain Gugger
03fda7b743
Add a job to test the documentation build ( #14645 )
...
* Add a job to the documentation build
* Add caching
* Test cache
2021-12-06 13:55:59 -05:00
Lysandre Debut
aada989ad5
Fix doc builder ( #14616 )
...
* Fix doc builder
* Fix doc builder
* Fix doc builder
2021-12-03 12:09:25 -05:00
Stas Bekman
e62091d5a7
[CI] move env print to util, add pt, nccl versions ( #14607 )
...
* move env print to util, add pt, nccl versions
* style
* version
* align
2021-12-03 08:18:36 -05:00
Lysandre Debut
77d87e732e
Adds a git pull instruction to the documentation builder ( #14597 )
...
* Adds a git pull instruction
* master -> main
2021-12-02 03:32:38 -05:00
Sylvain Gugger
4df7d05a87
Doc new front ( #14590 )
...
* Convert PretrainedConfig doc to Markdown
* Use syntax
* Add necessary doc files (#14496 )
* Doc fixes (#14499 )
* Fixes for the new front
* Convert DETR file for table
* Title is needed
* Simplify a bit
* Even simpler
* Remove imports
* Fix typo in toctree (#14516 )
* Fix checkpoints badge
* Update versions.yml format (#14517 )
* Doc new front github actions (#14512 )
* Doc new front github actions
* Fix docstring
* Fix feature extraction utils import (#14515 )
* Address Julien's comments
* Push to doc-builder
* Ready for merge
* Remove old build and deploy
* Doc misc fixes (#14583 )
* Rm versions.yml from doc
* Fix converting.rst
* Rm pretrained_models from toctree
* Fix index links (#14567 )
* Fix links in README
* Localized READMEs
* Fix copy script
* Fix find doc script
* Update README_ko.md
Co-authored-by: Julien Chaumond <julien@huggingface.co>
Co-authored-by: Julien Chaumond <julien@huggingface.co>
* Adapt build command to new CLI tools (#14578 )
* Fix typo
* Fix doc interlinks (#14589 )
* Convert PretrainedConfig doc to Markdown
* Use syntax
* Rm pattern <[a-z]+(.html).*>
* Rm huggingface.co/transformers/master
* Rm .html
* Rm .html from index.mdx
* Rm .html from model_summary.rst
* Update index.mdx rm html
* Update remove .html
* Fix inner doc links
* Fix interlink in preprocssing.rst
* Update pr_checks
Co-authored-by: Sylvain Gugger <sylvain.gugger@gmail.com>
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* Convert PretrainedConfig doc to Markdown
* Use syntax
* Add necessary doc files (#14496 )
* Doc fixes (#14499 )
* Fixes for the new front
* Convert DETR file for table
* Title is needed
* Simplify a bit
* Even simpler
* Remove imports
* Fix checkpoints badge
* Fix typo in toctree (#14516 )
* Update versions.yml format (#14517 )
* Doc new front github actions (#14512 )
* Doc new front github actions
* Fix docstring
* Fix feature extraction utils import (#14515 )
* Address Julien's comments
* Push to doc-builder
* Ready for merge
* Remove old build and deploy
* Doc misc fixes (#14583 )
* Rm versions.yml from doc
* Fix converting.rst
* Rm pretrained_models from toctree
* Fix index links (#14567 )
* Fix links in README
* Localized READMEs
* Fix copy script
* Fix find doc script
* Update README_ko.md
Co-authored-by: Julien Chaumond <julien@huggingface.co>
Co-authored-by: Julien Chaumond <julien@huggingface.co>
* Adapt build command to new CLI tools (#14578 )
* Fix typo
* Fix doc interlinks (#14589 )
* Convert PretrainedConfig doc to Markdown
* Use syntax
* Rm pattern <[a-z]+(.html).*>
* Rm huggingface.co/transformers/master
* Rm .html
* Rm .html from index.mdx
* Rm .html from model_summary.rst
* Update index.mdx rm html
* Update remove .html
* Fix inner doc links
* Fix interlink in preprocssing.rst
* Update pr_checks
Co-authored-by: Sylvain Gugger <sylvain.gugger@gmail.com>
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* Styling
Co-authored-by: Mishig Davaadorj <mishig.davaadorj@coloradocollege.edu>
Co-authored-by: Lysandre Debut <lysandre@huggingface.co>
Co-authored-by: Julien Chaumond <julien@huggingface.co>
2021-12-01 14:13:02 -05:00
Stas Bekman
d1fd64e7aa
clear ~/.cache/torch_extensions between builds ( #14520 )
2021-11-25 03:15:35 -05:00
NielsRogge
3772af49ce
[Tests] Improve vision tests ( #14458 )
...
* Improve tests
* Install vision for tf tests
2021-11-24 15:22:20 +01:00
Patrick von Platen
01b1466983
[TPU tests] Enable first TPU examples pytorch ( #14121 )
...
* up
* up
* fix
* up
* Update examples/pytorch/test_xla_examples.py
* correct labels
* up
* up
* up
* up
* up
* up
2021-10-28 01:22:28 +02:00
Lysandre Debut
990de2c17c
Remove pip 21.3 from installation candidates for model templates
2021-10-11 23:21:37 -04:00
