Ella Charlaix
39acfe84ba
Add deepspeed test to amd scheduled CI ( #27633 )
...
* add deepspeed scheduled test for amd
* fix image
* add dockerfile
* add comment
* enable tests
* trigger
* remove trigger for this branch
* trigger
* change runner env to trigger the docker build image test
* use new docker image
* remove test suffix from docker image tag
* replace test docker image with original image
* push new image
* Trigger
* add back amd tests
* fix typo
* add amd tests back
* fix
* comment until docker image build scheduled test fix
* remove deprecated deepspeed build option
* upgrade torch
* update docker & make tests pass
* Update docker/transformers-pytorch-deepspeed-amd-gpu/Dockerfile
* fix
* tmp disable test
* precompile deepspeed to avoid timeout during tests
* fix comment
* trigger deepspeed tests with new image
* comment tests
* trigger
* add sklearn dependency to fix slow tests
* enable back other tests
* final update
---------
Co-authored-by: Felix Marty <felix@hf.co>
Co-authored-by: Félix Marty <9808326+fxmarty@users.noreply.github.com>
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-12-11 16:33:36 +01:00
Yih-Dar
0f59d2f173
Fix AMD scheduled CI not triggered ( #27951 )
...
fix
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-12-11 16:22:10 +01:00
Yih-Dar
9f1f11a2e7
Show new failing tests in a more clear way in slack report ( #27881 )
...
* fix
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-12-07 15:09:30 +01:00
Maria Khalusova
d6392482bd
removed the delete doc workflows ( #27852 )
2023-12-06 01:30:56 -08:00
Yih-Dar
e0d2e69582
restructure AMD scheduled CI ( #27743 )
...
* fix
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-12-04 15:32:05 +01:00
Yih-Dar
62ab32b299
Remove check_runner_status.yml
( #27767 )
...
fix
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-11-30 10:17:25 +01:00
Yih-Dar
7a757bb694
Fix AMD Push CI not triggered ( #27732 )
...
* fix
* fix
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-11-28 09:30:21 +01:00
Yih-Dar
e11e26df93
Fix Past CI ( #27696 )
...
fix
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-11-27 09:11:58 +01:00
fxmarty
f93c1e9ece
Add RoCm scheduled CI & upgrade RoCm CI to PyTorch 2.1 ( #26940 )
...
* add scheduled ci on amdgpu
* fix likely typo
* more tests, avoid parallelism
* precise comment
* fix report channel
* trigger docker build on this branch
* fix
* fix
* run rocm scheduled ci
* fix
* fix
* fix
* fix
* fix
* fix
* fix
* fix
* fix
* fix
* fix
* fix
* fix
* fix
* fix
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-11-21 14:55:13 +01:00
Yih-Dar
d903abfccc
Fix AMD CI not showing GPU ( #27555 )
...
fix
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-11-17 10:44:37 +01:00
Yih-Dar
acb5b4aff5
Disable docker image build job latest-pytorch-amd
for now ( #27541 )
...
fix
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-11-16 17:00:46 +01:00
Yih-Dar
64e21ca2a4
Make some jobs run on the GitHub Actions runners ( #27512 )
...
fix
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-11-15 10:43:16 +01:00
Yih-Dar
00dc856233
At most 2 GPUs for CI ( #27435 )
...
At most 2 GPUs
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-11-10 16:19:06 +01:00
Mert Yanık
eb30a49b20
Translate index.md to Turkish ( #27093 )
...
* Add index.md for tukish language
* Fix index.md (huggingface/transformers#27088 )
* Add 'tr' to additional files
* Update docs/source/tr/_toctree.yml
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
* Update index.md
---------
Co-authored-by: Mert Yanık <mert.yanik@lcwaikiki.com>
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
2023-11-08 08:35:20 -05:00
Yih-Dar
eef7ea98c3
Update doctest workflow file ( #27306 )
...
fix
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-11-06 11:27:48 +01:00
Yih-Dar
9dc4ce9ea7
Disable CI runner check ( #27170 )
...
Disable runner check
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-10-31 11:59:21 +01:00
Aarya Balwadkar
a2f55a65cd
Hindi translation of pipeline_tutorial.md ( #26837 )
...
* hindi translation of pipeline_tutorial.md
* Update pipeline_tutorial.md
* Update build_documentation.yml
* Update build_pr_documentation.yml
* Updated build_documentation.yml
---------
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
2023-10-25 11:21:49 -07:00
Akhil
093848d3cc
Added Telugu [te] translations ( #26828 )
...
* Create index.md
* Create _toctree.yml
* Updated index.md in telugu
* Update _toctree.yml
* Create quicktour.md
* Update quicktour.md
* Create index.md
* Update quicktour.md
* Update docs/source/te/quicktour.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
* Delete docs/source/hi/index.md
* Update docs/source/te/quicktour.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
* Update docs/source/te/quicktour.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
* Update docs/source/te/quicktour.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
* Update docs/source/te/quicktour.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
* Update docs/source/te/quicktour.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
* Update docs/source/te/quicktour.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
* Update docs/source/te/quicktour.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
* Update docs/source/te/quicktour.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
* Update build_documentation.yml
Added telugu [te]
* Update build_pr_documentation.yml
Added Telugu [te]
* Update _toctree.yml
---------
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
2023-10-20 15:27:55 -07:00
Yih-Dar
12cc123359
Better way to run AMD CI with different flavors ( #26634 )
...
* Enable testing against mi250
* Change BERT to trigger tests
* Revert BERT's change
* AMD CI
* AMD CI
---------
Co-authored-by: Morgan Funtowicz <funtowiczmo@gmail.com>
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-10-16 16:24:30 +02:00
Rockerz
9b7668c03a
add japanese documentation ( #26138 )
...
* udpaet
* update
* Update docs/source/ja/autoclass_tutorial.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
* add codes workflows/build_pr_documentation.yml
* Create preprocessing.md
* added traning.md
* Create Model_sharing.md
* add quicktour.md
* new
* ll
* Create benchmark.md
* Create Tensorflow_model
* add
* add community.md
* add create_a_model
* create custom_model.md
* create_custom_tools.md
* create fast_tokenizers.md
* create
* add
* Update docs/source/ja/_toctree.yml
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
* md
* add
* commit
* add
* h
* Update docs/source/ja/peft.md
Co-authored-by: Younes Belkada <49240599+younesbelkada@users.noreply.github.com>
* Update docs/source/ja/_toctree.yml
Co-authored-by: Younes Belkada <49240599+younesbelkada@users.noreply.github.com>
* Update docs/source/ja/_toctree.yml
Co-authored-by: Younes Belkada <49240599+younesbelkada@users.noreply.github.com>
* Suggested Update
* add perf_train_gpu_one.md
* added perf based MD files
* Modify toctree.yml and Add transmartion to md codes
* Add `serialization.md` and edit `_toctree.yml`
* add task summary and tasks explained
* Add and Modify files starting from T
* Add testing.md
* Create main_classes files
* delete main_classes folder
* Add toctree.yml
* Update llm_tutorail.md
* Update docs/source/ja/_toctree.yml
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
* Update misspelled filenames
* Update docs/source/ja/_toctree.yml
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
* Update docs/source/ja/_toctree.yml
* Update docs/source/ja/_toctree.yml
* missplled file names inmrpovements
* Update _toctree.yml
* close tip block
* close another tip block
* Update docs/source/ja/quicktour.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
* Update docs/source/ja/pipeline_tutorial.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
* Update docs/source/ja/pipeline_tutorial.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
* Update docs/source/ja/preprocessing.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
* Update docs/source/ja/peft.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
* Update docs/source/ja/add_new_model.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
* Update docs/source/ja/testing.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
* Update docs/source/ja/task_summary.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
* Update docs/source/ja/tasks_explained.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
* Update glossary.md
* Update docs/source/ja/transformers_agents.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
* Update docs/source/ja/llm_tutorial.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
* Update docs/source/ja/create_a_model.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
* Update docs/source/ja/torchscript.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
* Update docs/source/ja/benchmarks.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
* Update docs/source/ja/troubleshooting.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
* Update docs/source/ja/troubleshooting.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
* Update docs/source/ja/troubleshooting.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
* Update docs/source/ja/add_new_model.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
* Update perf_torch_compile.md
* Update Year to default in en documentation
* Final Update
---------
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
Co-authored-by: Younes Belkada <49240599+younesbelkada@users.noreply.github.com>
2023-10-11 10:26:37 -07:00
Funtowicz Morgan
3632fb3c25
[AMD] Add initial version for run_tests_multi_gpu ( #26346 )
...
* Add initial version for run_tests_multi_gpu
* Trigger change in BERT
* fix typo setup -> setup_gpu
* Add tag mi210
* Enable multi-gpu jobs
* One more
* Use dynamic device allocation
* Attempt to fix syntax for docker create
* fix script path
* fix
* temp machine type
* fix label
* Enable multi-gpu tests
* Rename multi-amd-gpu to multi-gpu
* Let's not be lazy dude
* Update rocm-smi output
* Add gpu_flavour in the matrix
* Fix typos
* merge single/multi dispatch into the matrix
* Format.
* Revert BERT's change
---------
Co-authored-by: Guillaume LEGENDRE <glegendre01@gmail.com>
2023-10-03 11:13:45 +02:00
Yih-Dar
6ae71ec836
Update runs-on
in workflow files ( #26435 )
...
* update
* fix
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-09-27 19:25:52 +02:00
Funtowicz Morgan
2d71307dc0
Integrate AMD GPU in CI/CD environment ( #26007 )
...
* Add a Dockerfile for PyTorch + ROCm based on official AMD released artifact
* Add a new artifact single-amdgpu testing on main
* Attempt to test the workflow without merging.
* Changed BERT to check if things are triggered
* Meet the dependencies graph on workflow
* Revert BERT changes
* Add check_runners_amdgpu to correctly mount and check availability
* Rename setup to setup_gpu for CUDA and add setup_amdgpu for AMD
* Fix all the needs.setup -> needs.setup_[gpu|amdgpu] dependencies
* Fix setup dependency graph to use check_runner_amdgpu
* Let's do the runner status check only on AMDGPU target
* Update the Dockerfile.amd to put ourselves in / rather than /var/lib
* Restore the whole setup for CUDA too.
* Let's redisable them
* Change BERT to trigger tests
* Restore BERT
* Add torchaudio with rocm 5.6 to AMD Dockerfile (#26050 )
fix dockerfile
Co-authored-by: Felix Marty <felix@hf.co>
* Place AMD GPU tests in a separate workflow (correct branch) (#26105 )
AMDGPU CI lives in an other workflow
* Fix invalid job name is dependencies.
* Remove tests multi-amdgpu for now.
* Use single-amdgpu
* Use --net=host for now.
* Remote host networking.
* Removed duplicated check_runners_amdgpu step
* Let's tag machine-types with mi210 for now.
* Machine type should be only mi210
* Remove unnecessary push.branches item
* Apply review suggestions moving from `x-amdgpu` to `x-gpu` introducing `amd-gpu` and `miXXX` labels.
* Remove amdgpu from step names.
* finalize
* delete
---------
Co-authored-by: fxmarty <9808326+fxmarty@users.noreply.github.com>
Co-authored-by: Felix Marty <felix@hf.co>
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-09-20 14:48:49 +02:00
Yih-Dar
9219d1427b
pin pandas==2.0.3 ( #25875 )
...
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-08-30 18:10:01 +02:00
Yih-Dar
62399d6f35
Add flax installation in daily doctest workflow ( #25860 )
...
fix
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-08-30 15:13:50 +02:00
Lysandre Debut
8657ec68fc
Sets the stalebot to 10 AM CEST ( #25678 )
...
This sets the stale bot trigger time at 10 AM CEST rather than 5 PM CEST as all core maintainers on watch duty are now in the European timezone
2023-08-23 14:21:07 +02:00
Yih-Dar
b413e0610b
Remove utils/documentation_tests.txt
( #25680 )
...
* fix
* fix
* fix
* fix
* fix
* fix
* Apply suggestions from code review
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>
2023-08-23 11:14:45 +02:00
Sylvain Gugger
2defb6b048
More utils doc ( #25457 )
...
* Document and clean more utils.
* More documentation and fixes
* Switch to Lysandre's token
* Address review comments
* Actually put else
2023-08-17 07:58:35 +02:00
Sylvain Gugger
d561408cc3
Skip Add model like job ( #24865 )
2023-07-17 15:52:04 -04:00
Yih-Dar
11cb6e0f7e
Unpin DeepSpeed and require DS >= 0.9.3 ( #24541 )
...
* fix
* fix
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-06-28 14:01:22 +02:00
Yih-Dar
e84bf1f734
⚠️ Time to say goodbye to py37 ( #24091 )
...
* fix
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-06-28 07:22:39 +02:00
Sylvain Gugger
eb849f6604
Migrate doc files to Markdown. ( #24376 )
...
* Rename index.mdx to index.md
* With saved modifs
* Address review comment
* Treat all files
* .mdx -> .md
* Remove special char
* Update utils/tests_fetcher.py
Co-authored-by: Lysandre Debut <lysandre.debut@reseau.eseo.fr>
---------
Co-authored-by: Lysandre Debut <lysandre.debut@reseau.eseo.fr>
2023-06-20 18:07:47 -04:00
Yih-Dar
0b259a3b7e
Clean up disk sapce during docker image build for transformers-pytorch-gpu
( #24346 )
...
fix
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-06-19 12:54:02 +02:00
Yih-Dar
896a58de15
Byebye pytorch 1.9 ( #24080 )
...
byebye
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-06-16 16:38:23 +02:00
Mishig
ef010071ee
Update delete_doc_comment_trigger.yml ( #24084 )
...
fix base workflow name
2023-06-07 17:55:48 +02:00
Mishig
5c9394b54c
[doc build] Use secrets ( #24079 )
2023-06-07 17:33:39 +02:00
Yih-Dar
7631db0fdc
Pin deepspeed
to 0.9.2
for now ( #24024 )
...
fix
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-06-05 20:00:28 +02:00
Lysandre Debut
dc67da0182
Revert "Update stale.yml to use HuggingFaceBot" ( #23943 )
...
Revert "Update stale.yml to use HuggingFaceBot (#23941 )"
This reverts commit 5929f86ebb
.
2023-06-01 11:58:11 -04:00
Lysandre Debut
5929f86ebb
Update stale.yml to use HuggingFaceBot ( #23941 )
2023-06-01 10:54:50 -04:00
Yih-Dar
e69feab8a1
Update workflow files ( #23658 )
...
* fix
* fix
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-05-22 21:26:51 +02:00
Yih-Dar
9728f1134b
Fix confusing transformers
installation in CI ( #23465 )
...
* fix
* fix
* fix
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-05-19 22:10:18 +02:00
Yih-Dar
1f2c00d671
Fix DeepSpeed stuff in the nightly CI ( #23478 )
...
fix
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-05-19 20:31:55 +02:00
Yih-Dar
22a0769933
Update 3 docker files to use cu118 ( #23406 )
...
* fix
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-05-17 14:26:50 +02:00
Yih-Dar
6a6225beab
Fix doctest files fetch issue ( #23277 )
...
* fix
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-05-11 17:14:06 +02:00
Yih-Dar
9088fcae82
Bring back the PR Refactor doctests + add CI
to main
( #23271 )
...
* Revert "Revert "[Doctests] Refactor doctests + add CI" (#23245 )"
This reverts commit 69ee46243c
.
* try not expose HfDocTestParser
* move into testing_utils.py
* remove pytest install
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-05-10 22:00:48 +02:00
Sylvain Gugger
69ee46243c
Revert "[Doctests] Refactor doctests + add CI" ( #23245 )
...
Revert "[Doctests] Refactor doctests + add CI (#22987 )"
This reverts commit 627f44799a
.
2023-05-09 15:26:15 -04:00
Arthur
627f44799a
[Doctests] Refactor doctests + add CI ( #22987 )
...
* intiial commit
* new styling
* update
* just run doctest in CI
* remove more test for fast dev
* update
* update refs
* update path and fetch upstream
* update documentatyion trests
* typo
* parse pwd
* don't check for files that are in hidden folders
* just give paths relative to transformers
* update
* update
* update
* major refactoring
* make sure options is ok
* lest test that mdx is tested
* doctest glob
* nits
* update doctest nightly
* some cleaning
* run correct test on diff
* debug
* run on a single worker
* skip_cuda_test tampkate
* updates
* add rA and continue on failure
* test options
* parse `py` codeblock?
* we don't need to replace ignore results, don't remember whyu I put it
* cleanup
* more cleaning
* fix arg
* more cleaning
* clean an todo
* more pre-processing
* doctest-module has none so extra `- ` is needed
* remove logs
* nits
* doctest-modules ....
* oups
* let's use sugar
* make dataset go quiet
* add proper timeout
* nites
* spleling timeout
* update
* properly skip tests that have CUDSA
* proper skipping
* cleaning main and get tests to run
* remove make report?
* remove tee
* some updates
* tee was removed but is the full output still available?
* [all-test]
* only our tests
* don't touch tee in this PR
* no atee-sys
* proper sub
* monkey
* only replace call
* fix sub
* nits
* nits
* fix invalid syntax
* add skip cuda doctest env variable
* make sure all packages are installed
* move file
* update check repo
* revert changes
* nit
* finish cleanup
* fix re
* findall
* update don't test init files
* ignore pycache
* `-ignore-pycache` when running pytests
* try to fix the import missmatch error
* install dec
* pytest is required as doctest_utils imports things from it
* the only log issues were dataset, ignore results should work
* more cleaning
* Update .circleci/create_circleci_config.py
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* Apply suggestions from code review
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* [ydshieh] empty string if cuda is found
* [ydshieh] fix condition
* style
* [ydshieh] fix
* Add comment
* style
* style
* show failure
* trigger CI
---------
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
Co-authored-by: Yih-Dar <2521628+ydshieh@users.noreply.github.com>
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-05-09 20:34:48 +02:00
Yih-Dar
073baf7f22
Install accelerete@main
in PyTorch Past CI jobs ( #22963 )
...
fix
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-04-24 21:19:06 +02:00
Yih-Dar
aa4316757d
Change schedule CI time ( #22884 )
...
* fix
* Update .github/workflows/self-nightly-past-ci-caller.yml
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
2023-04-20 14:01:08 +02:00
Yih-Dar
648bd5a8aa
Show diff between 2 CI runs on Slack reports ( #22798 )
...
fix
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-04-19 19:27:37 +02:00
Yih-Dar
1ebc1dee92
Fix Past CI not running against the latest main
( #22823 )
...
fix
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-04-18 15:41:41 +02:00
Sylvain Gugger
e13d6ef7dc
Simplify update metadata job ( #22811 )
...
* Simplify update metadata job
* Match more branch names
* Install all what is necessary
* Install all what is necessary
* Forgot the dev
* Install less stuff
* This syntax?
2023-04-17 13:54:20 -04:00
Yih-Dar
656d41ab4c
Remove DS_BUILD_AIO=1
( #22741 )
...
fix
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-04-13 18:08:22 +02:00
Yih-Dar
2c22bc79c2
Make tiny model creation + pipeline testing more robust ( #22500 )
...
* Final Tiny things
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-04-06 17:45:55 +02:00
Yih-Dar
0fe6c6bdca
(Re-)Enable Nightly + Past CI ( #22393 )
...
* Enable Nightly + Past CI
* put schedule
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-03-30 21:06:35 +02:00
Yih-Dar
97440e9c75
Avoid using personal HF token in CI ( #22453 )
...
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-03-29 19:45:06 +02:00
Yih-Dar
8894b81742
Use real tokenizers if tiny version(s) creation has issue(s) ( #22428 )
...
Fix some tiny model creation issues
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-03-29 16:16:23 +02:00
Yih-Dar
e8cc02555e
Automatically create/update tiny models ( #22275 )
...
* Automatically create or update tiny models
* Skip failed tests
* update workflow file
* use revision
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-03-23 19:14:17 +01:00
Yih-Dar
ba9e0191de
Prepare daily CI for torch 2.0.0 ( #22135 )
...
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-03-13 22:21:15 +01:00
Yih-Dar
aab895c396
Make Slack CI reporting stronger ( #21823 )
...
* Use token
* Avoid failure
* better error
* Fix
* fix style
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-02-28 17:12:44 +01:00
Yih-Dar
ff143ae10e
Update doctest GH workflow file ( #21744 )
...
update
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-02-23 09:40:53 +01:00
Nolwenn Bernard
baf4bacb1f
[i18n-fr] Translate index page to French ( #21458 )
...
* Translate index page to French
* Fix indent
* Fix toctree
* Replace missing file by in_translation
* Add index
* Update docs/source/fr/index.mdx
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
---------
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
2023-02-06 12:25:49 -05:00
Yih-Dar
94db82573e
Fix (DeepSpeed) docker image build issue ( #21002 )
...
* Fix docker image build issue
* remove comment
* Add comment
* Update docker/transformers-pytorch-deepspeed-latest-gpu/Dockerfile
Co-authored-by: Stas Bekman <stas00@users.noreply.github.com>
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
Co-authored-by: Stas Bekman <stas00@users.noreply.github.com>
2023-01-04 21:28:33 +01:00
Yih-Dar
bf9a5882a7
Update some GH action versions ( #20537 )
...
* update actions versions
* update actions versions
* update actions versions
* update actions versions
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2022-12-06 16:54:40 +01:00
Yih-Dar
67d32f4649
Replace set-output
by $GITHUB_OUTPUT
( #20547 )
...
* remove set-output
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2022-12-05 18:25:13 +01:00
Yih-Dar
373bfe70a0
Change Doctests CI launch time ( #20523 )
...
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2022-12-01 16:38:41 +01:00
Yih-Dar
e8d448edcf
extract warnings in GH workflows ( #20487 )
...
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2022-11-29 15:58:54 +01:00
BFSS
05d80d856c
translate zh quicktour( #20095 ) ( #20181 )
...
* zh quicktour(#20095 )
* add zh to doc workflow
* remove untranslation from toctree
Co-authored-by: BeifangSusu <BeifangSusu@bfss.com>
2022-11-21 08:44:18 -05:00
Yih-Dar
700e0cd65f
Add missing report button for Example test ( #20293 )
...
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2022-11-17 15:55:00 +01:00
Mishig
e627e9b5ae
Complete doc migration ( #20267 )
2022-11-16 08:43:37 -05:00
Wonhyeong Seo
07d8d6e2f7
docs: translated index page to korean ( #20180 )
...
docs: i18n: first draft of index page
docs: fix: first revision of index page
docs: i18n: missed section - supported frameworks
docs: fix: second revision of index page
review by @ArthurZucker
refactor: remove untranslated files from korean
docs: fix: remove untranslated references from toctree.yml
feat: enable korean docs in gh actions
docs: feat: add in_translation page as placeholder
docs: bug: testing if internal toc need alphabet chars
docs: fix: custom english anchor for non-alphanumeric headings
review by @sgugger
docs: i18n: translate comments on install methods in _config.py
docs: refactor: more concise wording for translations
2022-11-14 12:09:21 -05:00
Mishig
e2a23b6ce9
Update github pr docs actions ( #20125 )
2022-11-08 10:37:24 -05:00
Yih-Dar
c06d555647
Show installed libraries and their versions in GA jobs ( #20069 )
...
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2022-11-04 18:03:18 +01:00
Yih-Dar
12ce2941c7
Fix docker image build ( #19759 )
...
* Use 2 jobs for the docker image build (latest torch + DS)
* fix
* Add comment
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2022-10-20 20:36:13 +02:00
Yih-Dar
5418e3cef0
Build Push CI images also in a daily basis ( #19532 )
...
* Build Push CI images also in a daily basis
* update
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2022-10-13 07:31:12 +02:00
Yih-Dar
e81cb010f8
Avoid Push CI failing to report due to many commits being merged ( #19496 )
...
* Change the depth to 20
* Add comment
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2022-10-12 09:25:05 +02:00
Yih-Dar
8d68878cc0
python3
instead of python
in push CI setup job (#19492 )
...
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2022-10-11 19:18:39 +02:00
Sylvain Gugger
9ac586b3c8
Rework pipeline tests ( #19366 )
...
* Rework pipeline tests
* Try to fix Flax tests
* Try to put it before
* Use a new decorator instead
* Remove ignore marker since it doesn't work
* Filter pipeline tests
* Woopsie
* Use the fitlered list
* Clean up and fake modif
* Remove init
* Revert fake modif
2022-10-07 18:01:58 -04:00
Sushrut1101
6dce9e0cdd
docker-build: Update actions/checkout to v3 ( #19288 )
2022-10-04 16:26:52 +02:00
Oscar Dominguez
cd024da6f8
ci(workflows): update actions/checkout to v3 ( #19280 )
...
in stale.yml
2022-10-04 16:07:53 +02:00
Oscar Dominguez
ca3ebc44e0
ci(stale.yml): upgrade actions/setup-python to v4 ( #19281 )
2022-10-04 16:07:33 +02:00
Yih-Dar
163cd15279
Add job names in Past CI artifacts ( #19235 )
...
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2022-09-29 19:18:24 +02:00
Yih-Dar
99c32493e0
Fix confusing working directory in Push CI ( #19234 )
...
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2022-09-29 08:36:46 +02:00
Yih-Dar
71fc331746
Separate Push CI images from Scheduled CI ( #19170 )
...
* separate images
* Fix condition
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2022-09-26 10:55:42 +02:00
Yih-Dar
0cea8d5555
Add offline runners info in the Slack report ( #19169 )
...
* send slack report for offline runners
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2022-09-23 19:23:05 +02:00
Yih-Dar
cc567e0063
Fix the wrong schedule ( #19117 )
...
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2022-09-20 13:46:55 +02:00
Yih-Dar
22264f933d
fix working dir ( #19101 )
...
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2022-09-19 07:09:24 -04:00
Yih-Dar
ba7f2173cc
Add runner availability check ( #19054 )
...
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2022-09-19 12:27:06 +02:00
Yih-Dar
7a8118947f
Add checks for more workflow jobs ( #18905 )
...
* add check for scheduled CI
* Add check to other CIs
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2022-09-07 12:51:37 +02:00
Yih-Dar
7d5fde991d
unpin slack_sdk version ( #18901 )
...
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2022-09-06 18:42:00 +02:00
Yih-Dar
ecdf9b06bc
Remove cached torch_extensions on CI runners ( #18868 )
...
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2022-09-02 18:17:58 +02:00
Yih-Dar
0ab465a5d2
pin Slack SDK to 3.18.1 to avoid failing issue ( #18869 )
...
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2022-09-02 16:49:08 +02:00
Lysandre Debut
ee407024c4
Update location identification ( #18834 )
2022-08-31 15:10:25 +02:00
Yih-Dar
e88e9ff045
Disable nightly CI temporarily ( #18820 )
...
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2022-08-30 18:33:09 +02:00
Yih-Dar
81ab11124f
Add checks for some workflow jobs ( #18583 )
...
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2022-08-16 13:53:47 +02:00
flozi00
5d3f037433
german docs translation ( #18544 )
...
* Create _config.py
* Create _toctree.yml
* Create index.mdx
not sure about "du / ihr" oder "sie"
* Create quicktour.mdx
* Update _toctree.yml
* Update build_documentation.yml
* Update build_pr_documentation.yml
* fix build
* Update index.mdx
* Update quicktour.mdx
* Create installation.mdx
* Update _toctree.yml
2022-08-11 09:52:27 -04:00
Yih-Dar
d2704c4143
Add machine type in the artifact of Examples directory job ( #18459 )
...
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2022-08-04 18:52:01 +02:00
Yih-Dar
c4c6b4dbda
Add PyTorch 1.11 to past CI ( #18302 )
...
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2022-07-26 15:47:23 +02:00
Yih-Dar
2c5747edfe
Update notification service ( #17921 )
...
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2022-07-21 15:03:50 +02:00
Yih-Dar
f681437203
Enable Past CI ( #17919 )
...
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2022-07-05 18:08:36 +02:00
Zachary Mueller
893ab12452
Auto-build Docker images before on-merge if setup.py was changed ( #17573 )
...
* Auto-build on setup modification
* Modify push-caller
* Make adjustments based on code review
2022-06-23 16:51:33 -04:00
Yih-Dar
8d634b70e0
Fix push CI artifact path ( #17788 )
...
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2022-06-23 12:31:22 +02:00
Joao Gante
0d0c392c45
CLI: use hub's create_commit
( #17755 )
...
* use create_commit
* better commit message and description
* touch setup.py to trigger cache update
* add hub version gating
2022-06-22 16:50:21 +01:00
Yih-Dar
6589e510fa
Attempt to change Push CI to workflow_run ( #17753 )
...
* Use workflow_run event for push CI
* change to workflow_run
* Add comments
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2022-06-18 08:35:03 +02:00
Yih-Dar
ca169dbdf1
Enable PyTorch nightly build CI ( #17335 )
...
* nightly build pytorch CI
* fix working dir
* change time and event name
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2022-06-17 16:42:27 +02:00
Yih-Dar
50415b84d6
Revert "Change push CI to run on workflow_run event ( #17692 )" ( #17717 )
...
This reverts commit b76290f44c
.
2022-06-15 18:42:43 +02:00
Yih-Dar
b76290f44c
Change push CI to run on workflow_run event ( #17692 )
...
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2022-06-15 17:43:31 +02:00
Yih-Dar
da0bed5f4a
Pre-build DeepSpeed ( #17607 )
...
* pre-build deepspeed
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2022-06-09 23:02:33 +02:00
Yih-Dar
9aa230aa2f
Use latest stable PyTorch/DeepSpeed for Push & Scheduled CI ( #17417 )
...
* update versions
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2022-06-07 11:53:05 +02:00
Yih-Dar
659b27fd26
Print more library versions in CI ( #17384 )
...
* print more lib. versions and just befor test runs
* update print_env_pt.py
* rename to print_env
* Disable warning + better job name
* print python version
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2022-06-02 10:24:16 +02:00
Yih-Dar
0932adb3e8
Split push CI into 2 workflows ( #17369 )
...
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2022-06-02 10:19:26 +02:00
Martina Fumanelli
dfc38463b8
Setup for Italian translation and add quicktour.mdx translation ( #17472 )
...
* Setup for Italian translation and add first document
- Add 'it' folder for files translated into Italian
- Add _config.py and _toctree.yml files
- Add translation of quicktour.mdx
* Fix style issue of italian documentation files
* Add 'it' to the languages section in the .github/workflows
* Remove - installation from _toctree for Italian
* Translation for index file
- Add index to _toctree.yml
- Add translation of index.mdx
* Fix typo in docs/source/it/index.mdx
* Translate code comments in docs/source/it/_config.py
Co-authored-by: Martina Fumanelli <martinafumanelli@Martinas-MBP.homenet.telecomitalia.it>
2022-05-31 09:57:43 -04:00
Yih-Dar
400b30936a
Docker image build in parallel ( #17434 )
...
* docker image build in parallel
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2022-05-31 15:39:03 +02:00
Yih-Dar
1ac2b8fa7f
Fix missing job action button in CI report ( #17270 )
...
* use matrix.machine_type
* fix job names used in job_link
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2022-05-17 08:31:06 +02:00
Yih-Dar
50d1867cf8
Add PR title to push CI report ( #17246 )
...
* add PR title to push CI report
* add link
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2022-05-13 21:50:40 +02:00
Yih-Dar
506899d147
Fix push CI channel ( #17242 )
...
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2022-05-13 20:59:56 +02:00
Yih-Dar
993553b2f1
fix --gpus option for docker ( #17235 )
...
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2022-05-13 17:26:26 +02:00
Yih-Dar
38043d8453
Update self-push workflow ( #17177 )
...
* update push ci
* install git-python
* update comment
* update deepspeed jobs
* fix report
* skip 2 more tests that require fairscale
* Fix changes in test_fetcher.py (to deal with `setup.py` is changed)
* set RUN_PT_TF_CROSS_TESTS=1 and final clean-up
* remove SIGOPT_API_TOKEN
* remove echo "$matrix_folders"
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2022-05-13 16:28:00 +02:00
Rafael Zimmer
85fc455972
Added translation of installation.mdx to Portuguese Issue #16824 ( #16979 )
...
* Added translation of installation.mdx to Portuguese, as well
as default templates of _toctree.yml and _config.py
* [ build_documentation.yml ] - Updated doc_builder to build
documentation in Portuguese.
[ pipeline_tutorial.mdx ] - Created translation for the pipeline_tutorial.mdx.
* [ build_pr_documentation.yml ] - Added pt language to pr_documentation builder.
[ pipeline_tutorial.mdx ] - Grammar changes.
* [ accelerate.mdx ] - Translated to Portuguese the acceleration tutorial.
* [ multilingual.mdx ] - Added portuguese translation for multilingual tutorial.
[ training.mdx ] - Added portuguese translation for training tutorial.
* [ preprocessing.mdx ] - WIP
* Update _toctree.yml
* Adding Pré-processamento to _toctree.yml
* Update accelerate.mdx
* Nits and eliminate preprocessing file while it is ready
Co-authored-by: Omar U. Espejel <espejelomar@gmail.com>
2022-05-13 07:55:44 -04:00
Yih-Dar
3212afa614
split single_gpu and multi_gpu ( #17083 )
...
* split single_gpu and multi_gpu
* update needs in send_result
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2022-05-09 07:13:07 -04:00
Yih-Dar
351cdbdfdc
Fix self-push CI report path in cat ( #17111 )
...
* fix report cat path
* fix report cat path
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2022-05-06 07:45:17 -07:00
Sylvain Gugger
dd16a113a4
Remove torchhub test ( #17097 )
2022-05-05 10:02:47 -04:00
Sylvain Gugger
bb8d40529e
Deprecate model templates ( #17062 )
...
* Deprecate model templates
* Address review comments
2022-05-04 09:36:38 -04:00
Sylvain Gugger
dd739f7045
Remove fetch in model templates test
2022-05-03 10:49:12 -04:00
Sylvain Gugger
6e17ba6aa5
Remove Python and use v2 action ( #17059 )
2022-05-03 10:12:17 -04:00
Yih-Dar
19420fd99e
Move test model folders ( #17034 )
...
* move test model folders (TODO: fix imports and others)
* fix (potentially partially) imports (in model test modules)
* fix (potentially partially) imports (in tokenization test modules)
* fix (potentially partially) imports (in feature extraction test modules)
* fix import utils.test_modeling_tf_core
* fix path ../fixtures/
* fix imports about generation.test_generation_flax_utils
* fix more imports
* fix fixture path
* fix get_test_dir
* update module_to_test_file
* fix get_tests_dir from wrong transformers.utils
* update config.yml (CircleCI)
* fix style
* remove missing imports
* update new model script
* update check_repo
* update SPECIAL_MODULE_TO_TEST_MAP
* fix style
* add __init__
* update self-scheduled
* fix add_new_model scripts
* check one way to get location back
* python setup.py build install
* fix import in test auto
* update self-scheduled.yml
* update slack notification script
* Add comments about artifact names
* fix for yolos
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2022-05-03 14:42:02 +02:00
Sylvain Gugger
4be8b95a9f
Disable Flax GPU tests on push ( #17042 )
2022-05-02 10:25:53 -04:00
Yih-Dar
992996e9ca
Add -e flag to some GH workflow yml files ( #16959 )
...
* Add -e flag
* add check
* create new keys
* run python setup.py build install
* add comments
* change to develop
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2022-04-27 21:44:21 +02:00
Sylvain Gugger
3785f4665a
Fix syntax error in TorchHub workflow
2022-04-18 07:54:00 -04:00
Joao Gante
6984848ed0
Create empty venv on cache miss ( #16816 )
2022-04-18 07:49:31 -04:00
Joao Gante
02de7a8e7f
CI: non-remote GH Actions now use a python venv ( #16789 )
2022-04-18 09:47:38 +01:00
Stas Bekman
ce2fef2ad2
[trainer / deepspeed] fix hyperparameter_search ( #16740 )
...
* [trainer / deepspeed] fix hyperparameter_search
* require optuna
* style
* oops
* add dep in the right place
* create deepspeed-testing dep group
* Trigger CI
2022-04-14 17:24:38 -07:00
Stas Bekman
3a16ab25c8
[self-scheduled ci] explain where dependencies are ( #16757 )
2022-04-13 12:28:02 -04:00
Joao Gante
0b8f697219
CI: setup-dependent pip cache ( #16751 )
...
* Setup-dependent pip cache
* Do not restore from old versions
2022-04-13 16:19:14 +01:00
Sylvain Gugger
b9a768b3ff
Enable doc in Spanish ( #16518 )
...
* Reorganize doc for multilingual support
* Fix style
* Style
* Toc trees
* Adapt templates
2022-04-04 10:25:46 -04:00
Lysandre Debut
eca77f4719
Updates the default branch from master to main ( #16326 )
...
* Updates the default branch from master to main
* Links from `master` to `main`
* Typo
* Update examples/flax/README.md
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
2022-03-23 03:46:59 -04:00
Patrick von Platen
c1af180dfe
Add Slack notification support for doc tests ( #16253 )
...
* up
* up
* up
* fix
* yeh
* ups
* Empty test commit
* correct quicktour
* correct
* correct
* up
* up
* uP
* uP
* up
* up
* uP
* up
* up
* up
* up
* up
* up
* up
* up
* up
* up
* Update src/transformers/models/van/modeling_van.py
* finish
* apply suggestions
* remove folder
* revert to daily testing
2022-03-21 11:33:18 +01:00
Yih-Dar
09013efdf1
Update step name ( #16189 )
...
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2022-03-16 11:19:38 -04:00
Sylvain Gugger
8bfd2fb8f0
Use templates ( #16142 )
...
* Use tempaltes for all doc building jobs
* Add this branch to the doc build
* Switch to main branch
2022-03-15 08:07:56 -04:00
Lysandre Debut
a01fe4cd32
Rebuild deepspeed ( #16081 )
...
* Rebuild deepspeed
* Apply suggestions from code review
Co-authored-by: Stas Bekman <stas00@users.noreply.github.com>
Co-authored-by: Stas Bekman <stas00@users.noreply.github.com>
2022-03-11 14:35:48 -05:00
Patrick von Platen
322c8533d7
Run daily test without time-out at least once ( #16077 )
2022-03-11 18:04:17 +01:00
Sylvain Gugger
10591399d6
Build the doc in a seperate folder then move it ( #16020 )
...
* Build the doc in a seperate folder then move it
* Allow job
* Is this it?
* Dislike comments?
* Copy instead of move
* Removing version built
* Typos
* No variable
* Take _versions.yml into account
* Finish main job and add dev job
* Forgot the run
* Fix syntax error
* Execute builder from the repo
* Typo
2022-03-10 07:44:29 -05:00
Lysandre Debut
e493a3a5e2
Fix github actions comment ( #16009 )
...
* Add issue number
* Dev
2022-03-09 08:39:03 -05:00
Patrick von Platen
c1aaa43935
[Doctests] Move doctests to new GPU & Fix bugs ( #15969 )
...
* test
* up
* up
* Empty test commit
* up
* update tests
* up
* fix some vision models
* correct
* correct docs
* Trigger notification
* finalize
* check
* correct quicktour
* Apply suggestions from code review
* improve doctests
* Trigger Build
* next try
* next try
* and again
* Output current clone information
* Output current clone information
* Correct path
* add tf round again
* revert to daily job
Co-authored-by: Lysandre <lysandre.debut@reseau.eseo.fr>
2022-03-09 13:09:56 +01:00
Sylvain Gugger
f5a080dd10
Do a pull in case docs were updated during build ( #15922 )
2022-03-08 07:19:41 -05:00
Sylvain Gugger
3d2242869d
Update delete-dev-doc job to match build-dev-doc ( #15891 )
...
* Update delete-dev-doc job to match build-dev-doc
* More debug info
* More debug info
* Stash if needed
* Remove the comment update
* Fix paths
* Wtf is going on..
* Fix git status test
* Try another way
* I don't understand what's happening
* Bash shell
* What's happening now...
* What's happening now...
* Try like this
* Back to trying to use bash
* And like that?
* Refine tests
* Stash after adding new files
* Stash after adding new files
* Proper commit sha and PR number
* Address review comments
2022-03-02 16:18:54 -05:00
Lysandre Debut
d1a29078c0
Remove stash for now ( #15882 )
2022-03-01 22:36:19 -05:00
Lysandre Debut
26426923b7
No self-hosted runner for dev documentation ( #15710 )
2022-03-01 14:05:54 -05:00
Mishig Davaadorj
00eaffc81f
Bump up doc node version to 16 ( #15874 )
2022-03-01 18:37:57 +01:00
Suraj Patil
afca0d5192
use python 3.7 for flax self-push tests ( #15865 )
...
* set python 3.7 for flax tests
* setup-python@v2
* python-dev
* install -y
* python3-dev
* install kenlm from source
* install cython
* cd to kenlm
* kenlm install
* don't install kenlm
* change flax pretrained to run flax tests
* cleanup
* remove python-dev
2022-03-01 18:26:30 +01:00
Lysandre Debut
54f0db4066
Add PT + TF automatic builds ( #15860 )
...
* Add PT + TF automatic builds
* Apply suggestions from code review
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
* Wrap up
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
2022-03-01 08:55:11 -05:00
Lysandre Debut
b7e292aebd
Fix the push run ( #15807 )
2022-02-24 19:30:17 +01:00
Lysandre Debut
bb7949b35a
Fix model templates ( #15806 )
...
* Fix model templates
* Update paths
2022-02-23 18:27:29 -05:00
Lysandre
309e87e25e
Docker images should only run on a daily basis
2022-02-23 18:01:44 -05:00
Lysandre
c475f3ce2d
Scheduled tests should only run on a daily basis
2022-02-23 17:52:22 -05:00
Eliott C
6336017c15
Fix build_documentation CI ( #15803 )
2022-02-23 21:53:51 +01:00
Lysandre Debut
a0e3480699
[Test refactor 5/5] Build docker images ( #15729 )
2022-02-23 15:48:19 -05:00
Lysandre Debut
4c737f0e40
[Test refactor 4/5] Improve the scheduled tests ( #15728 )
2022-02-23 15:48:05 -05:00
Lysandre Debut
29c10a41d0
[Test refactor 1/5] Per-folder tests reorganization ( #15725 )
...
* Per-folder tests reorganization
Co-authored-by: sgugger <sylvain.gugger@gmail.com>
Co-authored-by: Stas Bekman <stas@stason.org>
2022-02-23 15:46:28 -05:00
Eliott C
86636f52a9
Fix indent in doc-builder CI ( #15798 )
2022-02-23 20:01:33 +01:00
Eliott C
a1efc82362
HTML dev docs ( #15678 )
...
Co-authored-by: Pierric Cistac <Pierrci@users.noreply.github.com>
2022-02-23 19:43:22 +01:00
Eliott C
bc3379e12c
🔥 Remove build_doc_test github action ( #15680 )
2022-02-16 14:06:26 +01:00
Lysandre Debut
7029240927
Upgrade click version ( #15579 )
2022-02-09 10:28:43 -05:00
Patrick von Platen
e2b6e73fa2
[Flax tests] Disable scheduled GPU tests ( #15503 )
2022-02-03 17:12:14 +01:00
Ayush Chaurasia
c74f3d4c48
Add W&B backend for hyperparameter sweep ( #14582 )
...
# Add support for W&B hyperparameter sweep
This PR:
* allows using wandb for running hyperparameter search.
* The runs are visualized on W&B sweeps dashboard
* This supports runnning sweeps on parallel devices, all reporting to the same central dashboard.
### Usage
**To run new a hyperparameter search:**
```
trainer.hyperparameter_search(
backend="wandb",
project="transformers_sweep", # name of the project
n_trials=5,
metric="eval/loss", # metric to be optimized, default 'eval/loss'. A warning is raised if the passed metric is not found
)
```
This outputs a sweep id. Eg. `my_project/sweep_id`
**To run sweeps on parallel devices:**
Just pass sweep id which you want to run parallel
```
trainer.hyperparameter_search(
backend="wandb",
sweep_id = "my_project/sweep_id"
)
```
2022-02-02 14:06:14 -05:00
Patrick von Platen
9f831bdeaf
[DocTests Speech] Add doc tests for all speech models ( #15031 )
...
* fix_torch_device_generate_test
* remove @
* doc tests
* up
* up
* fix doctests
* adapt files
* finish refactor
* up
* save intermediate
* add more logic
* new change
* improve
* next try
* next try
* next try
* next try
* fix final spaces
* fix final spaces
* improve
* renaming
* correct more bugs
* finish wavlm
* add comment
* run on test runner
* finish all speech models
* adapt
* finish
2022-01-27 14:29:31 +01:00
Sylvain Gugger
e65bfc0971
Try without bad instruction
2022-01-24 15:55:29 -05:00
Sylvain Gugger
81156d20cd
Add model like ( #14992 )
...
* Add new model like command
* Bad doc-styler
* black and doc-styler, stop fighting!
* black and doc-styler, stop fighting!
* At last
* Clean up
* Typo
* Bad doc-styler
* Bad doc-styler
* All good maybe?
* Use constants
* Add doc and type hints
* More cleaning
* Add doc
* Fix Copied from
* Doc template
* Use typing.Pattern instead
* Framework-specific files
* Fixes
* Select frameworks clean model init
* Deal with frameworks in main init
* fixes
* Last fix
* Prompt user for info
* Delete exemple config
* Last fixes
* Add test config
* Fix bug with model_type included in each other
* Fixes
* More fixes
* More fixes
* Adapt config
* Remove print statements
* Will fix tokenization later, leave it broken for now
* Add test
* Quality
* Try this way
* Debug
* Maybe by setting the path?
* Let's try another way
* It should go better when actually passing the arg...
* Remove debug statements and style
* Fix config
* Add tests
* Test require the three backends
* intermediate commit
* Revamp pattern replacements and start work on feature extractors
* Adapt model info
* Finalize code for processors
* Fix in main init additions
* Finish questionnaire for processing classes
* Fix file name
* Fix for real
* Fix patterns
* Style
* Remove needless warnings
* Copied from should work now.
* Include Copied form in blocks
* Add test
* More fixes and tests
* Apply suggestions from code review
Co-authored-by: Lysandre Debut <lysandre@huggingface.co>
* Address review comment
Co-authored-by: Lysandre Debut <lysandre@huggingface.co>
2022-01-24 15:25:10 -05:00
Patrick von Platen
b7cb126ccc
[PyTorch-nightly-test] Fix Wav2Vec2 LM & Phoneme tests ( #15272 )
...
* [PyTorch-nightly-test] Fix Wav2Vec2 LM & Phoneme tests
* Update .github/workflows/self-nightly-scheduled.yml
* change lines
* Apply suggestions from code review
2022-01-24 10:53:53 +01:00
Lysandre Debut
ad7390636d
Tentative workflow improvement ( #15255 )
2022-01-20 13:51:19 -05:00
Lysandre Debut
6a3c883c8b
Fix PR number ( #15231 )
...
* Fix PR number
* Fix PR number
2022-01-19 11:00:16 -05:00
Lysandre Debut
653379c094
Build dev documentation ( #15210 )
...
* Wrap up
* Remove secret
* Fix path
* Typo
Revert image switch
* Specific token for comments
* Cleaner comments
* Correct PR number
* Explicit master install
* Force uninstall
2022-01-19 08:47:34 -05:00
Sylvain Gugger
5f3c57fc84
Check the repo consistency in model templates test ( #15141 )
...
* Check the repo consistency in model templates test
* Fix doc template
* Fix docstrings
* Fix last docstring
2022-01-14 04:52:38 -05:00
Lysandre Debut
6950ccec1b
doc-builder -> doc-build ( #15134 )
...
* Updated script
* Commit everything
* Ready for review!
* Update .github/workflows/build_documentation.yml
Co-authored-by: Julien Chaumond <julien@huggingface.co>
Co-authored-by: Julien Chaumond <julien@huggingface.co>
2022-01-13 06:02:24 -05:00
Lysandre Debut
444ea95a80
Print out durations of all scheduled tests ( #15102 )
2022-01-11 08:15:59 -05:00
NielsRogge
83c552d390
Add detectron2 to Github actions ( #15053 )
2022-01-06 08:53:58 -05:00
Patrick von Platen
dbac8899fe
[Tests] Correct Wav2Vec2 & WavLM tests ( #15015 )
...
* up
* up
* up
2022-01-03 20:19:04 +01:00
Sylvain Gugger
c075fb7855
Replace commit sha by commit url for update jobs ( #14852 )
...
* Replace commit sha by commit url for update jobs
* Typo
* Update .github/workflows/build_documentation.yml
Co-authored-by: Julien Chaumond <julien@huggingface.co>
* Apply review comments
Co-authored-by: Julien Chaumond <julien@huggingface.co>
2021-12-21 11:17:11 -05:00
Patrick von Platen
c4a96cecbc
Wav2Vec2 meets phonemes ( #14353 )
...
* up
* add tokenizer
* improve more
* finish tokenizer
* finish
* adapt speech recognition script
* adapt convert
* more fixes
* more fixes
* update phonemizer wav2vec2
* better naming
* fix more tests
* more fixes swedish
* correct tests
* finish
* improve script
* remove file
* up
* lets get those 100 model architectures until the end of the month
* make fix-copies
* correct more
* correct script
* more fixes
* more fixes
* add to docs
* Apply suggestions from code review
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* replace assert
* fix copies
* fix docs
* new try docs
* boom boom
* update
* add phonemizer to audio tests
* make fix-copies
* up
* upload models
* some changes
* Update tests/test_tokenization_wav2vec2_phoneme.py
Co-authored-by: Anton Lozhkov <aglozhkov@gmail.com>
* more fixes
* remove @
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
Co-authored-by: Anton Lozhkov <aglozhkov@gmail.com>
2021-12-17 19:56:44 +01:00
Sylvain Gugger
12e1b4c6df
Fix the build documentation job ( #14788 )
...
* Fix the build documentation job
* Fix install
* Address review comment
2021-12-16 09:35:20 -05:00
Sylvain Gugger
7e61d56a45
Fix the doc_build_test job ( #14774 )
...
* Fake new model
* Fix doc-building test job
* Is this the problem?
* Another try
* Typo
* Clean up
* Can we do without -e ?
* Clean setup
2021-12-15 03:40:17 -05:00
Sylvain Gugger
7533d30acd
Convert Trainer doc page to MarkDown ( #14753 )
...
* Convert Trainer doc page to MarkDown
* Fix repo consistency
* Fix the doc build test job
2021-12-13 13:09:50 -05:00
Sylvain Gugger
e4666bff06
Fix name
2021-12-13 12:01:37 -05:00
Sylvain Gugger
64e92ed224
Update transformers metadata ( #14724 )
...
* Wip on metadata update
* Most of the script
* Add a job to auto-update the transformers metadata
* Style
2021-12-13 11:46:03 -05:00
Sylvain Gugger
c3cd88a9ba
Small fixes for the doc ( #14751 )
2021-12-13 11:17:01 -05:00
Stas Bekman
8362d07d63
[CI/pt-nightly] switch to cuda-11.3 ( #14726 )
2021-12-13 09:53:48 -05:00
Sylvain Gugger
f46668282b
Fix path for notebooks
2021-12-10 15:03:17 -05:00
Sylvain Gugger
3b2d1652e4
Fix typo in branch name
2021-12-10 14:38:21 -05:00
Sylvain Gugger
1b75d7238c
Automatically build doc notebooks ( #14718 )
...
* Test workflow
* Build doc
* Make a clean build
* Add doc config
* Restore other workflows
* Final job
* Print something in else statements
* Pull before making changes
2021-12-10 14:20:56 -05:00
Sylvain Gugger
68e53e6fcd
Add a job to test doc building (for realsies this time) ( #14662 )
2021-12-09 07:01:03 -05:00
Lysandre Debut
2294071a0c
Fix doc builder ( #14676 )
2021-12-08 09:14:36 -05:00
Patrick von Platen
961732c276
[Wav2Vec2] PyCTCDecode Integration to support language model boosted decoding ( #14339 )
...
* up
* up
* up
* make it cleaner
* correct
* make styhahalal
* add more tests
* finish
* small fix
* make style
* up
* tryout to solve cicrle ci
* up
* fix more tests
* fix more tests
* apply sylvains suggestions
* fix import
* correct docs
* add pyctcdecode only to speech tests
* fix more tests
* add tf, flax and pt tests
* add pt
* fix last tests
* fix more tests
* Apply suggestions from code review
* change lines
* Apply suggestions from code review
Co-authored-by: Anton Lozhkov <aglozhkov@gmail.com>
* correct tests
* correct tests
* add doc string
Co-authored-by: Anton Lozhkov <aglozhkov@gmail.com>
2021-12-08 12:07:54 +01:00
Sylvain Gugger
381b05a3f5
Remove nonworking workflow for now
2021-12-06 17:25:28 -05:00
Sylvain Gugger
03fda7b743
Add a job to test the documentation build ( #14645 )
...
* Add a job to the documentation build
* Add caching
* Test cache
2021-12-06 13:55:59 -05:00
Lysandre Debut
aada989ad5
Fix doc builder ( #14616 )
...
* Fix doc builder
* Fix doc builder
* Fix doc builder
2021-12-03 12:09:25 -05:00
Stas Bekman
e62091d5a7
[CI] move env print to util, add pt, nccl versions ( #14607 )
...
* move env print to util, add pt, nccl versions
* style
* version
* align
2021-12-03 08:18:36 -05:00
Lysandre Debut
77d87e732e
Adds a git pull instruction to the documentation builder ( #14597 )
...
* Adds a git pull instruction
* master -> main
2021-12-02 03:32:38 -05:00
Sylvain Gugger
4df7d05a87
Doc new front ( #14590 )
...
* Convert PretrainedConfig doc to Markdown
* Use syntax
* Add necessary doc files (#14496 )
* Doc fixes (#14499 )
* Fixes for the new front
* Convert DETR file for table
* Title is needed
* Simplify a bit
* Even simpler
* Remove imports
* Fix typo in toctree (#14516 )
* Fix checkpoints badge
* Update versions.yml format (#14517 )
* Doc new front github actions (#14512 )
* Doc new front github actions
* Fix docstring
* Fix feature extraction utils import (#14515 )
* Address Julien's comments
* Push to doc-builder
* Ready for merge
* Remove old build and deploy
* Doc misc fixes (#14583 )
* Rm versions.yml from doc
* Fix converting.rst
* Rm pretrained_models from toctree
* Fix index links (#14567 )
* Fix links in README
* Localized READMEs
* Fix copy script
* Fix find doc script
* Update README_ko.md
Co-authored-by: Julien Chaumond <julien@huggingface.co>
Co-authored-by: Julien Chaumond <julien@huggingface.co>
* Adapt build command to new CLI tools (#14578 )
* Fix typo
* Fix doc interlinks (#14589 )
* Convert PretrainedConfig doc to Markdown
* Use syntax
* Rm pattern <[a-z]+(.html).*>
* Rm huggingface.co/transformers/master
* Rm .html
* Rm .html from index.mdx
* Rm .html from model_summary.rst
* Update index.mdx rm html
* Update remove .html
* Fix inner doc links
* Fix interlink in preprocssing.rst
* Update pr_checks
Co-authored-by: Sylvain Gugger <sylvain.gugger@gmail.com>
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* Convert PretrainedConfig doc to Markdown
* Use syntax
* Add necessary doc files (#14496 )
* Doc fixes (#14499 )
* Fixes for the new front
* Convert DETR file for table
* Title is needed
* Simplify a bit
* Even simpler
* Remove imports
* Fix checkpoints badge
* Fix typo in toctree (#14516 )
* Update versions.yml format (#14517 )
* Doc new front github actions (#14512 )
* Doc new front github actions
* Fix docstring
* Fix feature extraction utils import (#14515 )
* Address Julien's comments
* Push to doc-builder
* Ready for merge
* Remove old build and deploy
* Doc misc fixes (#14583 )
* Rm versions.yml from doc
* Fix converting.rst
* Rm pretrained_models from toctree
* Fix index links (#14567 )
* Fix links in README
* Localized READMEs
* Fix copy script
* Fix find doc script
* Update README_ko.md
Co-authored-by: Julien Chaumond <julien@huggingface.co>
Co-authored-by: Julien Chaumond <julien@huggingface.co>
* Adapt build command to new CLI tools (#14578 )
* Fix typo
* Fix doc interlinks (#14589 )
* Convert PretrainedConfig doc to Markdown
* Use syntax
* Rm pattern <[a-z]+(.html).*>
* Rm huggingface.co/transformers/master
* Rm .html
* Rm .html from index.mdx
* Rm .html from model_summary.rst
* Update index.mdx rm html
* Update remove .html
* Fix inner doc links
* Fix interlink in preprocssing.rst
* Update pr_checks
Co-authored-by: Sylvain Gugger <sylvain.gugger@gmail.com>
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* Styling
Co-authored-by: Mishig Davaadorj <mishig.davaadorj@coloradocollege.edu>
Co-authored-by: Lysandre Debut <lysandre@huggingface.co>
Co-authored-by: Julien Chaumond <julien@huggingface.co>
2021-12-01 14:13:02 -05:00
Stas Bekman
d1fd64e7aa
clear ~/.cache/torch_extensions between builds ( #14520 )
2021-11-25 03:15:35 -05:00
NielsRogge
3772af49ce
[Tests] Improve vision tests ( #14458 )
...
* Improve tests
* Install vision for tf tests
2021-11-24 15:22:20 +01:00