riaz.somc/transformers
1
0
Fork 0
mirror of https://github.com/huggingface/transformers.git synced 2025-07-29 17:22:25 +06:00
Code Issues Actions 22 Packages Projects Releases Wiki Activity
16,314 Commits 884 Branches 231 Tags 1.9 GiB
080e14b24c
Commit Graph

2485 Commits

Author SHA1 Message Date
Jay Zhou
79132d4cfe
Fix a typo in examples/pytorch/text-classification/run_classification.py (#29072) 2024-02-19 13:01:15 +00:00
Lysandre Debut
f497f564bb
Update all references to canonical models (#29001) 
* Script & Manual edition

* Update
 2024-02-16 08:16:58 +01:00
Klaus Hipp
fe3df9d5b3
[Docs] Add language identifiers to fenced code blocks (#28955) 
Add language identifiers to code blocks
 2024-02-12 10:48:31 -08:00
Alexey Fadeev
792819f6cf
Updated requirements for image-classification samples: datasets>=2.14.0 (#28974) 
Updated datasets requirements. Need a package version >= 2.14.0
 2024-02-12 14:57:25 +00:00
Yih-Dar
308d2b9004
Update the cache number (#28905) 
* fix

* fix

* fix

---------

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2024-02-07 16:37:09 +01:00
Klaus Hipp
1c31b7aa3b
[Docs] Add missing language options and fix broken links (#28852) 
* Add missing entries to the language selector

* Add links to the Colab and AWS Studio notebooks for ONNX

* Use anchor links in CONTRIBUTING.md

* Fix broken hyperlinks due to spaces

* Fix links to OpenAI research articles

* Remove confusing footnote symbols from author names, as they are also considered invalid markup
 2024-02-06 12:01:01 -08:00
dependabot[bot]
e83227d76e
Bump cryptography from 41.0.2 to 42.0.0 in /examples/research_projects/decision_transformer (#28879) 
Bump cryptography in /examples/research_projects/decision_transformer

Bumps [cryptography](https://github.com/pyca/cryptography) from 41.0.2 to 42.0.0.
- [Changelog](https://github.com/pyca/cryptography/blob/main/CHANGELOG.rst)
- [Commits](https://github.com/pyca/cryptography/compare/41.0.2...42.0.0)

---
updated-dependencies:
- dependency-name: cryptography
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2024-02-06 03:53:08 +01:00
dependabot[bot]
ca8944c4e3
Bump dash from 2.3.0 to 2.15.0 in /examples/research_projects/decision_transformer (#28845) 
Bump dash in /examples/research_projects/decision_transformer

Bumps [dash](https://github.com/plotly/dash) from 2.3.0 to 2.15.0.
- [Release notes](https://github.com/plotly/dash/releases)
- [Changelog](https://github.com/plotly/dash/blob/dev/CHANGELOG.md)
- [Commits](https://github.com/plotly/dash/compare/v2.3.0...v2.15.0)

---
updated-dependencies:
- dependency-name: dash
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2024-02-05 03:12:30 +01:00
Klaus Hipp
721ee783ca
[Docs] Fix spelling and grammar mistakes (#28825) 
* Fix typos and grammar mistakes in docs and examples

* Fix typos in docstrings and comments

* Fix spelling of `tokenizer` in model tests

* Remove erroneous spaces in decorators

* Remove extra spaces in Markdown link texts
 2024-02-02 08:45:00 +01:00
zspo
d98591a12b
[docs] fix some bugs about parameter description (#28806) 
Co-authored-by: p_spozzhang <p_spozzhang@tencent.com>
 2024-02-01 16:59:29 +00:00
Matt
74c9cfeaa7
Pin Torch to <2.2.0 (#28785) 
* Pin torch to <2.2.0

* Pin torchvision and torchaudio as well

* Playing around with versions to see if this helps

* twiddle something to restart the CI

* twiddle it back

* Try changing the natten version

* make fixup

* Revert "Try changing the natten version"

This reverts commit de0d6592c3.

* make fixup

* fix fix fix

* fix fix fix

---------

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2024-01-30 23:01:12 +01:00
Matt
415e9a0980
Add tf_keras imports to prepare for Keras 3 (#28588) 
* Port core files + ESM (because ESM code is odd)

* Search-replace in modelling code

* Fix up transfo_xl as well

* Fix other core files + tests (still need to add correct import to tests)

* Fix cookiecutter

* make fixup, fix imports in some more core files

* Auto-add imports to tests

* Cleanup, add imports to sagemaker tests

* Use correct exception for importing tf_keras

* Fixes in modeling_tf_utils

* make fixup

* Correct version parsing code

* Ensure the pipeline tests correctly revert to float32 after each test

* Ensure the pipeline tests correctly revert to float32 after each test

* More tf.keras -> keras

* Add dtype cast

* Better imports of tf_keras

* Add a cast for tf.assign, just in case

* Fix callback imports
 2024-01-30 17:26:36 +00:00
Klaus Hipp
39fa400969
Fix input data file extension in examples (#28741) 2024-01-29 10:06:31 +00:00
Steven Liu
abe0289e6d
[docs] Fix datasets in guides (#28715) 
* change datasets

* fix
 2024-01-26 09:29:07 -08:00
bofeng huang
deb2b59073
Fix lr_scheduler in no_trainer training scripts (#27872) 
* Fix lr_scheduler

* Fix lr scheduler
 2024-01-22 14:22:18 +00:00
jheitmann
f0acf7b6d8
Fix id2label assignment in run_classification.py (#28590) 2024-01-22 11:31:31 +00:00
Amy Roberts
b2748a6efd v4.38.dev.0 2024-01-19 10:43:28 +00:00
Yoach Lacombe
772307be76
Making CTC training example more general (#28582) 
* add w2v2bert compatibility

* Update examples/pytorch/speech-recognition/run_speech_recognition_ctc.py

Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>

---------

Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
 2024-01-18 17:01:49 +00:00
Yoach Lacombe
d2cdefb9ec
Add new meta w2v2-conformer BERT-like model (#28165) 
* first commit

* correct default value non causal

* update config and modeling code

* update converting checkpoint

* clean modeling and fix tests

* make style

* add new config parameters to docstring

* fix copied from statements

* Apply suggestions from code review

Co-authored-by: Sanchit Gandhi <93869735+sanchit-gandhi@users.noreply.github.com>

* make position_embeddings_type docstrings clearer

* clean converting script

* remove function not used

* clean modeling file

* apply suggestion for test file + add convert script to not_doctested

* modify tests according to review - cleaner logic and more tests

* Apply nit suggestions from code review

Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>

* add checker of valid position embeddings type

* instantiate new layer norm layer with the right eps

* fix freeze_feature_encoder since it can be None in some cases

* add test same output in convert script

* restore wav2vec2conformer and add new model

* create processor and FE + clean

* add new model code

* fix convert script and set default config parameters

* correct model id paths

* make style

* make fix-copies and cleaning files

* fix copied from statements

* complete .md and fixe copies

* clean convert script argument defaults

* fix config parameters docstrings

* fix config docstring

* add copied from and enrich FE tests

* fix copied from and repo-consistency

* add autotokenizer

* make test input length shorter and change docstring code

* fix docstrings and copied from

* add add_adapter to ASR training example

* make testing of adapters more robust

* adapt to multi adapter layers

* refactor input_values->input_features and remove w2v2-bert feature extractor

* remove pretraining model

* remove depreciated features and useless lines

* add copied from and ignore statements to modeling tests

* remove pretraining model #2

* change import in convert script

* change default in convert script

* update readme and remove useless line

* Update tests/models/wav2vec2_bert/test_processor_wav2vec2_bert.py

Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>

* refactor BERT to Bert for consistency

* remove useless ignore copy statement

* add persistent to buffer in rotary

* add eps in LayerNorm init and remove copied from

* add adapter activation parameters and add copied from statements

* Fix copied statements and add unitest.skip reasons

* add copied statement in test_processor

* refactor processor

* make style

* replace numpy random by torch rand

* remove expected output CTC

* improve converting script with processor class

* Apply suggestions from code review

Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>

* remove gumbel class

* remove tests related to previously deleted class

* Update src/transformers/models/wav2vec2_bert/configuration_wav2vec2_bert.py

Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>

* correct typos

* remove uused parameters

* update processor to takes both text and audio

* update checkpoints

* update expected output and add ctc expected output

* add label_attention_mask

* replace pt with np in processor tests

* fix typo

* revert to behaviour with labels_attention_mask

---------

Co-authored-by: Sanchit Gandhi <93869735+sanchit-gandhi@users.noreply.github.com>
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
 2024-01-18 13:37:34 +00:00
hugo-syn
5d8eb93eee
chore: Fix multiple typos (#28574) 2024-01-18 13:35:09 +00:00
regisss
0cdcd7a2b3
Remove task arg in load_dataset in image-classification example (#28408) 
* Remove `task` arg in `load_dataset` in image-classification example

* Manage case where "train" is not in dataset

* Add new args to manage image and label column names

* Similar to audio-classification example

* Fix README

* Update tests
 2024-01-16 08:04:08 +01:00
Timothy Cronin
ff86bc364d
improve dev setup comments and hints (#28495) 
* improve dev setup comments and hints

* fix tests for new dev setup hints
 2024-01-15 18:36:40 +00:00
Yih-Dar
64bdbd888c
Don't set finetuned_from if it is a local path (#28482) 
* fix

* fix

---------

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2024-01-15 11:38:20 +01:00
Joao Gante
4fb3d3a0f6
TF: purge TFTrainer (#28483) 2024-01-12 16:56:34 +00:00
dependabot[bot]
c45ef1c0d1
Bump jinja2 from 2.11.3 to 3.1.3 in /examples/research_projects/decision_transformer (#28457) 
Bump jinja2 in /examples/research_projects/decision_transformer

Bumps [jinja2](https://github.com/pallets/jinja) from 2.11.3 to 3.1.3.
- [Release notes](https://github.com/pallets/jinja/releases)
- [Changelog](https://github.com/pallets/jinja/blob/main/CHANGES.rst)
- [Commits](https://github.com/pallets/jinja/compare/2.11.3...3.1.3)

---
updated-dependencies:
- dependency-name: jinja2
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2024-01-12 15:28:55 +01:00
Alex Hedges
95091e1582
Set cache_dir for evaluate.load() in example scripts (#28422) 
While using `run_clm.py`,[^1] I noticed that some files were being added
to my global cache, not the local cache. I set the `cache_dir` parameter
for the one call to `evaluate.load()`, which partially solved the
problem. I figured that while I was fixing the one script upstream, I
might as well fix the problem in all other example scripts that I could.

There are still some files being added to my global cache, but this
appears to be a bug in `evaluate` itself. This commit at least moves
some of the files into the local cache, which is better than before.

To create this PR, I made the following regex-based transformation:
`evaluate\.load\((.*?)\)` -> `evaluate\.load\($1,
cache_dir=model_args.cache_dir\)`. After using that, I manually fixed
all modified files with `ruff` serving as useful guidance. During the
process, I removed one existing usage of the `cache_dir` parameter in a
script that did not have a corresponding `--cache-dir` argument
declared.

[^1]: I specifically used `pytorch/language-modeling/run_clm.py` from
v4.34.1 of the library. For the original code, see the following URL:
acc394c4f5/examples/pytorch/language-modeling/run_clm.py.
 2024-01-11 15:38:44 +01:00
dependabot[bot]
932ad8af7a
Bump fonttools from 4.31.1 to 4.43.0 in /examples/research_projects/decision_transformer (#28417) 
Bump fonttools in /examples/research_projects/decision_transformer

Bumps [fonttools](https://github.com/fonttools/fonttools) from 4.31.1 to 4.43.0.
- [Release notes](https://github.com/fonttools/fonttools/releases)
- [Changelog](https://github.com/fonttools/fonttools/blob/main/NEWS.rst)
- [Commits](https://github.com/fonttools/fonttools/compare/4.31.1...4.43.0)

---
updated-dependencies:
- dependency-name: fonttools
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2024-01-10 11:22:43 +01:00
Lysandre
3ed3e3190c Dev version 2023-12-13 18:29:31 +01:00
saswatmeher
a49f4acab3
Fix link in README.md of Image Captioning (#27969) 
Update the link for vision encoder decoder doc used by
FlaxVisionEncoderDecoderModel link.
 2023-12-12 08:07:15 -05:00
Adam Louly
4850aaba6f
fix no sequence length models error (#27522) 
* fix no sequence length models error

* block size check

---------

Co-authored-by: Adam Louly <adamlouly@microsoft.com@orttrainingdev9.d32nl1ml4oruzj4qz3bqlggovf.px.internal.cloudapp.net>
 2023-12-11 18:01:26 +00:00
Phuc Van Phan
0410a29a2d
fix: fix gradient accumulate step for learning rate (#27667) 2023-12-07 07:59:26 +01:00
Dave Berenbaum
fe41647afc
uses dvclive_test mode in examples/pytorch/test_accelerate_examples.py (#27763) 2023-11-30 14:52:03 +01:00
Peter Pan
ce31508134
docs: replace torch.distributed.run by torchrun (#27528) 
* docs: replace torch.distributed.run by torchrun

 `transformers` now officially support pytorch >= 1.10.
 The entrypoint `torchrun`` is present from 1.10 onwards.

Signed-off-by: Peter Pan <Peter.Pan@daocloud.io>

* Update src/transformers/trainer.py

with @ArthurZucker's suggestion

Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>

---------

Signed-off-by: Peter Pan <Peter.Pan@daocloud.io>
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>
 2023-11-27 16:26:33 +00:00
Mathias Nielsen
f31af3927f
[ examples] fix loading jsonl with load dataset in run translation example (#26924) 
* Renamed variable extension to builder_name

* If builder name is jsonl change to json to align with load_datasets

* Apply suggestions from code review

Co-authored-by: Quentin Lhoest <42851186+lhoestq@users.noreply.github.com>

---------

Co-authored-by: Quentin Lhoest <42851186+lhoestq@users.noreply.github.com>
 2023-11-20 15:45:42 +01:00
V.Prasanna kumar
ffbcfc0166
Broken links fixed related to datasets docs (#27569) 
fixed the broken links belogs to dataset library of transformers
 2023-11-17 13:44:09 -08:00
Arthur
651408a077
[Styling] stylify using ruff (#27144) 
* try to stylify using ruff

* might need to remove these changes?

* use ruf format andruff check

* use isinstance instead of type comparision

* use # fmt: skip

* use # fmt: skip

* nits

* soem styling changes

* update ci job

* nits isinstance

* more files update

* nits

* more nits

* small nits

* check and format

* revert wrong changes

* actually use formatter instead of checker

* nits

* well docbuilder is overwriting this commit

* revert notebook changes

* try to nuke docbuilder

* style

* fix feature exrtaction test

* remve `indent-width = 4`

* fixup

* more nits

* update the ruff version that we use

* style

* nuke docbuilder styling

* leve the print for detected changes

* nits

* Remove file I/O

Co-authored-by: charliermarsh
 <charlie.r.marsh@gmail.com>

* style

* nits

* revert notebook changes

* Add # fmt skip when possible

* Add # fmt skip when possible

* Fix

* More `  # fmt: skip` usage

* More `  # fmt: skip` usage

* More `  # fmt: skip` usage

* NIts

* more fixes

* fix tapas

* Another way to skip

* Recommended way

* Fix two more fiels

* Remove asynch
Remove asynch

---------

Co-authored-by: charliermarsh <charlie.r.marsh@gmail.com>
 2023-11-16 17:43:19 +01:00
Lucain
fd65aa9818
Set usedforsecurity=False in hashlib methods (FIPS compliance) (#27483) 
* Set usedforsecurity=False in hashlib methods (FIPS compliance)

* trigger ci

* tokenizers version

* deps

* bump hfh version

* let's try this
 2023-11-16 14:29:53 +00:00
Matt
4989e73e2f
Update the TF pin for 2.15 (#27375) 
* Move the TF pin for 2.15

* make fixup
 2023-11-16 13:47:43 +00:00
Phuc Van Phan
69c9b89fcb
docs: add docs for map, and add num procs to load_dataset (#27520) 2023-11-16 13:16:19 +00:00
Matt
2e72bbab2c
Incorrect setting for num_beams in translation and summarization examples (#27519) 
* Remove the torch main_process_first context manager from TF examples

* Correctly set num_beams=1 in our examples, and add a guard in GenerationConfig.validate()

* Update src/transformers/generation/configuration_utils.py

Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>

---------

Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
 2023-11-15 18:18:54 +00:00
Adam Louly
e6522e49a7
Fixing the failure of models without max_position_embeddings attribute. (#27499) 
fix max pos issue

Co-authored-by: Adam Louly <adamlouly@microsoft.com@orttrainingdev9.d32nl1ml4oruzj4qz3bqlggovf.px.internal.cloudapp.net>
 2023-11-15 18:16:42 +00:00
Zach Mueller
a85ea4b19a
Fix wav2vec2 params (#27515) 
Fix test
 2023-11-15 09:24:03 -05:00
Arthur
b97cab7e6d
Remove-auth-token (#27060) 
* don't use `use_auth_token`internally

* let's use token everywhere

* fixup
 2023-11-13 14:20:54 +01:00
Yih-Dar
c8b6052ff6
Final fix of the accelerate installation issue (#27408) 
* fix

* [test-all] commit

* fix

* [test-all] commit

* [test-all] commit

* fix

* fix

* fix

* fix

* fix

---------

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2023-11-09 18:52:29 +01:00
Dave Berenbaum
791ec370d1
Adds dvclive callback (#27352) 
* dvclive trainer callback

* style fixes

* dvclive link fixes
 2023-11-09 12:19:31 +00:00
Zach Mueller
e9adb0c9cf
Change thresh in test (#27378) 
Change thresh
 2023-11-09 04:44:36 -05:00
Zach Mueller
845aa832b7
Remove unused param from example script tests (#27354) 
Unused param
 2023-11-08 09:07:32 -05:00
Zach Mueller
efa57cb234
Fix example tests from failing (#27353) 
* Fix example tests from failing

* CHange thresh
 2023-11-08 07:45:21 -05:00
Hz, Ji
b6dbfee0a2
moving example of benchmarking to legacy dir (#27337) 
move example of benchmarking to legacy
 2023-11-08 09:27:37 +01:00
Lysandre
bc78fd1274 Dev version 2023-11-02 18:15:36 +01:00
Dong-geon Lee
25e6e9418c
Unify warning styles for better readability (#27184) 2023-10-31 18:12:14 +00:00
Hz, Ji
cd19b19378
make tests of pytorch_example device agnostic (#27081) 2023-10-30 14:56:41 +00:00
Gema Parreño
722e936491
[Typo fix] flag config in WANDB (#27130) 
typo fix flag config
 2023-10-29 18:22:26 +00:00
Lucain
66b088faf0
Provide alternative when warning on use_auth_token (#27105) 2023-10-27 14:32:54 +02:00
dependabot[bot]
3c2692407d
Bump urllib3 from 1.26.17 to 1.26.18 in /examples/research_projects/lxmert (#26888) 
Bump urllib3 in /examples/research_projects/lxmert

Bumps [urllib3](https://github.com/urllib3/urllib3) from 1.26.17 to 1.26.18.
- [Release notes](https://github.com/urllib3/urllib3/releases)
- [Changelog](https://github.com/urllib3/urllib3/blob/main/CHANGES.rst)
- [Commits](https://github.com/urllib3/urllib3/compare/1.26.17...1.26.18)

---
updated-dependencies:
- dependency-name: urllib3
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-10-26 09:10:29 +02:00
dependabot[bot]
9c5240af14
Bump werkzeug from 2.2.3 to 3.0.1 in /examples/research_projects/decision_transformer (#27072) 
Bump werkzeug in /examples/research_projects/decision_transformer

Bumps [werkzeug](https://github.com/pallets/werkzeug) from 2.2.3 to 3.0.1.
- [Release notes](https://github.com/pallets/werkzeug/releases)
- [Changelog](https://github.com/pallets/werkzeug/blob/main/CHANGES.rst)
- [Commits](https://github.com/pallets/werkzeug/compare/2.2.3...3.0.1)

---
updated-dependencies:
- dependency-name: werkzeug
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-10-26 08:56:28 +02:00
Michal Jamroz
e2d6d5ce57
Normalize only if needed (#26049) 
* Normalize only if needed

* Update examples/pytorch/image-classification/run_image_classification.py

Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>

* if else in one line

* within block

* one more place, sorry for mess

* import order

* Update examples/pytorch/image-classification/run_image_classification.py

Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>

* Update examples/pytorch/image-classification/run_image_classification_no_trainer.py

Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>

---------

Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
 2023-10-24 13:32:03 +01:00
YQ
f71c9ccf59
fix logit-to-multi-hot conversion in example (#26936) 
* fix logit to multi-hot converstion

* add comments

* typo
 2023-10-23 12:33:05 +02:00
Matt
cbd278f0f6
Pin Keras for now (#26904) 
* Pin Keras for now out of paranoia

* Add the keras pin to _tests_requirements.txt too

* Make sure the Keras version matches the TF one

* make fixup
 2023-10-19 14:39:31 +01:00
dependabot[bot]
bece55d8f9
Bump urllib3 from 1.26.17 to 1.26.18 in /examples/research_projects/decision_transformer (#26889) 
Bump urllib3 in /examples/research_projects/decision_transformer

Bumps [urllib3](https://github.com/urllib3/urllib3) from 1.26.17 to 1.26.18.
- [Release notes](https://github.com/urllib3/urllib3/releases)
- [Changelog](https://github.com/urllib3/urllib3/blob/main/CHANGES.rst)
- [Commits](https://github.com/urllib3/urllib3/compare/1.26.17...1.26.18)

---
updated-dependencies:
- dependency-name: urllib3
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-10-18 13:31:06 +02:00
dependabot[bot]
6d644d6852
Bump urllib3 from 1.26.17 to 1.26.18 in /examples/research_projects/visual_bert (#26890) 
Bump urllib3 in /examples/research_projects/visual_bert

Bumps [urllib3](https://github.com/urllib3/urllib3) from 1.26.17 to 1.26.18.
- [Release notes](https://github.com/urllib3/urllib3/releases)
- [Changelog](https://github.com/urllib3/urllib3/blob/main/CHANGES.rst)
- [Commits](https://github.com/urllib3/urllib3/compare/1.26.17...1.26.18)

---
updated-dependencies:
- dependency-name: urllib3
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-10-18 04:30:50 -07:00
Tom Aarsen
40ea9ab2a1
Add many missing spaces in adjacent strings (#26751) 
Add missing spaces in adjacent strings
 2023-10-12 10:28:40 +02:00
Zach Mueller
1d6a84749b
Fix checkpoint path in no_trainer scripts (#26733) 
checkpoint path
 2023-10-11 16:16:27 +02:00
Roy Hvaara
fc63914399
[JAX] Replace uses of jnp.array in types with jnp.ndarray. (#26703) 
`jnp.array` is a function, not a type:
https://jax.readthedocs.io/en/latest/_autosummary/jax.numpy.array.html
so it never makes sense to use `jnp.array` in a type annotation. Presumably the intent was to write `jnp.ndarray` aka `jax.Array`.

Co-authored-by: Peter Hawkins <phawkins@google.com>
 2023-10-10 21:35:16 +02:00
jheitmann
3eceaa3637
Fix source_prefix default value (#26654) 2023-10-10 20:49:10 +02:00
statelesshz
27597fea07
remove SharedDDP as it is deprecated (#25702) 
* remove SharedDDP as it was drepracated

* apply review suggestion

* make style

* Oops,forgot to remove the compute_loss context manager in Seq2SeqTrainer.

* remove the unnecessary conditional statement

* keep the logic of IPEX

* clean code

* mix precision setup & make fixup

---------

Co-authored-by: statelesshz <jihuazhong1@huawei.com>
 2023-10-06 16:03:11 +02:00
Phuc Van Phan
6015f91a5a
refactor: change default block_size (#26229) 
* refactor: change default block_size

* fix: return tf to origin

* fix: change files to origin

* rebase

* rebase

* rebase

* rebase

* rebase

* rebase

* rebase

* rebase

* refactor: add min block_size to files

* reformat: add min block_size for run_clm tf
 2023-10-04 15:31:38 +01:00
dependabot[bot]
fc296f419e
Bump pillow from 9.3.0 to 10.0.1 in /examples/research_projects/decision_transformer (#26580) 
Bump pillow in /examples/research_projects/decision_transformer

Bumps [pillow](https://github.com/python-pillow/Pillow) from 9.3.0 to 10.0.1.
- [Release notes](https://github.com/python-pillow/Pillow/releases)
- [Changelog](https://github.com/python-pillow/Pillow/blob/main/CHANGES.rst)
- [Commits](https://github.com/python-pillow/Pillow/compare/9.3.0...10.0.1)

---
updated-dependencies:
- dependency-name: pillow
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-10-04 11:52:46 +02:00
Lysandre
bd6205919a v4.35.0.dev0 2023-10-03 16:54:37 +02:00
dependabot[bot]
cf345d5f38
Bump urllib3 from 1.26.9 to 1.26.17 in /examples/research_projects/decision_transformer (#26554) 
Bump urllib3 in /examples/research_projects/decision_transformer

Bumps [urllib3](https://github.com/urllib3/urllib3) from 1.26.9 to 1.26.17.
- [Release notes](https://github.com/urllib3/urllib3/releases)
- [Changelog](https://github.com/urllib3/urllib3/blob/main/CHANGES.rst)
- [Commits](https://github.com/urllib3/urllib3/compare/1.26.9...1.26.17)

---
updated-dependencies:
- dependency-name: urllib3
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-10-03 08:55:12 +02:00
dependabot[bot]
6de6fdd06d
Bump urllib3 from 1.26.5 to 1.26.17 in /examples/research_projects/visual_bert (#26552) 
Bump urllib3 in /examples/research_projects/visual_bert

Bumps [urllib3](https://github.com/urllib3/urllib3) from 1.26.5 to 1.26.17.
- [Release notes](https://github.com/urllib3/urllib3/releases)
- [Changelog](https://github.com/urllib3/urllib3/blob/main/CHANGES.rst)
- [Commits](https://github.com/urllib3/urllib3/compare/1.26.5...1.26.17)

---
updated-dependencies:
- dependency-name: urllib3
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-10-03 08:55:01 +02:00
dependabot[bot]
e092b4ad68
Bump urllib3 from 1.26.5 to 1.26.17 in /examples/research_projects/lxmert (#26551) 
Bump urllib3 in /examples/research_projects/lxmert

Bumps [urllib3](https://github.com/urllib3/urllib3) from 1.26.5 to 1.26.17.
- [Release notes](https://github.com/urllib3/urllib3/releases)
- [Changelog](https://github.com/urllib3/urllib3/blob/main/CHANGES.rst)
- [Commits](https://github.com/urllib3/urllib3/compare/1.26.5...1.26.17)

---
updated-dependencies:
- dependency-name: urllib3
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-10-03 08:54:50 +02:00
Sanchit Gandhi
68e85fc822
[Flax Examples] Seq2Seq ASR Fine-Tuning Script (#21764) 
* from seq2seq speech

* [Flax] Example script for speech seq2seq

* tests and fixes

* make style

* fix: label padding tokens

* fix: label padding tokens over list

* update ln names for Whisper

* try datasets iter loader

* create readme and append results

* style

* make style

* adjust lr

* use pt dataloader

* make fast

* pin gen max len

* finish

* add pt to requirements for test

* fix pt -> torch

* add accelerate
 2023-09-29 16:42:58 +01:00
Phuc Van Phan
ba47efbfe4
docs: change assert to raise and some small docs (#26232) 
* docs: change assert to raise and some small docs

* docs: add rule and some document

* fix: fix bug

* fix: fix bug

* chorse: revert logging

* chorse: revert
 2023-09-28 10:14:17 +02:00
Phuc Van Phan
910faa3e1f
feat: adding num_proc to load_dataset (#26326) 
* feat: adding num_proc to load_dataset

* feat: add add_num_proc for run_mlm_flax

* feat: add num_proc for bart and t5

* chorse: remove
 2023-09-22 19:22:47 +02:00
Nino Risteski
7d4e0c23c8
Update README.md (#26198) 
Fixed a few typos
 2023-09-19 00:02:50 +02:00
Phuc Van Phan
8b5da9fc6e
refactor: change default block_size in block size > max position embeddings (#26069) 
* refactor: change default block_size when not initialize

* reformat: add the min of block size
 2023-09-18 16:47:57 +01:00
Phuc Van Phan
4fb64e285a
chore: correct update_step and correct gradient_accumulation_steps (#26068) 2023-09-12 18:31:23 +01:00
Phuc Van Phan
5af2c62696
docs: add space to docs (#26067) 
* docs: add space to docs

* docs: remove reduntant space
 2023-09-11 22:03:26 +01:00
Phuc Van Phan
9cebae64ad
docs: update link huggingface map (#26077) 2023-09-11 12:57:04 +01:00
Nino Risteski
f6295c6c53
Update README.md (#26003) 
fixed a typo
 2023-09-06 10:55:11 +01:00
Joao Gante
9a70d6e56f
Trainer: delegate default generation values to generation_config (#25987) 2023-09-05 14:47:00 +01:00
Traun Leyden
7011cd8667
Update RAG README.md with correct path to examples/seq2seq (#25953) 
Update README.md with correct path to examples/seq2seq
 2023-09-05 12:31:59 +01:00
Susnato Dhar
404ff8fc17
Fix typo (#25966) 
* Update feature_extraction_clap.py

* changed all lenght to length
 2023-09-05 10:12:25 +02:00
Lysandre
d8e13b3e04 v4.34.dev.0 2023-09-04 15:12:11 -04:00
Zach Mueller
be0e189bd3
Revert frozen training arguments (#25903) 
* Revert frozen training arguments

* TODO
 2023-09-01 11:24:12 -04:00
Younes Belkada
4b79697865
🚨🚨🚨 [Refactor] Move third-party related utility files into integrations/ folder 🚨🚨🚨 (#25599) 
* move deepspeed to `lib_integrations.deepspeed`

* more refactor

* oops

* fix slow tests

* Fix docs

* fix docs

* addess feedback

* address feedback

* final modifs for PEFT

* fixup

* ok now

* trigger CI

* trigger CI again

* Update docs/source/en/main_classes/deepspeed.md

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* import from `integrations`

* address feedback

* revert removal of `deepspeed` module

* revert removal of `deepspeed` module

* fix conflicts

* ooops

* oops

* add deprecation warning

* place it on the top

* put `FutureWarning`

* fix conflicts with not_doctested.txt

* add back `bitsandbytes` module with a depr warning

* fix

* fix

* fixup

* oops

* fix doctests

---------

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
 2023-08-25 17:13:34 +02:00
Phuc Van Phan
656e17f6f7
correct resume training steps number in progress bar (#25691) 
feat: correct update resume update with steps
 2023-08-23 20:09:14 +02:00
Matt
62396cff46
TF 2.14 compatibility (#25630) 
* Update the TF pin and see if anything breaks

* make fixup

* make fixup

* make fixup
 2023-08-22 13:13:38 +01:00
Sylvain Gugger
5c67682b16
v4.33.0.dev0 2023-08-21 07:07:04 -04:00
Zach Mueller
ca51499248
Make training args fully immutable (#25435) 
* Make training args fully immutable

* Working tests, PyTorch

* In test_trainer

* during testing

* Use proper dataclass way

* Fix test

* Another one

* Fix tf

* Lingering slow

* Exception

* Clean
 2023-08-15 11:47:47 -04:00
dependabot[bot]
7a94ea4c64
Bump tornado from 6.3.2 to 6.3.3 in /examples/research_projects/lxmert (#25511) 
Bumps [tornado](https://github.com/tornadoweb/tornado) from 6.3.2 to 6.3.3.
- [Changelog](https://github.com/tornadoweb/tornado/blob/master/docs/releases.rst)
- [Commits](https://github.com/tornadoweb/tornado/compare/v6.3.2...v6.3.3)

---
updated-dependencies:
- dependency-name: tornado
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-08-15 08:52:30 +02:00
dependabot[bot]
2552b8c5bd
Bump tornado from 6.3.2 to 6.3.3 in /examples/research_projects/visual_bert (#25512) 
Bump tornado in /examples/research_projects/visual_bert

Bumps [tornado](https://github.com/tornadoweb/tornado) from 6.3.2 to 6.3.3.
- [Changelog](https://github.com/tornadoweb/tornado/blob/master/docs/releases.rst)
- [Commits](https://github.com/tornadoweb/tornado/compare/v6.3.2...v6.3.3)

---
updated-dependencies:
- dependency-name: tornado
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-08-15 08:52:20 +02:00
dependabot[bot]
0ebe7ae160
Bump gitpython from 3.1.30 to 3.1.32 in /examples/research_projects/decision_transformer (#25467) 
Bump gitpython in /examples/research_projects/decision_transformer

Bumps [gitpython](https://github.com/gitpython-developers/GitPython) from 3.1.30 to 3.1.32.
- [Release notes](https://github.com/gitpython-developers/GitPython/releases)
- [Changelog](https://github.com/gitpython-developers/GitPython/blob/main/CHANGES)
- [Commits](https://github.com/gitpython-developers/GitPython/compare/3.1.30...3.1.32)

---
updated-dependencies:
- dependency-name: gitpython
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-08-13 19:47:16 +02:00
dependabot[bot]
2b22cde71e
Bump gitpython from 3.1.30 to 3.1.32 in /examples/research_projects/distillation (#25468) 
Bump gitpython in /examples/research_projects/distillation

Bumps [gitpython](https://github.com/gitpython-developers/GitPython) from 3.1.30 to 3.1.32.
- [Release notes](https://github.com/gitpython-developers/GitPython/releases)
- [Changelog](https://github.com/gitpython-developers/GitPython/blob/main/CHANGES)
- [Commits](https://github.com/gitpython-developers/GitPython/compare/3.1.30...3.1.32)

---
updated-dependencies:
- dependency-name: gitpython
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-08-13 19:47:04 +02:00
Gema Parreño
0acf56224b
Update run_translation.py broken link example Pytoch (#25461) 
* Update run_translation.py

Fixed link

* Update run_translation.py
 2023-08-11 15:41:24 +02:00
Yih-Dar
9c7b744795
Fix missing usage of token (#25382) 
* add missing tokens

* fix

---------

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2023-08-08 16:27:24 +02:00
Zach Mueller
01ab39b65f
Load state in else (#25318) 
* Load else

* New approach

* Propagate
 2023-08-08 05:41:00 -04:00
Phuc Van Phan
5fe36970e5
Adding more information in help parser on train_file and validation_file (#25324) 
chorse: adding new doc on train and val
 2023-08-07 17:56:13 +02:00
Jackmin801
145109382a
Allow trust_remote_code in example scripts (#25248) 
* pytorch examples

* pytorch mim no trainer

* cookiecutter

* flax examples

* missed line in pytorch run_glue

* tensorflow examples

* tensorflow run_clip

* tensorflow run_mlm

* tensorflow run_ner

* tensorflow run_clm

* pytorch example from_configs

* pytorch no trainer examples

* Revert "tensorflow run_clip"

This reverts commit 261f86ac1f.

* fix: duplicated argument
 2023-08-07 16:32:25 +02:00
Yih-Dar
149cb0cce2
Add token arugment in example scripts (#25172) 
* fix

* fix

* fix

* fix

* fix

* fix

* fix

---------

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2023-08-02 11:17:31 +02:00
Yih-Dar
d53b8ad780
Update use_auth_token -> token in example scripts (#25167) 
* pytorch examples

* tensorflow examples

* flax examples

---------

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2023-07-28 15:33:45 +02:00
Alan Ji
afa96fffdf
make run_generation more generic for other devices (#25133) 
* make run_generation more generic for other devices

* use Accelerate to support any device type it supports.

* make style

* fix error usage of accelerator.prepare_model

* use `PartialState` to make sure everything is running on the right device

---------

Co-authored-by: statelesshz <jihuazhong1@huawei.com>
 2023-07-28 08:20:10 -04:00
Lucain
6232c380f2
Fix .push_to_hub and cleanup get_full_repo_name usage (#25120) 
* Fix .push_to_hub and cleanup get_full_repo_name usage

* Do not rely on Python bool conversion magic

* request changes
 2023-07-28 11:40:08 +02:00
dependabot[bot]
f1deb21fce
Bump certifi from 2022.12.7 to 2023.7.22 in /examples/research_projects/visual_bert (#25097) 
Bump certifi in /examples/research_projects/visual_bert

Bumps [certifi](https://github.com/certifi/python-certifi) from 2022.12.7 to 2023.7.22.
- [Commits](https://github.com/certifi/python-certifi/compare/2022.12.07...2023.07.22)

---
updated-dependencies:
- dependency-name: certifi
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-07-25 17:25:14 -04:00
dependabot[bot]
45bde362d2
Bump certifi from 2022.12.7 to 2023.7.22 in /examples/research_projects/decision_transformer (#25098) 
Bump certifi in /examples/research_projects/decision_transformer

Bumps [certifi](https://github.com/certifi/python-certifi) from 2022.12.7 to 2023.7.22.
- [Commits](https://github.com/certifi/python-certifi/compare/2022.12.07...2023.07.22)

---
updated-dependencies:
- dependency-name: certifi
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-07-25 17:25:05 -04:00
dependabot[bot]
6b8dbc283c
Bump certifi from 2022.12.7 to 2023.7.22 in /examples/research_projects/lxmert (#25096) 
Bump certifi in /examples/research_projects/lxmert

Bumps [certifi](https://github.com/certifi/python-certifi) from 2022.12.7 to 2023.7.22.
- [Commits](https://github.com/certifi/python-certifi/compare/2022.12.07...2023.07.22)

---
updated-dependencies:
- dependency-name: certifi
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-07-25 17:24:50 -04:00
Alan Ji
c879318cc5
replace per_gpu_eval_batch_size with per_device_eval_batch_size in readme of multiple-choice task (#25078) 
replace `per_gpu_eval_batch_size` with `per_device_eval_batch_size`
in readme of multiple-choice
 2023-07-25 08:11:56 -04:00
Zach Mueller
aa1b09c5d1
Change logic for logging in the examples (#24956) 
Change logic
 2023-07-20 12:30:10 -04:00
dependabot[bot]
9f912ef62a
Bump aiohttp from 3.8.1 to 3.8.5 in /examples/research_projects/decision_transformer (#24954) 
Bump aiohttp in /examples/research_projects/decision_transformer

Bumps [aiohttp](https://github.com/aio-libs/aiohttp) from 3.8.1 to 3.8.5.
- [Release notes](https://github.com/aio-libs/aiohttp/releases)
- [Changelog](https://github.com/aio-libs/aiohttp/blob/v3.8.5/CHANGES.rst)
- [Commits](https://github.com/aio-libs/aiohttp/compare/v3.8.1...v3.8.5)

---
updated-dependencies:
- dependency-name: aiohttp
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-07-20 12:17:38 -04:00
dependabot[bot]
9859806608
Bump pygments from 2.11.2 to 2.15.0 in /examples/research_projects/decision_transformer (#24949) 
Bump pygments in /examples/research_projects/decision_transformer

Bumps [pygments](https://github.com/pygments/pygments) from 2.11.2 to 2.15.0.
- [Release notes](https://github.com/pygments/pygments/releases)
- [Changelog](https://github.com/pygments/pygments/blob/master/CHANGES)
- [Commits](https://github.com/pygments/pygments/compare/2.11.2...2.15.0)

---
updated-dependencies:
- dependency-name: pygments
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-07-20 07:43:48 -04:00
statelesshz
37d8611ac9
replace no_cuda with use_cpu in test_pytorch_examples (#24944) 
* replace no_cuda with use_cpu in test_pytorch_examples

* remove codes that never be used

* fix style
 2023-07-20 07:09:04 -04:00
ranchlai
8fd8c8e49e
Add multi-label text classification support to pytorch example (#24770) 
* Add text classification example

* set the problem type and finetuning task

* ruff reformated

* fix bug for unseting label_to_id for regression

* update README.md

* fixed finetuning task

* update comment

* check if label exists in feature before removing

* add useful logging
 2023-07-20 07:02:44 -04:00
Hwijeen Ahn
dd49404a89
check if eval dataset is dict (#24877) 
* check if eval dataset is dict

* formatting
 2023-07-18 13:33:41 -04:00
Sylvain Gugger
e9ad51306f
4.32.0.dev0 2023-07-17 13:30:44 -04:00
dependabot[bot]
38dfb86958
Bump cryptography from 41.0.0 to 41.0.2 in /examples/research_projects/decision_transformer (#24833) 
Bump cryptography in /examples/research_projects/decision_transformer

Bumps [cryptography](https://github.com/pyca/cryptography) from 41.0.0 to 41.0.2.
- [Changelog](https://github.com/pyca/cryptography/blob/main/CHANGELOG.rst)
- [Commits](https://github.com/pyca/cryptography/compare/41.0.0...41.0.2)

---
updated-dependencies:
- dependency-name: cryptography
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-07-17 07:17:17 -04:00
Bauke Brenninkmeijer
fc9e387dc0
Replacement of 20 asserts with exceptions (#24757) 
* initial replacements of asserts with errors/exceptions

* replace assert with exception in generation, align and bart

* reset formatting change

* reset another formatting issue

* Apply suggestion

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* don't touch this file

* change to 'is not False'

* fix type

---------

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
 2023-07-12 07:45:09 -04:00
Gema Parreño
4b26a61631
Fix loading dataset docs link in run_translation.py example (#24594) 
* fix loading dataset link

* Update examples/tensorflow/translation/run_translation.py

Co-authored-by: Matt <Rocketknight1@users.noreply.github.com>

* Update examples/tensorflow/translation/run_translation.py

Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>

---------

Co-authored-by: Matt <Rocketknight1@users.noreply.github.com>
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
 2023-07-03 15:21:21 +01:00
Jeroen Van Goey
fd8dcd0953
Udate link to RunHouse hardware setup documentation. (#24590) 
* Udate link to RunHouse hardware setup documentation.

* Fix link to hardware setup in other location as well
 2023-06-30 12:11:58 +01:00
Xiaoli Wang
239ace152b
Fix TypeError: Object of type int64 is not JSON serializable (#24340) 
* Fix TypeError: Object of type int64 is not JSON serializable

* Convert numpy.float64 and numpy.int64 to float and int for json serialization

* Black reformatted examples/pytorch/token-classification/run_ner_no_trainer.py

* * make style
 2023-06-27 12:15:49 +01:00
Matt
8e164c5400
Improved keras imports (#24448) 
* An end to accursed version-specific imports

* No more K.is_keras_tensor() either

* Update dependency tables

* Use a cleaner call context function getter

* Add a cap to <2.14

* Add cap to examples requirements too
 2023-06-23 19:09:34 +01:00
Patrick von Platen
1609a436ec
Add MMS CTC Fine-Tuning (#24281) 
* Add mms ctc fine tuning

* make style

* More fixes that are needed

* make fix-copies

* make draft for README

* add new file

* move to new file

* make style

* make style

* add quick test

* make style

* make style
 2023-06-15 01:10:27 +02:00
Ethan
f7d80cb3d2
Fix steps bugs in no trainer examples (#24197) 
Fix step bugs in no trainer + load checkpoint + grad acc
 2023-06-12 11:49:55 -04:00
Sylvain Gugger
ba695c1efd
v4.31.0.dev0 2023-06-07 16:49:00 -04:00
Zachary Mueller
cbf6bc2350
Oops, missed one (#24054) 
Oops
 2023-06-06 13:30:19 -04:00
Zachary Mueller
072188d638
Act on deprecations in Accelerate no_trainer examples (#24053) 
Act on deprecation
 2023-06-06 13:04:38 -04:00
dependabot[bot]
539e2281cd
Bump cryptography from 39.0.1 to 41.0.0 in /examples/research_projects/decision_transformer (#23964) 
Bump cryptography in /examples/research_projects/decision_transformer

Bumps [cryptography](https://github.com/pyca/cryptography) from 39.0.1 to 41.0.0.
- [Changelog](https://github.com/pyca/cryptography/blob/main/CHANGELOG.rst)
- [Commits](https://github.com/pyca/cryptography/compare/39.0.1...41.0.0)

---
updated-dependencies:
- dependency-name: cryptography
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-06-02 16:23:44 -04:00
Matt
167a0d8f87
Add an option to reduce compile() console spam (#23938) 
* Add an option to reduce compile() console spam

* Add annotations to the example scripts

* Add notes to the quicktour docs as well

* minor fix
 2023-06-02 15:28:52 +01:00
Sylvain Gugger
3ff443a6d9
Re-enable squad test (#23912) 
* Re-enable squad test

* [all-test]

* [all-test] Fix all test command

* Fix the all-test
 2023-05-31 13:44:26 -04:00
Sylvain Gugger
00f6ba0e7e
Skip failing test for now 2023-05-31 06:31:33 -04:00
Ran Ran
e724246935
Fix no such file or directory error (#23783) 
* Fix no such file or directory error

* Address comment

* Fix formatting issue
 2023-05-26 14:24:57 -04:00
dependabot[bot]
d685e330b5
Bump tornado from 6.0.4 to 6.3.2 in /examples/research_projects/visual_bert (#23767) 
Bump tornado in /examples/research_projects/visual_bert

Bumps [tornado](https://github.com/tornadoweb/tornado) from 6.0.4 to 6.3.2.
- [Changelog](https://github.com/tornadoweb/tornado/blob/master/docs/releases.rst)
- [Commits](https://github.com/tornadoweb/tornado/compare/v6.0.4...v6.3.2)

---
updated-dependencies:
- dependency-name: tornado
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-05-25 16:16:12 -04:00
dependabot[bot]
4b0e7ded1c
Bump tornado from 6.0.4 to 6.3.2 in /examples/research_projects/lxmert (#23766) 
Bumps [tornado](https://github.com/tornadoweb/tornado) from 6.0.4 to 6.3.2.
- [Changelog](https://github.com/tornadoweb/tornado/blob/master/docs/releases.rst)
- [Commits](https://github.com/tornadoweb/tornado/compare/v6.0.4...v6.3.2)

---
updated-dependencies:
- dependency-name: tornado
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-05-25 16:16:01 -04:00
Sylvain Gugger
6e4bc67099
Revamp test selection for the example tests (#23737) 
* Revamp test selection for the example tests

* Rename old XLA test and fake modif in run_glue

* Fixes

* Fake Trainer modif

* Remove fake modifs
 2023-05-25 09:38:21 -04:00
Wang, Yi
33687a3f61
add GPTJ/bloom/llama/opt into model list and enhance the jit support (#23291) 
Signed-off-by: Wang, Yi A <yi.a.wang@intel.com>
 2023-05-24 10:57:56 +01:00
zspo
003a0cf8cc
Fix some docs what layerdrop does (#23691) 
* Fix some docs what layerdrop does

* Update src/transformers/models/data2vec/configuration_data2vec_audio.py

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* Fix more docs

---------

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
 2023-05-23 14:50:40 -04:00
dependabot[bot]
aa30cd4f3f
Bump requests from 2.22.0 to 2.31.0 in /examples/research_projects/lxmert (#23668) 
Bump requests in /examples/research_projects/lxmert

Bumps [requests](https://github.com/psf/requests) from 2.22.0 to 2.31.0.
- [Release notes](https://github.com/psf/requests/releases)
- [Changelog](https://github.com/psf/requests/blob/main/HISTORY.md)
- [Commits](https://github.com/psf/requests/compare/v2.22.0...v2.31.0)

---
updated-dependencies:
- dependency-name: requests
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-05-23 05:31:53 -04:00
dependabot[bot]
9bf72ae564
Bump requests from 2.22.0 to 2.31.0 in /examples/research_projects/visual_bert (#23670) 
Bump requests in /examples/research_projects/visual_bert

Bumps [requests](https://github.com/psf/requests) from 2.22.0 to 2.31.0.
- [Release notes](https://github.com/psf/requests/releases)
- [Changelog](https://github.com/psf/requests/blob/main/HISTORY.md)
- [Commits](https://github.com/psf/requests/compare/v2.22.0...v2.31.0)

---
updated-dependencies:
- dependency-name: requests
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-05-23 05:31:30 -04:00
dependabot[bot]
ecc05f8c1e
Bump requests from 2.27.1 to 2.31.0 in /examples/research_projects/decision_transformer (#23673) 
Bump requests in /examples/research_projects/decision_transformer

Bumps [requests](https://github.com/psf/requests) from 2.27.1 to 2.31.0.
- [Release notes](https://github.com/psf/requests/releases)
- [Changelog](https://github.com/psf/requests/blob/main/HISTORY.md)
- [Commits](https://github.com/psf/requests/compare/v2.27.1...v2.31.0)

---
updated-dependencies:
- dependency-name: requests
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-05-23 05:28:09 -04:00
Zachary Mueller
b191d7db44
Update all no_trainer with skip_first_batches (#23664) 2023-05-22 14:49:31 -04:00
sshahrokhi
6f72e71f97
changing the requirements to a cpu torch version that works (#23483) 2023-05-22 12:58:55 -04:00
Boda Sadallah
a7920065f2
fix bug in group_texts function, that was inserting short batches (#23429) 
* fix bug in group_texts function, that was inserting short batches

* fully exclude short batches and return empty dict instead

* fix style
 2023-05-18 14:22:30 -04:00
Zachary Mueller
8a58809312
Fix translation no_trainer (#23407) 
* Fix translation
 2023-05-16 13:10:42 -04:00
Yih-Dar
d51296d9c2
skip test_run_squad_no_trainer for now (#23302) 
skip

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2023-05-11 19:26:48 +02:00
Hari
5d02e6bd20
Convert numpy arrays to lists before saving the evaluation metrics as json (#23268) 
* convert numpy array to list before writing to json

per_category_iou and per_category_accuracy  are ndarray in the eval_metrics

* code reformatted with make style
 2023-05-11 08:54:23 -04:00
Maria Khalusova
91f4c84a19
CTC example: updated trainer parameters to save tokenizer (#23243) 
trainer parameters changed to save tokenizer in addition to feature_extractor
 2023-05-10 07:45:10 -04:00
Sylvain Gugger
a0c0a78233
v4.30.0.dev0 2023-05-09 14:59:38 -04:00
Nicolas Patry
c34a525d2f
Proposed fix for TF example now running on safetensors. (#23208) 
* Proposed fix for TF example now running on safetensors.

* Adding more warnings and returning keys.

* Trigger CI

* Trigger CI

---------

Co-authored-by: Sylvain Gugger <Sylvain.gugger@gmail.com>
 2023-05-09 13:04:27 -04:00
Sebastian
1a8f61110e
fix: Update run_qa.py to work with deepset/germanquad (#23225) 
Call str on id to make sure any ints are converted into the expected format for squad datasets
 2023-05-09 09:20:10 -04:00
Sylvain Gugger
fd6970bc56
Skip failing test 2023-05-08 08:52:44 -04:00