riaz.somc/transformers
1
0
Fork 0
mirror of https://github.com/huggingface/transformers.git synced 2025-07-24 23:08:57 +06:00
Code Issues Actions 32 Packages Projects Releases Wiki Activity
13,875 Commits 887 Branches 229 Tags 2.1 GiB
3587769c08
Commit Graph

2300 Commits

Author SHA1 Message Date
Zach Mueller
be0e189bd3
Revert frozen training arguments (#25903) 
* Revert frozen training arguments

* TODO
 2023-09-01 11:24:12 -04:00
Younes Belkada
4b79697865
🚨🚨🚨 [Refactor] Move third-party related utility files into integrations/ folder 🚨🚨🚨 (#25599) 
* move deepspeed to `lib_integrations.deepspeed`

* more refactor

* oops

* fix slow tests

* Fix docs

* fix docs

* addess feedback

* address feedback

* final modifs for PEFT

* fixup

* ok now

* trigger CI

* trigger CI again

* Update docs/source/en/main_classes/deepspeed.md

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* import from `integrations`

* address feedback

* revert removal of `deepspeed` module

* revert removal of `deepspeed` module

* fix conflicts

* ooops

* oops

* add deprecation warning

* place it on the top

* put `FutureWarning`

* fix conflicts with not_doctested.txt

* add back `bitsandbytes` module with a depr warning

* fix

* fix

* fixup

* oops

* fix doctests

---------

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
 2023-08-25 17:13:34 +02:00
Phuc Van Phan
656e17f6f7
correct resume training steps number in progress bar (#25691) 
feat: correct update resume update with steps
 2023-08-23 20:09:14 +02:00
Matt
62396cff46
TF 2.14 compatibility (#25630) 
* Update the TF pin and see if anything breaks

* make fixup

* make fixup

* make fixup
 2023-08-22 13:13:38 +01:00
Sylvain Gugger
5c67682b16
v4.33.0.dev0 2023-08-21 07:07:04 -04:00
Zach Mueller
ca51499248
Make training args fully immutable (#25435) 
* Make training args fully immutable

* Working tests, PyTorch

* In test_trainer

* during testing

* Use proper dataclass way

* Fix test

* Another one

* Fix tf

* Lingering slow

* Exception

* Clean
 2023-08-15 11:47:47 -04:00
dependabot[bot]
7a94ea4c64
Bump tornado from 6.3.2 to 6.3.3 in /examples/research_projects/lxmert (#25511) 
Bumps [tornado](https://github.com/tornadoweb/tornado) from 6.3.2 to 6.3.3.
- [Changelog](https://github.com/tornadoweb/tornado/blob/master/docs/releases.rst)
- [Commits](https://github.com/tornadoweb/tornado/compare/v6.3.2...v6.3.3)

---
updated-dependencies:
- dependency-name: tornado
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-08-15 08:52:30 +02:00
dependabot[bot]
2552b8c5bd
Bump tornado from 6.3.2 to 6.3.3 in /examples/research_projects/visual_bert (#25512) 
Bump tornado in /examples/research_projects/visual_bert

Bumps [tornado](https://github.com/tornadoweb/tornado) from 6.3.2 to 6.3.3.
- [Changelog](https://github.com/tornadoweb/tornado/blob/master/docs/releases.rst)
- [Commits](https://github.com/tornadoweb/tornado/compare/v6.3.2...v6.3.3)

---
updated-dependencies:
- dependency-name: tornado
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-08-15 08:52:20 +02:00
dependabot[bot]
0ebe7ae160
Bump gitpython from 3.1.30 to 3.1.32 in /examples/research_projects/decision_transformer (#25467) 
Bump gitpython in /examples/research_projects/decision_transformer

Bumps [gitpython](https://github.com/gitpython-developers/GitPython) from 3.1.30 to 3.1.32.
- [Release notes](https://github.com/gitpython-developers/GitPython/releases)
- [Changelog](https://github.com/gitpython-developers/GitPython/blob/main/CHANGES)
- [Commits](https://github.com/gitpython-developers/GitPython/compare/3.1.30...3.1.32)

---
updated-dependencies:
- dependency-name: gitpython
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-08-13 19:47:16 +02:00
dependabot[bot]
2b22cde71e
Bump gitpython from 3.1.30 to 3.1.32 in /examples/research_projects/distillation (#25468) 
Bump gitpython in /examples/research_projects/distillation

Bumps [gitpython](https://github.com/gitpython-developers/GitPython) from 3.1.30 to 3.1.32.
- [Release notes](https://github.com/gitpython-developers/GitPython/releases)
- [Changelog](https://github.com/gitpython-developers/GitPython/blob/main/CHANGES)
- [Commits](https://github.com/gitpython-developers/GitPython/compare/3.1.30...3.1.32)

---
updated-dependencies:
- dependency-name: gitpython
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-08-13 19:47:04 +02:00
Gema Parreño
0acf56224b
Update run_translation.py broken link example Pytoch (#25461) 
* Update run_translation.py

Fixed link

* Update run_translation.py
 2023-08-11 15:41:24 +02:00
Yih-Dar
9c7b744795
Fix missing usage of token (#25382) 
* add missing tokens

* fix

---------

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2023-08-08 16:27:24 +02:00
Zach Mueller
01ab39b65f
Load state in else (#25318) 
* Load else

* New approach

* Propagate
 2023-08-08 05:41:00 -04:00
Phuc Van Phan
5fe36970e5
Adding more information in help parser on train_file and validation_file (#25324) 
chorse: adding new doc on train and val
 2023-08-07 17:56:13 +02:00
Jackmin801
145109382a
Allow trust_remote_code in example scripts (#25248) 
* pytorch examples

* pytorch mim no trainer

* cookiecutter

* flax examples

* missed line in pytorch run_glue

* tensorflow examples

* tensorflow run_clip

* tensorflow run_mlm

* tensorflow run_ner

* tensorflow run_clm

* pytorch example from_configs

* pytorch no trainer examples

* Revert "tensorflow run_clip"

This reverts commit 261f86ac1f.

* fix: duplicated argument
 2023-08-07 16:32:25 +02:00
Yih-Dar
149cb0cce2
Add token arugment in example scripts (#25172) 
* fix

* fix

* fix

* fix

* fix

* fix

* fix

---------

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2023-08-02 11:17:31 +02:00
Yih-Dar
d53b8ad780
Update use_auth_token -> token in example scripts (#25167) 
* pytorch examples

* tensorflow examples

* flax examples

---------

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2023-07-28 15:33:45 +02:00
Alan Ji
afa96fffdf
make run_generation more generic for other devices (#25133) 
* make run_generation more generic for other devices

* use Accelerate to support any device type it supports.

* make style

* fix error usage of accelerator.prepare_model

* use `PartialState` to make sure everything is running on the right device

---------

Co-authored-by: statelesshz <jihuazhong1@huawei.com>
 2023-07-28 08:20:10 -04:00
Lucain
6232c380f2
Fix .push_to_hub and cleanup get_full_repo_name usage (#25120) 
* Fix .push_to_hub and cleanup get_full_repo_name usage

* Do not rely on Python bool conversion magic

* request changes
 2023-07-28 11:40:08 +02:00
dependabot[bot]
f1deb21fce
Bump certifi from 2022.12.7 to 2023.7.22 in /examples/research_projects/visual_bert (#25097) 
Bump certifi in /examples/research_projects/visual_bert

Bumps [certifi](https://github.com/certifi/python-certifi) from 2022.12.7 to 2023.7.22.
- [Commits](https://github.com/certifi/python-certifi/compare/2022.12.07...2023.07.22)

---
updated-dependencies:
- dependency-name: certifi
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-07-25 17:25:14 -04:00
dependabot[bot]
45bde362d2
Bump certifi from 2022.12.7 to 2023.7.22 in /examples/research_projects/decision_transformer (#25098) 
Bump certifi in /examples/research_projects/decision_transformer

Bumps [certifi](https://github.com/certifi/python-certifi) from 2022.12.7 to 2023.7.22.
- [Commits](https://github.com/certifi/python-certifi/compare/2022.12.07...2023.07.22)

---
updated-dependencies:
- dependency-name: certifi
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-07-25 17:25:05 -04:00
dependabot[bot]
6b8dbc283c
Bump certifi from 2022.12.7 to 2023.7.22 in /examples/research_projects/lxmert (#25096) 
Bump certifi in /examples/research_projects/lxmert

Bumps [certifi](https://github.com/certifi/python-certifi) from 2022.12.7 to 2023.7.22.
- [Commits](https://github.com/certifi/python-certifi/compare/2022.12.07...2023.07.22)

---
updated-dependencies:
- dependency-name: certifi
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-07-25 17:24:50 -04:00
Alan Ji
c879318cc5
replace per_gpu_eval_batch_size with per_device_eval_batch_size in readme of multiple-choice task (#25078) 
replace `per_gpu_eval_batch_size` with `per_device_eval_batch_size`
in readme of multiple-choice
 2023-07-25 08:11:56 -04:00
Zach Mueller
aa1b09c5d1
Change logic for logging in the examples (#24956) 
Change logic
 2023-07-20 12:30:10 -04:00
dependabot[bot]
9f912ef62a
Bump aiohttp from 3.8.1 to 3.8.5 in /examples/research_projects/decision_transformer (#24954) 
Bump aiohttp in /examples/research_projects/decision_transformer

Bumps [aiohttp](https://github.com/aio-libs/aiohttp) from 3.8.1 to 3.8.5.
- [Release notes](https://github.com/aio-libs/aiohttp/releases)
- [Changelog](https://github.com/aio-libs/aiohttp/blob/v3.8.5/CHANGES.rst)
- [Commits](https://github.com/aio-libs/aiohttp/compare/v3.8.1...v3.8.5)

---
updated-dependencies:
- dependency-name: aiohttp
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-07-20 12:17:38 -04:00
dependabot[bot]
9859806608
Bump pygments from 2.11.2 to 2.15.0 in /examples/research_projects/decision_transformer (#24949) 
Bump pygments in /examples/research_projects/decision_transformer

Bumps [pygments](https://github.com/pygments/pygments) from 2.11.2 to 2.15.0.
- [Release notes](https://github.com/pygments/pygments/releases)
- [Changelog](https://github.com/pygments/pygments/blob/master/CHANGES)
- [Commits](https://github.com/pygments/pygments/compare/2.11.2...2.15.0)

---
updated-dependencies:
- dependency-name: pygments
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-07-20 07:43:48 -04:00
statelesshz
37d8611ac9
replace no_cuda with use_cpu in test_pytorch_examples (#24944) 
* replace no_cuda with use_cpu in test_pytorch_examples

* remove codes that never be used

* fix style
 2023-07-20 07:09:04 -04:00
ranchlai
8fd8c8e49e
Add multi-label text classification support to pytorch example (#24770) 
* Add text classification example

* set the problem type and finetuning task

* ruff reformated

* fix bug for unseting label_to_id for regression

* update README.md

* fixed finetuning task

* update comment

* check if label exists in feature before removing

* add useful logging
 2023-07-20 07:02:44 -04:00
Hwijeen Ahn
dd49404a89
check if eval dataset is dict (#24877) 
* check if eval dataset is dict

* formatting
 2023-07-18 13:33:41 -04:00
Sylvain Gugger
e9ad51306f
4.32.0.dev0 2023-07-17 13:30:44 -04:00
dependabot[bot]
38dfb86958
Bump cryptography from 41.0.0 to 41.0.2 in /examples/research_projects/decision_transformer (#24833) 
Bump cryptography in /examples/research_projects/decision_transformer

Bumps [cryptography](https://github.com/pyca/cryptography) from 41.0.0 to 41.0.2.
- [Changelog](https://github.com/pyca/cryptography/blob/main/CHANGELOG.rst)
- [Commits](https://github.com/pyca/cryptography/compare/41.0.0...41.0.2)

---
updated-dependencies:
- dependency-name: cryptography
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-07-17 07:17:17 -04:00
Bauke Brenninkmeijer
fc9e387dc0
Replacement of 20 asserts with exceptions (#24757) 
* initial replacements of asserts with errors/exceptions

* replace assert with exception in generation, align and bart

* reset formatting change

* reset another formatting issue

* Apply suggestion

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* don't touch this file

* change to 'is not False'

* fix type

---------

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
 2023-07-12 07:45:09 -04:00
Gema Parreño
4b26a61631
Fix loading dataset docs link in run_translation.py example (#24594) 
* fix loading dataset link

* Update examples/tensorflow/translation/run_translation.py

Co-authored-by: Matt <Rocketknight1@users.noreply.github.com>

* Update examples/tensorflow/translation/run_translation.py

Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>

---------

Co-authored-by: Matt <Rocketknight1@users.noreply.github.com>
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
 2023-07-03 15:21:21 +01:00
Jeroen Van Goey
fd8dcd0953
Udate link to RunHouse hardware setup documentation. (#24590) 
* Udate link to RunHouse hardware setup documentation.

* Fix link to hardware setup in other location as well
 2023-06-30 12:11:58 +01:00
Xiaoli Wang
239ace152b
Fix TypeError: Object of type int64 is not JSON serializable (#24340) 
* Fix TypeError: Object of type int64 is not JSON serializable

* Convert numpy.float64 and numpy.int64 to float and int for json serialization

* Black reformatted examples/pytorch/token-classification/run_ner_no_trainer.py

* * make style
 2023-06-27 12:15:49 +01:00
Matt
8e164c5400
Improved keras imports (#24448) 
* An end to accursed version-specific imports

* No more K.is_keras_tensor() either

* Update dependency tables

* Use a cleaner call context function getter

* Add a cap to <2.14

* Add cap to examples requirements too
 2023-06-23 19:09:34 +01:00
Patrick von Platen
1609a436ec
Add MMS CTC Fine-Tuning (#24281) 
* Add mms ctc fine tuning

* make style

* More fixes that are needed

* make fix-copies

* make draft for README

* add new file

* move to new file

* make style

* make style

* add quick test

* make style

* make style
 2023-06-15 01:10:27 +02:00
Ethan
f7d80cb3d2
Fix steps bugs in no trainer examples (#24197) 
Fix step bugs in no trainer + load checkpoint + grad acc
 2023-06-12 11:49:55 -04:00
Sylvain Gugger
ba695c1efd
v4.31.0.dev0 2023-06-07 16:49:00 -04:00
Zachary Mueller
cbf6bc2350
Oops, missed one (#24054) 
Oops
 2023-06-06 13:30:19 -04:00
Zachary Mueller
072188d638
Act on deprecations in Accelerate no_trainer examples (#24053) 
Act on deprecation
 2023-06-06 13:04:38 -04:00
dependabot[bot]
539e2281cd
Bump cryptography from 39.0.1 to 41.0.0 in /examples/research_projects/decision_transformer (#23964) 
Bump cryptography in /examples/research_projects/decision_transformer

Bumps [cryptography](https://github.com/pyca/cryptography) from 39.0.1 to 41.0.0.
- [Changelog](https://github.com/pyca/cryptography/blob/main/CHANGELOG.rst)
- [Commits](https://github.com/pyca/cryptography/compare/39.0.1...41.0.0)

---
updated-dependencies:
- dependency-name: cryptography
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-06-02 16:23:44 -04:00
Matt
167a0d8f87
Add an option to reduce compile() console spam (#23938) 
* Add an option to reduce compile() console spam

* Add annotations to the example scripts

* Add notes to the quicktour docs as well

* minor fix
 2023-06-02 15:28:52 +01:00
Sylvain Gugger
3ff443a6d9
Re-enable squad test (#23912) 
* Re-enable squad test

* [all-test]

* [all-test] Fix all test command

* Fix the all-test
 2023-05-31 13:44:26 -04:00
Sylvain Gugger
00f6ba0e7e
Skip failing test for now 2023-05-31 06:31:33 -04:00
Ran Ran
e724246935
Fix no such file or directory error (#23783) 
* Fix no such file or directory error

* Address comment

* Fix formatting issue
 2023-05-26 14:24:57 -04:00
dependabot[bot]
d685e330b5
Bump tornado from 6.0.4 to 6.3.2 in /examples/research_projects/visual_bert (#23767) 
Bump tornado in /examples/research_projects/visual_bert

Bumps [tornado](https://github.com/tornadoweb/tornado) from 6.0.4 to 6.3.2.
- [Changelog](https://github.com/tornadoweb/tornado/blob/master/docs/releases.rst)
- [Commits](https://github.com/tornadoweb/tornado/compare/v6.0.4...v6.3.2)

---
updated-dependencies:
- dependency-name: tornado
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-05-25 16:16:12 -04:00
dependabot[bot]
4b0e7ded1c
Bump tornado from 6.0.4 to 6.3.2 in /examples/research_projects/lxmert (#23766) 
Bumps [tornado](https://github.com/tornadoweb/tornado) from 6.0.4 to 6.3.2.
- [Changelog](https://github.com/tornadoweb/tornado/blob/master/docs/releases.rst)
- [Commits](https://github.com/tornadoweb/tornado/compare/v6.0.4...v6.3.2)

---
updated-dependencies:
- dependency-name: tornado
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-05-25 16:16:01 -04:00
Sylvain Gugger
6e4bc67099
Revamp test selection for the example tests (#23737) 
* Revamp test selection for the example tests

* Rename old XLA test and fake modif in run_glue

* Fixes

* Fake Trainer modif

* Remove fake modifs
 2023-05-25 09:38:21 -04:00
Wang, Yi
33687a3f61
add GPTJ/bloom/llama/opt into model list and enhance the jit support (#23291) 
Signed-off-by: Wang, Yi A <yi.a.wang@intel.com>
 2023-05-24 10:57:56 +01:00