riaz.somc/transformers
1
0
Fork 0
mirror of https://github.com/huggingface/transformers.git synced 2025-07-08 07:10:06 +06:00
Code Issues Actions 25 Packages Projects Releases Wiki Activity
15,053 Commits 859 Branches 225 Tags 2.1 GiB
3f9f749325
Commit Graph

2378 Commits

Author SHA1 Message Date
YQ
f71c9ccf59
fix logit-to-multi-hot conversion in example (#26936) 
* fix logit to multi-hot converstion

* add comments

* typo
 2023-10-23 12:33:05 +02:00
Matt
cbd278f0f6
Pin Keras for now (#26904) 
* Pin Keras for now out of paranoia

* Add the keras pin to _tests_requirements.txt too

* Make sure the Keras version matches the TF one

* make fixup
 2023-10-19 14:39:31 +01:00
dependabot[bot]
bece55d8f9
Bump urllib3 from 1.26.17 to 1.26.18 in /examples/research_projects/decision_transformer (#26889) 
Bump urllib3 in /examples/research_projects/decision_transformer

Bumps [urllib3](https://github.com/urllib3/urllib3) from 1.26.17 to 1.26.18.
- [Release notes](https://github.com/urllib3/urllib3/releases)
- [Changelog](https://github.com/urllib3/urllib3/blob/main/CHANGES.rst)
- [Commits](https://github.com/urllib3/urllib3/compare/1.26.17...1.26.18)

---
updated-dependencies:
- dependency-name: urllib3
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-10-18 13:31:06 +02:00
dependabot[bot]
6d644d6852
Bump urllib3 from 1.26.17 to 1.26.18 in /examples/research_projects/visual_bert (#26890) 
Bump urllib3 in /examples/research_projects/visual_bert

Bumps [urllib3](https://github.com/urllib3/urllib3) from 1.26.17 to 1.26.18.
- [Release notes](https://github.com/urllib3/urllib3/releases)
- [Changelog](https://github.com/urllib3/urllib3/blob/main/CHANGES.rst)
- [Commits](https://github.com/urllib3/urllib3/compare/1.26.17...1.26.18)

---
updated-dependencies:
- dependency-name: urllib3
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-10-18 04:30:50 -07:00
Tom Aarsen
40ea9ab2a1
Add many missing spaces in adjacent strings (#26751) 
Add missing spaces in adjacent strings
 2023-10-12 10:28:40 +02:00
Zach Mueller
1d6a84749b
Fix checkpoint path in no_trainer scripts (#26733) 
checkpoint path
 2023-10-11 16:16:27 +02:00
Roy Hvaara
fc63914399
[JAX] Replace uses of jnp.array in types with jnp.ndarray. (#26703) 
`jnp.array` is a function, not a type:
https://jax.readthedocs.io/en/latest/_autosummary/jax.numpy.array.html
so it never makes sense to use `jnp.array` in a type annotation. Presumably the intent was to write `jnp.ndarray` aka `jax.Array`.

Co-authored-by: Peter Hawkins <phawkins@google.com>
 2023-10-10 21:35:16 +02:00
jheitmann
3eceaa3637
Fix source_prefix default value (#26654) 2023-10-10 20:49:10 +02:00
statelesshz
27597fea07
remove SharedDDP as it is deprecated (#25702) 
* remove SharedDDP as it was drepracated

* apply review suggestion

* make style

* Oops,forgot to remove the compute_loss context manager in Seq2SeqTrainer.

* remove the unnecessary conditional statement

* keep the logic of IPEX

* clean code

* mix precision setup & make fixup

---------

Co-authored-by: statelesshz <jihuazhong1@huawei.com>
 2023-10-06 16:03:11 +02:00
Phuc Van Phan
6015f91a5a
refactor: change default block_size (#26229) 
* refactor: change default block_size

* fix: return tf to origin

* fix: change files to origin

* rebase

* rebase

* rebase

* rebase

* rebase

* rebase

* rebase

* rebase

* refactor: add min block_size to files

* reformat: add min block_size for run_clm tf
 2023-10-04 15:31:38 +01:00
dependabot[bot]
fc296f419e
Bump pillow from 9.3.0 to 10.0.1 in /examples/research_projects/decision_transformer (#26580) 
Bump pillow in /examples/research_projects/decision_transformer

Bumps [pillow](https://github.com/python-pillow/Pillow) from 9.3.0 to 10.0.1.
- [Release notes](https://github.com/python-pillow/Pillow/releases)
- [Changelog](https://github.com/python-pillow/Pillow/blob/main/CHANGES.rst)
- [Commits](https://github.com/python-pillow/Pillow/compare/9.3.0...10.0.1)

---
updated-dependencies:
- dependency-name: pillow
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-10-04 11:52:46 +02:00
Lysandre
bd6205919a v4.35.0.dev0 2023-10-03 16:54:37 +02:00
dependabot[bot]
cf345d5f38
Bump urllib3 from 1.26.9 to 1.26.17 in /examples/research_projects/decision_transformer (#26554) 
Bump urllib3 in /examples/research_projects/decision_transformer

Bumps [urllib3](https://github.com/urllib3/urllib3) from 1.26.9 to 1.26.17.
- [Release notes](https://github.com/urllib3/urllib3/releases)
- [Changelog](https://github.com/urllib3/urllib3/blob/main/CHANGES.rst)
- [Commits](https://github.com/urllib3/urllib3/compare/1.26.9...1.26.17)

---
updated-dependencies:
- dependency-name: urllib3
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-10-03 08:55:12 +02:00
dependabot[bot]
6de6fdd06d
Bump urllib3 from 1.26.5 to 1.26.17 in /examples/research_projects/visual_bert (#26552) 
Bump urllib3 in /examples/research_projects/visual_bert

Bumps [urllib3](https://github.com/urllib3/urllib3) from 1.26.5 to 1.26.17.
- [Release notes](https://github.com/urllib3/urllib3/releases)
- [Changelog](https://github.com/urllib3/urllib3/blob/main/CHANGES.rst)
- [Commits](https://github.com/urllib3/urllib3/compare/1.26.5...1.26.17)

---
updated-dependencies:
- dependency-name: urllib3
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-10-03 08:55:01 +02:00
dependabot[bot]
e092b4ad68
Bump urllib3 from 1.26.5 to 1.26.17 in /examples/research_projects/lxmert (#26551) 
Bump urllib3 in /examples/research_projects/lxmert

Bumps [urllib3](https://github.com/urllib3/urllib3) from 1.26.5 to 1.26.17.
- [Release notes](https://github.com/urllib3/urllib3/releases)
- [Changelog](https://github.com/urllib3/urllib3/blob/main/CHANGES.rst)
- [Commits](https://github.com/urllib3/urllib3/compare/1.26.5...1.26.17)

---
updated-dependencies:
- dependency-name: urllib3
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-10-03 08:54:50 +02:00
Sanchit Gandhi
68e85fc822
[Flax Examples] Seq2Seq ASR Fine-Tuning Script (#21764) 
* from seq2seq speech

* [Flax] Example script for speech seq2seq

* tests and fixes

* make style

* fix: label padding tokens

* fix: label padding tokens over list

* update ln names for Whisper

* try datasets iter loader

* create readme and append results

* style

* make style

* adjust lr

* use pt dataloader

* make fast

* pin gen max len

* finish

* add pt to requirements for test

* fix pt -> torch

* add accelerate
 2023-09-29 16:42:58 +01:00
Phuc Van Phan
ba47efbfe4
docs: change assert to raise and some small docs (#26232) 
* docs: change assert to raise and some small docs

* docs: add rule and some document

* fix: fix bug

* fix: fix bug

* chorse: revert logging

* chorse: revert
 2023-09-28 10:14:17 +02:00
Phuc Van Phan
910faa3e1f
feat: adding num_proc to load_dataset (#26326) 
* feat: adding num_proc to load_dataset

* feat: add add_num_proc for run_mlm_flax

* feat: add num_proc for bart and t5

* chorse: remove
 2023-09-22 19:22:47 +02:00
Nino Risteski
7d4e0c23c8
Update README.md (#26198) 
Fixed a few typos
 2023-09-19 00:02:50 +02:00
Phuc Van Phan
8b5da9fc6e
refactor: change default block_size in block size > max position embeddings (#26069) 
* refactor: change default block_size when not initialize

* reformat: add the min of block size
 2023-09-18 16:47:57 +01:00
Phuc Van Phan
4fb64e285a
chore: correct update_step and correct gradient_accumulation_steps (#26068) 2023-09-12 18:31:23 +01:00
Phuc Van Phan
5af2c62696
docs: add space to docs (#26067) 
* docs: add space to docs

* docs: remove reduntant space
 2023-09-11 22:03:26 +01:00
Phuc Van Phan
9cebae64ad
docs: update link huggingface map (#26077) 2023-09-11 12:57:04 +01:00
Nino Risteski
f6295c6c53
Update README.md (#26003) 
fixed a typo
 2023-09-06 10:55:11 +01:00
Joao Gante
9a70d6e56f
Trainer: delegate default generation values to generation_config (#25987) 2023-09-05 14:47:00 +01:00
Traun Leyden
7011cd8667
Update RAG README.md with correct path to examples/seq2seq (#25953) 
Update README.md with correct path to examples/seq2seq
 2023-09-05 12:31:59 +01:00
Susnato Dhar
404ff8fc17
Fix typo (#25966) 
* Update feature_extraction_clap.py

* changed all lenght to length
 2023-09-05 10:12:25 +02:00
Lysandre
d8e13b3e04 v4.34.dev.0 2023-09-04 15:12:11 -04:00
Zach Mueller
be0e189bd3
Revert frozen training arguments (#25903) 
* Revert frozen training arguments

* TODO
 2023-09-01 11:24:12 -04:00
Younes Belkada
4b79697865
🚨🚨🚨 [Refactor] Move third-party related utility files into integrations/ folder 🚨🚨🚨 (#25599) 
* move deepspeed to `lib_integrations.deepspeed`

* more refactor

* oops

* fix slow tests

* Fix docs

* fix docs

* addess feedback

* address feedback

* final modifs for PEFT

* fixup

* ok now

* trigger CI

* trigger CI again

* Update docs/source/en/main_classes/deepspeed.md

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* import from `integrations`

* address feedback

* revert removal of `deepspeed` module

* revert removal of `deepspeed` module

* fix conflicts

* ooops

* oops

* add deprecation warning

* place it on the top

* put `FutureWarning`

* fix conflicts with not_doctested.txt

* add back `bitsandbytes` module with a depr warning

* fix

* fix

* fixup

* oops

* fix doctests

---------

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
 2023-08-25 17:13:34 +02:00
Phuc Van Phan
656e17f6f7
correct resume training steps number in progress bar (#25691) 
feat: correct update resume update with steps
 2023-08-23 20:09:14 +02:00
Matt
62396cff46
TF 2.14 compatibility (#25630) 
* Update the TF pin and see if anything breaks

* make fixup

* make fixup

* make fixup
 2023-08-22 13:13:38 +01:00
Sylvain Gugger
5c67682b16
v4.33.0.dev0 2023-08-21 07:07:04 -04:00
Zach Mueller
ca51499248
Make training args fully immutable (#25435) 
* Make training args fully immutable

* Working tests, PyTorch

* In test_trainer

* during testing

* Use proper dataclass way

* Fix test

* Another one

* Fix tf

* Lingering slow

* Exception

* Clean
 2023-08-15 11:47:47 -04:00
dependabot[bot]
7a94ea4c64
Bump tornado from 6.3.2 to 6.3.3 in /examples/research_projects/lxmert (#25511) 
Bumps [tornado](https://github.com/tornadoweb/tornado) from 6.3.2 to 6.3.3.
- [Changelog](https://github.com/tornadoweb/tornado/blob/master/docs/releases.rst)
- [Commits](https://github.com/tornadoweb/tornado/compare/v6.3.2...v6.3.3)

---
updated-dependencies:
- dependency-name: tornado
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-08-15 08:52:30 +02:00
dependabot[bot]
2552b8c5bd
Bump tornado from 6.3.2 to 6.3.3 in /examples/research_projects/visual_bert (#25512) 
Bump tornado in /examples/research_projects/visual_bert

Bumps [tornado](https://github.com/tornadoweb/tornado) from 6.3.2 to 6.3.3.
- [Changelog](https://github.com/tornadoweb/tornado/blob/master/docs/releases.rst)
- [Commits](https://github.com/tornadoweb/tornado/compare/v6.3.2...v6.3.3)

---
updated-dependencies:
- dependency-name: tornado
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-08-15 08:52:20 +02:00
dependabot[bot]
0ebe7ae160
Bump gitpython from 3.1.30 to 3.1.32 in /examples/research_projects/decision_transformer (#25467) 
Bump gitpython in /examples/research_projects/decision_transformer

Bumps [gitpython](https://github.com/gitpython-developers/GitPython) from 3.1.30 to 3.1.32.
- [Release notes](https://github.com/gitpython-developers/GitPython/releases)
- [Changelog](https://github.com/gitpython-developers/GitPython/blob/main/CHANGES)
- [Commits](https://github.com/gitpython-developers/GitPython/compare/3.1.30...3.1.32)

---
updated-dependencies:
- dependency-name: gitpython
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-08-13 19:47:16 +02:00
dependabot[bot]
2b22cde71e
Bump gitpython from 3.1.30 to 3.1.32 in /examples/research_projects/distillation (#25468) 
Bump gitpython in /examples/research_projects/distillation

Bumps [gitpython](https://github.com/gitpython-developers/GitPython) from 3.1.30 to 3.1.32.
- [Release notes](https://github.com/gitpython-developers/GitPython/releases)
- [Changelog](https://github.com/gitpython-developers/GitPython/blob/main/CHANGES)
- [Commits](https://github.com/gitpython-developers/GitPython/compare/3.1.30...3.1.32)

---
updated-dependencies:
- dependency-name: gitpython
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-08-13 19:47:04 +02:00
Gema Parreño
0acf56224b
Update run_translation.py broken link example Pytoch (#25461) 
* Update run_translation.py

Fixed link

* Update run_translation.py
 2023-08-11 15:41:24 +02:00
Yih-Dar
9c7b744795
Fix missing usage of token (#25382) 
* add missing tokens

* fix

---------

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2023-08-08 16:27:24 +02:00
Zach Mueller
01ab39b65f
Load state in else (#25318) 
* Load else

* New approach

* Propagate
 2023-08-08 05:41:00 -04:00
Phuc Van Phan
5fe36970e5
Adding more information in help parser on train_file and validation_file (#25324) 
chorse: adding new doc on train and val
 2023-08-07 17:56:13 +02:00
Jackmin801
145109382a
Allow trust_remote_code in example scripts (#25248) 
* pytorch examples

* pytorch mim no trainer

* cookiecutter

* flax examples

* missed line in pytorch run_glue

* tensorflow examples

* tensorflow run_clip

* tensorflow run_mlm

* tensorflow run_ner

* tensorflow run_clm

* pytorch example from_configs

* pytorch no trainer examples

* Revert "tensorflow run_clip"

This reverts commit 261f86ac1f.

* fix: duplicated argument
 2023-08-07 16:32:25 +02:00
Yih-Dar
149cb0cce2
Add token arugment in example scripts (#25172) 
* fix

* fix

* fix

* fix

* fix

* fix

* fix

---------

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2023-08-02 11:17:31 +02:00
Yih-Dar
d53b8ad780
Update use_auth_token -> token in example scripts (#25167) 
* pytorch examples

* tensorflow examples

* flax examples

---------

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2023-07-28 15:33:45 +02:00
Alan Ji
afa96fffdf
make run_generation more generic for other devices (#25133) 
* make run_generation more generic for other devices

* use Accelerate to support any device type it supports.

* make style

* fix error usage of accelerator.prepare_model

* use `PartialState` to make sure everything is running on the right device

---------

Co-authored-by: statelesshz <jihuazhong1@huawei.com>
 2023-07-28 08:20:10 -04:00
Lucain
6232c380f2
Fix .push_to_hub and cleanup get_full_repo_name usage (#25120) 
* Fix .push_to_hub and cleanup get_full_repo_name usage

* Do not rely on Python bool conversion magic

* request changes
 2023-07-28 11:40:08 +02:00
dependabot[bot]
f1deb21fce
Bump certifi from 2022.12.7 to 2023.7.22 in /examples/research_projects/visual_bert (#25097) 
Bump certifi in /examples/research_projects/visual_bert

Bumps [certifi](https://github.com/certifi/python-certifi) from 2022.12.7 to 2023.7.22.
- [Commits](https://github.com/certifi/python-certifi/compare/2022.12.07...2023.07.22)

---
updated-dependencies:
- dependency-name: certifi
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-07-25 17:25:14 -04:00
dependabot[bot]
45bde362d2
Bump certifi from 2022.12.7 to 2023.7.22 in /examples/research_projects/decision_transformer (#25098) 
Bump certifi in /examples/research_projects/decision_transformer

Bumps [certifi](https://github.com/certifi/python-certifi) from 2022.12.7 to 2023.7.22.
- [Commits](https://github.com/certifi/python-certifi/compare/2022.12.07...2023.07.22)

---
updated-dependencies:
- dependency-name: certifi
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-07-25 17:25:05 -04:00
dependabot[bot]
6b8dbc283c
Bump certifi from 2022.12.7 to 2023.7.22 in /examples/research_projects/lxmert (#25096) 
Bump certifi in /examples/research_projects/lxmert

Bumps [certifi](https://github.com/certifi/python-certifi) from 2022.12.7 to 2023.7.22.
- [Commits](https://github.com/certifi/python-certifi/compare/2022.12.07...2023.07.22)

---
updated-dependencies:
- dependency-name: certifi
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-07-25 17:24:50 -04:00
Alan Ji
c879318cc5
replace per_gpu_eval_batch_size with per_device_eval_batch_size in readme of multiple-choice task (#25078) 
replace `per_gpu_eval_batch_size` with `per_device_eval_batch_size`
in readme of multiple-choice
 2023-07-25 08:11:56 -04:00
Zach Mueller
aa1b09c5d1
Change logic for logging in the examples (#24956) 
Change logic
 2023-07-20 12:30:10 -04:00
dependabot[bot]
9f912ef62a
Bump aiohttp from 3.8.1 to 3.8.5 in /examples/research_projects/decision_transformer (#24954) 
Bump aiohttp in /examples/research_projects/decision_transformer

Bumps [aiohttp](https://github.com/aio-libs/aiohttp) from 3.8.1 to 3.8.5.
- [Release notes](https://github.com/aio-libs/aiohttp/releases)
- [Changelog](https://github.com/aio-libs/aiohttp/blob/v3.8.5/CHANGES.rst)
- [Commits](https://github.com/aio-libs/aiohttp/compare/v3.8.1...v3.8.5)

---
updated-dependencies:
- dependency-name: aiohttp
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-07-20 12:17:38 -04:00
dependabot[bot]
9859806608
Bump pygments from 2.11.2 to 2.15.0 in /examples/research_projects/decision_transformer (#24949) 
Bump pygments in /examples/research_projects/decision_transformer

Bumps [pygments](https://github.com/pygments/pygments) from 2.11.2 to 2.15.0.
- [Release notes](https://github.com/pygments/pygments/releases)
- [Changelog](https://github.com/pygments/pygments/blob/master/CHANGES)
- [Commits](https://github.com/pygments/pygments/compare/2.11.2...2.15.0)

---
updated-dependencies:
- dependency-name: pygments
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-07-20 07:43:48 -04:00
statelesshz
37d8611ac9
replace no_cuda with use_cpu in test_pytorch_examples (#24944) 
* replace no_cuda with use_cpu in test_pytorch_examples

* remove codes that never be used

* fix style
 2023-07-20 07:09:04 -04:00
ranchlai
8fd8c8e49e
Add multi-label text classification support to pytorch example (#24770) 
* Add text classification example

* set the problem type and finetuning task

* ruff reformated

* fix bug for unseting label_to_id for regression

* update README.md

* fixed finetuning task

* update comment

* check if label exists in feature before removing

* add useful logging
 2023-07-20 07:02:44 -04:00
Hwijeen Ahn
dd49404a89
check if eval dataset is dict (#24877) 
* check if eval dataset is dict

* formatting
 2023-07-18 13:33:41 -04:00
Sylvain Gugger
e9ad51306f
4.32.0.dev0 2023-07-17 13:30:44 -04:00
dependabot[bot]
38dfb86958
Bump cryptography from 41.0.0 to 41.0.2 in /examples/research_projects/decision_transformer (#24833) 
Bump cryptography in /examples/research_projects/decision_transformer

Bumps [cryptography](https://github.com/pyca/cryptography) from 41.0.0 to 41.0.2.
- [Changelog](https://github.com/pyca/cryptography/blob/main/CHANGELOG.rst)
- [Commits](https://github.com/pyca/cryptography/compare/41.0.0...41.0.2)

---
updated-dependencies:
- dependency-name: cryptography
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-07-17 07:17:17 -04:00
Bauke Brenninkmeijer
fc9e387dc0
Replacement of 20 asserts with exceptions (#24757) 
* initial replacements of asserts with errors/exceptions

* replace assert with exception in generation, align and bart

* reset formatting change

* reset another formatting issue

* Apply suggestion

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* don't touch this file

* change to 'is not False'

* fix type

---------

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
 2023-07-12 07:45:09 -04:00
Gema Parreño
4b26a61631
Fix loading dataset docs link in run_translation.py example (#24594) 
* fix loading dataset link

* Update examples/tensorflow/translation/run_translation.py

Co-authored-by: Matt <Rocketknight1@users.noreply.github.com>

* Update examples/tensorflow/translation/run_translation.py

Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>

---------

Co-authored-by: Matt <Rocketknight1@users.noreply.github.com>
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
 2023-07-03 15:21:21 +01:00
Jeroen Van Goey
fd8dcd0953
Udate link to RunHouse hardware setup documentation. (#24590) 
* Udate link to RunHouse hardware setup documentation.

* Fix link to hardware setup in other location as well
 2023-06-30 12:11:58 +01:00
Xiaoli Wang
239ace152b
Fix TypeError: Object of type int64 is not JSON serializable (#24340) 
* Fix TypeError: Object of type int64 is not JSON serializable

* Convert numpy.float64 and numpy.int64 to float and int for json serialization

* Black reformatted examples/pytorch/token-classification/run_ner_no_trainer.py

* * make style
 2023-06-27 12:15:49 +01:00
Matt
8e164c5400
Improved keras imports (#24448) 
* An end to accursed version-specific imports

* No more K.is_keras_tensor() either

* Update dependency tables

* Use a cleaner call context function getter

* Add a cap to <2.14

* Add cap to examples requirements too
 2023-06-23 19:09:34 +01:00
Patrick von Platen
1609a436ec
Add MMS CTC Fine-Tuning (#24281) 
* Add mms ctc fine tuning

* make style

* More fixes that are needed

* make fix-copies

* make draft for README

* add new file

* move to new file

* make style

* make style

* add quick test

* make style

* make style
 2023-06-15 01:10:27 +02:00
Ethan
f7d80cb3d2
Fix steps bugs in no trainer examples (#24197) 
Fix step bugs in no trainer + load checkpoint + grad acc
 2023-06-12 11:49:55 -04:00
Sylvain Gugger
ba695c1efd
v4.31.0.dev0 2023-06-07 16:49:00 -04:00
Zachary Mueller
cbf6bc2350
Oops, missed one (#24054) 
Oops
 2023-06-06 13:30:19 -04:00
Zachary Mueller
072188d638
Act on deprecations in Accelerate no_trainer examples (#24053) 
Act on deprecation
 2023-06-06 13:04:38 -04:00
dependabot[bot]
539e2281cd
Bump cryptography from 39.0.1 to 41.0.0 in /examples/research_projects/decision_transformer (#23964) 
Bump cryptography in /examples/research_projects/decision_transformer

Bumps [cryptography](https://github.com/pyca/cryptography) from 39.0.1 to 41.0.0.
- [Changelog](https://github.com/pyca/cryptography/blob/main/CHANGELOG.rst)
- [Commits](https://github.com/pyca/cryptography/compare/39.0.1...41.0.0)

---
updated-dependencies:
- dependency-name: cryptography
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-06-02 16:23:44 -04:00
Matt
167a0d8f87
Add an option to reduce compile() console spam (#23938) 
* Add an option to reduce compile() console spam

* Add annotations to the example scripts

* Add notes to the quicktour docs as well

* minor fix
 2023-06-02 15:28:52 +01:00
Sylvain Gugger
3ff443a6d9
Re-enable squad test (#23912) 
* Re-enable squad test

* [all-test]

* [all-test] Fix all test command

* Fix the all-test
 2023-05-31 13:44:26 -04:00
Sylvain Gugger
00f6ba0e7e
Skip failing test for now 2023-05-31 06:31:33 -04:00
Ran Ran
e724246935
Fix no such file or directory error (#23783) 
* Fix no such file or directory error

* Address comment

* Fix formatting issue
 2023-05-26 14:24:57 -04:00
dependabot[bot]
d685e330b5
Bump tornado from 6.0.4 to 6.3.2 in /examples/research_projects/visual_bert (#23767) 
Bump tornado in /examples/research_projects/visual_bert

Bumps [tornado](https://github.com/tornadoweb/tornado) from 6.0.4 to 6.3.2.
- [Changelog](https://github.com/tornadoweb/tornado/blob/master/docs/releases.rst)
- [Commits](https://github.com/tornadoweb/tornado/compare/v6.0.4...v6.3.2)

---
updated-dependencies:
- dependency-name: tornado
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-05-25 16:16:12 -04:00
dependabot[bot]
4b0e7ded1c
Bump tornado from 6.0.4 to 6.3.2 in /examples/research_projects/lxmert (#23766) 
Bumps [tornado](https://github.com/tornadoweb/tornado) from 6.0.4 to 6.3.2.
- [Changelog](https://github.com/tornadoweb/tornado/blob/master/docs/releases.rst)
- [Commits](https://github.com/tornadoweb/tornado/compare/v6.0.4...v6.3.2)

---
updated-dependencies:
- dependency-name: tornado
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-05-25 16:16:01 -04:00
Sylvain Gugger
6e4bc67099
Revamp test selection for the example tests (#23737) 
* Revamp test selection for the example tests

* Rename old XLA test and fake modif in run_glue

* Fixes

* Fake Trainer modif

* Remove fake modifs
 2023-05-25 09:38:21 -04:00
Wang, Yi
33687a3f61
add GPTJ/bloom/llama/opt into model list and enhance the jit support (#23291) 
Signed-off-by: Wang, Yi A <yi.a.wang@intel.com>
 2023-05-24 10:57:56 +01:00
zspo
003a0cf8cc
Fix some docs what layerdrop does (#23691) 
* Fix some docs what layerdrop does

* Update src/transformers/models/data2vec/configuration_data2vec_audio.py

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* Fix more docs

---------

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
 2023-05-23 14:50:40 -04:00
dependabot[bot]
aa30cd4f3f
Bump requests from 2.22.0 to 2.31.0 in /examples/research_projects/lxmert (#23668) 
Bump requests in /examples/research_projects/lxmert

Bumps [requests](https://github.com/psf/requests) from 2.22.0 to 2.31.0.
- [Release notes](https://github.com/psf/requests/releases)
- [Changelog](https://github.com/psf/requests/blob/main/HISTORY.md)
- [Commits](https://github.com/psf/requests/compare/v2.22.0...v2.31.0)

---
updated-dependencies:
- dependency-name: requests
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-05-23 05:31:53 -04:00
dependabot[bot]
9bf72ae564
Bump requests from 2.22.0 to 2.31.0 in /examples/research_projects/visual_bert (#23670) 
Bump requests in /examples/research_projects/visual_bert

Bumps [requests](https://github.com/psf/requests) from 2.22.0 to 2.31.0.
- [Release notes](https://github.com/psf/requests/releases)
- [Changelog](https://github.com/psf/requests/blob/main/HISTORY.md)
- [Commits](https://github.com/psf/requests/compare/v2.22.0...v2.31.0)

---
updated-dependencies:
- dependency-name: requests
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-05-23 05:31:30 -04:00
dependabot[bot]
ecc05f8c1e
Bump requests from 2.27.1 to 2.31.0 in /examples/research_projects/decision_transformer (#23673) 
Bump requests in /examples/research_projects/decision_transformer

Bumps [requests](https://github.com/psf/requests) from 2.27.1 to 2.31.0.
- [Release notes](https://github.com/psf/requests/releases)
- [Changelog](https://github.com/psf/requests/blob/main/HISTORY.md)
- [Commits](https://github.com/psf/requests/compare/v2.27.1...v2.31.0)

---
updated-dependencies:
- dependency-name: requests
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-05-23 05:28:09 -04:00
Zachary Mueller
b191d7db44
Update all no_trainer with skip_first_batches (#23664) 2023-05-22 14:49:31 -04:00
sshahrokhi
6f72e71f97
changing the requirements to a cpu torch version that works (#23483) 2023-05-22 12:58:55 -04:00
Boda Sadallah
a7920065f2
fix bug in group_texts function, that was inserting short batches (#23429) 
* fix bug in group_texts function, that was inserting short batches

* fully exclude short batches and return empty dict instead

* fix style
 2023-05-18 14:22:30 -04:00
Zachary Mueller
8a58809312
Fix translation no_trainer (#23407) 
* Fix translation
 2023-05-16 13:10:42 -04:00
Yih-Dar
d51296d9c2
skip test_run_squad_no_trainer for now (#23302) 
skip

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2023-05-11 19:26:48 +02:00
Hari
5d02e6bd20
Convert numpy arrays to lists before saving the evaluation metrics as json (#23268) 
* convert numpy array to list before writing to json

per_category_iou and per_category_accuracy  are ndarray in the eval_metrics

* code reformatted with make style
 2023-05-11 08:54:23 -04:00
Maria Khalusova
91f4c84a19
CTC example: updated trainer parameters to save tokenizer (#23243) 
trainer parameters changed to save tokenizer in addition to feature_extractor
 2023-05-10 07:45:10 -04:00
Sylvain Gugger
a0c0a78233
v4.30.0.dev0 2023-05-09 14:59:38 -04:00
Nicolas Patry
c34a525d2f
Proposed fix for TF example now running on safetensors. (#23208) 
* Proposed fix for TF example now running on safetensors.

* Adding more warnings and returning keys.

* Trigger CI

* Trigger CI

---------

Co-authored-by: Sylvain Gugger <Sylvain.gugger@gmail.com>
 2023-05-09 13:04:27 -04:00
Sebastian
1a8f61110e
fix: Update run_qa.py to work with deepset/germanquad (#23225) 
Call str on id to make sure any ints are converted into the expected format for squad datasets
 2023-05-09 09:20:10 -04:00
Sylvain Gugger
fd6970bc56
Skip failing test 2023-05-08 08:52:44 -04:00
Ashwin Mathur
fc6c8b0eaa
Add no_trainer scripts to pre-train Vision Transformers (#23156) 
* Add run_mim_no_trainer.py draft from #20412

Add parse_args method and copy over other dependencies

Add Method call for sending telemetry

Initialize Accelerator

Make one log on every process

Set seed and Handle repository creation

Initialize dataset and Set validation split

Create Config

Adapt Config

Update Config

Create Feature Extractor

Create model

Set column names

Create transforms

Create mask generator

Create method to preprocess images

Shuffle datasets if needed and set transforms

Create Dataloaders

Add optimizer

Add learning rate scheduler

Prepare everything with our accelerator

Tie weights for TPU training

Recalculate training steps and training epochs

Set accelerator checkpointing steps

Initialize trackers and store configuration

Set total batch size

Fix typo: mlm -> mim

Log info at the start of training

Load in the weights and states from previous save

update the progress_bar if load from checkpoint

Define train loop

Add evaluation loop to training

Add to parse_args method

Push repo to hub

Save accelerator state

End training and save model and feature extractor

Remove unused imports

Fix trailing whitespace

* Update code based on comments, Rename feature_extractor to image_processor

* Fix linting

* Add argument for learning rate

* Add argument for setting number of training epochs

* Remove incorrect logger argument

* Convert max_train_steps to int for tqdm

---------

Co-authored-by: Saad Mahmud <shuvro.mahmud79@gmail.com>
 2023-05-05 13:22:49 -04:00
Robert Stone
b6933d76d2
Tidy Pytorch GLUE benchmark example (#23134) 
Migration to Evaluate for metric is not quite complete
 2023-05-03 15:50:41 -04:00
Alex Punnen
805db1fe13
num_noise_spans should be <= num_items #22246 (#22938) 2023-05-02 13:07:30 -04:00
regisss
bcedd0a471
Save the tokenizer and image preprocessor after training a model with the contrastive image-text example (#23035) 
Save tokenizer and image preprocessor
 2023-05-02 09:23:16 -04:00
dependabot[bot]
b8648290d2
Bump flask from 2.0.3 to 2.3.2 in /examples/research_projects/decision_transformer (#23094) 
Bump flask in /examples/research_projects/decision_transformer

Bumps [flask](https://github.com/pallets/flask) from 2.0.3 to 2.3.2.
- [Release notes](https://github.com/pallets/flask/releases)
- [Changelog](https://github.com/pallets/flask/blob/main/CHANGES.rst)
- [Commits](https://github.com/pallets/flask/compare/2.0.3...2.3.2)

---
updated-dependencies:
- dependency-name: flask
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-05-01 20:15:11 -04:00
Lingepumpe
5427250351
Avoid invalid escape sequences, use raw strings (#22936) 
* Avoid invalid escape sequences, use raw strings

* Integrate PR feedback
 2023-04-25 09:17:56 -04:00
Roy Hvaara
874c7caf19
Remove broken test_data symlink in legacy s2s examples (#22876) 2023-04-21 15:35:42 +01:00