riaz.somc/transformers
1
0
Fork 0
mirror of https://github.com/huggingface/transformers.git synced 2025-07-13 09:40:06 +06:00
Code Issues Actions 13 Packages Projects Releases Wiki Activity
9,397 Commits 863 Branches 226 Tags 1.9 GiB
473709fc76
Commit Graph

301 Commits

Author SHA1 Message Date
Sylvain Gugger
473709fc76
Use doc builder styler (#16412) 
* Config update

* Use doc-builder styler

* Cleanup

* Adapt import

* We need it there too!
 2022-03-28 07:45:18 -04:00
Yih-Dar
9d88be5778
bump cookiecutter version (#16387) 
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2022-03-24 11:08:31 -04:00
Lysandre Debut
eca77f4719
Updates the default branch from master to main (#16326) 
* Updates the default branch from master to main

* Links from `master` to `main`

* Typo

* Update examples/flax/README.md

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
 2022-03-23 03:46:59 -04:00
Suraj Patil
b25b92ac4f
update jax version and re-enable some tests (#16254) 2022-03-18 16:45:39 +01:00
Stas Bekman
580dd87c55
[Deepspeed] add support for bf16 mode (#14569) 
* [WIP] add support for bf16 mode

* prep for bf16

* prep for bf16

* fix; zero2/bf16 is ok

* check bf16 is available

* test fixes

* enable zero3_bf16

* config files

* docs

* split stage_dtype; merge back to non-dtype-specific config file

* fix doc

* cleanup

* cleanup

* bfloat16 => bf16 to match the PR changes

* s/zero_gather_fp16_weights_on_model_save/zero_gather_16bit_weights_on_model_save/; s/save_fp16_model/save_16bit_model/

* test fixes/skipping

* move

* fix

* Update docs/source/main_classes/deepspeed.mdx

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* backticks

* cleanup

* cleanup

* cleanup

* new version

* add note about grad accum in bf16

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
 2022-03-11 17:53:53 -08:00
Sylvain Gugger
79d28e80b6 v4.18.0.dev.0 2022-03-03 10:19:58 -05:00
Lysandre Debut
54f0db4066
Add PT + TF automatic builds (#15860) 
* Add PT + TF automatic builds

* Apply suggestions from code review

Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>

* Wrap up

Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
 2022-03-01 08:55:11 -05:00
SaulLu
e93763d420
fix CLIP fast tokenizer and change some properties of the slow version (#15067) 
Very big changes concerning the tokenizer fast of CLIP which did not correspond to the tokenizer slow of CLIP

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
 2022-02-18 10:21:30 +01:00
Alex Hedges
e1cbc073bf
Require tokenizers>=0.11.1 (#15266) 
`tokenizers` version that supports the feature to choose the direction of truncation
 2022-02-15 11:46:12 +01:00
Lysandre Debut
7732d0fe7a
Upgrade black to version ~=22.0 (#15565) 
* Upgrade black to version ~=22.0

* Check copies

* Fix code
 2022-02-09 09:28:57 -05:00
Stas Bekman
297602c7f4
[deepspeed] saving checkpoint fallback when fp16 weights aren't saved (#14948) 
* [deepspeed] saving checkpoint fallback when fp16 weights aren't saved

* Bump required deepspeed version to match usage when saving checkpoints

* update version

Co-authored-by: Mihai Balint <balint.mihai@gmail.com>
 2022-01-28 11:05:47 -08:00
Lysandre
eab338104d Docs for version v4.16.0 2022-01-27 13:11:51 -05:00
Lysandre
f87db5e412 Release: v4.16.0 2022-01-27 13:06:33 -05:00
Patrick von Platen
497346d07e
[ASR pipeline] correct with lm pipeline (#15200) 
* [ASR pipeline] correct with lm pipeline

* improve error
 2022-01-18 15:36:22 +01:00
Sylvain Gugger
6f8e644f09
Mark bad tokenizers version (#15188) 2022-01-17 15:20:58 -05:00
Joao Gante
ebc4edfe7a
update from keras2onnx to tf2onnx (#15162) 2022-01-14 17:35:39 +00:00
Nicolas Patry
08cb5718ec
Enabling tokenizers upgrade. (#14941) 
* Enabling `tokenizers` upgrade.

* Moved ugly comment.

* Tokenizers==0.11.1 needs an update to keep borrow checker

happy in highly contiguous calls.

* Support both 0.11.1 and 0.11.0
 2021-12-30 17:30:58 +01:00
Patrick von Platen
fa39ff9fc4 Docs for v4.16.0dev0 2021-12-22 20:39:44 +01:00
Patrick von Platen
05fa1a7ac1 Release: v4.15.0 2021-12-22 18:43:15 +01:00
Patrick von Platen
c4a96cecbc
Wav2Vec2 meets phonemes (#14353) 
* up

* add tokenizer

* improve more

* finish tokenizer

* finish

* adapt speech recognition script

* adapt convert

* more fixes

* more fixes

* update phonemizer wav2vec2

* better naming

* fix more tests

* more fixes swedish

* correct tests

* finish

* improve script

* remove file

* up

* lets get those 100 model architectures until the end of the month

* make fix-copies

* correct more

* correct script

* more fixes

* more fixes

* add to docs

* Apply suggestions from code review

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* replace assert

* fix copies

* fix docs

* new try docs

* boom boom

* update

* add phonemizer to audio tests

* make fix-copies

* up

* upload models

* some changes

* Update tests/test_tokenization_wav2vec2_phoneme.py

Co-authored-by: Anton Lozhkov <aglozhkov@gmail.com>

* more fixes

* remove @

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
Co-authored-by: Anton Lozhkov <aglozhkov@gmail.com>
 2021-12-17 19:56:44 +01:00
Sylvain Gugger
5061a9fd55
Post sphinx-clean up and contributing guide updates (#14790) 
* Clean up sphinx

* Update contributing guide

* Update docs README

* No example title

* Fix copies

* Update CONTRIBUTING.md

Co-authored-by: Lysandre Debut <lysandre@huggingface.co>

Co-authored-by: Lysandre Debut <lysandre@huggingface.co>
 2021-12-16 09:29:26 -05:00
Lysandre
7c9c41f43c Docs for v4.14.0 2021-12-15 18:29:53 +01:00
Lysandre
960d8cb41d Release: v4.14.0 2021-12-15 18:20:35 +01:00
Lysandre
ab31b3e41b Docs for v4.14.0dev0 2021-12-09 17:09:23 +01:00
Lysandre
4da3a696e4 Release: v4.13.0 2021-12-09 16:55:21 +01:00
Patrick von Platen
961732c276
[Wav2Vec2] PyCTCDecode Integration to support language model boosted decoding (#14339) 
* up

* up

* up

* make it cleaner

* correct

* make styhahalal

* add more tests

* finish

* small fix

* make style

* up

* tryout to solve cicrle ci

* up

* fix more tests

* fix more tests

* apply sylvains suggestions

* fix import

* correct docs

* add pyctcdecode only to speech tests

* fix more tests

* add tf, flax and pt tests

* add pt

* fix last tests

* fix more tests

* Apply suggestions from code review

* change lines

* Apply suggestions from code review

Co-authored-by: Anton Lozhkov <aglozhkov@gmail.com>

* correct tests

* correct tests

* add doc string

Co-authored-by: Anton Lozhkov <aglozhkov@gmail.com>
 2021-12-08 12:07:54 +01:00
Stas Bekman
956a483173
[deepspeed] zero inference (#14253) 
* [deepspeed] zero inference

* only z3 makes sense for inference

* fix and style

* docs

* rework

* fix test

* Apply suggestions from code review

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* responding to suggestions

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
 2021-11-23 14:09:15 -08:00
Lysandre Debut
331c3d2aa0
Add GitPython to quality tools (#14459) 
* Update setup.py

* Update setup.py

* Update setup.py

* Remove GitPython install
 2021-11-19 08:43:48 -05:00
Sylvain Gugger
b567510cff
Debug doc (#14424) 
* Create branch for tests

* Pin first upgrade

* Really pin

* Polish fix
 2021-11-16 18:58:07 -05:00
Sylvain Gugger
46d0cdae40
Support for TF >= 2.7 (#14345) 2021-11-09 18:49:29 -05:00
Suraj Patil
85a4bda4f4
bump flax version (#14343) 2021-11-09 22:15:22 +05:30
Sylvain Gugger
a14d62b0b1
Pin TF until tests are fixed (#14283) 
* Pin TF until tests are fixed

* Also pin TF CPU
 2021-11-04 21:15:42 -04:00
Matt
b90a48f654
Removing Keras version pinning (#14280) 
* Removing Keras version pinning

* make fixup
 2021-11-04 17:58:28 +00:00
Sylvain Gugger
1a674ce679
Add more instructions to the release guide (#14263) 
* Add more instructions to the release guide

* Apply suggestions from code review

Co-authored-by: Lysandre Debut <lysandre@huggingface.co>

* Address review comment

Co-authored-by: Lysandre Debut <lysandre@huggingface.co>
 2021-11-03 17:45:41 -04:00
Sylvain Gugger
a1c15ea855
Pin Keras cause they messed their release (#14262) 
* Pin Keras cause they messed their release

* Put != instead of <

* Try this way

* Back to the beginning but more agressive
 2021-11-03 15:03:09 -04:00
Sylvain Gugger
558f8543ba
Update Transformers to huggingface_hub >= 0.1.0 (#14251) 
* Update Transformers to huggingface_hub >= 0.1.0

* Forgot to save...

* Style

* Fix test
 2021-11-02 18:58:42 -04:00
Lysandre Debut
63d91f449c
Torch 1.10 (#14169) 
* Torch 1.10

* torch scatter for 1.10

* style

* Skip tests
ok
 2021-10-29 13:43:43 -04:00
Lysandre
b8fad022a0 v4.13.0.dev0 2021-10-28 12:56:46 -04:00
Lysandre
62bf536631 Release v4.12.0 2021-10-28 12:09:49 -04:00
Sylvain Gugger
0f502682fb Pin PyTorch to make CI green 2021-10-21 11:59:23 -04:00
Patrick von Platen
bdf31d6e0a
[Speech] Move all examples to new audio feature (#14045) 
* up

* up

* up

* finish
 2021-10-18 12:52:40 +02:00
Lysandre
11c69b8045 Docs for version v4.11.0 2021-09-27 14:19:38 -04:00
Lysandre
dc193c906d Release: v4.11.0 2021-09-27 14:14:09 -04:00
kding1
6a3a197fcd
Add SigOpt HPO to transformers trainer api (#13572) 
* add sigopt hpo to transformers.

Signed-off-by: Ding, Ke <ke.ding@intel.com>

* extend sigopt changes to test code and others..

Signed-off-by: Ding, Ke <ke.ding@intel.com>

* Style.

* fix style for sigopt integration.

Signed-off-by: Ding, Ke <ke.ding@intel.com>

* Add necessary information to run unittests on SigOpt.

Co-authored-by: Morgan Funtowicz <funtowiczmo@gmail.com>
 2021-09-23 17:01:51 +02:00
Stas Bekman
bec2e3f55c
[deepspeed] replaced deprecated init arg (#13587) 
* [deepspeed] replaced deprecated init arg

* Trigger CI
 2021-09-16 12:12:16 -07:00
elishowk
c783e14887
upgrade sentencepiece version (#13564) 2021-09-15 15:25:03 +02:00
Sylvain Gugger
3081d3868e
Push to hub when saving checkpoints (#13503) 
* Push to hub when saving checkpoints

* Add model card

* Revert partial model card

* Small fix for checkpoint

* Add tests

* Add documentation

* Fix tests

* Bump huggingface_hub

* Fix test
 2021-09-14 08:02:15 -04:00
Anurag Kumar
92d4ef9ab0
Update setup.py (#13421) 2021-09-06 17:32:24 -04:00
Shiv Dhar
75858ca156
Update version of packaging package (#13454) 2021-09-06 17:19:02 -04:00
Li-Huai (Allan) Lin
85cb447766 Revert "Correct wrong function signatures on the docs website (#13198)" 
This reverts commit ffecfea949.
 2021-09-01 09:17:08 -04:00