riaz.somc/transformers
1
0
Fork 0
mirror of https://github.com/huggingface/transformers.git synced 2025-07-23 06:20:22 +06:00
Code Issues Actions 34 Packages Projects Releases Wiki Activity
14,387 Commits 880 Branches 229 Tags 2 GiB
00d8502b7a
Commit Graph

161 Commits

Author SHA1 Message Date
Lysandre
bc78fd1274 Dev version 2023-11-02 18:15:36 +01:00
Lucain
66b088faf0
Provide alternative when warning on use_auth_token (#27105) 2023-10-27 14:32:54 +02:00
Tom Aarsen
40ea9ab2a1
Add many missing spaces in adjacent strings (#26751) 
Add missing spaces in adjacent strings
 2023-10-12 10:28:40 +02:00
Roy Hvaara
fc63914399
[JAX] Replace uses of jnp.array in types with jnp.ndarray. (#26703) 
`jnp.array` is a function, not a type:
https://jax.readthedocs.io/en/latest/_autosummary/jax.numpy.array.html
so it never makes sense to use `jnp.array` in a type annotation. Presumably the intent was to write `jnp.ndarray` aka `jax.Array`.

Co-authored-by: Peter Hawkins <phawkins@google.com>
 2023-10-10 21:35:16 +02:00
Lysandre
bd6205919a v4.35.0.dev0 2023-10-03 16:54:37 +02:00
Sanchit Gandhi
68e85fc822
[Flax Examples] Seq2Seq ASR Fine-Tuning Script (#21764) 
* from seq2seq speech

* [Flax] Example script for speech seq2seq

* tests and fixes

* make style

* fix: label padding tokens

* fix: label padding tokens over list

* update ln names for Whisper

* try datasets iter loader

* create readme and append results

* style

* make style

* adjust lr

* use pt dataloader

* make fast

* pin gen max len

* finish

* add pt to requirements for test

* fix pt -> torch

* add accelerate
 2023-09-29 16:42:58 +01:00
Phuc Van Phan
910faa3e1f
feat: adding num_proc to load_dataset (#26326) 
* feat: adding num_proc to load_dataset

* feat: add add_num_proc for run_mlm_flax

* feat: add num_proc for bart and t5

* chorse: remove
 2023-09-22 19:22:47 +02:00
Phuc Van Phan
8b5da9fc6e
refactor: change default block_size in block size > max position embeddings (#26069) 
* refactor: change default block_size when not initialize

* reformat: add the min of block size
 2023-09-18 16:47:57 +01:00
Phuc Van Phan
5af2c62696
docs: add space to docs (#26067) 
* docs: add space to docs

* docs: remove reduntant space
 2023-09-11 22:03:26 +01:00
Phuc Van Phan
9cebae64ad
docs: update link huggingface map (#26077) 2023-09-11 12:57:04 +01:00
Lysandre
d8e13b3e04 v4.34.dev.0 2023-09-04 15:12:11 -04:00
Sylvain Gugger
5c67682b16
v4.33.0.dev0 2023-08-21 07:07:04 -04:00
Jackmin801
145109382a
Allow trust_remote_code in example scripts (#25248) 
* pytorch examples

* pytorch mim no trainer

* cookiecutter

* flax examples

* missed line in pytorch run_glue

* tensorflow examples

* tensorflow run_clip

* tensorflow run_mlm

* tensorflow run_ner

* tensorflow run_clm

* pytorch example from_configs

* pytorch no trainer examples

* Revert "tensorflow run_clip"

This reverts commit 261f86ac1f.

* fix: duplicated argument
 2023-08-07 16:32:25 +02:00
Yih-Dar
149cb0cce2
Add token arugment in example scripts (#25172) 
* fix

* fix

* fix

* fix

* fix

* fix

* fix

---------

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2023-08-02 11:17:31 +02:00
Yih-Dar
d53b8ad780
Update use_auth_token -> token in example scripts (#25167) 
* pytorch examples

* tensorflow examples

* flax examples

---------

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2023-07-28 15:33:45 +02:00
Lucain
6232c380f2
Fix .push_to_hub and cleanup get_full_repo_name usage (#25120) 
* Fix .push_to_hub and cleanup get_full_repo_name usage

* Do not rely on Python bool conversion magic

* request changes
 2023-07-28 11:40:08 +02:00
Sylvain Gugger
e9ad51306f
4.32.0.dev0 2023-07-17 13:30:44 -04:00
Bauke Brenninkmeijer
fc9e387dc0
Replacement of 20 asserts with exceptions (#24757) 
* initial replacements of asserts with errors/exceptions

* replace assert with exception in generation, align and bart

* reset formatting change

* reset another formatting issue

* Apply suggestion

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* don't touch this file

* change to 'is not False'

* fix type

---------

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
 2023-07-12 07:45:09 -04:00
Sylvain Gugger
ba695c1efd
v4.31.0.dev0 2023-06-07 16:49:00 -04:00
sshahrokhi
6f72e71f97
changing the requirements to a cpu torch version that works (#23483) 2023-05-22 12:58:55 -04:00
Sylvain Gugger
a0c0a78233
v4.30.0.dev0 2023-05-09 14:59:38 -04:00
Alex Punnen
805db1fe13
num_noise_spans should be <= num_items #22246 (#22938) 2023-05-02 13:07:30 -04:00
Sylvain Gugger
888c4a2ae0
v4.29.0.dev0 2023-04-12 20:04:29 -04:00
Sylvain
ef28df0572 Fix quality due to ruff release 2023-03-22 20:45:08 -04:00
Connor Henderson
8e6c34b390
fix: Allow only test_file in pytorch and flax summarization (#22293) 
allow only test_file in pytorch and flax summarization
 2023-03-22 10:46:56 +00:00
Sylvain Gugger
ebdb185bef
v4.28.0.dev0 2023-03-14 13:49:10 -04:00
Aaron Gokaslan
5e8c8eb5ba
Apply ruff flake8-comprehensions (#21694) 2023-02-22 09:14:54 +01:00
Sylvain Gugger
6f79d26442
Update quality tooling for formatting (#21480) 
* Result of black 23.1

* Update target to Python 3.7

* Switch flake8 to ruff

* Configure isort

* Configure isort

* Apply isort with line limit

* Put the right black version

* adapt black in check copies

* Fix copies
 2023-02-06 18:10:56 -05:00
Sylvain Gugger
7119bb052a
v4.27.0.dev0 2023-01-23 16:52:35 -05:00
amyeroberts
4bc18e7a83
Update examples with image processors (#21155) 
* Update examples to use image processors

* Small fixes

* Resolve conflicts
 2023-01-19 15:14:58 +00:00
Sylvain Gugger
05e72aa0c4
Adapt repository creation to latest hf_hub (#21158) 
* Adapt repository creation to latest hf_hub

* Update all examples

* Fix other tests, add Flax examples

* Address review comments
 2023-01-18 11:14:00 -05:00
milyiyo
3b309818e7
Refactor the function get_results (#20999) 2023-01-04 12:05:36 -05:00
fzyzcjy
ae3cbbcaf6
Fix tiny typo (#20841) 
* Fix typo

* Update README.md

* Update run_mlm_flax_stream.py

* Update README.md
 2022-12-20 03:17:59 -05:00
Sylvain Gugger
60d1f31bb0
v4.26.0.dev0 2022-12-01 16:19:33 -05:00
Katie Le
667ccea722
Replace assertion with ValueError exceptions in run_image_captioning_flax.py (#20365) 
* replace 4 asserts with ValueError exception for control flow

* Update examples/flax/image-captioning/run_image_captioning_flax.py

Co-authored-by: Sanchit Gandhi <93869735+sanchit-gandhi@users.noreply.github.com>

* Update examples/flax/image-captioning/run_image_captioning_flax.py

Co-authored-by: Sanchit Gandhi <93869735+sanchit-gandhi@users.noreply.github.com>

* reformatted file

* uninstalled trasformers and applied make style

Co-authored-by: Bibi <Bibi@katies-mac.local>
Co-authored-by: Sanchit Gandhi <93869735+sanchit-gandhi@users.noreply.github.com>
 2022-11-28 15:06:25 +00:00
Sylvain Gugger
c3a93d8d82
v4.25.0.dev0 2022-10-31 21:48:40 -04:00
Duong A. Nguyen
4212bb0d60
[Re-submit] Compute true loss Flax examples (#19504) 
* Compute true loss

* fixup

* final

* final

* final

* Update examples/flax/language-modeling/run_bart_dlm_flax.py

Co-authored-by: Sanchit Gandhi <93869735+sanchit-gandhi@users.noreply.github.com>

* jax.tree_map => jax.tree_util.tree_map

* Compute true loss

* final

* fixup

* final

* final

* Update examples/flax/language-modeling/run_bart_dlm_flax.py

Co-authored-by: Sanchit Gandhi <93869735+sanchit-gandhi@users.noreply.github.com>

* jax.tree_map => jax.tree_util.tree_map

Co-authored-by: Sanchit Gandhi <93869735+sanchit-gandhi@users.noreply.github.com>
 2022-10-13 11:33:36 +01:00
Lysandre
10100979ed Dev version 2022-10-10 17:25:40 -04:00
Kaiyu Yang
e150c4e2fe
Fix the error message in run_t5_mlm_flax.py (#19282) 2022-10-10 14:51:11 +01:00
Lysandre
16913b3c92 Dev version 2022-09-14 14:58:20 -04:00
Sanchit Gandhi
e6f221c8d4
[JAX] Replace all jax.tree_* calls with jax.tree_util.tree_* (#18361) 
* [JAX] Replace all jax.tree_* calls with jax.tree_util.tree_*

* fix double tree_util
 2022-09-09 15:18:56 +02:00
Karim Foda
d6eeb87170
Flax Remat for LongT5 (#17994) 
* [Flax] Add remat (gradient checkpointing)

* fix variable naming in test

* flip: checkpoint using a method

* fix naming

* fix class naming

* apply PVP's suggestions from code review

* add gradient_checkpointing to examples

* Add gradient_checkpointing to run_mlm_flax

* Add remat to longt5

* Add gradient checkpointing test longt5

* Fix args errors

* Fix remaining tests

* Make fixup & quality fixes

* replace kwargs

* remove unecessary kwargs

* Make fixup changes

* revert long_t5_flax changes

* Remove return_dict and copy to LongT5

* Remove test_gradient_checkpointing

Co-authored-by: sanchit-gandhi <sanchit@huggingface.co>
 2022-08-14 16:27:13 +01:00
Julien Chaumond
9129fd0377
transformers-cli login => huggingface-cli login (#18490) 
* zero chance anyone's using that constant no?

* `transformers-cli login` => `huggingface-cli login`

* `transformers-cli repo create` => `huggingface-cli repo create`

* `make style`
 2022-08-06 09:42:55 +02:00
Duong A. Nguyen
3909d7f139
Add Flax BART pretraining script (#18297) 
* add bart pretraining flax script

* fixup

* add bart pretraining flax script

* add BART to README

* add BART to README

* add BART to README

* add BART to README

* add BART to README

* add bos eos document

* Update README.md

* Update README.md

* Update examples/flax/language-modeling/run_bart_dlm_flax.py

Co-authored-by: Sanchit Gandhi <93869735+sanchit-gandhi@users.noreply.github.com>

* final

* final

* final

* remove use_auth_token ing from_config

Co-authored-by: Sanchit Gandhi <93869735+sanchit-gandhi@users.noreply.github.com>
 2022-08-01 12:06:30 -04:00
Sylvain Gugger
941d233153
Fix ROUGE add example check and update README (#18398) 
* Fix ROUGE add example check and update README

* Stay consistent in values
 2022-08-01 11:14:49 -04:00
Sylvain Gugger
986526a0e4
Replace as_target context managers by direct calls (#18325) 
* Preliminary work on tokenizers

* Quality + fix tests

* Treat processors

* Fix pad

* Remove all uses of  in tests, docs and examples

* Replace all as_target_tokenizer

* Fix tests

* Fix quality

* Update examples/flax/image-captioning/run_image_captioning_flax.py

Co-authored-by: amyeroberts <amy@huggingface.co>

* Style

Co-authored-by: amyeroberts <amy@huggingface.co>
 2022-07-29 08:09:09 -04:00
Vijay S Kalmath
da503ea02f
Migrate metrics used in flax examples to Evaluate (#18348) 
Currently, tensorflow examples use the `load_metric` function from
Datasets library, commit migrates function call to `load` function
from Evaluate library.
 2022-07-28 15:06:23 -04:00
Lysandre
c89a592e87 Dev version 2022-07-27 17:13:57 +02:00
Sanchit Gandhi
7490a97cac
[Flax] Fix incomplete batches in example scripts (#17863) 
* [Flax] Fix incomplete batches in example scripts

* fix dataloader batching

* convert jnp batch idxs to np array

* add missing `pad_shard_unpad` to final prediction generate step

* only `pad_shard_unpad` at inference time

* merge conflicts

* remove incomplete batch step from eval

* fix run_qa.py

* add `pad_shard_unpad` to run_flax_ner.py

* add `pad_shard_unpad` to run_flax_glue.py

* add `pad_shard_unpad` to run_image_classification.py

* make style

* fix mlm flax eval batches

* remove redundant imports
 2022-07-27 15:50:47 +01:00
Duong A. Nguyen
170fcaa604
Generalize decay_mask_fn to apply mask to all LayerNorm params (#18273) 
* generalize decay_mask_fn to find all layernorm params

* fixup

* generalising decay_mask_fn
 2022-07-27 12:23:57 +01:00