riaz.somc/transformers
1
0
Fork 0
mirror of https://github.com/huggingface/transformers.git synced 2025-07-16 19:18:24 +06:00
Code Issues Actions 34 Packages Projects Releases Wiki Activity
8,821 Commits 872 Branches 227 Tags 2 GiB
7d45a2e81c
Commit Graph

87 Commits

Author SHA1 Message Date
Lysandre
eab338104d Docs for version v4.16.0 2022-01-27 13:11:51 -05:00
Lysandre
f87db5e412 Release: v4.16.0 2022-01-27 13:06:33 -05:00
Kamal Raj
d1f5ca1afd
[FLAX] glue training example refactor (#13815) 
* refactor run_flax_glue.py

* updated readme

* rm unused import and args typo fix

* refactor

* make consistent arg name across task

* has_tensorboard check

* argparse -> argument dataclasses

* refactor according to review

* fix
 2022-01-19 12:04:51 +01:00
Stas Bekman
762416ffa8
[examples/flax/language-modeling] set loglevel (#15129) 2022-01-13 15:17:28 +01:00
Yih-Dar
9f89fa02ed
Add Flax image captioning example (#14864) 
* add image captioning example

* update README

* fix style & quality

* simplify

* apply review suggestions

* Apply suggestions from code review

Co-authored-by: Suraj Patil <surajp815@gmail.com>

* Apply suggestions from code review

Co-authored-by: Suraj Patil <surajp815@gmail.com>

* Apply review suggestions

* add comments about using np instead jax array

* remove unused lines

* add model creation script

* only support from_pretrained

* fix style

* fix

* not use cache_dir when creating model

* fix tokenizer creation

* update README

* fix quality

* apply suggestion

* simplify some blocks

* Update examples/flax/image-captioning/README.md


* Update examples/flax/image-captioning/run_image_captioning_flax.py

Co-authored-by: Suraj Patil <surajp815@gmail.com>

* apply suggestion

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
Co-authored-by: Suraj Patil <surajp815@gmail.com>
 2022-01-06 14:00:54 +01:00
Patrick von Platen
fa39ff9fc4 Docs for v4.16.0dev0 2021-12-22 20:39:44 +01:00
Patrick von Platen
05fa1a7ac1 Release: v4.15.0 2021-12-22 18:43:15 +01:00
Lysandre
7c9c41f43c Docs for v4.14.0 2021-12-15 18:29:53 +01:00
Lysandre
960d8cb41d Release: v4.14.0 2021-12-15 18:20:35 +01:00
Yih-Dar
a94105f95f
Fix preprocess_function in run_summarization_flax.py (#14769) 
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2021-12-15 11:36:28 +01:00
Benjamin Minixhofer
2a606f9974
Make data shuffling in run_clm_flax.py respect global seed (#13410) 
* use jax and jnp instead of numpy in data_loader

* return batches as np.ndarray
 2021-12-14 11:04:43 +01:00
Suraj Patil
6a025487a6
[Flax examples] remove dependancy on pytorch training args (#14636) 
* use custom training arguments

* update tests
 2021-12-12 09:19:12 +05:30
Lysandre
ab31b3e41b Docs for v4.14.0dev0 2021-12-09 17:09:23 +01:00
Lysandre
4da3a696e4 Release: v4.13.0 2021-12-09 16:55:21 +01:00
Suraj Patil
75ae287aec
fix flax examples tests (#14646) 
* make tensorboard optional

* update test_fetcher for flax examples

* make the tests slow
 2021-12-07 00:34:27 +05:30
Suraj Patil
cbe6026536
fix flax example tests (#14643) 2021-12-06 23:14:37 +05:30
Julien Chaumond
6cdc3a7844
[urls to hub] Replace outdated model tags with their now-canonical pipeline types (#14617) 
* Replace outdated model tags with their now-canonical pipeline types

* spam the CI till it's green
 2021-12-06 04:35:01 -05:00
Suraj Patil
c5bd732ac6
Add Flax example tests (#14599) 
* add test for glue

* add tests for clm

* fix clm test

* add summrization tests

* more tests

* fix few tests

* add test for t5 mlm

* fix t5 mlm test

* fix tests for multi device

* cleanup

* ci job

* fix metric file name

* make t5 more robust
 2021-12-06 10:48:58 +05:30
Rahul Nadkarni
8332327dca
Fix sentinel token IDs in data collator for Flax T5 pretraining script (#14477) 2021-11-29 17:30:17 +01:00
Kamal Raj
2bd950ca47
[Flax] token-classification model steps enumerate start from 1 (#14547) 
* step start from 1

* Updated cur_step calcualtion
 2021-11-29 21:55:59 +05:30
Nicholas Broad
69e16abf98
Switch from using sum for flattening lists of lists in group_texts (#14472) 
* remove sum for list flattening

* change to chain(*)

* make chain object a list

* delete empty lines

per sgugger's suggestions

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

Co-authored-by: Nicholas Broad <nicholas@nmbroad.com>
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
 2021-11-22 16:17:26 -05:00
Suraj Patil
85a4bda4f4
bump flax version (#14343) 2021-11-09 22:15:22 +05:30
Lysandre
b8fad022a0 v4.13.0.dev0 2021-10-28 12:56:46 -04:00
Lysandre
62bf536631 Release v4.12.0 2021-10-28 12:09:49 -04:00
Dhananjay Shettigar
319beb64eb
#12789 Replace assert statements with exceptions (#13909) 
* #12789 Replace assert statements with exceptions

* fix-copies: made copy changes to utils_qa.py in examples/pytorch/question-answering and examples/tensorflow/question-answering

* minor refactor for clarity
 2021-10-07 09:09:01 -04:00
Yih-Dar
a6ea244f99
Fix: save checkpoint after each epoch and push checkpoint to the hub (#13872) 
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2021-10-05 16:30:13 +05:30
Suraj Patil
7db2a79b38
[examples/flax] use Repository API for push_to_hub (#13672) 
* use Repository for push_to_hub

* update readme

* update other flax scripts

* update readme

* update qa example

* fix push_to_hub call

* fix typo

* fix more typos

* update readme

* use abosolute path to get repo name

* fix glue script
 2021-09-30 16:38:07 +05:30
Lysandre
11c69b8045 Docs for version v4.11.0 2021-09-27 14:19:38 -04:00
Lysandre
dc193c906d Release: v4.11.0 2021-09-27 14:14:09 -04:00
Kamal Raj
78807d86eb
[FLAX] Question Answering Example (#13649) 
* flax qa example

* Updated README:  Added Large model

* added utils_qa.py FULL_COPIES

* Updates:
1. Copyright Year updated
2. added dtype arg
3. passing seed and dtype to load model
4. Check eval flag before running eval

* updated README

* updated code comment
 2021-09-21 18:34:48 +05:30
Avital Oliver
51e5eca612
Add long overdue link to the Google TRC project (#13501) 
* Add long-overdue link to the Google TRC project

* Apply suggestions from code review

Co-authored-by: Suraj Patil <surajp815@gmail.com>
Co-authored-by: Stefan Schweter <stefan@schweter.it>
 2021-09-14 13:41:55 +05:30
Stefan Schweter
09549aa18c
examples: minor fixes in flax example readme (#13502) 2021-09-10 11:45:57 +05:30
Kamal Raj
1c191efc3a
flax ner example (#13365) 
* flax ner example

* added task to README

* updated readme

* 1. ArgumentParser -> HfArgumentParser
2. step-wise logging,eval and save

* added requirements.txt

* added progress bar

* updated README

* added check_min_version

* updated training data permuattion with JAX

* added metric lib to requirements

* updated readme table

* fixed imports
 2021-09-09 10:12:57 +05:30
Stefan Schweter
4046e66e40
examples: only use keep_linebreaks when reading TXT files (#13320) 
* examples: only use keep_linebreaks when reading TXT files for all CLM examples

* examples: only use keep_linebreaks when reading TXT files for all CLM examples

* examples: only use keep_linebreaks when reading TXT files for all CLM examples
 2021-08-28 16:22:29 +02:00
Stefan Schweter
319d840b46
examples: add keep_linebreaks option to CLM examples (#13150) 
* examples: add keep_linebreaks option to text dataset loader for all CLM examples

* examples: introduce new keep_linebreaks option as data argument in CLM examples
 2021-08-27 11:35:45 +02:00
Patrick von Platen
13a9c9a354
[Flax] Refactor gpt2 & bert example docs (#13024) 
* fix_torch_device_generate_test

* remove @

* improve docs for clm

* speed-ups

* correct t5 example as well

* push final touches

* Update examples/flax/language-modeling/README.md

* correct docs for mlm

* Update examples/flax/language-modeling/README.md

Co-authored-by: Patrick von Platen <patrick@huggingface.co>
 2021-08-09 13:37:50 +02:00
abhishek thakur
3ff2cde5ca
tfhub.de -> tfhub.dev (#12565) 2021-08-09 08:11:17 +02:00
Patrick von Platen
2e4082364e
[Flax T5] Speed up t5 training (#13012) 
* fix_torch_device_generate_test

* remove @

* update

* up

* fix

* remove f-stings

* correct readme

* up

Co-authored-by: Patrick von Platen <patrick@huggingface.co>
 2021-08-06 11:21:37 +02:00
Chungman Lee
75b8990d90
fix typo in example/text-classification README (#12974) 
* fix typo in example/text-classification README

* add space to align the table
 2021-08-02 12:58:43 +02:00
Stefan Schweter
3d4b3bc3fd
examples: use correct way to get vocab size in flax lm readme (#12947) 2021-07-30 21:57:53 +05:30
Stefan Schweter
d3c3e722d6
[FLAX] Minor fixes in CLM example (#12914) 
* readme: fix retrieval of vocab size for flax clm example

* examples: fix flax clm example when using training/evaluation files
 2021-07-27 19:48:04 +05:30
Patrick von Platen
13fefdf340
Update README.md 
cc @patil-suraj
 2021-07-20 13:51:15 +02:00
fgaim
66197adc98
Flax MLM: Allow validation split when loading dataset from local file (#12689) 
* Allow validation split when loading dataset from local file

* Flax clm & t5, enable validation split for datasets loaded from local file
 2021-07-20 13:38:25 +02:00
Patrick von Platen
f4399ec570
Update README.md 2021-07-14 12:54:31 +01:00
Nick Doiron
5803a2a7ac
Add ByT5 option to example run_t5_mlm_flax.py (#12634) 
* Allow ByT5 type in Flax T5 script

* use T5TokenizerFast

* change up tokenizer config

* model_args

* reorder imports

* Update run_t5_mlm_flax.py
 2021-07-13 13:39:57 +01:00
Bhadresh Savani
de23ecea36
added test file (#12630) 2021-07-12 12:15:14 +05:30
Patrick von Platen
deecdd4939
[Flax] Fix cur step flax examples (#12608) 
* fix_torch_device_generate_test

* remove @

* fix save problem
 2021-07-09 13:51:28 +01:00
Sylvain Gugger
6f1adc4334
Fix group_lengths for short datasets (#12558) 2021-07-08 07:23:41 -04:00
Ibraheem Moosa
122d7dc34f
Remove logging of GPU count etc logging. (#12569) 
Successfully logging this requires Pytorch. For the purposes of this script we are not using Pytorch.
 2021-07-07 23:05:47 +01:00
Patrick von Platen
7d321b7689
[Flax] Allow retraining from save checkpoint (#12559) 
* fix_torch_device_generate_test

* remove @

* finish
 2021-07-07 19:13:43 +05:30