riaz.somc/transformers
1
0
Fork 0
mirror of https://github.com/huggingface/transformers.git synced 2025-07-04 13:20:12 +06:00
Code Issues Actions 21 Packages Projects Releases Wiki Activity
19,216 Commits 857 Branches 224 Tags 2 GiB
3d15606e64
Commit Graph

159 Commits

Author SHA1 Message Date
Zachary Mueller
febe42b5da
Update no_trainer scripts with new Accelerate functionalities (#16617) 
Adds logging and save/loading to the Accelerate scripts

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
 2022-04-06 15:29:32 -04:00
Lysandre Debut
a180efe7fd Dev version 2022-04-06 11:08:12 -04:00
Karim Foda
24a85cca61
Add use_auth to load_datasets for private datasets to PT and TF examples (#16521) 
* fix formatting and remove use_auth

* Add use_auth_token to Flax examples
 2022-04-04 10:27:45 -04:00
Stas Bekman
a73281e3e4
[examples] max samples can't be bigger than the len of dataset (#16501) 
* [examples] max samples can't be bigger than then len of dataset

* do tf and flax
 2022-03-30 12:33:16 -07:00
Sylvain Gugger
4975002df5
Reorganize file utils (#16264) 
* Split file_utils in several submodules

* Fixes

* Add back more objects

* More fixes

* Who exactly decided to import that from there?

* Second suggestion to code with code review

* Revert wront move

* Fix imports

* Adapt all imports

* Adapt all imports everywhere

* Revert this import, will fix in a separate commit
 2022-03-23 10:26:33 -04:00
Lysandre Debut
eca77f4719
Updates the default branch from master to main (#16326) 
* Updates the default branch from master to main

* Links from `master` to `main`

* Typo

* Update examples/flax/README.md

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
 2022-03-23 03:46:59 -04:00
Sylvain Gugger
79d28e80b6 v4.18.0.dev.0 2022-03-03 10:19:58 -05:00
Sylvain Gugger
45cac3fade
Fix labels stored in model config for token classification examples (#15482) 
* Playing

* Properly set labels in model config for token classification example

* Port to run_ner_no_trainer

* Quality
 2022-02-02 14:23:43 -05:00
Lysandre
eab338104d Docs for version v4.16.0 2022-01-27 13:11:51 -05:00
Lysandre
f87db5e412 Release: v4.16.0 2022-01-27 13:06:33 -05:00
Patrick von Platen
457dd4392b
[Examples] Correct run ner label2id for fine-tuned models (#15017) 
* up

* up

* make style

* apply sylvains suggestions

* apply changes to accelerate as well

* more changes

* Apply suggestions from code review

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
 2022-01-24 21:18:04 +01:00
Patrick von Platen
fa39ff9fc4 Docs for v4.16.0dev0 2021-12-22 20:39:44 +01:00
Patrick von Platen
05fa1a7ac1 Release: v4.15.0 2021-12-22 18:43:15 +01:00
Lysandre
7c9c41f43c Docs for v4.14.0 2021-12-15 18:29:53 +01:00
Lysandre
960d8cb41d Release: v4.14.0 2021-12-15 18:20:35 +01:00
Lysandre
ab31b3e41b Docs for v4.14.0dev0 2021-12-09 17:09:23 +01:00
Lysandre
4da3a696e4 Release: v4.13.0 2021-12-09 16:55:21 +01:00
Kamal Raj
803a8cd18f
updated readme with proper arguments (#14624) 2021-12-05 22:12:51 -05:00
Sylvain Gugger
08a5f57567
Add new LFS prune API (#14294) 2021-11-05 18:58:51 -04:00
Lysandre
b8fad022a0 v4.13.0.dev0 2021-10-28 12:56:46 -04:00
Lysandre
62bf536631 Release v4.12.0 2021-10-28 12:09:49 -04:00
Jangwon Park
41dad89f70
chore: typo on ner accelerate example code (#14150) 2021-10-26 16:23:41 -04:00
Sylvain Gugger
f875fb0e5f
Fix label attribution in token classification examples (#14055) 2021-10-20 07:55:14 -04:00
Patrick von Platen
44eb8bdeea
map only on one process (#13810) 2021-09-30 18:52:53 +02:00
Lysandre
11c69b8045 Docs for version v4.11.0 2021-09-27 14:19:38 -04:00
Lysandre
dc193c906d Release: v4.11.0 2021-09-27 14:14:09 -04:00
Gunjan Chhablani
38580455de
Add model card creation snippet to example scripts (#13730) 
* Update run_glue.py

* Update run_glue.py

* Add model creation snippet to other scripts

* Fix style
 2021-09-24 15:51:46 +02:00
Sylvain Gugger
b7d264be0d
Add push_to_hub to no_trainer examples (#13659) 
* Add push_to_hub to no_trainer examples

* Quality

* Document integration

* Roll out to other examples
 2021-09-21 13:13:30 -04:00
Lysandre
5ee67a4412 Docs for v4.10.0 2021-08-31 16:02:31 +02:00
Lysandre
d12bbe4942 Release: v4.10.0 2021-08-31 15:53:10 +02:00
Allan Lin
91ff480e26
Update namespaces inside torch.utils.data to the latest. (#13167) 
* Update torch.utils.data namespaces to the latest.

* Format

* Update Dataloader.

* Style
 2021-08-19 14:29:51 +02:00
Sylvain Gugger
fd85734e0e
Add option to set max_len in run_ner (#12929) 2021-07-28 09:38:12 -04:00
Sylvain Gugger
303989de0e
Add accelerate to examples requirements (#12888) 2021-07-26 09:57:34 -04:00
Lysandre
40de2d5a4f Docs for v4.10.0dev0 2021-07-22 12:52:25 +02:00
Lysandre
72aee83ced Release: v4.9.0 2021-07-22 12:11:55 +02:00
Bhadresh Savani
04dbea31a9
[Examples] Added context manager to datasets map (#12367) 
* added cotext manager to datasets map

* fixed style and spaces

* fixed warning of deprecation

* changed desc
 2021-06-28 09:14:00 -07:00
Bhadresh Savani
539ee456d4
[Examples] Replicates the new --log_level feature to all trainer-based pytorch (#12359) 
* added log_level

* fix comment

* fixed log_level

* Trigger CI

* Unfied logging

* simplified args for log_level
 2021-06-25 14:58:42 -07:00
Stas Bekman
4a872caef4
remove extra white space from log format (#12360) 2021-06-25 13:20:14 -07:00
michal pitr
d4ce31e839
fixed typo (#12356) 2021-06-25 07:49:29 -04:00
Sylvain Gugger
2150dfed31 v4.9.0.dev0 2021-06-23 13:31:19 -04:00
Sylvain Gugger
9252a5127f Release: v4.8.0 2021-06-23 13:25:56 -04:00
Bhavitvya Malik
e43e11260f
update desc for map in all examples (#12226) 
* update desc for map in all examples

* added plm

* suggestions
 2021-06-17 15:37:31 -04:00
Lysandre
0daadc1919 Docs for v4.8.0 2021-06-17 18:17:42 +02:00
Lysandre
7a6c9fab8e Release: v4.7.0 2021-06-17 17:57:42 +02:00
Sylvain Gugger
7d7ceca396
Model card defaults (#12122) 
* [WIP] Model card defaults

* finetuned_from default value

* Add all mappings to the mapping file

* Be more defensive on finetuned_from arg

* Add default task tag

* Separate tags from tasks

* Edge case for dataset

* Apply suggestions from code review

Co-authored-by: Lysandre Debut <lysandre@huggingface.co>

Co-authored-by: Lysandre Debut <lysandre@huggingface.co>
 2021-06-15 16:01:37 -04:00
kumapo
955b2b97a6
Enable add_prefix_space if model_type is roberta or gpt2 (#12116) 2021-06-15 09:33:21 -04:00
Sylvain Gugger
b8ab541340
Don't log anything before logging is setup in examples (#12121) 
* Don't log anything before logging is setup in examples

* Last example
 2021-06-14 08:03:33 -04:00
Sylvain Gugger
d72e5a3a6d Fix quality 2021-06-10 09:27:11 -04:00
kumapo
472a867626
Add text_column_name and label_column_name to run_ner and run_ner_no_trainer args (#12083) 
* Add text_column_name and label_column_name to run_ner args

* Minor fix: grouping for text and label column name
 2021-06-10 08:03:20 -04:00
Koichi Yasuoka
82a2b76c95
Update run_ner.py with id2label config (#12001) 2021-06-09 07:27:05 -04:00
Sylvain Gugger
f086652b16
Add option to log only once in multinode training (#11819) 
* Add option to long only once in multinode training

* Use an alternate property
 2021-05-25 08:03:43 -04:00
Patrick von Platen
cebb96f53a
Add more subsections to main doc (#11758) 
* add headers to main doc

* Apply suggestions from code review

* update

* upload
 2021-05-18 14:38:56 +01:00
Lysandre
d77eb0cf92 Docs for v4.7.0.dev0 2021-05-12 17:08:35 +02:00
Lysandre
64e78564a5 Release: v4.6.0 2021-05-12 17:03:03 +02:00
Sylvain Gugger
a135f59536
Auto modelcard (#11599) 
* Autogenerate model cards from the Trainer

* ModelCard deprecated

* Fix test

* Style

* Apply suggestions from code review

Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>

* Address review comments

* Quality

* With all metadata

* Metadata

* Post-merge conflict mess

* Data args and all examples

* Default license and languages when possible

Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
 2021-05-11 11:30:34 -04:00
Sylvain Gugger
b29eb247d3
Split checkpoint from model_name_or_path in examples (#11492) 
* Split checkpoint from model_name_or_path in examples

* Address review comments

* Address review comments
 2021-04-29 18:33:47 -04:00
Bhadresh Savani
1d30ec95c7
[Examples] Fixes inconsistency around eval vs val and predict vs test (#11380) 
* added changes for uniformity

* modified files

* corrected typo

* fixed qa scripts

* fix typos

* fixed predict typo in qa no trainer

* fixed test file

* reverted trainer changes

* reverted trainer changes in custom exmaples

* updated readme

* added changes in deepspeed test

* added changes for predict and eval
 2021-04-26 09:24:31 -07:00
Sylvain Gugger
bf2e0cf70b
Trainer push to hub (#11328) 
* Initial support for upload to hub

* push -> upload

* Fixes + examples

* Fix torchhub test

* Torchhub test I hate you

* push_model_to_hub -> push_to_hub

* Apply mixin to other pretrained models

* Remove ABC inheritance

* Add tests

* Typo

* Run tests

* Install git-lfs

* Change approach

* Add push_to_hub to all

* Staging test suite

* Typo

* Maybe like this?

* More deps

* Cache

* Adapt name

* Quality

* MOAR tests

* Put it in testing_utils

* Docs + torchhub last hope

* Styling

* Wrong method

* Typos

* Update src/transformers/file_utils.py

Co-authored-by: Julien Chaumond <julien@huggingface.co>

* Address review comments

* Apply suggestions from code review

Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>

Co-authored-by: Julien Chaumond <julien@huggingface.co>
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
 2021-04-23 09:17:37 -04:00
Sylvain Gugger
dabeb15292
Examples reorg (#11350) 
* Base move

* Examples reorganization

* Update references

* Put back test data

* Move conftest

* More fixes

* Move test data to test fixtures

* Update path

* Apply suggestions from code review

Co-authored-by: Lysandre Debut <lysandre@huggingface.co>

* Address review comments and clean

Co-authored-by: Lysandre Debut <lysandre@huggingface.co>
 2021-04-21 11:11:20 -04:00