riaz.somc/transformers
1
0
Fork 0
mirror of https://github.com/huggingface/transformers.git synced 2025-07-30 17:52:35 +06:00
Code Issues Actions 30 Packages Projects Releases Wiki Activity
8,392 Commits 885 Branches 232 Tags 2.1 GiB
96cc02b51b
Commit Graph

8392 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Stas Bekman
72a6bf33c0
[Bert, et al] fix early device assignment (#14447) 
* fix early device assignment

* more models
 2021-11-18 11:47:49 -08:00
Sylvain Gugger
83ef8bcac2
Fix finite IterableDataset test on multiple GPUs (#14445) 2021-11-18 10:25:06 -05:00
NielsRogge
da36c557f7
Add ImageGPT (#14240) 
* First draft

* More improvements

* Improve conversion script

* Fix init weights for layer norm

* Fix correct model for conversion script

* Don't tie input and output embeddings

* Add print statements for debugging

* Add print statements for debugging

* Fix vocab size of model

* Improve documentation, remove fast tokenizer

* Add ImageGPTForImageClassification, improve docs

* Fix docs issue

* Set verbosity level back to info

* Improve tests

* Fix tests and add figure

* Delete tokenizer file

* Remove ImageGPTTokenizer from init files

* Remove ImageGPTLayer from init files

* Remove ImageGPT tokenizer from docs

* First draft of ImageGPTFeatureExtractor

* Fix typo

* Fix bug

* More improvements

* Apply suggestions from code review, add tests for feature extractor

* Fix layernorm

* Update save_pretrained method

* Fix issue

* Make all tests of ImageGPTFeatureExtractor pass

* Update code examples

* Rename model inputs to pixel_values

* Improve code examples

* Update init_weights to post_init

* Fix post_init
 2021-11-18 16:24:34 +01:00
Sylvain Gugger
d83b0e0c07
Add a post init method to all models (#14431) 
* Add a post init method to all models

* Fix tests

* Fix last tests

* Fix templates

* Add comment

* Forgot to save
 2021-11-18 08:38:09 -05:00
NielsRogge
08816de16a
Fix code example (#14441) 2021-11-18 11:26:54 +01:00
William Held
01f8e639d3
Recover Deleted XNLI Instructions (#14437) 2021-11-17 20:16:47 -05:00
N
1991da07f7
[WIP] Ensure TF model configs can be converted to proper JSON (#14415) 
* test: make sure model configs are jsonifiable

* fix: return python dict instead of config object

* fix: accept pretrained config and use correct class

* Re-enabling slow tests and applying them to core models only

* Re-enabling slow tests and applying them to core models only

* Add new test file to fetcher

* Remove tooslow tests from test_modeling_tf_common.py

* make style

* Style fixes

* Style fixes

* Style fixes

* Style fixes

* Adding core tests to GPT2 and BART

* Removing unused imports

Co-authored-by: niklas.fruehauf <niklas.fruehauf@sovanta.com>
Co-authored-by: matt <rocketknight1@gmail.com>
 2021-11-17 20:24:39 +00:00
Patrick von Platen
754202de4f
[Bart] Fix docs (#14434) 2021-11-17 19:02:33 +01:00
Antonio Carlos Falcão Petri
7544efc92e
[Gradient checkpoining] Update Wav2Vec scripts (#14036) 
Co-authored-by: Stas Bekman <stas@stason.org>
 2021-11-17 18:37:21 +01:00
Lysandre
c6c075544d Docs for version v4.12.5 2021-11-17 11:39:12 -05:00
NielsRogge
a2864a50e7
Improve semantic segmentation models (#14355) 
* Improve tests

* Improve documentation

* Add ignore_index attribute

* Add semantic_ignore_index to BEiT model

* Add segmentation maps argument to BEiTFeatureExtractor

* Simplify SegformerFeatureExtractor and corresponding tests

* Improve tests

* Apply suggestions from code review

* Minor docs improvements

* Streamline segmentation map tests of SegFormer and BEiT

* Improve reduce_labels docs and test

* Fix code quality

* Fix code quality again
 2021-11-17 15:29:58 +01:00
Patrick von Platen
700a748fe6
[Wav2Vec2] Add New Wav2Vec2 Translation (#14392) 
* add new wav2vec2 translation

* correct

* up

* add tests

* correct end copy

* correct more

* up

* correct unispeech sat

* finish

* finalize

* finish

* up
 2021-11-17 14:38:56 +01:00
Sylvain Gugger
b567510cff
Debug doc (#14424) 
* Create branch for tests

* Pin first upgrade

* Really pin

* Polish fix
 2021-11-16 18:58:07 -05:00
Lysandre
888fb21159 Docs for v4.12.4 2021-11-16 17:40:58 -05:00
Valentin
a33168aa78
Avoid looping when data exhausted (#14413) 
* stop training when a finite IterableDataset is exhausted

when using an iterable dataset num_epochs is set to
sys.maxsize to make sure all data is consumed
likewise we want to set max_steps high enough
but still stop when all data is consumed

(cherry picked from commit 6f0e1d6363153da9051e93acffe1cbab3a3f3b12)

* fix typo flase -> false

* add test for stopping training on exhausted finite iterable dataset

* remove redundant gradient_accumulation_steps

* run make style

reformat training_args docstring
 2021-11-16 16:50:04 -05:00
Sylvain Gugger
3e8d17e66d
Add forward method to dummy models (#14419) 
* Add forward method to dummy models

* Fix quality
 2021-11-16 09:24:40 -05:00
Sylvain Gugger
040fd47162
Fix gradient_checkpointing backward compatibility (#14408) 
* Fix gradient_checkpointing backward compatibility

* Remove needless line

* make sure mask prob is big enough and length small enough

* Fix tests

Co-authored-by: patrickvonplaten <patrick.v.platen@gmail.com>
 2021-11-16 08:58:42 -05:00
Lysandre Debut
1cc453d33c
Allow per-version configurations (#14344) 
* Allow per-version configurations

* Update tests/test_configuration_common.py

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* Update tests/test_configuration_common.py

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
 2021-11-15 16:38:02 -05:00
Patrick von Platen
76d0d41e51
[Wav2Vec2] Make sure that gradient checkpointing is only run if needed (#14407) 
* [Wav2Vec2] Make sure that gradient checkpointing is only run if needed

* make fix-copies
 2021-11-15 21:03:10 +01:00
Eldar Kurtic
9fd937ead1
Replace BertLayerNorm with LayerNorm (#14385) 
Running Movement pruning experiments with the newest HuggingFace would crash due to non-existing BertLayerNorm.
 2021-11-15 13:25:10 -05:00
Yih-Dar
a67d47b40c
Fix weight loading issue (#14016) 
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2021-11-15 17:48:40 +01:00
NielsRogge
74e6111ba7
Fix test and docs (#14399) 2021-11-15 17:35:33 +01:00
Patrick von Platen
4ce74edf51
[Speech2Text2] Enable tokenizers (#14390) 
* [Speech2Text2] Enable tokenizers

* minor fix

* Apply suggestions from code review

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
 2021-11-15 16:34:11 +01:00
Matt
267867e851
Quick fix to TF summarization example (#14401) 2021-11-15 13:45:51 +00:00
Stas Bekman
29dfb2dbb1
[doc] performance and parallelism updates (#14391) 
* [doc] performance and parallelism doc update

* improve

* improve
 2021-11-14 17:19:15 -08:00
nbertagnolli
790cdc2e55
Raise exceptions instead of using asserts in modeling_openai #12789 (#14386) 
* Raise exceptions instead of using asserts for control flow in modeling_openai #12789

* reformatted file
 2021-11-13 21:34:34 -05:00
Suraj Patil
2e60276b38
[M2M100Tokenizer] fix _build_translation_inputs (#14382) 
* add return_tensors paramter

* fix test

* Apply suggestions from code review

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* style

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
 2021-11-13 20:57:12 +05:30
Suraj Patil
3165930402
support wmt21 tokenizer in m2m100 tokenizer (#14376) 2021-11-13 14:21:58 +05:30
Li-Huai (Allan) Lin
280a811ecb
Use AlbertConverter for FNet instead of using FNet's own converter (#14365) 
* Add normalizer to FNetConverter

* Style

* Directly use AlbertConverter
 2021-11-12 19:46:40 +01:00
Patrick von Platen
55f49c5f4b
[Wav2Vec2 Example] Improve fine-tuning script (#14373) 
* improve some stuff

* finish

* correct last
 2021-11-12 16:35:57 +01:00
Suraj Patil
21546e59a6
fix docs (#14377) 2021-11-12 15:56:41 +05:30
Nicolas Patry
ed5d15518b
Adding support for raw python generator in addition to Dataset for pipelines (#14352) 
* Adding support for raw python `generator` in addition to `Dataset`

The main goal is to ease the create of streaming data to the pipe.

`Dataset` is more involved and pytorch specific.

This PR, provides a way to use a python iterator too.
This enabled #14250 but can be proposed as a standalone PR.

```python
from transformers import pipeline

def read_data(filename):
    with open(filename, 'r') as f:
        for line in f:
            yield f

pipe = pipeline("text-classification")
for classified in pipe(read_data("large_file.txt")):
    print("Success ! ", classified)
```

The main caveat of this, is the interaction with `DataLoader` with
`num_workers>1`. When you have multiple workers, each receive a copy
of the generator (like `IterableDataset`). That means the naive Iterator
will fail since all workers iterate on all items of the generator.

There are ways to do clever "skipping", but it could be bad still
because all workers still do have to pass through all items of the
generator (they just ignore items they don't handle), depending on
the case it might be bad.

Using `num_workers=1` is the simplest fix and if the cost of loading
your data is small enough should be good enough. In the above example
trying to do smart tricks to skip some lines is unlikely to be a net
positive for instance.

If there are better ways to do "jumps" on some data, then using
`Dataset` is more advised (since then differents workers can just jump
themselves).

* Adding iterator support for `tf` too.
 2021-11-12 09:20:40 +01:00
Stas Bekman
77262ef750
fix --gradient_checkpointing (#13964) 2021-11-11 17:50:21 +01:00
Suraj Patil
3d607df8f4
fix loading flax bf16 weights in pt (#14369) 
* fix loading flax bf16 weights in pt

* fix clip test

* fix t5 test

* add logging statement

* Update src/transformers/modeling_flax_pytorch_utils.py

Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>

* switch back to native any

* fix check for bf16 weights

Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
 2021-11-11 21:20:49 +05:30
Matt
7f20bf0d43
Fixing requirements for TF LM models and use correct model mappings (#14372) 
* Fixing requirements for TF LM models and use correct model mappings

* make style
 2021-11-11 15:34:00 +00:00
Matt
4c35c8d89c
Experimenting with adding proper get_config() and from_config() methods (#14361) 
* Experimenting with adding proper get_config() and from_config() methods

* Adding a test for get/from config

* Fix test for get/from config
 2021-11-11 14:21:50 +00:00
Suraj Patil
b1dbdf22ef
pass params to encode (#14370) 2021-11-11 17:16:24 +05:30
Suraj Patil
e92190c0f8
Fix Flax params dtype (#13098) 
* fix inits

* fix embed dtype

* fix embed dtype

* add test to check default dtype

* quality

* add type conversion methods for flax models

* more robust casting

* cast sinusoidal positions

* update pegasus

* update albert

* update test

* make sure dtype is passed to every module

* style

* fix electra dense

* fix t5

* quality

* add more tests

* better name

* use the dtype for lm head computation

* fix albert

* style

* fix albert embed dtype

* more tests

* fix vision enc-dec

* cleanup

* fix embed dtype pegasus

* fix default param test

* doc

* update template

* fix final_logits_bias dtype

* Apply suggestions from code review

Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>

* fix doc

* fix doc

* add detailed docstring for dtype parameter

* remove un-necessary import

Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
 2021-11-11 14:45:20 +05:30
Stas Bekman
1c76a51615
solve the port conflict (#14362) 2021-11-10 19:11:45 -08:00
Li-Huai (Allan) Lin
9e37c5cdf8
Fix list index out of range when padding nested empty lists (#13876) 
* Fix index out of range when padding

* Apply suggestions from code review

* Style
 2021-11-10 21:34:52 +01:00
Chang Wang
bec02ff209
enhance rewrite state_dict missing _metadata (#14348) 2021-11-10 07:25:41 -05:00
Ella Charlaix
2b0d9389f8
Add notebook INC quantization for text classification tasks (#14293) 
* Add notebook applying Intel Neural Compressor quantization for text classification tasks

* Add Optimum notebooks section
 2021-11-10 12:49:43 +01:00
Li-Huai (Allan) Lin
ea163d0948
Fix fast tokenization problems (#13930) 
* Fix albert mask token tokenization.

* Ensure special tokans sanitized.

* Style

* Fix

* Apply suggestions from code review
 2021-11-10 11:16:45 +01:00
Nicolas Patry
5c153079e2
Adding some quality of life for pipeline function. (#14322) 
* Adding some quality of life for `pipeline` function.

* Update docs/source/main_classes/pipelines.rst

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* Update src/transformers/pipelines/__init__.py

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* Improve the tests.

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
 2021-11-10 10:18:35 +01:00
Elad Segal
321eb56222
BatchFeature: Convert List[np.ndarray] to np.ndarray before converting to pytorch tensors (#14306) 
* update

* style fix

* retrigger checks

* check first element

* fix syntax error

* Update src/transformers/feature_extraction_utils.py

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* remove import

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
 2021-11-09 22:23:08 -05:00
Sylvain Gugger
46d0cdae40
Support for TF >= 2.7 (#14345) 2021-11-09 18:49:29 -05:00
Patrick von Platen
e81d8d7fa9
[Bert2Bert] allow bert2bert + relative embeddings (#14324) 
* [Bert2Bert] allow bert2bert + relative embeddings

* up

* Update README_ko.md

* up

* up
 2021-11-09 14:26:58 -05:00
Steven Liu
e4d8f517b9
Rewrite guides for fine-tuning with Datasets (#13923) 
* rewrite guides for fine-tuning with datasets

* simple qa code example

* use anonymous rST links

* style
 2021-11-09 14:12:50 -05:00
Suraj Patil
85a4bda4f4
bump flax version (#14343) 2021-11-09 22:15:22 +05:30
Yih-Dar
babd0b9a5e
remove test_model_various_embeddings (#14341) 
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2021-11-09 11:30:17 -05:00