riaz.somc/transformers
1
0
Fork 0
mirror of https://github.com/huggingface/transformers.git synced 2025-07-31 02:02:21 +06:00
Code Issues Actions 33 Packages Projects Releases Wiki Activity
8,353 Commits 887 Branches 232 Tags 2.1 GiB
11f65d4158
Commit Graph

8353 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Stas Bekman
11f65d4158
[test] add test for --config_overrides (#14466) 
* add test for --config_overrides

* remove unneeded parts of the test
 2021-11-22 11:33:43 -05:00
Daniel Stancl
e0e2da1194
Improve a add-new-pipeline docs a bit (#14485) 2021-11-22 10:35:49 -05:00
Nicolas Patry
a4553e6c64
Moving pipeline tests from Narsil to hf-internal-testing. (#14463) 
* Moving everything to `hf-internal-testing`.

* Fixing test values.

* Moving to other repo.

* Last touch?
 2021-11-22 04:40:45 -05:00
Sylvain Gugger
1a92bc5788
Fix dummy objects for quantization (#14478) 
* Fix dummy objects for quantization

* Add more models
 2021-11-21 17:39:20 -05:00
Alexander Measure
c9d2cf855a
add Tuple as possible type hint for EvalPredictions label_ids (#14473) 
* Update trainer_utils.py

* add Tuple type hints to all label_ids outputs

affects EvalLoopOutput and PredicctionOutput
 2021-11-21 10:31:09 -05:00
Shang Zhang
a59e7c1ed4
Add QDQBert model and quantization examples of SQUAD task (#14066) 
* clean up branch for add-qdqbert-model

* README update for QAT example; update docstrings in modeling_qdqbert.py

* Update qdqbert.rst

* Update README.md

* Update README.md

* calibration data using traning set; QAT example runs in fp32

* re-use BERTtokenizer for qdqbert

* Update docs/source/model_doc/qdqbert.rst

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* Update docs/source/model_doc/qdqbert.rst

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* Update docs/source/model_doc/qdqbert.rst

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* remove qdqbert tokenizer

* Update qdqbert.rst

* update evaluate-hf-trt-qa.py

* update configuration_qdqbert.py

* update modeling_qdqbert.py: add copied statement; replace assert with ValueError

* update copied from statement

* add is_quantization_available; run make fix-copies

* unittest add require_quantization

* add backend dependency to qdqbert model

* update README; update evaluate script; make style

* lint

* docs qdqbert update

* circleci build_doc add pytorch-quantization for qdqbert

* update README

* update example readme with instructions to upgrade TensorRT to 8.2

* Update src/transformers/models/qdqbert/configuration_qdqbert.py

Co-authored-by: Lysandre Debut <lysandre@huggingface.co>

* Update src/transformers/models/qdqbert/configuration_qdqbert.py

Co-authored-by: Lysandre Debut <lysandre@huggingface.co>

* Update src/transformers/models/qdqbert/configuration_qdqbert.py

Co-authored-by: Lysandre Debut <lysandre@huggingface.co>

* Update src/transformers/models/qdqbert/configuration_qdqbert.py

Co-authored-by: Lysandre Debut <lysandre@huggingface.co>

* change quantization to pytorch_quantization for backend requirement

* feed_forward_chunking not supported in QDQBert

* make style

* update model docstrings and comments in testing scripts

* rename example to quantization-qdqbert; rename example scripts from qat to quant

* Update src/transformers/models/qdqbert/modeling_qdqbert.py

Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>

* rm experimental functions in quant_trainer

* qa cleanup

* make fix-copies for docs index.rst

* fix doctree; use post_init() for qdqbert

* fix early device assignment for qdqbert

* fix CI:Model templates runner

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
Co-authored-by: Lysandre Debut <lysandre@huggingface.co>
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
 2021-11-19 13:33:39 -05:00
Nicolas Patry
81fe8afaac
Adding support for hidden_states and attentions in unbatching (#14420) 
support.
 2021-11-19 15:37:52 +01:00
Patrick von Platen
f25a9332e8
[Generation] Allow inputs_embeds as an input (#14443) 
* up

* finalize

* finalize

* finish

* Update src/transformers/generation_utils.py

* apply feedback
 2021-11-19 15:35:06 +01:00
NielsRogge
0490b98877
[ImageGPT] Small fixes (#14460) 
* Add integration test

* Fix typo
 2021-11-19 15:15:02 +01:00
Lysandre Debut
331c3d2aa0
Add GitPython to quality tools (#14459) 
* Update setup.py

* Update setup.py

* Update setup.py

* Remove GitPython install
 2021-11-19 08:43:48 -05:00
Patrick von Platen
efea0f868b
[Speech Recognition] More examples 
Add more XLS-R training runs to the official examples
 2021-11-18 23:42:02 +01:00
Stas Bekman
72a6bf33c0
[Bert, et al] fix early device assignment (#14447) 
* fix early device assignment

* more models
 2021-11-18 11:47:49 -08:00
Sylvain Gugger
83ef8bcac2
Fix finite IterableDataset test on multiple GPUs (#14445) 2021-11-18 10:25:06 -05:00
NielsRogge
da36c557f7
Add ImageGPT (#14240) 
* First draft

* More improvements

* Improve conversion script

* Fix init weights for layer norm

* Fix correct model for conversion script

* Don't tie input and output embeddings

* Add print statements for debugging

* Add print statements for debugging

* Fix vocab size of model

* Improve documentation, remove fast tokenizer

* Add ImageGPTForImageClassification, improve docs

* Fix docs issue

* Set verbosity level back to info

* Improve tests

* Fix tests and add figure

* Delete tokenizer file

* Remove ImageGPTTokenizer from init files

* Remove ImageGPTLayer from init files

* Remove ImageGPT tokenizer from docs

* First draft of ImageGPTFeatureExtractor

* Fix typo

* Fix bug

* More improvements

* Apply suggestions from code review, add tests for feature extractor

* Fix layernorm

* Update save_pretrained method

* Fix issue

* Make all tests of ImageGPTFeatureExtractor pass

* Update code examples

* Rename model inputs to pixel_values

* Improve code examples

* Update init_weights to post_init

* Fix post_init
 2021-11-18 16:24:34 +01:00
Sylvain Gugger
d83b0e0c07
Add a post init method to all models (#14431) 
* Add a post init method to all models

* Fix tests

* Fix last tests

* Fix templates

* Add comment

* Forgot to save
 2021-11-18 08:38:09 -05:00
NielsRogge
08816de16a
Fix code example (#14441) 2021-11-18 11:26:54 +01:00
William Held
01f8e639d3
Recover Deleted XNLI Instructions (#14437) 2021-11-17 20:16:47 -05:00
N
1991da07f7
[WIP] Ensure TF model configs can be converted to proper JSON (#14415) 
* test: make sure model configs are jsonifiable

* fix: return python dict instead of config object

* fix: accept pretrained config and use correct class

* Re-enabling slow tests and applying them to core models only

* Re-enabling slow tests and applying them to core models only

* Add new test file to fetcher

* Remove tooslow tests from test_modeling_tf_common.py

* make style

* Style fixes

* Style fixes

* Style fixes

* Style fixes

* Adding core tests to GPT2 and BART

* Removing unused imports

Co-authored-by: niklas.fruehauf <niklas.fruehauf@sovanta.com>
Co-authored-by: matt <rocketknight1@gmail.com>
 2021-11-17 20:24:39 +00:00
Patrick von Platen
754202de4f
[Bart] Fix docs (#14434) 2021-11-17 19:02:33 +01:00
Antonio Carlos Falcão Petri
7544efc92e
[Gradient checkpoining] Update Wav2Vec scripts (#14036) 
Co-authored-by: Stas Bekman <stas@stason.org>
 2021-11-17 18:37:21 +01:00
Lysandre
c6c075544d Docs for version v4.12.5 2021-11-17 11:39:12 -05:00
NielsRogge
a2864a50e7
Improve semantic segmentation models (#14355) 
* Improve tests

* Improve documentation

* Add ignore_index attribute

* Add semantic_ignore_index to BEiT model

* Add segmentation maps argument to BEiTFeatureExtractor

* Simplify SegformerFeatureExtractor and corresponding tests

* Improve tests

* Apply suggestions from code review

* Minor docs improvements

* Streamline segmentation map tests of SegFormer and BEiT

* Improve reduce_labels docs and test

* Fix code quality

* Fix code quality again
 2021-11-17 15:29:58 +01:00
Patrick von Platen
700a748fe6
[Wav2Vec2] Add New Wav2Vec2 Translation (#14392) 
* add new wav2vec2 translation

* correct

* up

* add tests

* correct end copy

* correct more

* up

* correct unispeech sat

* finish

* finalize

* finish

* up
 2021-11-17 14:38:56 +01:00
Sylvain Gugger
b567510cff
Debug doc (#14424) 
* Create branch for tests

* Pin first upgrade

* Really pin

* Polish fix
 2021-11-16 18:58:07 -05:00
Lysandre
888fb21159 Docs for v4.12.4 2021-11-16 17:40:58 -05:00
Valentin
a33168aa78
Avoid looping when data exhausted (#14413) 
* stop training when a finite IterableDataset is exhausted

when using an iterable dataset num_epochs is set to
sys.maxsize to make sure all data is consumed
likewise we want to set max_steps high enough
but still stop when all data is consumed

(cherry picked from commit 6f0e1d6363153da9051e93acffe1cbab3a3f3b12)

* fix typo flase -> false

* add test for stopping training on exhausted finite iterable dataset

* remove redundant gradient_accumulation_steps

* run make style

reformat training_args docstring
 2021-11-16 16:50:04 -05:00
Sylvain Gugger
3e8d17e66d
Add forward method to dummy models (#14419) 
* Add forward method to dummy models

* Fix quality
 2021-11-16 09:24:40 -05:00
Sylvain Gugger
040fd47162
Fix gradient_checkpointing backward compatibility (#14408) 
* Fix gradient_checkpointing backward compatibility

* Remove needless line

* make sure mask prob is big enough and length small enough

* Fix tests

Co-authored-by: patrickvonplaten <patrick.v.platen@gmail.com>
 2021-11-16 08:58:42 -05:00
Lysandre Debut
1cc453d33c
Allow per-version configurations (#14344) 
* Allow per-version configurations

* Update tests/test_configuration_common.py

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* Update tests/test_configuration_common.py

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
 2021-11-15 16:38:02 -05:00
Patrick von Platen
76d0d41e51
[Wav2Vec2] Make sure that gradient checkpointing is only run if needed (#14407) 
* [Wav2Vec2] Make sure that gradient checkpointing is only run if needed

* make fix-copies
 2021-11-15 21:03:10 +01:00
Eldar Kurtic
9fd937ead1
Replace BertLayerNorm with LayerNorm (#14385) 
Running Movement pruning experiments with the newest HuggingFace would crash due to non-existing BertLayerNorm.
 2021-11-15 13:25:10 -05:00
Yih-Dar
a67d47b40c
Fix weight loading issue (#14016) 
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2021-11-15 17:48:40 +01:00
NielsRogge
74e6111ba7
Fix test and docs (#14399) 2021-11-15 17:35:33 +01:00
Patrick von Platen
4ce74edf51
[Speech2Text2] Enable tokenizers (#14390) 
* [Speech2Text2] Enable tokenizers

* minor fix

* Apply suggestions from code review

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
 2021-11-15 16:34:11 +01:00
Matt
267867e851
Quick fix to TF summarization example (#14401) 2021-11-15 13:45:51 +00:00
Stas Bekman
29dfb2dbb1
[doc] performance and parallelism updates (#14391) 
* [doc] performance and parallelism doc update

* improve

* improve
 2021-11-14 17:19:15 -08:00
nbertagnolli
790cdc2e55
Raise exceptions instead of using asserts in modeling_openai #12789 (#14386) 
* Raise exceptions instead of using asserts for control flow in modeling_openai #12789

* reformatted file
 2021-11-13 21:34:34 -05:00
Suraj Patil
2e60276b38
[M2M100Tokenizer] fix _build_translation_inputs (#14382) 
* add return_tensors paramter

* fix test

* Apply suggestions from code review

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* style

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
 2021-11-13 20:57:12 +05:30
Suraj Patil
3165930402
support wmt21 tokenizer in m2m100 tokenizer (#14376) 2021-11-13 14:21:58 +05:30
Li-Huai (Allan) Lin
280a811ecb
Use AlbertConverter for FNet instead of using FNet's own converter (#14365) 
* Add normalizer to FNetConverter

* Style

* Directly use AlbertConverter
 2021-11-12 19:46:40 +01:00
Patrick von Platen
55f49c5f4b
[Wav2Vec2 Example] Improve fine-tuning script (#14373) 
* improve some stuff

* finish

* correct last
 2021-11-12 16:35:57 +01:00
Suraj Patil
21546e59a6
fix docs (#14377) 2021-11-12 15:56:41 +05:30
Nicolas Patry
ed5d15518b
Adding support for raw python generator in addition to Dataset for pipelines (#14352) 
* Adding support for raw python `generator` in addition to `Dataset`

The main goal is to ease the create of streaming data to the pipe.

`Dataset` is more involved and pytorch specific.

This PR, provides a way to use a python iterator too.
This enabled #14250 but can be proposed as a standalone PR.

```python
from transformers import pipeline

def read_data(filename):
    with open(filename, 'r') as f:
        for line in f:
            yield f

pipe = pipeline("text-classification")
for classified in pipe(read_data("large_file.txt")):
    print("Success ! ", classified)
```

The main caveat of this, is the interaction with `DataLoader` with
`num_workers>1`. When you have multiple workers, each receive a copy
of the generator (like `IterableDataset`). That means the naive Iterator
will fail since all workers iterate on all items of the generator.

There are ways to do clever "skipping", but it could be bad still
because all workers still do have to pass through all items of the
generator (they just ignore items they don't handle), depending on
the case it might be bad.

Using `num_workers=1` is the simplest fix and if the cost of loading
your data is small enough should be good enough. In the above example
trying to do smart tricks to skip some lines is unlikely to be a net
positive for instance.

If there are better ways to do "jumps" on some data, then using
`Dataset` is more advised (since then differents workers can just jump
themselves).

* Adding iterator support for `tf` too.
 2021-11-12 09:20:40 +01:00
Stas Bekman
77262ef750
fix --gradient_checkpointing (#13964) 2021-11-11 17:50:21 +01:00
Suraj Patil
3d607df8f4
fix loading flax bf16 weights in pt (#14369) 
* fix loading flax bf16 weights in pt

* fix clip test

* fix t5 test

* add logging statement

* Update src/transformers/modeling_flax_pytorch_utils.py

Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>

* switch back to native any

* fix check for bf16 weights

Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
 2021-11-11 21:20:49 +05:30
Matt
7f20bf0d43
Fixing requirements for TF LM models and use correct model mappings (#14372) 
* Fixing requirements for TF LM models and use correct model mappings

* make style
 2021-11-11 15:34:00 +00:00
Matt
4c35c8d89c
Experimenting with adding proper get_config() and from_config() methods (#14361) 
* Experimenting with adding proper get_config() and from_config() methods

* Adding a test for get/from config

* Fix test for get/from config
 2021-11-11 14:21:50 +00:00
Suraj Patil
b1dbdf22ef
pass params to encode (#14370) 2021-11-11 17:16:24 +05:30
Suraj Patil
e92190c0f8
Fix Flax params dtype (#13098) 
* fix inits

* fix embed dtype

* fix embed dtype

* add test to check default dtype

* quality

* add type conversion methods for flax models

* more robust casting

* cast sinusoidal positions

* update pegasus

* update albert

* update test

* make sure dtype is passed to every module

* style

* fix electra dense

* fix t5

* quality

* add more tests

* better name

* use the dtype for lm head computation

* fix albert

* style

* fix albert embed dtype

* more tests

* fix vision enc-dec

* cleanup

* fix embed dtype pegasus

* fix default param test

* doc

* update template

* fix final_logits_bias dtype

* Apply suggestions from code review

Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>

* fix doc

* fix doc

* add detailed docstring for dtype parameter

* remove un-necessary import

Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
 2021-11-11 14:45:20 +05:30
Stas Bekman
1c76a51615
solve the port conflict (#14362) 2021-11-10 19:11:45 -08:00