riaz.somc/transformers
1
0
Fork 0
mirror of https://github.com/huggingface/transformers.git synced 2025-07-24 23:08:57 +06:00
Code Issues Actions 32 Packages Projects Releases Wiki Activity
5,115 Commits 887 Branches 229 Tags 2.1 GiB
49e9be0639
Commit Graph

5115 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Morgan Funtowicz
b105f2c6b3 Update ONNX doc to match the removal of --optimize argument. 
Signed-off-by: Morgan Funtowicz <funtowiczmo@gmail.com>
 2020-08-21 10:37:09 +02:00
Sylvain Gugger
e5f452275b
Trainer automatically drops unused columns in nlp datasets (#6449) 
* Add a classmethod to easily build a Trainer from nlp dataset and metric

* Fix docstrings

* Split train/eval

* Formatting

* Log dropped columns + docs

* Authorize callable activations

* Poc for auto activation

* Be framework-agnostic

* Formatting

* Remove class method

* Remove unnecessary code
 2020-08-20 16:29:14 -04:00
Sam Shleifer
5bf4465e6c
Regression test for pegasus bugfix (#6606) 2020-08-20 15:34:43 -04:00
sgugger
86c07e634f One last threshold to raise 2020-08-20 14:23:09 -04:00
Sylvain Gugger
e8af90c052
Move threshold up for flaky test with Electra (#6622) 
* Move threshold up for flaky test with Electra

* Update above as well
 2020-08-20 13:59:40 -04:00
Ivan Dolgov
953958372a
XLNet Bug when training with apex 16-bit precision (#6567) 
* xlnet fp16 bug fix

* comment cast added

* Update modeling_xlnet.py

Co-authored-by: Kevin Canwen Xu <canwenxu@126.com>
 2020-08-21 01:34:23 +08:00
Patrick von Platen
505f2d749e
[Tests] fix attention masks in Tests (#6621) 
* fix distilbert

* fix typo
 2020-08-20 13:23:47 -04:00
Denisa Roberts
c9454507cf
Add tests for Reformer tokenizer (#6485) 2020-08-20 18:58:44 +02:00
Joe Davison
f9d280a959
TFTrainer dataset doc & fix evaluation bug (#6618) 
* TFTrainer dataset doc & fix evaluation bug

discussed in #6551

* add docstring to test/eval datasets
 2020-08-20 12:11:36 -04:00
Sylvain Gugger
573bdb0a5d
Add tests to Trainer (#6605) 
* Add tests to Trainer

* Test if removing long breaks everything

* Remove ugly hack

* Fix distributed test

* Use float for number of epochs
 2020-08-20 11:13:50 -04:00
Joe Davison
039d8d65fc
add intro to nlp lib & dataset links to custom datasets tutorial (#6583) 
* add intro to nlp lib + links

* unique links...
 2020-08-20 10:32:51 -04:00
sgugger
b3e54698dd Fix CI 2020-08-20 08:34:02 -04:00
Prajjwal Bhargava
33bf426498
removed redundant arg in prepare_inputs (#6614) 
* removed redundant arg in prepare_inputs

* made same change in prediction_loop
 2020-08-20 08:23:35 -04:00
Romain Rigaux
cabfdfafc0
Docs copy button misses ... prefixed code (#6518) 
Tested in a local build of the docs.

e.g. Just above https://huggingface.co/transformers/task_summary.html#causal-language-modeling

Copy will copy the full code, e.g.

for token in top_5_tokens:
     print(sequence.replace(tokenizer.mask_token, tokenizer.decode([token])))

Instead of currently only:

for token in top_5_tokens:


>>> for token in top_5_tokens:
...     print(sequence.replace(tokenizer.mask_token, tokenizer.decode([token])))
Distilled models are smaller than the models they mimic. Using them instead of the large versions would help reduce our carbon footprint.
Distilled models are smaller than the models they mimic. Using them instead of the large versions would help increase our carbon footprint.
Distilled models are smaller than the models they mimic. Using them instead of the large versions would help decrease our carbon footprint.
Distilled models are smaller than the models they mimic. Using them instead of the large versions would help offset our carbon footprint.
Distilled models are smaller than the models they mimic. Using them instead of the large versions would help improve our carbon footprint.

Docs for the option fix:
https://sphinx-copybutton.readthedocs.io/en/latest/
 2020-08-20 17:35:06 +08:00
Stas Bekman
61b5ee11e3
lighter 'make test' (#6512) 2020-08-20 17:24:25 +08:00
Siddharth Jain
3c3c46f563
Typo fix in 04-onnx-export (#6595) 2020-08-20 16:17:16 +08:00
Oren Amsalem
93c5c9a528
[cleanup] remove confusing newline (#6603) 2020-08-20 00:33:36 -04:00
Sylvain Gugger
18ca0e9140
Fix #6575 (#6596) 2020-08-19 13:04:33 -04:00
Suraj Patil
7581884dee
[BartTokenizerFast] add prepare_seq2seq_batch (#6543) 2020-08-19 10:37:48 -04:00
Patrick von Platen
8bcceaceff
fix model outputs test (#6593) 2020-08-19 16:18:51 +02:00
Sam Shleifer
9a86321b11
tf generation utils: remove unused kwargs (#6591) 2020-08-19 09:37:45 -04:00
Pradhy729
2a7402cbd3
Feed forward chunking others (#6365) 
* Feed forward chunking for Distilbert & Albert

* Added ff chunking for many other models

* Change model signature

* Added chunking for XLM

* Cleaned up by removing some variables.

* remove test_chunking flag

Co-authored-by: patrickvonplaten <patrick.v.platen@gmail.com>
 2020-08-19 14:31:10 +02:00
Patrick von Platen
fe0b85e77a
[EncoderDecoder] Add functionality to tie encoder decoder weights (#6538) 
* start adding tie encoder to decoder functionality

* finish model tying

* make style

* Apply suggestions from code review

* fix t5 list including cross attention

* apply sams suggestions

* Update src/transformers/modeling_encoder_decoder.py

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* add max depth break point

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
 2020-08-19 14:23:45 +02:00
Sam Shleifer
ab42d74850
Fix bart base test (#6587) 2020-08-18 21:28:10 -04:00
Sam Shleifer
1529bf9680
add BartConfig.force_bos_token_to_be_generated (#6526) 
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
 2020-08-18 19:15:50 -04:00
Patrick von Platen
974bb4af26
[Model card] Bert2GPT2 EncoderDecoder model (#6569) 
* Bert2GPT2 EncoderDecoder model

* Update README.md
 2020-08-18 19:28:17 +02:00
Suraj Patil
6f972e1423
update xnli-mt url (#6580) 2020-08-18 13:10:47 -04:00
Suraj Patil
fb6844aff5
[Pegasus Doc] minor typo (#6579) 
Minor typo correction
@sshleifer
 2020-08-18 12:47:47 -04:00
Manuel Romero
aaab9ab187
Create README.md (#6556) 2020-08-18 12:43:20 -04:00
Manuel Romero
1dfce0f08a
Create README.md (#6557) 2020-08-18 12:42:14 -04:00
Romain Rigaux
7516bcf273
[docs] Fix number of 'ug' occurrences in tokenizer_summary (#6574) 2020-08-18 10:23:25 -04:00
Romain Rigaux
5a5af22ed5
[docs] Fix wrong newline in the middle of a paragraph (#6573) 2020-08-18 10:22:43 -04:00
Stas Bekman
7659a8eb37
fix incorrect codecov reports (#6553) 
As discussed at https://github.com/huggingface/transformers/issues/6317 codecov currently sends an invalid report when it fails to find a code coverage report for the base it checks against, so this gets fixed by:

-  require_base: yes        # don't report if there is no base coverage report

let's add this for clarity, this supposedly is already the default.

-  require_head: yes        # don't report if there is no head coverage report 

and perhaps no point reporting on doc changes as they don't make any difference and it just generates noise:

-  require_changes: true    # only comment if there was change in coverage
 2020-08-18 10:21:13 -04:00
Stefan Schweter
cfa26d2b41
github: add @stefan-it to bug-report template for all token-classification related bugs (#6489) 2020-08-18 08:38:54 -04:00
Philip May
1fdf372f8c
Small typo fixes for model card: electra-base-german-uncased (#6555) 
* Update README.md

* Update model_cards/german-nlp-group/electra-base-german-uncased/README.md

Co-authored-by: Julien Chaumond <chaumond@gmail.com>
 2020-08-18 08:21:52 -04:00
Ali Modarressi
5a81195ea9
Fixed label datatype for STS-B (#6492) 
* fixed label datatype for sts-b

* naming update

* make style

* make style
 2020-08-18 08:09:39 -04:00
Sam Shleifer
12d7624199
[marian] converter supports models from new Tatoeba project (#6342) 2020-08-17 23:55:42 -04:00
Jim Regan
fb7330b30e
update with #s of sentences/tokens (#6546) 2020-08-17 16:48:05 -04:00
onepointconsulting
63144701ed
Added first model card (#6530) 
* Added first model card

* Add metadata

Co-authored-by: Julien Chaumond <chaumond@gmail.com>
 2020-08-17 16:24:10 -04:00
Ikram Ali
98ee802023
[model_cards] Add model cards for Urduhack model (roberta-urdu-small) (#6536) 
* [model_cards] roberta-urdu-small added.

* [model_cards] typo fixed.

* Tweak license format (yaml expects a simple string)

Co-authored-by: Ikram Ali <mrikram1989>
Co-authored-by: Julien Chaumond <chaumond@gmail.com>
 2020-08-17 16:04:29 -04:00
Jim Regan
3a302904cb
[model_cards] Add a new model for Irish (#6544) 2020-08-17 15:56:56 -04:00
Julien Chaumond
07971d8b18 [model_cards] Fix yaml for cedpsam/chatbot_fr 2020-08-17 21:33:32 +02:00
Suraj Patil
407da12ef1
[T5Tokenizer] add prepare_seq2seq_batch method (#6122) 
* tests
 2020-08-17 13:57:19 -04:00
Suraj Patil
c9564f5343
[Doc] add more MBart and other doc (#6490) 
* add mbart example

* add Pegasus and MBart in readme

* typo

* add MBart in Pretrained models

* add pre-proc doc

* add DPR in readme

* fix indent

* doc fix
 2020-08-17 12:30:26 -04:00
Stas Bekman
f68c873100
replace _ with __ rst links (#6541) 2020-08-17 12:27:02 -04:00
sgugger
7ca6ab67fc Fix CI 2020-08-17 12:20:40 -04:00
Stas Bekman
b732e7e111
[doc] multiple corrections to "Summary of the tasks" (#6509) 
* [doc] multiple corrections to "Summary of the tasks"

* fix indentation

* correction

* fix links, add links to examples/seq2seq/README.md instead of non-existing script
 2020-08-17 11:49:16 -04:00
Suraj Patil
2a77813d53
[BartTokenizer] add prepare s2s batch (#6212) 
Co-authored-by: sgugger <sylvain.gugger@gmail.com>
 2020-08-17 11:44:46 -04:00
Stas Bekman
84d33317ae
[doc] make the text more readable, fix some typos, add some disambiguation (#6508) 
* [doc] make the text more readable, fix some typos, add some disambiguation

* Update docs/source/glossary.rst

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
 2020-08-17 11:07:58 -04:00
Joe Davison
d0c2389f48
add custom datasets tutorial (#6466) 
* add custom datasets tutorial

* python -> bash code blocks

* Apply suggestions from code review

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* minor review feedback changes

* add working native QA snippet

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
 2020-08-17 09:15:34 -04:00