riaz.somc/transformers
1
0
Fork 0
mirror of https://github.com/huggingface/transformers.git synced 2025-07-28 16:52:24 +06:00
Code Issues Actions 18 Packages Projects Releases Wiki Activity
3,606 Commits 889 Branches 231 Tags 2.1 GiB
c10decf7a0
Commit Graph

242 Commits

Author SHA1 Message Date
Sam Shleifer
39371ee454
[Bart/Memory] don't create lm_head (#3323) 
* delete lm_head, skips weight tying
* Fixed s3
 2020-03-26 18:40:39 -04:00
sakares saengkaew
1a6c546c6f
Add missing token classification for XLM (#3277) 
* Add the missing token classification for XLM

* fix styling

* Add XLMForTokenClassification to AutoModelForTokenClassification class

* Fix docstring typo for non-existing class

* Add the missing token classification for XLM

* fix styling

* fix styling

* Add XLMForTokenClassification to AutoModelForTokenClassification class

* Fix docstring typo for non-existing class

* Add missing description for AlbertForTokenClassification

* fix styling

* Add missing docstring for AlBert

* Slow tests should be slow

Co-authored-by: Sakares Saengkaew <s.sakares@gmail.com>
Co-authored-by: LysandreJik <lysandre.debut@reseau.eseo.fr>
 2020-03-26 10:22:13 -04:00
Patrick von Platen
022e8fab97
Adds translation pipeline (#3419) 
* fix merge conflicts

* add t5 summarization example

* change parameters for t5 summarization

* make style

* add first code snippet for translation

* only add prefixes

* add prefix patterns

* make style

* renaming

* fix conflicts

* remove unused patterns

* solve conflicts

* fix merge conflicts

* remove translation example

* remove summarization example

* make sure tensors are in numpy for float comparsion

* re-add t5 config

* fix t5 import config typo

* make style

* remove unused numpy statements

* update doctstring

* import translation pipeline
 2020-03-26 13:50:58 +01:00
Patrick von Platen
9c683ef01e
Add t5 to pipeline(task='summarization') (#3413) 
* solve conflicts

* move warnings below

* incorporate changes

* add pad_to_max_length to pipelines

* add bug fix for T5 beam search

* add prefix patterns

* make style

* fix conflicts

* adapt pipelines for task specific parameters

* improve docstring

* remove unused patterns
 2020-03-26 11:03:13 +01:00
Patrick von Platen
e392ba6938
Add camembert integration tests (#3375) 
* add integration tests for camembert

* use jplu/tf-camembert fro the moment

* make style
 2020-03-24 10:18:37 +01:00
Patrick von Platen
95e00d0808
Clean special token init in modeling_....py (#3264) 
* make style

* fix conflicts
 2020-03-20 21:41:04 +01:00
Patrick von Platen
bbf26c4e61
Support T5 Generation (#3228) 
* fix conflicts

* update bart max length test

* correct spelling mistakes

* implemented model specific encode function

* fix merge conflicts

* better naming

* save intermediate state -> need to rethink strucuture a bit

* leave tf problem as it is for now

* current version

* add layers.pop

* remove ipdb

* make style

* clean return cut decoding

* remove ipdbs

* Fix restoring layers in the decoders that doesnt exists.

* push good intermediate solution for now

* fix conflicts

* always good to refuse to merge conflicts when rebasing

* fix small bug

* improve function calls

* remove unused file

* add correct scope behavior for t5_generate

Co-authored-by: Morgan Funtowicz <funtowiczmo@gmail.com>
 2020-03-19 23:18:23 +01:00
Sam Shleifer
ad7233fc01
[BART] cleanup: remove redundant kwargs, improve docstrings (#3319) 2020-03-19 11:16:51 -04:00
Lysandre Debut
d6afbd323d
XLM-R Tokenizer now passes common tests + Integration tests (#3198) 
* XLM-R now passes common tests + Integration tests

* Correct mask index

* Model input names

* Style

* Remove text preprocessing

* Unneccessary import
 2020-03-18 09:52:49 -04:00
Patrick von Platen
292186a3e7
Adding LM Head to Transfo-XL and first step to fixing problem with Adaptive Embeddings in TransfoXL (#3286) 
* first commit

* work in progress

* make language generation task pass

* update to working version for LM

* delete print

* remove dead code

* make style
 2020-03-18 09:24:27 -04:00
Sam Shleifer
38a555a83c
Add Summarization to Pipelines (#3128) 
* passing

* Undo stupid chg

* docs

* undo rename

* delete-cruft

* only import if you have torch

* Dont rely on dict ordering

* Fix dict ordering upstream

* docstring link

* docstring link

* remove trailing comma for 3.5 compat

* new name

* delegate kwarging

* Update kwargs
 2020-03-17 18:04:21 -04:00
Patrick von Platen
e8f44af5bf
[generate] do_sample default back to False (#3298) 
* change do_samples back

* None better default as boolean

* adapt do_sample to True in test example

* make style
 2020-03-17 10:52:37 -04:00
Sam Shleifer
b2c1a447fe
[BART] Delete redundant unit test (#3302) 2020-03-16 23:09:10 -04:00
Sam Shleifer
5ea8ba67b4
[BART] Remove unused kwargs (#3279) 
* Remove unused kwargs
* dont call forward in tests
 2020-03-15 23:00:44 -04:00
Thomas Wolf
3814e167d9
Merge pull request #3225 from patrickvonplaten/finalize_merge_bart_generate_into_default_generate 
Complete merge Seq-2-Seq generation into default generation
 2020-03-14 15:08:59 +01:00
Sam Shleifer
2bd79e23de
[BART] FP16 testing fixes (#3266) 2020-03-13 19:48:26 -04:00
Patrick von Platen
6a82f774f2 fix typo 2020-03-12 21:10:51 +01:00
Patrick von Platen
f1c71da115 fix eos_token_ids in test 2020-03-12 21:00:54 +01:00
Patrick von Platen
6047f46b19 re-add eos token to get good bart results 2020-03-12 20:17:50 +01:00
Patrick von Platen
ac303eae46 fix problem with half 2020-03-11 12:24:30 +01:00
Patrick von Platen
bc9d5d917c make all tensors half precision 2020-03-11 12:15:38 +01:00
Patrick von Platen
a332cc9f7f finalize generation merge 2020-03-11 11:53:36 +01:00
Patrick von Platen
7351a8dbaf re-add scoring filtering 2020-03-11 11:06:56 +01:00
Patrick von Platen
374deef48d fixed typo 2020-03-11 11:06:56 +01:00
patrickvonplaten
41b437ea3a add draft version of propsoed changes for ROGUE score 2020-03-11 11:06:56 +01:00
patrickvonplaten
a5751f7578 fix bug with attention_mask as optional input argument 2020-03-11 11:06:56 +01:00
patrickvonplaten
d880a5fbde finalized PR 2020-03-11 11:06:56 +01:00
patrickvonplaten
2acfe63964 best current version and make style 2020-03-11 11:06:56 +01:00
patrickvonplaten
c62444da39 fix conflicts 2020-03-11 11:06:56 +01:00
Patrick von Platen
77e6775065 add current changes 2020-03-11 11:06:56 +01:00
Patrick von Platen
421216997b comment out stuff 2020-03-11 11:06:56 +01:00
Patrick von Platen
7a11e925cf work in progress 2020-03-11 11:06:56 +01:00
Patrick von Platen
aceb3fbaf4 only do output_past=True for language generation in bart 2020-03-11 11:06:56 +01:00
Patrick von Platen
7cba11fb9b better naming 2020-03-11 11:06:56 +01:00
Patrick von Platen
ff648221bd fix conflicts 2020-03-11 11:06:56 +01:00
Patrick von Platen
c0d9dd3ba9 refactored code a bit and made more generic 2020-03-11 11:06:56 +01:00
Patrick von Platen
d8e2b3c547 fix conflicts 2020-03-11 11:06:56 +01:00
Patrick von Platen
31f2437f07
Merge pull request #3191 from patrickvonplaten/add_integration_tests_lm_generate_torch_tf 
Add integration tests lm generate torch tf
 2020-03-10 11:29:17 +01:00
Julien Chaumond
cbf8f5d32b [model upload] Support for organizations 2020-03-09 17:33:57 -04:00
Lysandre
525b6b1c54 TFQA pipeline marked as slow test 2020-03-09 16:52:30 -04:00
Lysandre Debut
5164ea91a7
Skipping outputs (#3116) 
* Minimal example

* Proposal 2

* Proposal 2 for fast tokenizers

* Typings

* Docs

* Revert "Docs" for easier review

This reverts commit eaf0f97062e809887704a542144c537f769d5223.

* Remove unnecessary assignments

* Tests

* Fix faulty type

* Remove prints

* return_outputs -> model_input_names

* Revert "Revert "Docs" for easier review"

This reverts commit 6fdc69408102bf695797f2dfddbb6350c6b9e722.

* code quality
 2020-03-09 13:48:58 -04:00
Patrick von Platen
efb619235c add print statement to avoid code quality problem 2020-03-09 15:31:21 +01:00
Patrick von Platen
b12541c4dc test ctrl 2020-03-09 13:58:01 +00:00
Patrick von Platen
b73dd1a0e4 fix typo in test xlm tf 2020-03-09 11:34:31 +01:00
Patrick von Platen
4620caa864 fix if use lang embeddings in tf xlm 2020-03-09 11:18:54 +01:00
patrickvonplaten
fbd02d4693 fixed all tests, still need to check ctrl tf and pt and xlm tf 2020-03-08 21:45:55 +01:00
patrickvonplaten
b4a3a64744 fix xlnet & transfotests 2020-03-08 16:25:03 +01:00
patrickvonplaten
66c827656f fix typo in test gpt2 2020-03-08 15:35:08 +01:00
patrickvonplaten
314bdc7c14 fix typo in test 2020-03-08 15:34:20 +01:00
patrickvonplaten
575976144a updated all tests 2020-03-08 15:29:10 +01:00