riaz.somc/transformers
1
0
Fork 0
mirror of https://github.com/huggingface/transformers.git synced 2025-07-31 10:12:23 +06:00
Code Issues Actions 23 Packages Projects Releases Wiki Activity
11,686 Commits 887 Branches 232 Tags 2.1 GiB
cd2457809f
Commit Graph

11686 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Alara Dirik
cd2457809f
Improve OWL-ViT postprocessing (#20980) 
* add post_process_object_detection method

* style changes
 2023-01-03 19:25:09 +03:00
Yih-Dar
e901914da7
Fix for LXMERT (#20986) 
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2023-01-03 17:16:52 +01:00
Yih-Dar
8f09dd89f6
Avoid CI runs under users' own CircleCI personal account (#20981) 
* Avoid null CI

* Avoid null CI

* rename

* more clear error message

* Update .circleci/config.yml

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* clean up

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
 2023-01-03 16:19:38 +01:00
Anna Krogager
7b0727a401
Ignore errors when deleting old checkpoints in trainer (#20984) 2023-01-03 10:10:59 -05:00
samuelpullely
15c68c67f4
Enable decoder_attention_mask in generate function (#20726) 
* Enable `decoder_attention_mask` in `generate` function

* Make style corrections

* Run `make repo-consistency`

* Add integration test
 2023-01-03 09:59:08 -05:00
JeongYeon Nam
a9653400d3
Fix valid ratio for Deformable Detr (#20958) 
* fix: valid ratio has right value

* chore: remove unnecessary line

Co-authored-by: Jeongyeon Nam <jy.nam@navercorp.com>
 2023-01-03 09:43:26 -05:00
Wang, Yi
9c9fe89f84
[run_clm example] add torch_dtype option for model load. (#20971) 
* [run_clm example] add torch_dtype option for model load.
for BLOOM 175B model. peak memory will reduce about 350G for inference. the weight of BLOOM in model hub is bfloat16

Signed-off-by: Wang, Yi A <yi.a.wang@intel.com>

* add other type in option

* fix style

Signed-off-by: Wang, Yi A <yi.a.wang@intel.com>
 2023-01-03 09:33:11 -05:00
Yih-Dar
e697c912c2
Remove more unused attributes in config classes (#20858) 
Remove more unused attributes in config classes

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2023-01-03 14:37:40 +01:00
NielsRogge
9c6f7485a6
Add GIT (GenerativeImage2Text) (#20295) 
* First draft

* Make model instantiation work

* Fix copied from statement

* More fixes

* Add correct output head

* Improve configuration

* Add conversion script

* Improve conversion script

* Remove token_type_ids

* Fix conversion of projection layers

* Convert all weights

* Use cats image

* Make logits match

* Generate caption on cats image

* Add GITProcessor

* Update conversion script

* Add support for more checkpoints

* Fix conversion script

* Add initial tests

* Remove cross-attention

* More improvements

* Remove is_decoder

* Improve model tests

* Improve tests

* Improve model outputs

* Fix model outputs equivalence

* Fix more tests

* Remove unused code

* Use generate to generate text, no use of cache for now

* Use generate more appropriately

* Fix config tests

* Fix style

* Add support for use_cache

Co-authored-by: Joao Gante <joaofranciscocardosogante@gmail.com>

* Fix style

* Fix GIT vision encoder

* Update README

* Fix integration test

* Set bos and eos token ids

* Improve docs

* Improve code

* Add support for provided attention_mask

* Add copied from statement

* Fix gradient checkpointing test

* Set model_input_names

* Investigate model_input_names

* Remove script

* Fix model inputs

* Fix docstring

* Rename GIT to Git

* Support more models

* Add support for textvqa model

* Add video support

* Extend conversion script for video

* Add support for large variant

* Add support for more models

* Fix config archive map

* Update integration test

* Fix README

* Fix CLIP mean and std

* Update processor

* Fix use_cache for video, thanks @gante

* Remove print statements

* Remove assertion

* Add processor tests

* Fix model_input_names

* Use Auto API for processor

* Fix processor tests

* Fix integration test

* Fix pipeline test

* Make tests faster

* Update conversion script

* Update conversion script

* Convert more checkpoints

* Update conversion script

* Fix typo

* Update docstrings

* Improve code snippets

* Fix doc tests

* Add more code examplesé

* Fix doc tests

* Add integration tests

* Fix unused variable

* revert

* Add GIT to Japanese README

Co-authored-by: Niels Rogge <nielsrogge@Nielss-MacBook-Pro.local>
Co-authored-by: Joao Gante <joaofranciscocardosogante@gmail.com>
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2023-01-03 14:17:18 +01:00
Alara Dirik
305f41e4de
Fix post_process_object_detection method descriptions (#20977) 
fix post_process_object_detection descriptions
 2023-01-03 15:56:02 +03:00
Konstantin Kotik
367fdf3330
MinNewTokensLengthLogitsProcessor for .generate method #20814 (#20892) 
* feat: add min new length logit processor

* test: add min new length logit processor

* docs: add MinNewTokensLengthLogitsProcessor

* feat: import MinNewTokensLengthLogitsProcessor

* fix: update pytorch dummy objects

* refactor & fix: rename attributes and var and get rid of dynamic attribute

* tests: align test with new interface

* docs: fix typo

* docs: minor clarification

* Empty-Commit

* empty commit

* run automated quality edits

Co-authored-by: Joao Gante <joao@huggingface.co>
 2023-01-03 06:29:02 -05:00
Joao Gante
4fd89e4978
Generate: delete unused TF _reorder_cache (#20964) 2023-01-03 10:54:56 +00:00
ivanllt
a3e8d3cb1c
Fix T5 docstring (#20957) 
Fix start_docstring for deparallelize method
 2023-01-03 05:53:33 -05:00
Joao Gante
588faad106
Generate: TF XLA beam sample (#20927) 
* beam sample in beam search

* rag now works with the updated beam search

* delete legacy (non-XLA) generation code related to beam sample
 2023-01-02 10:25:44 +00:00
Hao Wang
375801d5e6
update pyknp to rhoknp (#20890) 
* update pyknp to rhoknp

* fix linter

* fix linter

* fix linter

* fix linter

* fix linter

* support rhoknp==1.1.0, fix testcase
 2022-12-31 01:22:26 -05:00
bofeng huang
092d4d49dd
Add generate kwargs to AutomaticSpeechRecognitionPipeline (#20952) 
* Add generate kwargs to AutomaticSpeechRecognitionPipeline

* Add test for generation kwargs
 2022-12-31 01:13:39 -05:00
bofeng huang
47c9b22d08
Add generate kwargs to AutomaticSpeechRecognitionPipeline (#20952) 
* Add generate kwargs to AutomaticSpeechRecognitionPipeline

* Add test for generation kwargs
 2022-12-31 01:13:28 -05:00
Stas Bekman
9e6da0a7ed
[trainer: distributed_concat] ensure all_gather's inputs are contiguous (#20951) 
[trainer: distributed_concat] ensure all_gather's input are contiguous
 2022-12-30 21:55:12 -08:00
Samuel Xu
17292440c0
Fixing DistilBert error message (#20945) 
Fixing error message
 2022-12-30 03:44:09 -05:00
bofeng huang
881fa716c8
Fix error message in WhisperFeatureExtractor (#20936) 
* Fix error message

* Fix code quality
 2022-12-30 02:37:37 -05:00
Matthew McDermott
491a33d138
Adds type checking to PreTrainedConfig. (#20926) 2022-12-30 02:35:01 -05:00
ivanllt
8637316e5e
Remove Bert tokenizer dependency from DistillBert (slow/fast) tokenizers (#20933) 2022-12-29 02:36:27 -05:00
bofeng huang
fe65657de1
Fix FP16 inference in TextGenerationPipeline (#20913) 
* add torch_dtype attribute to Pipeline

* Use torch_dtype to cast input tensor type in AutomaticSpeechRecognitionPipeline

* Fix code quality

* Add TextGenerationPipeline fp16 test

* Fix code quality

* Remove useless require in tests

Co-authored-by: Nicolas Patry <patry.nicolas@protonmail.com>

Co-authored-by: Nicolas Patry <patry.nicolas@protonmail.com>
 2022-12-29 02:19:25 -05:00
Harsh Trivedi
11c49ed23b
Load the state dict on CPU to prevent unnecessary GPU memory surge (#20920) 
load the state dict on cpu.
 2022-12-29 02:18:03 -05:00
Alex Hedges
0b686a8a1e
Remove non-breaking spaces (#20929) 
* Remove non-breaking space in comment

It was likely added unintionally.

* Remove remaining non-breaking spaces
 2022-12-29 02:12:40 -05:00
Joao Gante
bbcd961897
Generate: correctly detect default max length (#20911) 
correctly detect default max length
 2022-12-28 10:05:25 +00:00
Akshaya Annavajhala
5f9b2ce0ea
Avoid collisions in writing metrics via 2 APIs - azureml + mlflow (#20837) 
* Avoid collisions in writing metrics via 2 APIs - azureml + mlflow

MLflow tracking API is enabled by default in AzureML and HF MLflow integration is more fully featured. I'd remove the AzureML integration but leaving the current behavior for backwards compatibility (though it should really be removed)

* Trigger CI
 2022-12-28 02:24:54 -05:00
Yih-Dar
5fa0b17c3d
[Past CI] 🔥 Leave Past CI failures in the past 🔥 (#20861) 
* torch.jit._state

* Fix past CI

* Fix for perceiver

* Fix REALM

* Fix for Bloom

* Fix for SwinMode

* Fix for TrajectoryTransformerModel

* Fix for test_wav2vec2_with_lm

* make style

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2022-12-27 18:37:25 +01:00
Eli Simhayev
e35bc46af6
fix docs typos in "add_new_model" (#20900) 
fix Jupyter typos
 2022-12-27 02:49:15 -05:00
Kamal Raj Kanakarajan
d1b3011292
Update flan-t5 original model link (#20897) 
Update flan-t5.mdx
 2022-12-27 02:26:14 -05:00
Younes Belkada
accad48e5b
[ T5] fix fp16 loading issue (#20878) 
* fix fp16 loading issue

* add backward compatibility

* better refactor

* better readability

- remove `force_upcast_dtype` as it is used once
- use `inspect`
- add `TODO`
 2022-12-26 10:01:03 +01:00
Nathan Barry
47146721b8
typo fix (#20891) 2022-12-26 02:06:23 -05:00
Márton Makrai
3830b3f74a
Fixes typo in the help text for --max_length (#20883) 2022-12-24 02:07:06 -05:00
Arthur
a081f292ca
[RobertaPreLayernom] Fixes the CI daily test (#20886) 
get correct checkpoint
 2022-12-23 19:55:17 +01:00
Younes Belkada
cab7799f7b
Add japanese translation of template (#20870) 
* add japanese translation of template

* fix japanese translation

- fix special cases
- fix typos
- manually translate special cases

Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>

Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>
 2022-12-23 14:39:42 +01:00
Jasmijn Bastings
efed8a2794
Add script to convert T5X T5 (v1.0 and v1.1) checkpoints to PyTorch (#20801) 
* Add script to convert T5X T5 (v1.0 and v1.1) checkpoints to PyTorch

* Remove unnecessary check and update docstring

* Format docstring

* Fix whitespace in docstring
 2022-12-23 14:36:46 +01:00
Nicolas Patry
f7f0ec2f54
Adding support for fp16 for asr pipeline. (#20864) 
* Supporting `fp16` for asr pipeline

* Adding test.

* Style.

* Oops.

* Flake8 update ?

* Fixing flake8 ?

* Revert "Flake8 update ?"

This reverts commit 0b917fcb52.

* Style (acctidentally deleted flake8 F401.)

* Move to a bigger test (no small whisper model, and s2t doesn't seem to
accept torch_dtype=fp16).

Also we need to use a GPU to actually compute on fp16.

* Using BatchFeature capability.
 2022-12-23 10:18:45 +01:00
Syed Abdul Gaffar Shakhadri
15bc776fec
Add Onnx Config for PoolFormer (#20868) 
poolformer onnx

Co-authored-by: syed <syed.abdul@sandlogic.com>
 2022-12-23 01:30:57 -05:00
Sourab Mangrulkar
4a4cd6cd02
having new model entries in Hindi for Hindi README (#20869) 2022-12-23 12:00:48 +05:30
Younes Belkada
52dd2b61bf
[MobileNet-v2] Fix ONNX typo (#20860) 
* fix typo `onnx`

* fix test
 2022-12-22 18:52:54 +01:00
Younes Belkada
4d10ffd506
[FSMT] Make it compatible with xxxForConditionalGeneration models (#20825) 
* add `get_encoder` and `get_decoder`

* add additional kwargs support

* fix condition

* add better checks

* better checks

* fix embed positions

* better test to consider padding

* fix debug statement

* Apply suggestions from code review

Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>

* add arguments on docstring

Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>
 2022-12-22 11:11:19 +01:00
dhansmair
2222740f50
change strings to f-strings in image_processing_utils.py (#20865) 
change strings to f-strings
 2022-12-22 02:06:50 -05:00
Joao Gante
829e889418
Generate: post-generate config doctest fix (#20804) 
* fix doctests

* revert unwanted change
 2022-12-21 19:18:45 +00:00
Yih-Dar
39e620c134
Update HubertModelIntegrationTest.test_inference_keyword_spotting (#20863) 
fix ci

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2022-12-21 18:40:14 +01:00
Arthur
4a433e321f
Add-warning-tokenizer (#20826) 
* add fast not use warning

* update
 2022-12-21 18:18:34 +01:00
Arthur
76d02feadb
Fix doctest (#20843) 
* fix doc for generation, dinat, nat and prelayernorm

* style

* update

* fix cpies

* use auto config and auto tokenizer

Co-authored-by: sgugger <sylvain.gugger@gmail.com>

* als modify roberta and the depending models

Co-authored-by: sgugger <sylvain.gugger@gmail.com>
 2022-12-21 16:34:31 +01:00
Mohit Sharma
aaa6296de2
Fix whisper export (#20800) 
* fix_whisper_export

* update input

* update input
 2022-12-21 16:28:42 +01:00
Yih-Dar
3090e70857
Fix past CI by skipping LevitModelTest.test_problem_types (#20859) 
* Fix past CI

* Fix past CI

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2022-12-21 14:29:13 +01:00
Maria Khalusova
04c560225b
Adding evaluate to the list of libraries required in generated notebooks (#20850) 
Adding `evaluate` to the list of libraries to be installed for every generated notebook in transformers
 2022-12-21 14:04:08 +01:00
İdil Sülo
0ae58204c6
Add visual prompt to processor of CLIPSeg model (#20816) 
Adds visual_prompt argument to CLIPSegProcessor to enable image-guided segmentation
 2022-12-21 15:23:45 +03:00