Arthur
fb78769b9c
[MT5] Fix CONFIG_MAPPING issue leading it to load umt5 class ( #24678 )
...
* update
* add umt5 to auto tokenizer mapping
* nits
* fixup
* fix failing torch test
2023-07-07 11:33:54 +09:00
Zach Mueller
fded6f4186
Fix integration with Accelerate and failing test ( #24691 )
...
Fix integration
2023-07-06 14:12:16 -04:00
Yih-Dar
bbf3090848
Avoid import sentencepiece_model_pb2 in utils.__init__.py ( #24689 )
...
fix
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-07-06 16:30:23 +02:00
Sourab Mangrulkar
66a378429d
DeepSpeed/FSDP ckpt saving utils fixes and FSDP training args fixes ( #24591 )
...
* update ds and fsdp ckpt logic
* refactoring
* fix 🐛
* resolve comment
* fix issue with overriding of the fsdp config set by accelerate
2023-07-06 15:03:25 +05:30
Zhao Tianyu
392740452e
Add dropouts to GPT-NeoX ( #24680 )
...
* add attention dropout, post attention dropout, post mlp dropout to gpt-neox
* fix typo
* add documentation
* fix too long line
* ran Checking/fixing src/transformers/models/gpt_neox/configuration_gpt_neox.py src/transformers/models/gpt_neox/modeling_gpt_neox.py
python utils/custom_init_isort.py
python utils/sort_auto_mappings.py
doc-builder style src/transformers docs/source --max_len 119 --path_to_docs docs/source
python utils/check_doc_toc.py --fix_and_overwrite
running deps_table_update
updating src/transformers/dependency_versions_table.py
python utils/check_copies.py
python utils/check_table.py
python utils/check_dummies.py
python utils/check_repo.py
Checking all models are included.
Checking all models are public.
Checking all models are properly tested.
Checking all objects are properly documented.
Checking all models are in at least one auto class.
Checking all names in auto name mappings are defined.
Checking all keys in auto name mappings are defined in `CONFIG_MAPPING_NAMES`.
Checking all auto mappings could be imported.
Checking all objects are equally (across frameworks) in the main __init__.
python utils/check_inits.py
python utils/check_config_docstrings.py
python utils/check_config_attributes.py
python utils/check_doctest_list.py
python utils/update_metadata.py --check-only
python utils/check_task_guides.py
2023-07-06 10:26:36 +01:00
Yuchao Dai
fb3b22c3b9
LlamaTokenizer should be picklable ( #24681 )
...
* LlamaTokenizer should be picklable
* make fixup
2023-07-06 10:21:27 +01:00
Matt
9a5d468ba0
Add Nucleotide Transformer notebooks and restructure notebook list ( #24669 )
...
* Add Nucleotide Transformer notebooks and restructure lists
* Add missing linebreak!
2023-07-05 18:28:47 +01:00
Rafael Padilla
3df3b9d4bf
Fix model referenced and results in documentation. Model mentioned was inaccessible ( #24609 )
2023-07-05 13:25:36 -03:00
Yih-Dar
050ef14516
Unpin huggingface_hub ( #24667 )
...
* fix
* fix
* fix
* [test all] commit
* [test all] commit
* [test all] commit
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-07-05 16:49:10 +02:00
Nripesh Niketan
bd9dfc23b9
Add is_torch_mps_available function to utils ( #24660 )
...
* Add mps function utils
* black formating
* format fix
* Added MPS functionality to transformers
* format fix
2023-07-05 16:02:20 +02:00
Yih-Dar
ee339bad01
Fix VisionTextDualEncoderIntegrationTest ( #24661 )
...
fix
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-07-05 13:44:30 +02:00
Yih-Dar
d211a84aca
Fix EncodecModelTest::test_multi_gpu_data_parallel_forward ( #24663 )
...
fix
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-07-05 11:37:46 +02:00
Sylvain Gugger
469f4d0c29
Make warning disappear for remote code in pipelines ( #24603 )
...
* Make warning disappear for remote code in pipelines
* Make sure it works twice in a row
* No need for that
2023-07-04 19:03:14 -04:00
Sylvain Gugger
b19c7b5ccf
Add finetuned_from property in the autogenerated model card ( #24528 )
...
* Add finetuned_from tag in the autogenerated model card
* Update name
2023-07-04 17:58:31 -04:00
Rafael Padilla
ea9caf7aba
Update warning messages reffering to post_process_object_detection ( #24649 )
...
* including the threshold alert in warning messages.
* Updating doc owlvit.md including post_process_object_detection function with threshold.
* fix
2023-07-04 16:47:57 -03:00
amyeroberts
f3e96235a3
documentation_tests.txt - sort filenames alphabetically ( #24647 )
...
* Sort filenames alphabetically
* Add check for order
2023-07-04 17:06:05 +01:00
Prathik Rao
a3b402ff9a
llama fp16 torch.max bug fix ( #24561 )
...
* open llama fp16 bug fix
* bug fix
* bug fixed
* make style
* Update modeling_llama.py
* apply formatting
* Address amy's comment
---------
Co-authored-by: Prathik Rao <prathikrao@microsoft.com@orttrainingdev8.d32nl1ml4oruzj4qz3bqlggovf.px.internal.cloudapp.net>
Co-authored-by: root <root@orttrainingdev8.d32nl1ml4oruzj4qz3bqlggovf.px.internal.cloudapp.net>
2023-07-04 16:05:12 +01:00
Sanchit Gandhi
4e94566018
Fix audio feature extractor deps ( #24636 )
...
* Fix audio feature extractor deps
* use audio utils window over torch window
2023-07-04 16:03:27 +01:00
Shahad Mahmud
cd4584e3c8
precompiled_charsmap checking before adding to the normalizers' list for XLNetTokenizerFast conversion. ( #24618 )
...
* precompiled_charsmap checking before adding to the normalizers' list.
* precompiled_charsmap checking for all Sentencepiece tokenizer models
* precompiled_charsmap checking for SPM tokenizer models - correct formatting
2023-07-04 02:51:42 +02:00
Joao Gante
f4e4b4d0e2
Generate: force cache with inputs_embeds forwarding ( #24639 )
2023-07-03 18:18:49 +01:00
Joao Gante
9934bb1f42
Generate: multi-device support for contrastive search ( #24635 )
2023-07-03 16:08:20 +01:00
Gema Parreño
4b26a61631
Fix loading dataset docs link in run_translation.py example ( #24594 )
...
* fix loading dataset link
* Update examples/tensorflow/translation/run_translation.py
Co-authored-by: Matt <Rocketknight1@users.noreply.github.com>
* Update examples/tensorflow/translation/run_translation.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
---------
Co-authored-by: Matt <Rocketknight1@users.noreply.github.com>
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
2023-07-03 15:21:21 +01:00
Yih-Dar
6eedfa6dd1
Pin Pillow for now ( #24633 )
...
fix
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-07-03 12:24:46 +02:00
Eli Simhayev
fc7ce2ebc5
[Time-Series] Added blog-post to tips ( #24482 )
...
* [Time-Series] Added blog-post to tips
* added Resources to time series models docs
* removed "with Bert"
2023-07-03 10:07:25 +02:00
Nayeon Han
e16191a8ac
🌐 [i18n-KO] Translated perplexity.mdx to Korean ( #23850 )
...
* docs: ko: `perplexity.mdx`
* translate comment
* reference english file
* change extension
* update toctree
2023-07-03 08:50:27 +02:00
Arthur
799df10aef
[Umt5] Add google's umt5 to transformers ( #24477 )
...
* add tokenization template
* update conversion script
* update modeling code
* update
* update convert checkpoint
* update modeling
* revert changes on convert script
* new conversion script for new format
* correct position bias
* cleaning a bit
* Credit co authors
Co-authored-by: agemagician
<ahmed.elnaggar@tum.de>
Co-authored-by: stefan-it
<>
* styling
* Add docq
* fix copies
* add co author
* Other Author
* Merge branch 'main' of https://github.com/huggingface/transformers into add-umt5
* add testing
* nit
* Update docs/source/en/model_doc/umt5.mdx
Co-authored-by: Stefan Schweter <stefan@schweter.it>
* fix t5
* actual fix?
* revert wrong changes
* remove
* update test
* more fixes
* revert some changes
* add SPIECE_UNDERLINE
* add a commone xample
* upfate
* fix copies
* revert changes on t5 conversion script
* revert bytefallback changes since there was no addition yet
* fixup
* fixup
* ingore umt5 cutom testing folder
* fix readmes
* revertT5 changes
* same outputs
* fixup
* update example
* Apply suggestions from code review
* style
* draft addition of all new files
* current update
* fix attention and stuff
* finish refactoring
* auto config
* fixup
* more nits
* add umt5 to init
* use md format
* Update README.md
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* revert changes on mt5
* revert mt4 changes
* update test
* more fixes
* add to mapping
* fix-copies
* fix copies
* foix retain grad
* fix some tests
* nits
* done
* Update src/transformers/models/umt5/modeling_umt5.py
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* Update docs/source/en/model_doc/umt5.md
* Update src/transformers/models/umt5/__init__.py
* Update docs/source/en/model_doc/umt5.md
Co-authored-by: Stefan Schweter <stefan@schweter.it>
* Update src/transformers/models/umt5/modeling_umt5.py
* update conversion script + use google checkpoints
* nits
* update test and modelling
* stash slow convert
* update fixupd
* don't change slow
---------
Co-authored-by: stefan-it <>
Co-authored-by: Stefan Schweter <stefan@schweter.it>
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
2023-07-03 07:38:21 +02:00
ydshieh
66ded238cd
fix pydantic install command
2023-07-01 09:29:21 +02:00
Serge Matveenko
d51aa48a76
Limit Pydantic to V1 in dependencies ( #24596 )
...
* Limit Pydantic to V1 in dependencies
Pydantic is about to release V2 release which will break a lot of things. This change prevents `transformers` to be used with Pydantic V2 to avoid breaking things.
* more
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-07-01 00:04:03 +02:00
Yih-Dar
299aafe55f
Use protobuf 4 ( #24599 )
...
* fix
* fix
* fix
* fix
* fix
* fix
* fix
* fix
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-06-30 20:56:55 +02:00
Stas Bekman
49e812d12b
[several models] improve readability ( #24585 )
...
* [modeling_clip.py] improve readability
* apply to other models
* fix
2023-06-30 11:27:27 -07:00
Matt
134caef31a
Speed up TF tests by reducing hidden layer counts ( #24595 )
...
* hidden layers, huh, what are they good for (absolutely nothing)
* Some tests break with 1 hidden layer, use 2
* Use 1 hidden layer in a few slow models
* Use num_hidden_layers=2 everywhere
* Slightly higher tol for groupvit
* Slightly higher tol for groupvit
2023-06-30 16:30:33 +01:00
Yih-Dar
3441ad7d43
Make (TF) CI faster (test only a subset of model classes) ( #24592 )
...
* fix
* fix
* fix
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-06-30 16:54:54 +02:00
JB (Don)
78a2b19fc8
Show a warning for missing attention masks when pad_token_id is not None ( #24510 )
...
* Adding warning messages to BERT for missing attention masks
These warning messages when there are pad tokens within the input ids and
no attention masks are given. The warning message should only show up once.
* Adding warning messages to BERT for missing attention masks
These warning messages are shown when the pad_token_id is not None
and no attention masks are given. The warning message should only
show up once.
* Ran fix copies to copy over the changes to some of the other models
* Add logger.warning_once.cache_clear() to the test
* Shows warning when there are no attention masks and input_ids start/end with pad tokens
* Using warning_once() instead and fix indexing in input_ids check
---------
Co-authored-by: JB Lau <hckyn@voyager2.local>
2023-06-30 08:19:39 -04:00
Jeroen Van Goey
fd8dcd0953
Udate link to RunHouse hardware setup documentation. ( #24590 )
...
* Udate link to RunHouse hardware setup documentation.
* Fix link to hardware setup in other location as well
2023-06-30 12:11:58 +01:00
Arthur
b52a03cd3b
⚠️ ⚠️ [T5Tokenize] Fix T5 family tokenizers ⚠️ ⚠️ ( #24565 )
...
* don't add space before single letter chars that don't have a merge
* fix the fix
* fixup
* add a test
* more testing
* fixup
* hack to make sure fast is also fixed
* update switch transformers test
* revert convert slow
* Update src/transformers/models/t5/tokenization_t5.py
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* add typechecking
* quality
---------
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
2023-06-30 07:00:43 +02:00
Sourab Mangrulkar
9e28750287
fix peft ckpts not being pushed to hub ( #24578 )
...
* fix push to hub for peft ckpts
* oops
2023-06-30 00:07:44 +05:30
MS Kim(tony9402)
232c898f9f
Fix annotations ( #24582 )
...
* fix annotations
* fix annotations
* fix annotations
* fix annotations
* fix annotations
* fix annotations
* fix annotations
* fix annotations
* fix annotations
* fix annotations
* fix annotations
* fix annotations
* fix annotations
* fix annotations
* fix annotations
* fix annotations
* fix annotations
* fix annotations
* fix annotations
* fix annotations
* fix annotations
* fix annotations
* fix annotations
2023-06-29 14:17:35 -04:00
Yih-Dar
c817bc44e2
Check all objects are equally in the main __init__ file ( #24573 )
...
* fix
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-06-29 17:49:59 +02:00
Sylvain Gugger
8c4471d1fc
Fix ESM models buffers ( #24576 )
...
* Fix ESM models buffers
* Remove modifs
* Tied weights keys are needed silly
* quality
2023-06-29 10:55:21 -04:00
amyeroberts
b324557aac
Removal of deprecated vision methods and specify deprecation versions ( #24570 )
...
* Removal of deprecated methods and specify versions
* Fix tests
2023-06-29 15:09:51 +01:00
Yih-Dar
77db28dc52
Update some torchscript tests after #24505 ( #24566 )
...
* fix
* fix
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-06-29 16:05:24 +02:00
Sanchit Gandhi
1c1c90756d
Add Musicgen ( #24109 )
...
* Add Audiocraft
* add cross attention
* style
* add for lm
* convert and verify
* introduce t5
* split configs
* load t5 + lm
* clean conversion
* copy from t5
* style
* start pattern provider
* make generation work
* style
* fix pos embs
* propagate shape changes
* propagate shape changes
* style
* delay pattern: pad tokens at end
* audiocraft -> musicgen
* fix inits
* add mdx
* style
* fix pad token in processor
* override generate and add todos
* add init to test
* undo pattern delay mask after gen
* remove cfg logits processor
* remove cfg logits processor
* remove logits processor in favour of mask
* clean pos embs
* make fix copies
* update readmes
* clean pos emb
* refactor encoder/decoder
* make fix copies
* update conversion
* fix config imports
* update config docs
* make style
* send pattern mask to device
* pattern mask with delay
* recover prompted audio tokens
* fix docstrings
* laydown test file
* pattern edge case
* remove t5 ref
* add processing class
* config refactor
* better pattern comment
* check if mask is not present
* check if mask is not present
* refactor to auto class
* remove encoder configs
* fix processor
* processor import
* start updating conversion
* start updating tests
* make style
* convert t5, encodec, lm
* convert as composite
* also convert processor
* run generate
* classifier free gen
* comments and clean up
* make style
* docs for logit proc
* docstring for uncond gen
* start lm tests
* work tests
* let the lm generate
* refactor: reshape inside forward
* undo greedy loop changes
* from_enc_dec -> from_sub_model
* fix input id shapes in docstrings
* Apply suggestions from code review
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
* undo generate changes
* from sub model config
* Update src/transformers/models/musicgen/modeling_musicgen.py
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
* make generate work again
* generate uncond -> get uncond inputs
* remove prefix allowed tokens fn
* better error message
* logit proc checks
* Apply suggestions from code review
Co-authored-by: Joao Gante <joaofranciscocardosogante@gmail.com>
* make decoder only tests work
* composite fast tests
* make style
* uncond generation
* feat extr padding
* make audio prompt work
* fix inputs docstrings
* unconditional inputs: dict -> model output
* clean up tests
* more clean up tests
* make style
* t5 encoder -> auto text encoder
* remove comments
* deal with frames
* fix auto text
* slow tests
* nice mdx
* remove can generate
* todo - hub id
* convert m/l
* make fix copies
* only import generation with torch
* ignore decoder from tests
* don't wrap uncond inputs
* make style
* cleaner uncond inputs
* add example to musicgen forward
* fix docs
* ignore MusicGen Model/ForConditionalGeneration in auto mapping
* add doc section to toctree
* add to doc tests
* add processor tests
* fix push to hub in conversion
* tips for decoder only loading
* Apply suggestions from code review
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* fix conversion for s / m / l checkpoints
* import stopping criteria from module
* remove from pipeline tests
* fix uncond docstring
* decode audio method
* fix docs
* org: sanchit-gandhi -> facebook
* fix max pos embeddings
* remove auto doc (not compatible with shapes)
* bump max pos emb
* make style
* fix doc
* fix config doc
* fix config doc
* ignore musicgen config from docstring
* make style
* fix config
* fix config for doctest
* consistent from_sub_models
* don't automap decoder
* fix mdx save audio file
* fix mdx save audio file
* processor batch decode for audio
* remove keys to ignore
* update doc md
* update generation config
* allow changes for default generation config
* update tests
* make style
* fix docstring for uncond
* fix processor test
* fix processor test
---------
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
Co-authored-by: Joao Gante <joaofranciscocardosogante@gmail.com>
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
2023-06-29 14:48:59 +01:00
Sylvain Gugger
2dc5e1a120
Revert "Fix typing annotations for FSDP and DeepSpeed in TrainingArguments" ( #24574 )
...
Revert "Fix typing annotations for FSDP and DeepSpeed in TrainingArguments (#24549 )"
This reverts commit c5e29d4381
2023-06-29 08:14:43 -04:00
Joao Gante
4f1b31c2ee
Docs: 4 bit doc corrections ( #24572 )
...
4 bit doc corrections
2023-06-29 13:13:20 +01:00
MS Kim(tony9402)
1fd52e6e60
Fix annotations ( #24571 )
...
* fix annotations
* fix copies
2023-06-29 08:05:19 -04:00
MS Kim(tony9402)
63cc30e71b
Fix Typo ( #24559 )
2023-06-29 08:04:07 -04:00
amyeroberts
ae454f41d4
Update old existing feature extractor references ( #24552 )
...
* Update old existing feature extractor references
* Typo
* Apply suggestions from code review
* Apply suggestions from code review
* Apply suggestions from code review
* Address comments from review - update 'feature extractor'
Co-authored by: Yih-Dar <2521628+ydshieh@users.noreply.github.com>
2023-06-29 10:17:36 +01:00
Pasquale De Marinis
10c2ac7bc6
Fixed OwlViTModel inplace operations ( #24529 )
...
* fixed OwlViTModel inplace operations
* fixed operands order in owlvit
2023-06-29 10:17:26 +02:00
condor-cp
66954ea25e
Update masked_language_modeling.md ( #24560 )
...
See https://github.com/huggingface/transformers/issues/24546
2023-06-28 17:54:20 -04:00
Yih-Dar
fd6735102a
Make PT/Flax tests could be run on GPU ( #24557 )
...
fix
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-06-28 20:11:01 +02:00
