Jacky Lee
4302b27719
Fix typos in translated quicktour docs ( #35302 )
...
* fix: quicktour typos
* fix: one more
2024-12-17 09:32:00 -08:00
谭九鼎
5bfb40bc8e
docs: HUGGINGFACE_HUB_CACHE -> HF_HUB_CACHE ( #34904 )
2024-11-26 09:37:18 -08:00
Fanli Lin
25f510a9c6
[docs] update not-working model revision ( #34682 )
...
update revision
2024-11-11 07:09:31 -08:00
Yih-Dar
f0e640adfa
Drop support for Python 3.8 ( #34314 )
...
* drop python 3.8
* update docker files
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2024-10-24 11:16:55 +02:00
amyeroberts
1de7dc7403
Skip tests properly ( #31308 )
...
* Skip tests properly
* [test_all]
* Add 'reason' as kwarg for skipTest
* [test_all] Fix up
* [test_all]
2024-06-26 21:59:08 +01:00
Younes Belkada
f5590deaa8
Docs / Quantization: Replace all occurences of load_in_8bit with bnb config ( #31136 )
...
Replace all occurences of `load_in_8bit` with bnb config
2024-05-30 16:47:35 +02:00
Lucain
c3044ec2f3
Use HF_HUB_OFFLINE + fix has_file in offline mode ( #31016 )
...
* Fix has_file in offline mode
* harmonize env variable for offline mode
* Switch to HF_HUB_OFFLINE
* fix test
* revert test_offline to test TRANSFORMERS_OFFLINE
* Add new offline test
* merge conflicts
* docs
2024-05-29 11:55:43 +01:00
clinty
bdbe166211
Fix broken link to Transformers notebooks ( #30512 )
...
Co-authored-by: Clint Adams <clint@debian.org>
2024-04-29 10:57:51 +01:00
Lysandre Debut
d4e92f1a21
Remove add-new-model in favor of add-new-model-like ( #30424 )
...
* Remove add-new-model in favor of add-new-model-like
* nits
2024-04-24 09:38:18 +02:00
Matt
696ededd2b
Remove old TF port docs ( #30426 )
...
* Remove old TF port guide
* repo-consistency
* Remove some translations as well for consistency
* Remove some translations as well for consistency
2024-04-23 16:06:20 +01:00
Zach Mueller
60d5f8f9f0
🚨 🚨 🚨 Deprecate evaluation_strategy to eval_strategy 🚨 🚨 🚨 ( #30190 )
...
* Alias
* Note alias
* Tests and src
* Rest
* Clean
* Change typing?
* Fix tests
* Deprecation versions
2024-04-18 12:49:43 -04:00
Hafedh
0eaef0c709
add push_to_hub to pipeline ( #29172 )
...
* add `push_to_hub` to pipeline
* fix docs
* format with ruff
* update save_pretrained
* update save_pretrained
* remove unnecessary comment
* switch to push_to_hub method in DynamicPipelineTester
* remove unused imports
* update docs for add_new_pipeline
* fix docs for add_new_pipeline
* add comment
* fix italien docs
* changes to token retrieval for pipelines
* Update src/transformers/pipelines/base.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
---------
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
2024-04-16 15:34:04 +01:00
Utkarsha Gupte
0201f6420b
[ #29174 ] ImportError Fix: Trainer with PyTorch requires accelerate>=0.20.1 Fix ( #29888 )
...
* ImportError: Trainer with PyTorch requires accelerate>=0.20.1 Fix
Adding the evaluate and accelerate installs at the beginning of the cell to fix the issue
* ImportError Fix: Trainer with PyTorch requires accelerate>=0.20.1
* Import Error Fix
* Update installation.md
* Update quicktour.md
* rollback other lang changes
* Update _config.py
* updates for other languages
* fixing error
* Tutorial Update
* Update tokenization_utils_base.py
* Just use an optimizer string to pass the doctest?
---------
Co-authored-by: Matt <rocketknight1@gmail.com>
2024-04-08 14:21:16 +01:00
Yitong Huang
873d9bb3cc
Make torch xla available on GPU ( #29334 )
...
* add USE_TORCH_XLA env
* rename torch_tpu to torch_xla
* better is_torch_xla_available; fix some fsdp and performance issues
* fix format
* fix bug when pjrt_device is cpu
* fix bug
* fix the deprecation handling
---------
Co-authored-by: anw90 <ang868@gmail.com>
Co-authored-by: wangang.wa <wangang.wa@alibaba-inc.com>
2024-03-11 14:07:16 +00:00
Lysandre Debut
f497f564bb
Update all references to canonical models ( #29001 )
...
* Script & Manual edition
* Update
2024-02-16 08:16:58 +01:00
Klaus Hipp
d90acc1643
[i18n-de] Translate CONTRIBUTING.md to German ( #28954 )
...
* Translate contributing.md to German
* Fix formatting issues in contributing.md
* Address review comments
* Fix capitalization
2024-02-12 13:39:20 -08:00
Klaus Hipp
2749e479f3
[Docs] Fix broken links and syntax issues ( #28918 )
...
* Fix model documentation links in attention.md
* Fix external link syntax
* Fix target anchor names of section links
* Fix copyright statement comments
* Fix documentation headings
2024-02-08 14:13:35 -08:00
Klaus Hipp
33df036917
[Docs] Revert translation of '@slow' decorator ( #28912 )
2024-02-08 03:31:47 +01:00
Klaus Hipp
1c31b7aa3b
[Docs] Add missing language options and fix broken links ( #28852 )
...
* Add missing entries to the language selector
* Add links to the Colab and AWS Studio notebooks for ONNX
* Use anchor links in CONTRIBUTING.md
* Fix broken hyperlinks due to spaces
* Fix links to OpenAI research articles
* Remove confusing footnote symbols from author names, as they are also considered invalid markup
2024-02-06 12:01:01 -08:00
Klaus Hipp
4830f26965
[Docs] Fix backticks in inline code and documentation links ( #28875 )
...
Fix backticks in code blocks and documentation links
2024-02-06 11:15:44 -08:00
Klaus Hipp
721ee783ca
[Docs] Fix spelling and grammar mistakes ( #28825 )
...
* Fix typos and grammar mistakes in docs and examples
* Fix typos in docstrings and comments
* Fix spelling of `tokenizer` in model tests
* Remove erroneous spaces in decorators
* Remove extra spaces in Markdown link texts
2024-02-02 08:45:00 +01:00
Hankyeol Kyung
995a7ce9a8
Fix broken link on page ( #28451 )
...
* [docs] Fix broken link
Signed-off-by: Hankyeol Kyung <kghnkl0103@gmail.com>
* [docs] Use shorter domain
Signed-off-by: Hankyeol Kyung <kghnkl0103@gmail.com>
---------
Signed-off-by: Hankyeol Kyung <kghnkl0103@gmail.com>
2024-01-11 09:26:13 -08:00
Kevin Herro
5d36025ca1
README: install transformers from conda-forge channel ( #28313 )
...
Switch to the conda-forge channel for transformer installation,
as the huggingface channel does not offer the latest version.
Fixes #28248
2024-01-04 09:36:16 -08:00
Peter Pan
ce31508134
docs: replace torch.distributed.run by torchrun ( #27528 )
...
* docs: replace torch.distributed.run by torchrun
`transformers` now officially support pytorch >= 1.10.
The entrypoint `torchrun`` is present from 1.10 onwards.
Signed-off-by: Peter Pan <Peter.Pan@daocloud.io>
* Update src/transformers/trainer.py
with @ArthurZucker's suggestion
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>
---------
Signed-off-by: Peter Pan <Peter.Pan@daocloud.io>
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>
2023-11-27 16:26:33 +00:00
V.Prasanna kumar
ffbcfc0166
Broken links fixed related to datasets docs ( #27569 )
...
fixed the broken links belogs to dataset library of transformers
2023-11-17 13:44:09 -08:00
Florian Zimmermeister
9ed538f2e6
[i18n-DE] contribute chapter ( #26481 )
...
* start working on next chapter
* finish testing
* Update docs/source/de/testing.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
* Update docs/source/de/testing.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
* Update docs/source/de/testing.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
---------
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
2023-10-02 09:56:40 -07:00
Florian Zimmermeister
ef81759e31
[i18n-DE] Complete first toc chapter ( #26311 )
...
* initial
* toctree
* add tf model
* run scripts
* peft
* llm and agents
* Update docs/source/de/peft.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
* Update docs/source/de/peft.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
* Update docs/source/de/peft.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
* Update docs/source/de/run_scripts.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
* Update docs/source/de/run_scripts.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
* Update docs/source/de/transformers_agents.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
* Update docs/source/de/transformers_agents.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
---------
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
2023-09-27 11:33:05 -07:00
lishukan
c385de2441
[TYPO] fix typo/format in quicktour.md ( #25519 )
...
* fix_all_language_quicktour
* give up ! before bash command
---------
Co-authored-by: lishukan <lishukan@dxy.cn>
2023-08-16 08:03:23 +02:00
Sanchit Gandhi
e93103632b
Add bloom flax ( #25094 )
...
* First commit
* step 1 working
* add alibi
* placeholder for `scan`
* add matrix mult alibi
* beta scaling factor for bmm
* working v1 - simple forward pass
* move layer_number from attribute to arg in call
* partial functioning scan
* hacky working scan
* add more modifs
* add test
* update scan for new kwarg order
* fix position_ids problem
* fix bug in attention layer
* small fix
- do the alibi broadcasting only once
* prelim refactor
* finish refactor
* alibi shifting
* incorporate dropout_add to attention module
* make style
* make padding work again
* update
* remove bogus file
* up
* get generation to work
* clean code a bit
* added small tests
* adding albii test
* make CI tests pass:
- change init weight
- add correct tuple for output attention
- add scan test
- make CI tests work
* fix few nits
* fix nit onnx
* fix onnx nit
* add missing dtype args to nn.Modules
* remove debugging statements
* fix scan generate
* Update modeling_flax_bloom.py
* Update test_modeling_flax_bloom.py
* Update test_modeling_flax_bloom.py
* Update test_modeling_flax_bloom.py
* fix small test issue + make style
* clean up
* Update tests/models/bloom/test_modeling_flax_bloom.py
Co-authored-by: Sanchit Gandhi <93869735+sanchit-gandhi@users.noreply.github.com>
* fix function name
* small fix test
* forward contrib credits from PR17761
* Fix failing test
* fix small typo documentation
* fix non passing test
- remove device from build alibi
* refactor call
- refactor `FlaxBloomBlockCollection` module
* make style
* upcast to fp32
* cleaner way to upcast
* remove unused args
* remove layer number
* fix scan test
* make style
* fix i4 casting
* fix slow test
* Update src/transformers/models/bloom/modeling_flax_bloom.py
Co-authored-by: Sanchit Gandhi <93869735+sanchit-gandhi@users.noreply.github.com>
* remove `layer_past`
* refactor a bit
* fix `scan` slow test
* remove useless import
* major changes
- remove unused code
- refactor a bit
- revert import `torch`
* major refactoring
- change build alibi
* remove scan
* fix tests
* make style
* clean-up alibi
* add integration tests
* up
* fix batch norm conversion
* style
* style
* update pt-fx cross tests
* update copyright
* Update src/transformers/modeling_flax_pytorch_utils.py
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* per-weight check
* style
* line formats
---------
Co-authored-by: younesbelkada <younesbelkada@gmail.com>
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
Co-authored-by: Younes Belkada <49240599+younesbelkada@users.noreply.github.com>
Co-authored-by: haileyschoelkopf <haileyschoelkopf@users.noreply.github.com>
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
2023-07-27 18:24:56 +01:00
Arthur
799df10aef
[Umt5] Add google's umt5 to transformers ( #24477 )
...
* add tokenization template
* update conversion script
* update modeling code
* update
* update convert checkpoint
* update modeling
* revert changes on convert script
* new conversion script for new format
* correct position bias
* cleaning a bit
* Credit co authors
Co-authored-by: agemagician
<ahmed.elnaggar@tum.de>
Co-authored-by: stefan-it
<>
* styling
* Add docq
* fix copies
* add co author
* Other Author
* Merge branch 'main' of https://github.com/huggingface/transformers into add-umt5
* add testing
* nit
* Update docs/source/en/model_doc/umt5.mdx
Co-authored-by: Stefan Schweter <stefan@schweter.it>
* fix t5
* actual fix?
* revert wrong changes
* remove
* update test
* more fixes
* revert some changes
* add SPIECE_UNDERLINE
* add a commone xample
* upfate
* fix copies
* revert changes on t5 conversion script
* revert bytefallback changes since there was no addition yet
* fixup
* fixup
* ingore umt5 cutom testing folder
* fix readmes
* revertT5 changes
* same outputs
* fixup
* update example
* Apply suggestions from code review
* style
* draft addition of all new files
* current update
* fix attention and stuff
* finish refactoring
* auto config
* fixup
* more nits
* add umt5 to init
* use md format
* Update README.md
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* revert changes on mt5
* revert mt4 changes
* update test
* more fixes
* add to mapping
* fix-copies
* fix copies
* foix retain grad
* fix some tests
* nits
* done
* Update src/transformers/models/umt5/modeling_umt5.py
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* Update docs/source/en/model_doc/umt5.md
* Update src/transformers/models/umt5/__init__.py
* Update docs/source/en/model_doc/umt5.md
Co-authored-by: Stefan Schweter <stefan@schweter.it>
* Update src/transformers/models/umt5/modeling_umt5.py
* update conversion script + use google checkpoints
* nits
* update test and modelling
* stash slow convert
* update fixupd
* don't change slow
---------
Co-authored-by: stefan-it <>
Co-authored-by: Stefan Schweter <stefan@schweter.it>
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
2023-07-03 07:38:21 +02:00
amyeroberts
ae454f41d4
Update old existing feature extractor references ( #24552 )
...
* Update old existing feature extractor references
* Typo
* Apply suggestions from code review
* Apply suggestions from code review
* Apply suggestions from code review
* Address comments from review - update 'feature extractor'
Co-authored by: Yih-Dar <2521628+ydshieh@users.noreply.github.com>
2023-06-29 10:17:36 +01:00
Sylvain Gugger
eb849f6604
Migrate doc files to Markdown. ( #24376 )
...
* Rename index.mdx to index.md
* With saved modifs
* Address review comment
* Treat all files
* .mdx -> .md
* Remove special char
* Update utils/tests_fetcher.py
Co-authored-by: Lysandre Debut <lysandre.debut@reseau.eseo.fr>
---------
Co-authored-by: Lysandre Debut <lysandre.debut@reseau.eseo.fr>
2023-06-20 18:07:47 -04:00
Shubhamai
900677487d
Flax Regnet ( #21867 )
...
* initial commit
* review changes
* post model PR merge
* updating doc
2023-04-04 12:41:12 -04:00
Shubhamai
a0cbbba31f
Resnet flax ( #21472 )
...
* [WIP] flax resnet
* added pretrained flax models, results reproducible
* Added pretrained flax models, results reproducible
* working on tests
* no real code change, just some comments
* [flax] adding support for batch norm layers
* fixing bugs related to pt+flax integration
* removing loss from modeling flax output class
* fixing classifier tests
* fixing comments, model output
* cleaning comments
* review changes
* review changes
* Apply suggestions from code review
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* renaming Flax to PyTorch
---------
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
2023-03-24 19:45:57 +00:00
Alara Dirik
cdddfbffa1
Add ConvNeXT V2 ( #21679 )
...
* Add ConvNeXt V2 to transformers
* TF model is separated from the PR to fix issues
2023-03-14 12:08:14 +03:00
Alara Dirik
269b054939
Add ALIGN to transformers ( #21741 )
...
Adds the ALIGN model to transformers. ALIGN is introduced in "Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision" by Chao Jia, Yinfei Yang, Ye Xia, Yi-Ting Chen, Zarana Parekh, Hieu Pham, Quoc V. Le, Yunhsuan Sung, Zhen Li, Tom Duerig.
2023-03-01 21:23:31 +03:00
Alara Dirik
49ab16239c
Add EfficientNet ( #21563 )
...
* Add EfficientNet to transformers
2023-02-20 16:37:11 +03:00
tanreinama
f56174ac5b
add GPTSAN model (reopen) ( #21291 )
...
* add GPTSAN-Japanese
* add GPTSAN
* add GPTSAN
* add GPTSAN
* add GPTSAN
* add GPTSAN
* add GPTSAN
* add GPTSAN
* add GPTSAN
* add GPTSAN
* add GPTSAN
* add GPTSAN
* add GPTSAN
* add GPTSAN
* add GPTSAN
* add GPTSAN
* add GPTSAN
* add GPTSAN
* add GPTSAN
* add GPTSAN (update for review)
* add GPTSAN
* add GPTSAN
* add GPTSAN
* add GPTSAN
* add GPTSAN
* add GPTSAN
* add GPTSAN
* add GPTSAN
* add GPTSAN
* add GPTSAN
* add GPTSAN
* add GPTSAN
* add GPTSAN
* add GPTSAN
* add GPTSAN
* add GPTSAN
* add GPTSAN
* add GPTSAN
* add GPTSAN
* add GPTSAN
* fix typo in comment text
* add GPTSAN
* add GPTSAN
* add GPTSAN
* add GPTSAN
* fix document and comments
* fix class name GPTSAN->GPTSan
* fix import and test for tokenizer
2023-02-20 11:25:27 +01:00
Stefan Schweter
7e51a441e4
Add XLM-V to Model Doc ( #21498 )
...
* doc: introduce new section for XLM-V model
* doc: mention more details for XLM-V integration
* docs: paper abstract in italics, model identifier for base model added
* doc: mention new XLM-V support
* auto: add XLM-V mapping
* doc: run make fix-copies ;)
2023-02-07 16:43:19 -05:00
Irene López
7dbee87e09
Fix PushToHubCallback import in Share a model docs ( #21457 )
...
docs: update PushToHubCallback import in docs
2023-02-06 09:26:22 -05:00
Jitesh Jain
5b949623c7
Add OneFormer Model ( #20577 )
...
* Add Oneformer Model
* Add OneFormer Tests
* Add UNIVERSAL_SEGMENTATION_MAPPING
* Fix config
* 🐛 Fix error encountered while writing tests
* 🔨 Fix instance segmentation post processing
* Format Files and Add Documentation
* Add Documentation mdx file
* Run make fixup
* Run make fix-copies
* Remove unnecessary code
* Format modeling_oneformer.py
* Add OneFormer to ImageSegmentationPipeline
* Format files
* Add Demo link to Readme
* Fix fomatting errors
* Fix test failures
* Update Table in index.mdx
* Fix version
* Fix style
* Remove OneFormer from TF
* Fix Imports
* Fix dummy objects
* Fix tests
* Add newline
* Remove OneFormerFeatureExtractor
* Remove CUDA Kernels
* Use AutoBackbone for Swin
* Fix description
* Use Image Processor
* Fix copies
* Fix formatting
* Fix import order
* Fix flake8 errors
* Fix doc errors
* Add Hindi Readme entry
* Update supported backbones
* Update supported backbones
* Undo Changes
* Fix type of config
* Fix isort
* Fix auto.mdx
* Fix swin config
* Replace DinatBackbone with AutoBackbone
* Use SwinBackbone
* Use SwinBackbone
* Fix conversion script
* Fix arguments
* Add argument description
* Fix style
* Add OneFormerProcessor
* Fix OneFormerProcessor Tests
* Fix mapping
* Fix imports
* Fix inits
* Fix style
* Fix comment
* Fix docstring
* Move OneFormer to MultiModal
* Fix Copies
* Remove size divisor
* Fix check_repo.py
* Fix copies
* Add Processor for Testing Pipeline
* Fix padding for tokens
* Fix variables
* Fix formatting with correct black version
* Add Image Processor Test
* Apply suggestions
* Revert common modeling
* Add check for task
* Fix conversion script
* Fix initialization order
* Fix tests
* Undo Pipeline Changes
* Fix layers in MLP
* Fix copies
* Update image paths
* Fix copies
* Apply suggestions
2023-01-19 09:31:07 +01:00
Alara Dirik
2411f0e465
Add Mask2Former ( #20792 )
...
* Adds Mask2Former to transformers
Co-authored-by: Shivalika Singh <shivalikasingh95@gmail.com>
Co-authored-by: Shivalika Singh <73357305+shivalikasingh95@users.noreply.github.com>
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
2023-01-16 20:37:07 +03:00
Matt
c1b9a11dd4
Convert tokenizer outputs for Keras in doc example ( #20732 )
...
* Convert tokenizer outputs for Keras in doc example
* Das deutsche Beispiel auch korrigieren
2022-12-12 16:14:04 +00:00
Raj Rajhans
22d7161a52
fix: "BigSicence" typo in docs ( #20331 )
2022-11-21 09:44:54 -05:00
Wonhyeong Seo
8fadfd5035
[docs] set overflowing image width to auto-scale ( #20197 )
...
* docs: fix: set overflowing image width to auto-scale
* docs: fix: new language Korean is also affected
* docs: fix: unnecessary line break in index page
2022-11-15 01:13:40 +01:00
Joao Gante
f270b960d6
Generate: move generation_*.py src files into generation/*.py ( #20096 )
...
* move generation_*.py src files into generation/*.py
* populate generation.__init__ with lazy loading
* move imports and references from generation.xxx.object to generation.object
2022-11-09 15:34:08 +00:00
flozi00
fa4eeb4fd3
german training, accelerate and model sharing ( #19171 )
...
* correct spelling in README
* processing
* german training
* accelerate
* german model sharing
* build doc
* ttf links
* casing
2022-09-23 14:52:09 -04:00
flozi00
de26241645
german processing ( #19121 )
...
* correct spelling in README
* processing
2022-09-20 09:18:21 -04:00
flozi00
ae219532e3
german autoclass ( #19049 )
...
* german autoclass
* Update _toctree.yml
2022-09-16 16:16:00 -04:00
Devlee247
9832ac7c73
Fix LayoutXLM wrong link in README ( #18932 )
...
* fix LayoutXLM wrong link in README
* fix LayoutXLM worng link in index.mdx
2022-09-08 07:32:41 -04:00
