Klaus Hipp
721ee783ca
[Docs] Fix spelling and grammar mistakes ( #28825 )
...
* Fix typos and grammar mistakes in docs and examples
* Fix typos in docstrings and comments
* Fix spelling of `tokenizer` in model tests
* Remove erroneous spaces in decorators
* Remove extra spaces in Markdown link texts
2024-02-02 08:45:00 +01:00
Hankyeol Kyung
995a7ce9a8
Fix broken link on page ( #28451 )
...
* [docs] Fix broken link
Signed-off-by: Hankyeol Kyung <kghnkl0103@gmail.com>
* [docs] Use shorter domain
Signed-off-by: Hankyeol Kyung <kghnkl0103@gmail.com>
---------
Signed-off-by: Hankyeol Kyung <kghnkl0103@gmail.com>
2024-01-11 09:26:13 -08:00
Kevin Herro
5d36025ca1
README: install transformers from conda-forge channel ( #28313 )
...
Switch to the conda-forge channel for transformer installation,
as the huggingface channel does not offer the latest version.
Fixes #28248
2024-01-04 09:36:16 -08:00
Peter Pan
ce31508134
docs: replace torch.distributed.run by torchrun ( #27528 )
...
* docs: replace torch.distributed.run by torchrun
`transformers` now officially support pytorch >= 1.10.
The entrypoint `torchrun`` is present from 1.10 onwards.
Signed-off-by: Peter Pan <Peter.Pan@daocloud.io>
* Update src/transformers/trainer.py
with @ArthurZucker's suggestion
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>
---------
Signed-off-by: Peter Pan <Peter.Pan@daocloud.io>
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>
2023-11-27 16:26:33 +00:00
V.Prasanna kumar
ffbcfc0166
Broken links fixed related to datasets docs ( #27569 )
...
fixed the broken links belogs to dataset library of transformers
2023-11-17 13:44:09 -08:00
Florian Zimmermeister
9ed538f2e6
[i18n-DE] contribute chapter ( #26481 )
...
* start working on next chapter
* finish testing
* Update docs/source/de/testing.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
* Update docs/source/de/testing.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
* Update docs/source/de/testing.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
---------
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
2023-10-02 09:56:40 -07:00
Florian Zimmermeister
ef81759e31
[i18n-DE] Complete first toc chapter ( #26311 )
...
* initial
* toctree
* add tf model
* run scripts
* peft
* llm and agents
* Update docs/source/de/peft.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
* Update docs/source/de/peft.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
* Update docs/source/de/peft.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
* Update docs/source/de/run_scripts.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
* Update docs/source/de/run_scripts.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
* Update docs/source/de/transformers_agents.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
* Update docs/source/de/transformers_agents.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
---------
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
2023-09-27 11:33:05 -07:00
lishukan
c385de2441
[TYPO] fix typo/format in quicktour.md ( #25519 )
...
* fix_all_language_quicktour
* give up ! before bash command
---------
Co-authored-by: lishukan <lishukan@dxy.cn>
2023-08-16 08:03:23 +02:00
Sanchit Gandhi
e93103632b
Add bloom flax ( #25094 )
...
* First commit
* step 1 working
* add alibi
* placeholder for `scan`
* add matrix mult alibi
* beta scaling factor for bmm
* working v1 - simple forward pass
* move layer_number from attribute to arg in call
* partial functioning scan
* hacky working scan
* add more modifs
* add test
* update scan for new kwarg order
* fix position_ids problem
* fix bug in attention layer
* small fix
- do the alibi broadcasting only once
* prelim refactor
* finish refactor
* alibi shifting
* incorporate dropout_add to attention module
* make style
* make padding work again
* update
* remove bogus file
* up
* get generation to work
* clean code a bit
* added small tests
* adding albii test
* make CI tests pass:
- change init weight
- add correct tuple for output attention
- add scan test
- make CI tests work
* fix few nits
* fix nit onnx
* fix onnx nit
* add missing dtype args to nn.Modules
* remove debugging statements
* fix scan generate
* Update modeling_flax_bloom.py
* Update test_modeling_flax_bloom.py
* Update test_modeling_flax_bloom.py
* Update test_modeling_flax_bloom.py
* fix small test issue + make style
* clean up
* Update tests/models/bloom/test_modeling_flax_bloom.py
Co-authored-by: Sanchit Gandhi <93869735+sanchit-gandhi@users.noreply.github.com>
* fix function name
* small fix test
* forward contrib credits from PR17761
* Fix failing test
* fix small typo documentation
* fix non passing test
- remove device from build alibi
* refactor call
- refactor `FlaxBloomBlockCollection` module
* make style
* upcast to fp32
* cleaner way to upcast
* remove unused args
* remove layer number
* fix scan test
* make style
* fix i4 casting
* fix slow test
* Update src/transformers/models/bloom/modeling_flax_bloom.py
Co-authored-by: Sanchit Gandhi <93869735+sanchit-gandhi@users.noreply.github.com>
* remove `layer_past`
* refactor a bit
* fix `scan` slow test
* remove useless import
* major changes
- remove unused code
- refactor a bit
- revert import `torch`
* major refactoring
- change build alibi
* remove scan
* fix tests
* make style
* clean-up alibi
* add integration tests
* up
* fix batch norm conversion
* style
* style
* update pt-fx cross tests
* update copyright
* Update src/transformers/modeling_flax_pytorch_utils.py
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* per-weight check
* style
* line formats
---------
Co-authored-by: younesbelkada <younesbelkada@gmail.com>
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
Co-authored-by: Younes Belkada <49240599+younesbelkada@users.noreply.github.com>
Co-authored-by: haileyschoelkopf <haileyschoelkopf@users.noreply.github.com>
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
2023-07-27 18:24:56 +01:00
Arthur
799df10aef
[Umt5] Add google's umt5 to transformers ( #24477 )
...
* add tokenization template
* update conversion script
* update modeling code
* update
* update convert checkpoint
* update modeling
* revert changes on convert script
* new conversion script for new format
* correct position bias
* cleaning a bit
* Credit co authors
Co-authored-by: agemagician
<ahmed.elnaggar@tum.de>
Co-authored-by: stefan-it
<>
* styling
* Add docq
* fix copies
* add co author
* Other Author
* Merge branch 'main' of https://github.com/huggingface/transformers into add-umt5
* add testing
* nit
* Update docs/source/en/model_doc/umt5.mdx
Co-authored-by: Stefan Schweter <stefan@schweter.it>
* fix t5
* actual fix?
* revert wrong changes
* remove
* update test
* more fixes
* revert some changes
* add SPIECE_UNDERLINE
* add a commone xample
* upfate
* fix copies
* revert changes on t5 conversion script
* revert bytefallback changes since there was no addition yet
* fixup
* fixup
* ingore umt5 cutom testing folder
* fix readmes
* revertT5 changes
* same outputs
* fixup
* update example
* Apply suggestions from code review
* style
* draft addition of all new files
* current update
* fix attention and stuff
* finish refactoring
* auto config
* fixup
* more nits
* add umt5 to init
* use md format
* Update README.md
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* revert changes on mt5
* revert mt4 changes
* update test
* more fixes
* add to mapping
* fix-copies
* fix copies
* foix retain grad
* fix some tests
* nits
* done
* Update src/transformers/models/umt5/modeling_umt5.py
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* Update docs/source/en/model_doc/umt5.md
* Update src/transformers/models/umt5/__init__.py
* Update docs/source/en/model_doc/umt5.md
Co-authored-by: Stefan Schweter <stefan@schweter.it>
* Update src/transformers/models/umt5/modeling_umt5.py
* update conversion script + use google checkpoints
* nits
* update test and modelling
* stash slow convert
* update fixupd
* don't change slow
---------
Co-authored-by: stefan-it <>
Co-authored-by: Stefan Schweter <stefan@schweter.it>
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
2023-07-03 07:38:21 +02:00
amyeroberts
ae454f41d4
Update old existing feature extractor references ( #24552 )
...
* Update old existing feature extractor references
* Typo
* Apply suggestions from code review
* Apply suggestions from code review
* Apply suggestions from code review
* Address comments from review - update 'feature extractor'
Co-authored by: Yih-Dar <2521628+ydshieh@users.noreply.github.com>
2023-06-29 10:17:36 +01:00
Sylvain Gugger
eb849f6604
Migrate doc files to Markdown. ( #24376 )
...
* Rename index.mdx to index.md
* With saved modifs
* Address review comment
* Treat all files
* .mdx -> .md
* Remove special char
* Update utils/tests_fetcher.py
Co-authored-by: Lysandre Debut <lysandre.debut@reseau.eseo.fr>
---------
Co-authored-by: Lysandre Debut <lysandre.debut@reseau.eseo.fr>
2023-06-20 18:07:47 -04:00
Shubhamai
900677487d
Flax Regnet ( #21867 )
...
* initial commit
* review changes
* post model PR merge
* updating doc
2023-04-04 12:41:12 -04:00
Shubhamai
a0cbbba31f
Resnet flax ( #21472 )
...
* [WIP] flax resnet
* added pretrained flax models, results reproducible
* Added pretrained flax models, results reproducible
* working on tests
* no real code change, just some comments
* [flax] adding support for batch norm layers
* fixing bugs related to pt+flax integration
* removing loss from modeling flax output class
* fixing classifier tests
* fixing comments, model output
* cleaning comments
* review changes
* review changes
* Apply suggestions from code review
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* renaming Flax to PyTorch
---------
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
2023-03-24 19:45:57 +00:00
Alara Dirik
cdddfbffa1
Add ConvNeXT V2 ( #21679 )
...
* Add ConvNeXt V2 to transformers
* TF model is separated from the PR to fix issues
2023-03-14 12:08:14 +03:00
Alara Dirik
269b054939
Add ALIGN to transformers ( #21741 )
...
Adds the ALIGN model to transformers. ALIGN is introduced in "Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision" by Chao Jia, Yinfei Yang, Ye Xia, Yi-Ting Chen, Zarana Parekh, Hieu Pham, Quoc V. Le, Yunhsuan Sung, Zhen Li, Tom Duerig.
2023-03-01 21:23:31 +03:00
Alara Dirik
49ab16239c
Add EfficientNet ( #21563 )
...
* Add EfficientNet to transformers
2023-02-20 16:37:11 +03:00
tanreinama
f56174ac5b
add GPTSAN model (reopen) ( #21291 )
...
* add GPTSAN-Japanese
* add GPTSAN
* add GPTSAN
* add GPTSAN
* add GPTSAN
* add GPTSAN
* add GPTSAN
* add GPTSAN
* add GPTSAN
* add GPTSAN
* add GPTSAN
* add GPTSAN
* add GPTSAN
* add GPTSAN
* add GPTSAN
* add GPTSAN
* add GPTSAN
* add GPTSAN
* add GPTSAN
* add GPTSAN (update for review)
* add GPTSAN
* add GPTSAN
* add GPTSAN
* add GPTSAN
* add GPTSAN
* add GPTSAN
* add GPTSAN
* add GPTSAN
* add GPTSAN
* add GPTSAN
* add GPTSAN
* add GPTSAN
* add GPTSAN
* add GPTSAN
* add GPTSAN
* add GPTSAN
* add GPTSAN
* add GPTSAN
* add GPTSAN
* add GPTSAN
* fix typo in comment text
* add GPTSAN
* add GPTSAN
* add GPTSAN
* add GPTSAN
* fix document and comments
* fix class name GPTSAN->GPTSan
* fix import and test for tokenizer
2023-02-20 11:25:27 +01:00
Stefan Schweter
7e51a441e4
Add XLM-V to Model Doc ( #21498 )
...
* doc: introduce new section for XLM-V model
* doc: mention more details for XLM-V integration
* docs: paper abstract in italics, model identifier for base model added
* doc: mention new XLM-V support
* auto: add XLM-V mapping
* doc: run make fix-copies ;)
2023-02-07 16:43:19 -05:00
Irene López
7dbee87e09
Fix PushToHubCallback import in Share a model docs ( #21457 )
...
docs: update PushToHubCallback import in docs
2023-02-06 09:26:22 -05:00
Jitesh Jain
5b949623c7
Add OneFormer Model ( #20577 )
...
* Add Oneformer Model
* Add OneFormer Tests
* Add UNIVERSAL_SEGMENTATION_MAPPING
* Fix config
* 🐛 Fix error encountered while writing tests
* 🔨 Fix instance segmentation post processing
* Format Files and Add Documentation
* Add Documentation mdx file
* Run make fixup
* Run make fix-copies
* Remove unnecessary code
* Format modeling_oneformer.py
* Add OneFormer to ImageSegmentationPipeline
* Format files
* Add Demo link to Readme
* Fix fomatting errors
* Fix test failures
* Update Table in index.mdx
* Fix version
* Fix style
* Remove OneFormer from TF
* Fix Imports
* Fix dummy objects
* Fix tests
* Add newline
* Remove OneFormerFeatureExtractor
* Remove CUDA Kernels
* Use AutoBackbone for Swin
* Fix description
* Use Image Processor
* Fix copies
* Fix formatting
* Fix import order
* Fix flake8 errors
* Fix doc errors
* Add Hindi Readme entry
* Update supported backbones
* Update supported backbones
* Undo Changes
* Fix type of config
* Fix isort
* Fix auto.mdx
* Fix swin config
* Replace DinatBackbone with AutoBackbone
* Use SwinBackbone
* Use SwinBackbone
* Fix conversion script
* Fix arguments
* Add argument description
* Fix style
* Add OneFormerProcessor
* Fix OneFormerProcessor Tests
* Fix mapping
* Fix imports
* Fix inits
* Fix style
* Fix comment
* Fix docstring
* Move OneFormer to MultiModal
* Fix Copies
* Remove size divisor
* Fix check_repo.py
* Fix copies
* Add Processor for Testing Pipeline
* Fix padding for tokens
* Fix variables
* Fix formatting with correct black version
* Add Image Processor Test
* Apply suggestions
* Revert common modeling
* Add check for task
* Fix conversion script
* Fix initialization order
* Fix tests
* Undo Pipeline Changes
* Fix layers in MLP
* Fix copies
* Update image paths
* Fix copies
* Apply suggestions
2023-01-19 09:31:07 +01:00
Alara Dirik
2411f0e465
Add Mask2Former ( #20792 )
...
* Adds Mask2Former to transformers
Co-authored-by: Shivalika Singh <shivalikasingh95@gmail.com>
Co-authored-by: Shivalika Singh <73357305+shivalikasingh95@users.noreply.github.com>
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
2023-01-16 20:37:07 +03:00
Matt
c1b9a11dd4
Convert tokenizer outputs for Keras in doc example ( #20732 )
...
* Convert tokenizer outputs for Keras in doc example
* Das deutsche Beispiel auch korrigieren
2022-12-12 16:14:04 +00:00
Raj Rajhans
22d7161a52
fix: "BigSicence" typo in docs ( #20331 )
2022-11-21 09:44:54 -05:00
Wonhyeong Seo
8fadfd5035
[docs] set overflowing image width to auto-scale ( #20197 )
...
* docs: fix: set overflowing image width to auto-scale
* docs: fix: new language Korean is also affected
* docs: fix: unnecessary line break in index page
2022-11-15 01:13:40 +01:00
Joao Gante
f270b960d6
Generate: move generation_*.py src files into generation/*.py ( #20096 )
...
* move generation_*.py src files into generation/*.py
* populate generation.__init__ with lazy loading
* move imports and references from generation.xxx.object to generation.object
2022-11-09 15:34:08 +00:00
flozi00
fa4eeb4fd3
german training, accelerate and model sharing ( #19171 )
...
* correct spelling in README
* processing
* german training
* accelerate
* german model sharing
* build doc
* ttf links
* casing
2022-09-23 14:52:09 -04:00
flozi00
de26241645
german processing ( #19121 )
...
* correct spelling in README
* processing
2022-09-20 09:18:21 -04:00
flozi00
ae219532e3
german autoclass ( #19049 )
...
* german autoclass
* Update _toctree.yml
2022-09-16 16:16:00 -04:00
Devlee247
9832ac7c73
Fix LayoutXLM wrong link in README ( #18932 )
...
* fix LayoutXLM wrong link in README
* fix LayoutXLM worng link in index.mdx
2022-09-08 07:32:41 -04:00
flozi00
359f7b4b8d
Create pipeline_tutorial.mdx german docs ( #18625 )
...
* Create pipeline_tutorial.mdx
* Update _toctree.yml
2022-09-01 09:57:59 +02:00
flozi00
5d3f037433
german docs translation ( #18544 )
...
* Create _config.py
* Create _toctree.yml
* Create index.mdx
not sure about "du / ihr" oder "sie"
* Create quicktour.mdx
* Update _toctree.yml
* Update build_documentation.yml
* Update build_pr_documentation.yml
* fix build
* Update index.mdx
* Update quicktour.mdx
* Create installation.mdx
* Update _toctree.yml
2022-08-11 09:52:27 -04:00
