Orr Zohar
4397dfcb71
SmolVLM2 ( #36126 )
...
* smolvlm init
* updates
* fixing bugs
* minimal run, no checks
* minimal run, no checks
* passing first check + adding url support
* updating video dataloading logic
* fixing image logic
* trying modular, but fails
* modular is working, changing processor to match PR comments and general transformers logic
* fixing kwargs
* offloading video loading logic to image_util
* fixing circleci code formatting errors
* fixing circleci code formatting errors
* fixing circleci code formatting errors
* fixing circleci code formatting errors
* fixing circleci code formatting errors
* fixing circleci code formatting errors
* fixing circleci code formatting errors
* fixing circleci code formatting errors
* fixing circleci code formatting errors
* fixing circleci code formatting errors
* fixing circleci code formatting errors
* fixing circleci code formatting errors
* fixing circleci code formatting errors
* fixing circleci code formatting errors
* update
* add idefics3-based tests
* add keyword to all
* add PreTrainedModel
* updateing video loading logic
* working inference
* updates for PR comments
* updates for PR comments
* moving SmolVLMPretrainedModel higher to fix import error
* CI test pass
* CI test pass
* removing lambda
* CI test pass
* CI test pass
* CI test pass
* CI test pass
* CI test pass
* CI test pass
* processor tests
* add example in docs
* typo
* fix copies
* skip compile tests - sdpa for VisionTransformer
* fix init
* raise import error for num2words
* update doc for FA2
* more doc fix
* CI
* updates for PR comments
* Update docs/source/en/model_doc/smolvlm.md
Co-authored-by: Pedro Cuenca <pedro@huggingface.co>
* Update docs/source/en/model_doc/smolvlm.md
Co-authored-by: Pedro Cuenca <pedro@huggingface.co>
* Update docs/source/en/model_doc/smolvlm.md
Co-authored-by: Joshua Lochner <admin@xenova.com>
* Update docs/source/en/model_doc/smolvlm.md
Co-authored-by: Pedro Cuenca <pedro@huggingface.co>
* Update docs/source/en/model_doc/smolvlm.md
Co-authored-by: Pedro Cuenca <pedro@huggingface.co>
* fixing processor -- tokenizer not defined properly, (gpt2 tokenizer), and does not have the attributes of fake image token, etc
* adding smolvlm to VQA models
* removing vqa auto class
* Update src/transformers/models/smolvlm/processing_smolvlm.py
Co-authored-by: Joshua Lochner <admin@xenova.com>
* removing smolvlmvisiontransformer from index.md
* my bad, video processing had typos
* fixing docs
* renaming params in SmolVLMModel.inputs_merger
* removing un-needed dtype/device in model forward
* ruff for CI
* update docs
* Update docs/source/en/model_doc/smolvlm.md
Co-authored-by: Pedro Cuenca <pedro@huggingface.co>
* return cache position
* return cache position
* return cache also in modular
* needed to run modular again
* fix training tests
* push vectorized inputs merger
* format
* format
* reduce number of mappings
* addressing PR comments
* happy CI, happy me :)
* skip non-nested images
* adjust integration test for smaller GPUs
* format
* fix kwargs in chat template apply
* skip this for now
---------
Co-authored-by: raushan <raushan@huggingface.co>
Co-authored-by: Pablo <pablo.montalvo.leroux@gmail.com>
Co-authored-by: Pedro Cuenca <pedro@huggingface.co>
Co-authored-by: Joshua Lochner <admin@xenova.com>
2025-02-20 15:00:26 +01:00
kkscilife
09261ccf12
[Bugfix] fix file name of docstring in utils/check_table.py ( #36108 )
...
fix file name
Co-authored-by: kkscilife <qa-caif-cicd@pjlab.org.cn>
2025-02-10 15:48:02 +00:00
Billel Mokeddem
6c08b3b6e5
Add Falcon3 documentation ( #35307 )
...
* Add Falcon3 documentation
* Update Falcon3 documentation
* Change Falcon to Falcon3
* Update docs and run make fix-copies
* Add blog post and huggingface models links
2024-12-17 14:23:13 +01:00
Tony Wu
f33a0cebb3
Add ColPali to 🤗 transformers ( #33736 )
...
* feat: run `add-new-model-like`
* feat: add paligemma code with "copied from"
* feat: add ColPaliProcessor
* feat: add ColPaliModel
* feat: add ColPaliConfig
* feat: rename `ColPaliForConditionalGeneration` to `ColPaliModel`
* fixup modeling colpali
* fix: fix root import shortcuts
* fix: fix `modeling_auto` dict
* feat: comment out ColPali test file
* fix: fix typos from `add-new-model-like`
* feat: explicit the forward input args
* feat: move everything to `modular_colpali.py`
* fix: put back ColPaliProcesor
* feat: add auto-generated files
* fix: run `fix-copies`
* fix: remove DOCStRING constants to make modular converter work
* fix: fix typo + modular converter
* fix: add missing imports
* feat: no more errors when loading ColPaliModel
* fix: remove unused args in forward + tweak doc
* feat: rename `ColPaliModel` to `ColPaliForRetrieval`
* fix: apply `fix-copies`
* feat: add ColPaliProcessor to `modular_colpali`
* fix: run make quality + make style
* fix: remove duplicate line in configuration_auto
* feat: make ColPaliModel inehrit from PaliGemmaForConditionalGeneration
* fix: tweak and use ColPaliConfig
* feat: rename `score` to `post_process_retrieval`
* build: run modular formatter + make style
* feat: convert colpali weights + fixes
* feat: remove old weight converter file
* feat: add and validate tests
* feat: replace harcoded path to "vidore/colpali-v1.2-hf" in tests
* fix: add bfloat16 conversion in weight converter
* feat: replace pytest with unittest in modeling colpali test
* feat: add sanity check for weight conversion (doesn't work yet)
* feat: add shape sanity check in weigth converter
* feat: make ColPaliProcessor args explicit
* doc: add doc for ColPali
* fix: trying to fix output mismatch
* feat: tweaks
* fix: ColPaliModelOutput inherits from ModelOutput instead of PaliGemmaCausalLMOutputWithPast
* fix: address comments on PR
* fix: adapt tests to the Hf norm
* wip: try things
* feat: add `__call__` method to `ColPaliProcessor`
* feat: remove need for dummy image in `process_queries`
* build: run new modular converter
* fix: fix incorrect method override
* Fix tests, processing, modular, convert
* fix tokenization auto
* hotfix: manually fix processor -> fixme once convert modular is fixed
* fix: convert weights working
* feat: rename and improve convert weight script
* feat: tweaks
* fest: remove `device` input for `post_process_retrieval`
* refactor: remove unused `get_torch_device`
* Fix all tests
* docs: update ColPali model doc
* wip: fix convert weights to hf
* fix logging modular
* docs: add acknowledgements in model doc
* docs: add missing docstring to ColPaliProcessor
* docs: tweak
* docs: add doc for `ColPaliForRetrievalOutput.forward`
* feat: add modifications from colpali-engine v0.3.2 in ColPaliProcessor
* fix: fix and upload colapli hf weights
* refactor: rename `post_process_retrieval` to `score_retrieval`
* fix: fix wrong typing for `score_retrieval`
* test: add integration test for ColPali
* chore: rerun convert modular
* build: fix root imports
* Update docs/source/en/index.md
Co-authored-by: Yoni Gozlan <74535834+yonigozlan@users.noreply.github.com>
* fix: address PR comments
* wip: reduce the prediction gap in weight conversion
* docs: add comment in weight conversion script
* docs: add example for `ColPaliForRetrieval.forward`
* tests: change dataset path to the new one in hf-internal
* fix: colpali weight conversion works
* test: add fine-grained check for ColPali integration test
* fix: fix typos in convert weight script
* docs: move input docstring in a variable
* fix: remove hardcoded torch device in test
* fix: run the new modular refactor
* docs: fix python example for ColPali
* feat: add option to choose `score_retrieval`'s output dtype and device
* docs: update doc for `score_retrieval`
* feat: add `patch_size` property in ColPali model
* chore: run `make fix-copies`
* docs: update description for ColPali cookbooks
* fix: remove `ignore_index` methods
* feat: remove non-transformers specific methods
* feat: update `__init__.py` to new hf format
* fix: fix root imports in transformers
* feat: remove ColPali's inheritance from PaliGemma
* Fix CI issues
* nit remove prints
* feat: remove ColPali config and model from `modular_colpali.py`
* feat: add `ColPaliPreTrainedModel` and update modeling and configuration code
* fix: fix auto-removed imports in root `__init__.py`
* fix: various fixes
* fix: fix `_init_weight`
* temp: comment `AutoModel.from_config` for experiments
* fix: add missing `output_attentions` arg in ColPali's forward
* fix: fix `resize_token_embeddings`
* fix: make `input_ids` optional in forward
* feat: rename `projection_layer` to `embedding_proj_layer`
* wip: fix convert colpali weight script
* fix tests and convert weights from original repo
* fix unprotected import
* fix unprotected torch import
* fix style
* change vlm_backbone_config to vlm_config
* fix unprotected import in modular this time
* fix: load config from Hub + tweaks in convert weight script
* docs: move example usage from model docstring to model markdown
* docs: fix input docstring for ColPali's forward method
* fix: use `sub_configs` for ColPaliConfig
* fix: remove non-needed sanity checks in weight conversion script + tweaks
* fix: fix issue with `replace_return_docstrings` in ColPali's `forward`
* docs: update docstring for `ColPaliConfig`
* test: change model path in ColPali test
* fix: fix ColPaliConfig
* fix: fix weight conversion script
* test: fix expected weights for ColPali model
* docs: update ColPali markdown
* docs: fix minor typo in ColPaliProcessor
* Fix tests and add _no_split_modules
* add text_config to colpali config
* [run slow] colpali
* move inputs to torch_device in integration test
* skip test_model_parallelism
* docs: clarify quickstart snippet in ColPali's model card
* docs: update ColPali's model card
---------
Co-authored-by: yonigozlan <yoni.gozlan@huggingface.co>
Co-authored-by: Yoni Gozlan <74535834+yonigozlan@users.noreply.github.com>
2024-12-17 11:26:43 +01:00
Yoni Gozlan
94f18cf23c
Add OmDet-Turbo ( #31843 )
...
* Add template with add-new-model-like
* Add rough OmDetTurboEncoder and OmDetTurboDecoder
* Add working OmDetTurbo convert to hf
* Change OmDetTurbo encoder to RT-DETR encoder
* Add swin timm backbone as default, add always partition fix for swin timm
* Add labels and tasks caching
* Fix make fix-copies
* Format omdet_turbo
* fix Tokenizer tests
* Fix style and quality
* Reformat omdet_turbo
* Fix quality, style, copies
* Standardize processor kwargs
* Fix style
* Add output_hidden_states and ouput_attentions
* Add personalize multi-head attention, improve docstrings
* Add integrated test and fix copy, style, quality
* Fix unprotected import
* Cleanup comments and fix unprotected imports
* Add fix different prompts in batch (key_padding_mask)
* Add key_padding_mask to custom multi-head attention module
* Replace attention_mask by key_padding_mask
* Remove OmDetTurboModel and refactor
* Refactor processing of classes and abstract use of timm backbone
* Add testing, fix output attentions and hidden states, add cache for anchors generation
* Fix copies, style, quality
* Add documentation, conver key_padding_mask to attention_mask
* revert changes to backbone_utils
* Fic docstrings rst
* Fix unused argument in config
* Fix image link documentation
* Reorder config and cleanup
* Add tokenizer_init_kwargs in merge_kwargs of the processor
* Change AutoTokenizer to CLIPTokenizer in convert
* Fix init_weights
* Add ProcessorMixin tests, Fix convert while waiting on uniform kwargs
* change processor kwargs and make task input optional
* Fix omdet docs
* Remove unnecessary tests for processor kwargs
* Replace nested BatchEncoding output of the processor by a flattened BatchFeature
* Make modifications from Pavel review
* Add changes Amy review
* Remove unused param
* Remove normalize_before param, Modify processor call docstring
* Remove redundant decoder class, add gradient checkpointing for decoder
* Remove commented out code
* Fix inference in fp16 and add fp16 integrated test
* update omdet md doc
* Add OmdetTurboModel
* fix caching and nit
* add OmDetTurboModel to tests
* nit change repeated key test
* Improve inference speed in eager mode
* fix copies
* Fix nit
* remove OmdetTurboModel
* [run-slow] omdet_turbo
* [run-slow] omdet_turbo
* skip dataparallel test
* [run-slow] omdet_turbo
* update weights to new path
* remove unnecessary config in class
---------
Co-authored-by: Ubuntu <ubuntu@ip-172-31-91-248.ec2.internal>
2024-09-25 13:26:28 -04:00
Yunfei Chu
16ed0640be
Add Qwen2-Audio ( #32137 )
...
* add qwen2audio
* Update check_repo.py
* fix style
* fix test
* fix style
* add model size
* Qwen2AudioEncoderModel->Qwen2AudioEncoder; add copy info
* Update src/transformers/models/qwen2_audio/modeling_qwen2_audio.py
Co-authored-by: Yoach Lacombe <52246514+ylacombe@users.noreply.github.com>
* Update src/transformers/models/qwen2_audio/modeling_qwen2_audio.py
Co-authored-by: Yoach Lacombe <52246514+ylacombe@users.noreply.github.com>
* Update src/transformers/models/qwen2_audio/modeling_qwen2_audio.py
Co-authored-by: Yoach Lacombe <52246514+ylacombe@users.noreply.github.com>
* switch the attention_mask and the feature_attention_mask
* add to PRIVATE_MODELS in check_repo.py; add to MODEL_NAMES_TO_IGNORE in check_table.py
* fix initialization
* update chat_template
* fix consistency issue after copy
* add docstrings to _merge_input_ids_with_audio_features
* add copied from to prepare_inputs_for_generation
* add more details to docs
* rm comment
* add init_std
* Update src/transformers/models/qwen2_audio/modeling_qwen2_audio.py
Co-authored-by: Yoach Lacombe <52246514+ylacombe@users.noreply.github.com>
* Update src/transformers/models/qwen2_audio/modeling_qwen2_audio.py
Co-authored-by: Yoach Lacombe <52246514+ylacombe@users.noreply.github.com>
* Update src/transformers/models/qwen2_audio/modeling_qwen2_audio.py
Co-authored-by: Yoach Lacombe <52246514+ylacombe@users.noreply.github.com>
* Update src/transformers/models/qwen2_audio/modeling_qwen2_audio.py
Co-authored-by: Yoach Lacombe <52246514+ylacombe@users.noreply.github.com>
* update
* Update docs/source/en/model_doc/qwen2_audio.md
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* update tests
* rm ignore_index
* update processor
* rm ffmpeg_read
* Update tests/models/qwen2_audio/test_modeling_qwen2_audio.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Update docs/source/en/model_doc/qwen2_audio.md
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Update docs/source/en/model_doc/qwen2_audio.md
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Update docs/source/en/model_doc/qwen2_audio.md
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* update
* typo
* [run_slow] qwen2_audio
* [run_slow] qwen2_audio
* [run_slow] qwen2_audio
* fix quality
* [run_slow] qwen2_audio
* [run_slow] qwen2_audio
* [run_slow] qwen2_audio
* add official model
---------
Co-authored-by: Yoach Lacombe <52246514+ylacombe@users.noreply.github.com>
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
2024-08-08 15:47:24 +02:00
Arthur
673440d073
update ruff version ( #30932 )
...
* update ruff version
* fix research projects
* Empty
* Fix errors
---------
Co-authored-by: Lysandre <lysandre@huggingface.co>
2024-05-22 06:40:15 +02:00
Arthur
89c510d842
Add llama3 ( #30334 )
...
* nuke
* add co-author
* add co-author
* update card
* fixup and fix copies to please our ci
* nit fixup
* super small nits
* remove tokenizer_path from call to `write_model`
* always safe serialize by default
---------
Co-authored-by: pcuenca <pcuenca@users.noreply.github.com>
Co-authored-by: xenova <xenova@users.noreply.github.com>
2024-04-24 10:11:19 +02:00
Taylor Jackle Spriggs
ee3af60be0
Add support for fine-tuning CLIP-like models using contrastive-image-text example ( #29070 )
...
* add support for siglip and chinese-clip model training with contrastive-image-text example
* codebase fixups
2024-02-20 12:08:31 +00:00
NielsRogge
3b742ea84c
Add SigLIP ( #26522 )
...
* Add first draft
* Use appropriate gelu function
* More improvements
* More improvements
* More improvements
* Convert checkpoint
* More improvements
* Improve docs, remove print statements
* More improvements
* Add link
* remove unused masking function
* begin tokenizer
* do_lower_case
* debug
* set split_special_tokens=True
* Remove script
* Fix style
* Fix rebase
* Use same design as CLIP
* Add fast tokenizer
* Add SiglipTokenizer to init, remove extra_ids
* Improve conversion script
* Use smaller inputs in conversion script
* Update conversion script
* More improvements
* Add processor to conversion script
* Add tests
* Remove print statements
* Add tokenizer tests
* Fix more tests
* More improvements related to weight initialization
* More improvements
* Make more tests pass
* More improvements
* More improvements
* Add copied from
* Add canonicalize_text
* Enable fast tokenizer tests
* More improvements
* Fix most slow tokenizer tests
* Address comments
* Fix style
* Remove script
* Address some comments
* Add copied from to tests
* Add more copied from
* Add more copied from
* Add more copied from
* Remove is_flax_available
* More updates
* Address comment
* Remove SiglipTokenizerFast for now
* Add caching
* Remove umt5 test
* Add canonicalize_text inside _tokenize, thanks Arthur
* Fix image processor tests
* Skip tests which are not applicable
* Skip test_initialization
* More improvements
* Compare pixel values
* Fix doc tests, add integration test
* Add do_normalize
* Remove causal mask and leverage ignore copy
* Fix attention_mask
* Fix remaining tests
* Fix dummies
* Rename temperature and bias
* Address comments
* Add copied from to tokenizer tests
* Add SiglipVisionModel to auto mapping
* Add copied from to image processor tests
* Improve doc
* Remove SiglipVisionModel from index
* Address comments
* Improve docs
* Simplify config
* Add first draft
* Make it like mistral
* More improvements
* Fix attention_mask
* Fix output_attentions
* Add note in docs
* Convert multilingual model
* Convert large checkpoint
* Convert more checkpoints
* Add pipeline support, correct image_mean and image_std
* Use padding=max_length by default
* Make processor like llava
* Add code snippet
* Convert more checkpoints
* Set keep_punctuation_string=None as in OpenCLIP
* Set normalized=False for special tokens
* Fix doc test
* Update integration test
* Add figure
* Update organization
* Happy new year
* Use AutoModel everywhere
---------
Co-authored-by: patil-suraj <surajp815@gmail.com>
2024-01-08 18:17:16 +01:00
Younes Belkada
44b5506d29
[Llava] Add Llava to transformers ( #27662 )
...
* add model like
* logits match
* minor fixes
* fixes
* up
* up
* add todo
* llava processor
* keep the processor simple
* add conversion script
* fixup
* fix copies
* up
* add to index
* fix config + logits
* fix
* refactor
* more refactor
* more refactor
* fix copies
* add authors
* v1 tests
* add `LlavaProcessor` in init
* remove unneeded import
* up
* up
* docs
* up
* fix CI
* fix CI
* add attention mask in test
* make fixup
* remove the vision model
* that' s the dirty way to do it
* nits
* nits
* updates
* add more tests
* add input tests
* fixup
* more styling
* nits
* updates amd cleanup
* fixup the generation expected results
* fix the testing script
* some cleanup and simplification which does not work yet but almost there!
* make correct dispatch operations
* vectorize works for batch of images and text
* last todos
* nits
* update test and modeling code
* remove useless function for now
* fix few issues
* fix generation
* some nits
* add bakllava
* nits
* remove duplicated code
* finis merge
* cleanup
* missed this line
* fill the todos
* add left padding offset
* add left and rignt padding logic
* bool to properly index
* make sure
* more cleanups
* batch is fixed 😉
* add correct device for tensor creation
* fix some dtype missmatch
* ruff
* update conversion script
* Update src/transformers/__init__.py
* fa 2 support + fix conversion script
* more
* correct reshaping
* fix test dict
* fix copies by ignoring
* fix nit
* skip clip vision model
* fixup
* fixup
* LlavaForVisionText2Text -> LlavaForCausalLM
* update
* fix
* raise correct errors
* fix
* docs
* nuke for now
* nits here and there
* fixup
* fix remaining tests
* update LlavaForConditionalGeneration instead of CausalLM
* fixups
* pipeline support
* slow and piepline tests
* supports batch
* nits
* cleanup
* fix first integration tests
* add pad token where needed
* correct etsts
* fixups
* update pipeline testr
* fix quality
* nits
* revert unneeded change
* nit
* use BatchFeature
* from ...feature_extraction_utils import BatchFeature
* nits
* nits
* properly update
* more f*** nits
* fix copies
* comment
* keep slow test slow
* Update src/transformers/models/llava/processing_llava.py
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>
* add piepline example
* add pixel values in docstrign
* update pr doctest
* fix
* fix slow tests
* remove hack
* fixup
* small note
* forward contrib credits from PR25789
* forward contrib credits from original implementation and work
* add arthur
* Update src/transformers/models/llava/processing_llava.py
Co-authored-by: Lysandre Debut <hi@lysand.re>
* update docstring
* nit
* move to not doctested because of timeout issues
* fixup
* add description
* more
* fix-copies
* fix docs
* add beam search
* add more comments
* add typehints on processor
* add speedup plot
* update slow tests and docs
* push test
* push batched test
* fix batched generation with different number of images
* remove benchmark due to a bug
* fix test
* fix copies
* add gcolab demo
---------
Co-authored-by: Arthur Zucker <arthur.zucker@gmail.com>
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>
Co-authored-by: shauray8 <shauray8@users.noreply.github.com>
Co-authored-by: haotian-liu <haotian-liu@users.noreply.github.com>
Co-authored-by: Lysandre Debut <hi@lysand.re>
2023-12-07 09:30:47 +01:00
Juarez Bochi
fdd86eed3b
Add madlad-400 MT models ( #27471 )
...
* Add madlad-400 models
* Add madlad-400 to the doc table
* Update docs/source/en/model_doc/madlad-400.md
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Fill missing details in documentation
* Update docs/source/en/model_doc/madlad-400.md
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Do not doctest madlad-400
Tests are timing out.
---------
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
2023-11-28 13:19:50 +00:00
Maria Khalusova
18fbeec824
[docs] Update to scripts building index.md ( #26546 )
...
* build the table in index.md with links to the model_doc
* removed list generation on index.md
* fixed missing models
* make style
2023-10-05 10:20:41 -04:00
Maria Khalusova
546e7679e7
[docs] removed MaskFormerSwin and TimmBackbone from the table on index.md ( #26347 )
...
removed MaskFormerSwin and TimmBackbone from the table
2023-09-25 09:41:59 -04:00
Sylvain Gugger
2defb6b048
More utils doc ( #25457 )
...
* Document and clean more utils.
* More documentation and fixes
* Switch to Lysandre's token
* Address review comments
* Actually put else
2023-08-17 07:58:35 +02:00
Sylvain Gugger
640e1b6c6f
Remove tokenizers from the doc table ( #24963 )
2023-07-21 09:41:36 -04:00
Sylvain Gugger
eb849f6604
Migrate doc files to Markdown. ( #24376 )
...
* Rename index.mdx to index.md
* With saved modifs
* Address review comment
* Treat all files
* .mdx -> .md
* Remove special char
* Update utils/tests_fetcher.py
Co-authored-by: Lysandre Debut <lysandre.debut@reseau.eseo.fr>
---------
Co-authored-by: Lysandre Debut <lysandre.debut@reseau.eseo.fr>
2023-06-20 18:07:47 -04:00
Maria Khalusova
2eaaf17a0b
Export to ONNX doc refocused on using optimum, added tflite ( #23434 )
...
* doc refocused on using optimum, tflite
* minor updates to fix checks
* Apply suggestions from code review
Co-authored-by: regisss <15324346+regisss@users.noreply.github.com>
* TFLite to separate page, added links
* Removed the onnx list builder
* make style
* Update docs/source/en/serialization.mdx
Co-authored-by: regisss <15324346+regisss@users.noreply.github.com>
---------
Co-authored-by: regisss <15324346+regisss@users.noreply.github.com>
2023-05-24 08:13:23 -04:00
Connor Henderson
0f96c26de6
refactor: Make direct_transformers_import util ( #21652 )
...
* refactor: Make direct_import util
* edit direct import fn
* add docstring
* make import function specific to transformers only
* edit doc string
2023-02-16 11:32:32 -05:00
Sylvain Gugger
9d1116e995
Update deprecated load_module ( #21651 )
2023-02-15 15:57:24 -05:00
Yulv-git
95113d1365
Fix some typos. ( #17560 )
...
* Fix some typos.
Signed-off-by: Yulv-git <yulvchi@qq.com>
* Fix typo.
Signed-off-by: Yulv-git <yulvchi@qq.com>
* make fixup.
2022-07-11 05:00:13 -04:00
Sylvain Gugger
b9a768b3ff
Enable doc in Spanish ( #16518 )
...
* Reorganize doc for multilingual support
* Fix style
* Style
* Toc trees
* Adapt templates
2022-04-04 10:25:46 -04:00
Alberto Bégué
cb7ed6e083
Add Tensorflow handling of ONNX conversion ( #13831 )
...
* Add TensorFlow support for ONNX export
* Change documentation to mention conversion with Tensorflow
* Refactor export into export_pytorch and export_tensorflow
* Check model's type instead of framework installation to choose between TF and Pytorch
Co-authored-by: Lysandre Debut <lysandre@huggingface.co>
Co-authored-by: Alberto Bégué <alberto.begue@della.ai>
Co-authored-by: lewtun <lewis.c.tunstall@gmail.com>
2022-02-10 11:18:41 +01:00
lewtun
6775b211b6
Remove Longformers from ONNX-supported models ( #15273 )
2022-02-07 17:32:13 +01:00
lewtun
16f0b7d72c
Update ONNX docs ( #14904 )
...
* Remove docs for deprecated ONNX export
* Tidy up the CLI help messages
* Revamp ONNX docs
* Update auto-config table
* Use DistilBERT as example for consistency
* Wrap up first pass at ONNX docs
* Fix table check
* Add tweaks and introduction
* Add cross-ref
* Fix missing import
* Fix style
* Add permalinks to ONNX configs
* Clarify role of OrderedDict
* Update docs/source/serialization.mdx
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* Add doctest syntax to code blocks
* Remove permalinks
* Revert "Remove permalinks"
This reverts commit 099701daf0
2022-01-11 18:06:05 +01:00
Yih-Dar
0a03a86813
fix model table cell text alignment ( #14999 )
...
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2022-01-10 06:44:11 -05:00
Sylvain Gugger
207594be81
Convert rst files ( #14888 )
...
* Convert all tutorials and guides
* Convert all remaining rst to mdx
* Track and fix bad links
2021-12-22 16:14:35 -05:00
Sylvain Gugger
4df7d05a87
Doc new front ( #14590 )
...
* Convert PretrainedConfig doc to Markdown
* Use syntax
* Add necessary doc files (#14496 )
* Doc fixes (#14499 )
* Fixes for the new front
* Convert DETR file for table
* Title is needed
* Simplify a bit
* Even simpler
* Remove imports
* Fix typo in toctree (#14516 )
* Fix checkpoints badge
* Update versions.yml format (#14517 )
* Doc new front github actions (#14512 )
* Doc new front github actions
* Fix docstring
* Fix feature extraction utils import (#14515 )
* Address Julien's comments
* Push to doc-builder
* Ready for merge
* Remove old build and deploy
* Doc misc fixes (#14583 )
* Rm versions.yml from doc
* Fix converting.rst
* Rm pretrained_models from toctree
* Fix index links (#14567 )
* Fix links in README
* Localized READMEs
* Fix copy script
* Fix find doc script
* Update README_ko.md
Co-authored-by: Julien Chaumond <julien@huggingface.co>
Co-authored-by: Julien Chaumond <julien@huggingface.co>
* Adapt build command to new CLI tools (#14578 )
* Fix typo
* Fix doc interlinks (#14589 )
* Convert PretrainedConfig doc to Markdown
* Use syntax
* Rm pattern <[a-z]+(.html).*>
* Rm huggingface.co/transformers/master
* Rm .html
* Rm .html from index.mdx
* Rm .html from model_summary.rst
* Update index.mdx rm html
* Update remove .html
* Fix inner doc links
* Fix interlink in preprocssing.rst
* Update pr_checks
Co-authored-by: Sylvain Gugger <sylvain.gugger@gmail.com>
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* Convert PretrainedConfig doc to Markdown
* Use syntax
* Add necessary doc files (#14496 )
* Doc fixes (#14499 )
* Fixes for the new front
* Convert DETR file for table
* Title is needed
* Simplify a bit
* Even simpler
* Remove imports
* Fix checkpoints badge
* Fix typo in toctree (#14516 )
* Update versions.yml format (#14517 )
* Doc new front github actions (#14512 )
* Doc new front github actions
* Fix docstring
* Fix feature extraction utils import (#14515 )
* Address Julien's comments
* Push to doc-builder
* Ready for merge
* Remove old build and deploy
* Doc misc fixes (#14583 )
* Rm versions.yml from doc
* Fix converting.rst
* Rm pretrained_models from toctree
* Fix index links (#14567 )
* Fix links in README
* Localized READMEs
* Fix copy script
* Fix find doc script
* Update README_ko.md
Co-authored-by: Julien Chaumond <julien@huggingface.co>
Co-authored-by: Julien Chaumond <julien@huggingface.co>
* Adapt build command to new CLI tools (#14578 )
* Fix typo
* Fix doc interlinks (#14589 )
* Convert PretrainedConfig doc to Markdown
* Use syntax
* Rm pattern <[a-z]+(.html).*>
* Rm huggingface.co/transformers/master
* Rm .html
* Rm .html from index.mdx
* Rm .html from model_summary.rst
* Update index.mdx rm html
* Update remove .html
* Fix inner doc links
* Fix interlink in preprocssing.rst
* Update pr_checks
Co-authored-by: Sylvain Gugger <sylvain.gugger@gmail.com>
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* Styling
Co-authored-by: Mishig Davaadorj <mishig.davaadorj@coloradocollege.edu>
Co-authored-by: Lysandre Debut <lysandre@huggingface.co>
Co-authored-by: Julien Chaumond <julien@huggingface.co>
2021-12-01 14:13:02 -05:00
Sylvain Gugger
7d83655da9
Autodocument the list of ONNX-supported models ( #13884 )
2021-10-05 22:43:16 -04:00
Omar Sanseviero
c066598c23
Fix frameworks table so it's alphabetical ( #13118 )
...
* Fix frameworks table so it's alphabetical
* Update index.rst
* Don't differentiate when sorting between upper and lower case
2021-08-16 15:45:19 +02:00
Sylvain Gugger
9870093f7b
[WIP] Disentangle auto modules from other modeling files ( #13023 )
...
* Initial work
* All auto models
* All tf auto models
* All flax auto models
* Tokenizers
* Add feature extractors
* Fix typos
* Fix other typo
* Use the right config
* Remove old mapping names and update logic in AutoTokenizer
* Update check_table
* Fix copies and check_repo script
* Fix last test
* Add back name
* clean up
* Update template
* Update template
* Forgot a )
* Use alternative to fixup
* Fix TF model template
* Address review comments
* Address review comments
* Style
2021-08-06 13:12:30 +02:00
Patrick von Platen
32dbb2d954
make style ( #11442 )
2021-04-26 13:50:34 +02:00
Lysandre Debut
2ae7388eee
Check table as independent script ( #8976 )
2020-12-07 19:55:12 -05:00
