LiamSwayne
12bb853ccd
[documentation] grammatical fixes in image_classification.mdx ( #24141 )
...
Update image_classification.mdx
2023-06-09 16:59:44 +01:00
Arthur
5af3a1aa48
[lamaTokenizerFast] Update documentation ( #24132 )
...
* Update documentation
* nits
2023-06-09 16:30:20 +02:00
Elliott Wang
e2972dffdd
PLAM => PaLM ( #24129 )
2023-06-09 12:32:16 +01:00
Sylvain Gugger
c3572e6bfb
Add AzureOpenAiAgent ( #24058 )
...
* Add AzureOpenAiAgent
* quality
* Update src/transformers/tools/agents.py
Co-authored-by: Lysandre Debut <lysandre.debut@reseau.eseo.fr>
---------
Co-authored-by: Lysandre Debut <lysandre.debut@reseau.eseo.fr>
2023-06-07 16:34:53 -04:00
amyeroberts
a717e0318c
Add TimmBackbone model ( #22619 )
...
* Add test_backbone for convnext
* Add TimmBackbone model
* Add check for backbone type
* Tidying up - config checks
* Update convnextv2
* Tidy up
* Fix indices & clearer comment
* Exceptions for config checks
* Correclty update config for tests
* Safer imports
* Safer safer imports
* Fix where decorators go
* Update import logic and backbone tests
* More import fixes
* Fixup
* Only import all_models if torch available
* Fix kwarg updates in from_pretrained & main rebase
* Tidy up
* Add tests for AutoBackbone
* Tidy up
* Fix import error
* Fix up
* Install nattan in doc_test_job
* Revert back to setting self._out_xxx directly
* Bug fix - out_indices mapping from out_features
* Fix tests
* Dont accept output_loading_info for Timm models
* Set out_xxx and don't remap
* Use smaller checkpoint for test
* Don't remap timm indices - check out_indices based on stage names
* Skip test as it's n/a
* Apply suggestions from code review
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* Cleaner imports / spelling is hard
---------
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
2023-06-06 17:11:30 +01:00
Wonhyeong Seo
7da3ce04a6
🌐 [i18n-KO] Translated bertology.mdx to Korean ( #23968 )
...
* docs: ko: `bertology.mdx`
* feat: nmt draft
* fix: manual edits
* fix: resolve suggestions
Co-authored-by: Hyeonseo Yun <0525yhs@gmail.com>
---------
Co-authored-by: Hyeonseo Yun <0525yhs@gmail.com>
2023-06-06 09:08:45 -04:00
Wonhyeong Seo
c938597657
🌐 [i18n-KO] Translated language-modeling.mdx ( #23969 )
...
* docs: ko: `language_modeling.mdx`
* feat: nmt draft
* fix: manual edits
* fix: add inline toc
* fix: typo in toc_tree.yml
* fix: resolve suggestions
Co-authored-by: Sohyun Sim <96299403+sim-so@users.noreply.github.com>
---------
Co-authored-by: Sohyun Sim <96299403+sim-so@users.noreply.github.com>
2023-06-06 09:08:26 -04:00
Hyeonseo Yun
649ffbf575
🌐 [i18n-KO] Translated tasks_explained.mdx to Korean ( #23844 )
...
* docs: ko: tasks_explained.mdx
* feat: nmt and manual edit `tasks_explained.mdx`
* revised: resolve suggestions task_explained.mdx
* fixed: added draft of reference docs
Co-Authored-By: Kihoon Son <75935546+KIHOON71@users.noreply.github.com>
Co-Authored-By: Nayeon Han <nayeon2.han@gmail.com>
* revised: resolve suggestions(voca, spell check) task_explained.mdx
Co-Authored-By: Sohyun Sim <96299403+sim-so@users.noreply.github.com>
* revised: remove duplicate sentence in task_explained.mdx
* fixed: remove draft of reference docs
- I think it will be confusing in the translation process.
- This issue is included in #23971 .
---------
Co-authored-by: Kihoon Son <75935546+KIHOON71@users.noreply.github.com>
Co-authored-by: Nayeon Han <nayeon2.han@gmail.com>
Co-authored-by: Sohyun Sim <96299403+sim-so@users.noreply.github.com>
2023-06-05 12:02:03 -04:00
Eli Simhayev
bacaab1629
Added time-series blogs to the models ( #23857 )
...
* added blogs to docs
* removed new-line
2023-06-02 12:32:34 -04:00
Matt
167a0d8f87
Add an option to reduce compile() console spam ( #23938 )
...
* Add an option to reduce compile() console spam
* Add annotations to the example scripts
* Add notes to the quicktour docs as well
* minor fix
2023-06-02 15:28:52 +01:00
Kihoon Son
2fdba73a99
🌐 [i18n-KO] Translated object_detection.mdx to Korean ( #23164 )
...
* translated object_detection.mdx
Co-Authored-By: Hyeonseo Yun <0525_hhgus@naver.com>
Co-Authored-By: Nayeon Han <nayeon2.han@gmail.com>
Co-Authored-By: simso <3035487+simso@users.noreply.github.com>
Co-Authored-By: Gabriel Yang <gabrielwithhappy@gmail.com>
Co-Authored-By: Wonhyeong Seo <wonhseo@kakao.com>
Co-Authored-By: Jungnerd <46880056+jungnerd@users.noreply.github.com>
* Apply suggestions from code review
Co-authored-by: Hyeonseo Yun <0525yhs@gmail.com>
Co-authored-by: Wonhyeong Seo <wonhseo@kakao.com>
Co-authored-by: Sohyun Sim <96299403+sim-so@users.noreply.github.com>
---------
Co-authored-by: Hyeonseo Yun <0525_hhgus@naver.com>
Co-authored-by: Nayeon Han <nayeon2.han@gmail.com>
Co-authored-by: simso <3035487+simso@users.noreply.github.com>
Co-authored-by: Gabriel Yang <gabrielwithhappy@gmail.com>
Co-authored-by: Wonhyeong Seo <wonhseo@kakao.com>
Co-authored-by: Jungnerd <46880056+jungnerd@users.noreply.github.com>
Co-authored-by: Hyeonseo Yun <0525yhs@gmail.com>
Co-authored-by: Sohyun Sim <96299403+sim-so@users.noreply.github.com>
2023-06-02 07:43:55 -04:00
Patrick von Platen
dcb5e18c9e
add new mms functions to doc ( #23954 )
2023-06-02 11:35:52 +01:00
Shehan Munasinghe
07c54413ac
Add MobileViTv2 ( #22820 )
...
* generated code from add-new-model-like
* Add code for modeling, config, and weight conversion
* add tests for image-classification, update modeling and config
* add code, tests for semantic-segmentation
* make style, make quality, make fix-copies
* make fix-copies
* Update modeling_mobilevitv2.py
fix bugs
* Update _toctree.yml
* update modeling, config
fix bugs
* Edit docs - fix bug MobileViTv2v2 -> MobileViTv2
* Update mobilevitv2.mdx
* update docstrings
* Update configuration_mobilevitv2.py
make style
* Update convert_mlcvnets_to_pytorch.py
remove unused options
* Update convert_mlcvnets_to_pytorch.py
make style
* Add suggestions from code review
Co-Authored-By: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* make style, make quality
* Add suggestions from code review
Co-Authored-By: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Add suggestions from code review
Remove MobileViTv2ImageProcessor
Co-Authored-By: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* make style
* Add suggestions from code review
Rename MobileViTv2 -> MobileViTV2
Co-Authored-By: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Add suggestions from code review
Co-Authored-By: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Update modeling_mobilevitv2.py
make style
* Update serialization.mdx
* Update modeling_mobilevitv2.py
---------
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
2023-06-02 10:37:02 +01:00
Patrick von Platen
5dfd407b37
[MMS] Scaling Speech Technology to 1,000+ Languages | Add attention adapter to Wav2Vec2 ( #23813 )
...
* add fine-tuned with adapter layer
* Add set_target_lang to tokenizer
* Implement load adapter
* add tests
* make style
* Apply suggestions from code review
* Update src/transformers/models/wav2vec2/tokenization_wav2vec2.py
* make fix-copies
* Apply suggestions from code review
* make fix-copies
* make style again
* mkae style again
* fix doc string
* Update tests/models/wav2vec2/test_tokenization_wav2vec2.py
* Apply suggestions from code review
* fix
* Correct wav2vec2 adapter
* mkae style
* Update src/transformers/models/wav2vec2/modeling_wav2vec2.py
Co-authored-by: Sanchit Gandhi <93869735+sanchit-gandhi@users.noreply.github.com>
* add more nice docs
* finish
* finish
* Apply suggestions from code review
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Apply suggestions from code review
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Apply suggestions from code review
* all finish
---------
Co-authored-by: Sanchit Gandhi <93869735+sanchit-gandhi@users.noreply.github.com>
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
2023-06-02 10:30:24 +01:00
Marc Sun
e03a9cc0cd
Modify device_map behavior when loading a model using from_pretrained ( #23922 )
...
* Modify device map behavior for 4/8 bits model
* Remove device_map arg for training 4/8 bit model
* Remove index
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* Add Exceptions
* Modify comment
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* Fix formatting
* Get current device with accelerate
* Revert "Get current device with accelerate"
This reverts commit 46f0079910
2023-06-01 13:21:22 -04:00
Brendon Soong
d1fa349e78
#23675 Registering Malay language ( #23689 )
...
* #23675 Registering Malay language
* removing untranslated files
* some translate
* more updates to toctree
* inc index
* additional translations for toctree
* translations of more sections
* removing untranslated file
* translated index.mdx to malay
2023-06-01 13:17:27 -04:00
Denisa Roberts
88f50a1e89
Add TensorFlow implementation of EfficientFormer ( #22620 )
...
* Add tf code for efficientformer
* Fix return dict bug - return last hidden state after last stage
* Fix corresponding return dict bug
* Override test tol
* Change default values of training to False
* Set training to default False X3
* Rm axis from ln
* Set init in dense projection
* Rm debug stuff
* Make style; all tests pass.
* Modify year to 2023
* Fix attention biases codes
* Update the shape list logic
* Add a batch norm eps config
* Remove extract comments in test files
* Add conditional attn and hidden states return for serving output
* Change channel dim checking logic
* Add exception for withteacher model in training mode
* Revert layer count for now
* Add layer count for conditional layer naming
* Transpose for conv happens only in main layer
* Make tests smaller
* Make style
* Update doc
* Rm from_pt
* Change to actual expect image class label
* Remove stray print in tests
* Update image processor test
* Remove the old serving output logic
* Make style
* Make style
* Complete test
2023-05-31 10:43:12 +01:00
Sohyun Sim
015829e6c4
🌐 [i18n-KO] Translated pad_truncation.mdx to Korean ( #23823 )
...
* docs: ko: pad_truncation.mdx
* feat: manual draft
* fix: resolve suggestions
Co-authored-by: Hyeonseo Yun <0525yhs@gmail.com>
---------
Co-authored-by: Hyeonseo Yun <0525yhs@gmail.com>
2023-05-31 10:23:59 +02:00
peridotml
62ba64b90a
Adds a FlyteCallback ( #23759 )
...
* initial flyte callback
* lint
* logs should still be saved to Flyte even if pandas isn't install (unlikely)
* cr - flyte team
* add docs for Flytecallback
* fix doc string - cr sgugger
* Apply suggestions from code review
cr - sgugger fix doc strings
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
---------
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
2023-05-30 10:08:07 -04:00
Hyeonseo Yun
867316670a
🌐 [i18n-KO] Translated troubleshooting.mdx to Korean ( #23166 )
...
* docs: ko: troubleshooting.mdx
* revised: fix _toctree.yml #23112
* feat: nmt draft `troubleshooting.mdx`
* fix: manual edits `troubleshooting.mdx`
* revised: resolve suggestions troubleshooting.mdx
Co-authored-by: Sohyun Sim <96299403+sim-so@users.noreply.github.com>
---------
Co-authored-by: Sohyun Sim <96299403+sim-so@users.noreply.github.com>
2023-05-30 09:49:47 -04:00
Kihoon Son
192aa04783
[i18n-KO] Translated video_classification.mdx to Korean ( #23026 )
...
* task/video_classification translated
Co-Authored-By: Hyeonseo Yun <0525_hhgus@naver.com>
Co-Authored-By: Gabriel Yang <gabrielwithhappy@gmail.com>
Co-Authored-By: Sohyun Sim <96299403+sim-so@users.noreply.github.com>
Co-Authored-By: Nayeon Han <nayeon2.han@gmail.com>
Co-Authored-By: Wonhyeong Seo <wonhseo@kakao.com>
Co-Authored-By: Jungnerd <46880056+jungnerd@users.noreply.github.com>
* Update docs/source/ko/tasks/video_classification.mdx
Co-authored-by: Jungnerd <46880056+jungnerd@users.noreply.github.com>
* Update docs/source/ko/tasks/video_classification.mdx
Co-authored-by: Jungnerd <46880056+jungnerd@users.noreply.github.com>
* Update docs/source/ko/tasks/video_classification.mdx
Co-authored-by: Jungnerd <46880056+jungnerd@users.noreply.github.com>
* Update docs/source/ko/tasks/video_classification.mdx
Co-authored-by: Jungnerd <46880056+jungnerd@users.noreply.github.com>
* Update docs/source/ko/tasks/video_classification.mdx
Co-authored-by: Jungnerd <46880056+jungnerd@users.noreply.github.com>
* Update docs/source/ko/tasks/video_classification.mdx
Co-authored-by: Jungnerd <46880056+jungnerd@users.noreply.github.com>
* Update docs/source/ko/tasks/video_classification.mdx
Co-authored-by: Jungnerd <46880056+jungnerd@users.noreply.github.com>
* Update docs/source/ko/tasks/video_classification.mdx
Co-authored-by: Jungnerd <46880056+jungnerd@users.noreply.github.com>
* Update docs/source/ko/tasks/video_classification.mdx
Co-authored-by: Sohyun Sim <96299403+sim-so@users.noreply.github.com>
* Update docs/source/ko/tasks/video_classification.mdx
Co-authored-by: Sohyun Sim <96299403+sim-so@users.noreply.github.com>
* Apply suggestions from code review
Co-authored-by: Sohyun Sim <96299403+sim-so@users.noreply.github.com>
Co-authored-by: Hyeonseo Yun <0525yhs@gmail.com>
Co-authored-by: Jungnerd <46880056+jungnerd@users.noreply.github.com>
Co-authored-by: Gabriel Yang <gabrielwithhappy@gmail.com>
* Update video_classification.mdx
* Update _toctree.yml
* Update _toctree.yml
* Update _toctree.yml
* Update _toctree.yml
---------
Co-authored-by: Hyeonseo Yun <0525_hhgus@naver.com>
Co-authored-by: Gabriel Yang <gabrielwithhappy@gmail.com>
Co-authored-by: Sohyun Sim <96299403+sim-so@users.noreply.github.com>
Co-authored-by: Nayeon Han <nayeon2.han@gmail.com>
Co-authored-by: Wonhyeong Seo <wonhseo@kakao.com>
Co-authored-by: Jungnerd <46880056+jungnerd@users.noreply.github.com>
Co-authored-by: Hyeonseo Yun <0525yhs@gmail.com>
2023-05-30 09:28:44 -04:00
Kihoon Son
a077f710f3
🌐 [i18n-KO] Translated fast_tokenizers.mdx to Korean ( #22956 )
...
* docs: ko: fast_tokenizer.mdx
content - translated
Co-Authored-By: Gabriel Yang <gabrielwithhappy@gmail.com>
Co-Authored-By: Nayeon Han <nayeon2.han@gmail.com>
Co-Authored-By: Hyeonseo Yun <0525_hhgus@naver.com>
Co-Authored-By: Sohyun Sim <96299403+sim-so@users.noreply.github.com>
Co-Authored-By: Jungnerd <46880056+jungnerd@users.noreply.github.com>
Co-Authored-By: Wonhyeong Seo <wonhseo@kakao.com>
* Update docs/source/ko/fast_tokenizers.mdx
Co-authored-by: Sohyun Sim <96299403+sim-so@users.noreply.github.com>
* Update docs/source/ko/fast_tokenizers.mdx
Co-authored-by: Sohyun Sim <96299403+sim-so@users.noreply.github.com>
* Update docs/source/ko/fast_tokenizers.mdx
Co-authored-by: Sohyun Sim <96299403+sim-so@users.noreply.github.com>
* Update docs/source/ko/fast_tokenizers.mdx
Co-authored-by: Sohyun Sim <96299403+sim-so@users.noreply.github.com>
* Update docs/source/ko/fast_tokenizers.mdx
Co-authored-by: Sohyun Sim <96299403+sim-so@users.noreply.github.com>
* Update docs/source/ko/fast_tokenizers.mdx
Co-authored-by: Sohyun Sim <96299403+sim-so@users.noreply.github.com>
* Update docs/source/ko/fast_tokenizers.mdx
Co-authored-by: Hyeonseo Yun <0525yhs@gmail.com>
* Update fast_tokenizers.mdx
* Update fast_tokenizers.mdx
* Update fast_tokenizers.mdx
* Update fast_tokenizers.mdx
* Update _toctree.yml
---------
Co-authored-by: Gabriel Yang <gabrielwithhappy@gmail.com>
Co-authored-by: Nayeon Han <nayeon2.han@gmail.com>
Co-authored-by: Hyeonseo Yun <0525_hhgus@naver.com>
Co-authored-by: Sohyun Sim <96299403+sim-so@users.noreply.github.com>
Co-authored-by: Jungnerd <46880056+jungnerd@users.noreply.github.com>
Co-authored-by: Wonhyeong Seo <wonhseo@kakao.com>
Co-authored-by: Hyeonseo Yun <0525yhs@gmail.com>
2023-05-30 09:27:40 -04:00
Eli Simhayev
4b6a5a7caa
[Time-Series] Autoformer model ( #21891 )
...
* ran `transformers-cli add-new-model-like`
* added `AutoformerLayernorm` and `AutoformerSeriesDecomposition`
* added `decomposition_layer` in `init` and `moving_avg` to config
* added `AutoformerAutoCorrelation` to encoder & decoder
* removed caninical self attention `AutoformerAttention`
* added arguments in config and model tester. Init works! 😁
* WIP autoformer attention with autocorrlation
* fixed `attn_weights` size
* wip time_delay_agg_training
* fixing sizes and debug time_delay_agg_training
* aggregation in training works! 😁
* `top_k_delays` -> `top_k_delays_index` and added `contiguous()`
* wip time_delay_agg_inference
* finish time_delay_agg_inference 😎
* added resize to autocorrelation
* bug fix: added the length of the output signal to `irfft`
* `attention_mask = None` in the decoder
* fixed test: changed attention expected size, `test_attention_outputs` works!
* removed unnecessary code
* apply AutoformerLayernorm in final norm in enc & dec
* added series decomposition to the encoder
* added series decomp to decoder, with inputs
* added trend todos
* added autoformer to README
* added to index
* added autoformer.mdx
* remove scaling and init attention_mask in the decoder
* make style
* fix copies
* make fix-copies
* inital fix-copies
* fix from https://github.com/huggingface/transformers/pull/22076
* make style
* fix class names
* added trend
* added d_model and projection layers
* added `trend_projection` source, and decomp layer init
* added trend & seasonal init for decoder input
* AutoformerModel cannot be copied as it has the decomp layer too
* encoder can be copied from time series transformer
* fixed generation and made distrb. out more robust
* use context window to calculate decomposition
* use the context_window for decomposition
* use output_params helper
* clean up AutoformerAttention
* subsequences_length off by 1
* make fix copies
* fix test
* added init for nn.Conv1d
* fix IGNORE_NON_TESTED
* added model_doc
* fix ruff
* ignore tests
* remove dup
* fix SPECIAL_CASES_TO_ALLOW
* do not copy due to conv1d weight init
* remove unused imports
* added short summary
* added label_length and made the model non-autoregressive
* added params docs
* better doc for `factor`
* fix tests
* renamed `moving_avg` to `moving_average`
* renamed `factor` to `autocorrelation_factor`
* make style
* Update src/transformers/models/autoformer/configuration_autoformer.py
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
* Update src/transformers/models/autoformer/configuration_autoformer.py
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
* fix configurations
* fix integration tests
* Update src/transformers/models/autoformer/configuration_autoformer.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* fixing `lags_sequence` doc
* Revert "fixing `lags_sequence` doc"
This reverts commit 21e34911e3
2023-05-30 10:23:32 +02:00
amitportnoy
d61d747627
Update trainer.mdx class_weights example ( #23787 )
...
class_weights tensor should follow model's device
2023-05-26 08:36:33 -04:00
Arthur
8d28dba35d
[OPT] Doc nit, using fast is fine ( #23789 )
...
small doc nit
2023-05-26 14:30:32 +02:00
Sylvain Gugger
9850e6ddab
Enable prompts on the Hub ( #23662 )
...
* Enable prompts on the Hub
* Update src/transformers/tools/prompts.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Address review comments
---------
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
2023-05-24 16:09:13 -04:00
Jungnerd
50a56bedb6
fix: delete duplicate sentences in document_question_answering.mdx ( #23735 )
...
fix: delete duplicate sentence
2023-05-24 11:20:50 -04:00
Maria Khalusova
2eaaf17a0b
Export to ONNX doc refocused on using optimum, added tflite ( #23434 )
...
* doc refocused on using optimum, tflite
* minor updates to fix checks
* Apply suggestions from code review
Co-authored-by: regisss <15324346+regisss@users.noreply.github.com>
* TFLite to separate page, added links
* Removed the onnx list builder
* make style
* Update docs/source/en/serialization.mdx
Co-authored-by: regisss <15324346+regisss@users.noreply.github.com>
---------
Co-authored-by: regisss <15324346+regisss@users.noreply.github.com>
2023-05-24 08:13:23 -04:00
Tim Dettmers
9d73b92269
4-bit QLoRA via bitsandbytes (4-bit base model + LoRA) ( #23479 )
...
* Added lion and paged optimizers and made original tests pass.
* Added tests for paged and lion optimizers.
* Added and fixed optimizer tests.
* Style and quality checks.
* Initial draft. Some tests fail.
* Fixed dtype bug.
* Fixed bug caused by torch_dtype='auto'.
* All test green for 8-bit and 4-bit layers.
* Added fix for fp32 layer norms and bf16 compute in LLaMA.
* Initial draft. Some tests fail.
* Fixed dtype bug.
* Fixed bug caused by torch_dtype='auto'.
* All test green for 8-bit and 4-bit layers.
* Added lion and paged optimizers and made original tests pass.
* Added tests for paged and lion optimizers.
* Added and fixed optimizer tests.
* Style and quality checks.
* Fixing issues for PR #23479 .
* Added fix for fp32 layer norms and bf16 compute in LLaMA.
* Reverted variable name change.
* Initial draft. Some tests fail.
* Fixed dtype bug.
* Fixed bug caused by torch_dtype='auto'.
* All test green for 8-bit and 4-bit layers.
* Added lion and paged optimizers and made original tests pass.
* Added tests for paged and lion optimizers.
* Added and fixed optimizer tests.
* Style and quality checks.
* Added missing tests.
* Fixup changes.
* Added fixup changes.
* Missed some variables to rename.
* revert trainer tests
* revert test trainer
* another revert
* fix tests and safety checkers
* protect import
* simplify a bit
* Update src/transformers/trainer.py
* few fixes
* add warning
* replace with `load_in_kbit = load_in_4bit or load_in_8bit`
* fix test
* fix tests
* this time fix tests
* safety checker
* add docs
* revert torch_dtype
* Apply suggestions from code review
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* multiple fixes
* update docs
* version checks and multiple fixes
* replace `is_loaded_in_kbit`
* replace `load_in_kbit`
* change methods names
* better checks
* oops
* oops
* address final comments
---------
Co-authored-by: younesbelkada <younesbelkada@gmail.com>
Co-authored-by: Younes Belkada <49240599+younesbelkada@users.noreply.github.com>
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
2023-05-24 12:52:45 +02:00
Nayeon Han
1fe1e3caa4
🌐 [i18n-KO] Translated tasks/monocular_depth_estimation.mdx to Korean ( #23621 )
...
docs: ko: `tasks/monocular_depth_estimation`
Co-authored-by: Hyeonseo Yun <0525yhs@gmail.com>
Co-authored-by: Sohyun Sim <96299403+sim-so@users.noreply.github.com>
Co-authored-by: Gabriel Yang <gabrielwithhappy@gmail.com>
Co-authored-by: Wonhyeong Seo <wonhseo@kakao.com>
Co-authored-by: Jungnerd <46880056+jungnerd@users.noreply.github.com>
2023-05-23 15:54:39 +02:00
Joshua Lochner
12ec7f0c20
Remove erroneous img closing tag ( #23646 )
...
See https://github.com/huggingface/transformers/pull/23625
2023-05-22 09:28:26 -04:00
Matt
1c460a5273
TF port of the Segment Anything Model (SAM) ( #22970 )
...
* First commit
* Add auto-translation with GPT-4
* make fixup
* Add a functional layernorm for TF
* Add all the auxiliary imports etc.
* Add the extra processor and tests
* rebase to main
* Add all the needed fixes to the GPT code
* make fixup
* Make convolutions channels-last so they run on CPU
* make fixup
* Fix final issues
* Fix other models affected by test change
* Clarify comment on the sparse_prompt_embeddings check
* Refactor functional_layernorm, use shape_list in place of .shape in some places
* Remove deprecated torch-alike code
* Update tests/models/sam/test_modeling_tf_sam.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Update tests/models/sam/test_modeling_tf_sam.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Refactor processor with common methods and separated private methods
* make fixup
* Quietly delete the file that didn't do anything (sorry Sylvain)
* Refactor the processor tests into one file
* make fixup
* Clean up some unnecessary indirection
* Fix TF mask postprocessing
* Add more processor equivalence tests
* Refactor generate_crop_boxes to use framework-neutral np code
* Make the serving output correctly conditional
* Fix error message line length
* Use dict keys rather than indices internally in both TF and PT SAM call/forward
* Return dicts internally in the call/forward methods
* Revert changes to common tests and just override check_pt_tf_outputs
* Revert changes to other model tests
* Clarify comments for functional layernorm
* Add missing transpose from PT code
* Removed unused copied from in PT code
* Remove overrides for tests that don't exist in TF
* Fix transpose and update tests for PT and TF to check pred_masks
* Add training flag
* Update tests to use TF checkpoints
* Update index.mdx
* Add missing cross-test decorator
* Remove optional extra asterisks
* Revert return_dict changes in PT code
* Update src/transformers/models/sam/modeling_tf_sam.py
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* Remove None return annotations on init methods
* Update tests/models/sam/test_processor_sam.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Fix input_boxes shapes
* make fixup
---------
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
2023-05-19 14:14:13 +01:00
Julien Chaumond
3cf01b2060
README: Fix affiliation for MEGA ( #23394 )
...
* README: Fix affiliation for MEGA
* Fix quality
---------
Co-authored-by: Lysandre <lysandre@huggingface.co>
2023-05-19 11:03:07 +02:00
Sylvain Gugger
cf43200861
Add local agent ( #23438 )
...
* Add local agent
* Document LocalAgent
2023-05-18 11:09:55 -04:00
Joao Gante
db13634183
TF: GPT2 with native embedding layers ( #23436 )
2023-05-18 14:46:40 +01:00
Nayeon Han
8cfae44093
🌐 [i18n-KO] Translated tasks/zero_shot_object_detection.mdx to Korean ( #23430 )
...
docs: ko: zero_shot_object_detection
2023-05-18 08:52:17 -04:00
Joao Gante
5b1ad0eb73
Docs: add link to assisted generation blog post ( #23397 )
2023-05-16 18:54:34 +01:00
Sohyun Sim
728c5e82cc
🌐 [i18n-KO] Translated asr.mdx to Korean ( #23106 )
...
* docs: ko: task/asr.mdx
* feat: manual draft
* fix: resolve suggestions
Co-authored-by: Hyeonseo Yun <0525yhs@gmail.com>
---------
Co-authored-by: Hyeonseo Yun <0525yhs@gmail.com>
2023-05-16 09:22:56 -04:00
Yih-Dar
21741e8c7e
Update test_batched_inference_image_captioning_conditioned ( #23391 )
...
* fix
* fix
* fix test + add more docs
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
Co-authored-by: younesbelkada <younesbelkada@gmail.com>
2023-05-16 14:49:24 +02:00
richardachen
65b885027a
Typo suggestion ( #23360 )
...
Update graphormer.mdx
Typo suggestion
2023-05-15 12:04:16 +01:00
Shehan Munasinghe
c045249049
Add swiftformer ( #22686 )
...
* Commit the automatically generated code
using add-new-model-like
* Update description at swiftformer.mdx file
* remove autogenerated code for MaskedImageModeling
* update weight conversion scripts
* Update modeling_swiftformer.py
* update configuration_swiftformer.py
* Update test_modeling_swiftformer.py
* update modeling code - remove einops dependency
* Update _toctree.yml
* update modeling code - remove copied from comments
* update docs
* Revert "update docs"
This reverts commit c2e05e2998
2023-05-12 11:52:31 +01:00
Freddy Boulton
662751b4e2
Fix typo in gradio-tools docs ( #23305 )
...
Fix typo
2023-05-11 14:31:28 -04:00
Sylvain Gugger
f76fb3aeea
Fix broken links in the agent docs ( #23297 )
2023-05-11 14:26:19 -04:00
Lysandre Debut
71b19ee251
Agents extras ( #23301 )
...
* Agents extras
* Add to docs
2023-05-11 14:25:51 -04:00
Mishig
436dc779a5
Update transformers_agents.mdx ( #23289 )
...
Make `huggingface-tools` to [`huggingface-tools`](https://huggingface.co/huggingface-tools )
2023-05-11 08:54:02 -04:00
Mishig
125516977d
Update custom_tools.mdx: fix link ( #23292 )
...
Wrong parantheses
2023-05-11 08:50:04 -04:00
Yih-Dar
9088fcae82
Bring back the PR Refactor doctests + add CI to main ( #23271 )
...
* Revert "Revert "[Doctests] Refactor doctests + add CI" (#23245 )"
This reverts commit 69ee46243c
2023-05-10 22:00:48 +02:00
Sylvain Gugger
eb5b5ce641
Render custom tool docs a bit better ( #23269 )
...
* Try on a couple of blocks to see
* Build the doc please
* Build the doc please
* Build the doc please
* add more
* Finish with all
* Style
2023-05-10 11:58:20 -04:00
Sylvain Gugger
f93509b114
Refine documentation for Tools ( #23266 )
...
* refine documentation for Tools
* + one bugfix
2023-05-10 11:03:53 -04:00
Patrick von Platen
996f127a90
Improve Docs of Custom Tools and Agents ( #23255 )
...
* Improve docs
* correct tip format
* Apply suggestions from code review
* Apply suggestions from code review
* Apply suggestions from code review
* Correct grammer & spelling
* Improve code style
* make style ruff
* make style final
2023-05-10 08:55:26 -04:00
