s-JoL
c2c99dc7ef
add open-llama model with ckpt ( #22795 )
...
* update Open-Llama model
* update
* update format
* update doc
* update
* update stable embedding test
* update test case
* update format
* update readme
* fix typo
* update name
* remove tokenizer and update format
* remove convert_open_llama_weights_to_hf
* update warning and doc_string
---------
Co-authored-by: songliang.bayesian <songliang.bayesian@bytedance.com>
2023-04-28 11:01:32 -04:00
Maria Khalusova
521a8ffa53
[docs] Doc TOC updates ( #23049 )
...
* first draft of toc restructure
* polishing based on feedback
2023-04-28 09:24:28 -04:00
Hyeonseo Yun
4893d919f1
🌐 [i18n-KO] Translated model_sharing.mdx to Korean ( #22991 )
...
* docs: ko: init: model_sharing.mdx
* docs: ko: trans: model_sharing.mdx
Co-Authored-By: Kihoon Son <75935546+KIHOON71@users.noreply.github.com>
Co-Authored-By: Sohyun Sim <96299403+sim-so@users.noreply.github.com>
Co-Authored-By: Gabriel Yang <gabrielwithhappy@gmail.com>
Co-Authored-By: Nayeon Han <nayeon2.han@gmail.com>
Co-Authored-By: Wonhyeong Seo <wonhseo@kakao.com>
Co-Authored-By: Jungnerd <46880056+jungnerd@users.noreply.github.com>
* docs: ko: revised: apply code reviews model_sharing.mdx
Co-authored-by: Wonhyeong Seo <wonhseo@kakao.com>
Co-authored-by: Sohyun Sim <96299403+sim-so@users.noreply.github.com>
* docs: ko: revised: apply aditional reviews model_sharing.mdx
1. Natural Expression
2. `파인 튜닝` to `미세 조정`
3. Glossary Sync
Co-Authored-By: Sohyun Sim <96299403+sim-so@users.noreply.github.com>
Co-Authored-By: Nayeon Han <nayeon2.han@gmail.com>
Co-Authored-By: Wonhyeong Seo <wonhseo@kakao.com>
* docs: ko: revised: apply aditional reviews in model_sharing.mdx
1. Spell check
2. Natural Expression
3. Sync Glossary
Co-Authored-By: Gabriel Yang <gabrielwithhappy@gmail.com>
* docs: ko: revised: `프로그래밍 방식` to `API` in model_sharing.mdx
Co-Authored-By: Wonhyeong Seo <wonhseo@kakao.com>
---------
Co-authored-by: Kihoon Son <75935546+KIHOON71@users.noreply.github.com>
Co-authored-by: Sohyun Sim <96299403+sim-so@users.noreply.github.com>
Co-authored-by: Gabriel Yang <gabrielwithhappy@gmail.com>
Co-authored-by: Nayeon Han <nayeon2.han@gmail.com>
Co-authored-by: Wonhyeong Seo <wonhseo@kakao.com>
Co-authored-by: Jungnerd <46880056+jungnerd@users.noreply.github.com>
2023-04-28 09:20:33 -04:00
Ehsan M. Kermani
a0e7332839
Fix CLAP link across all READMEs ( #23032 )
...
* Fix CLAP link across all READMEs
* Fix copy only for en
2023-04-27 18:07:02 -04:00
peter-sk
d65b14ed67
added GPTNeoForTokenClassification ( #22908 )
...
* added GPTNeoForTokenClassification
* add to top-level init
* fixup
* test
* more fixup
* add to gpt_neo.mdx
* repo consistency
* dummy copy
* fix copies
* optax >= 0.1.5 assumes jax.Array exists - which it doesn't for jax <= 0.3.6
* merge with main made this superfluous
* added classifier_dropout
* remove legacy code
* removed fmt:on/off
removed expected_outputs
* doc style fix
* classifier_dropout is always in config
---------
Co-authored-by: Prof. Peter Schneider-Kamp <jps@ordbogen.com>
2023-04-27 12:10:03 -04:00
peter-sk
614e191c4d
added GPTNeoXForTokenClassification ( #23002 )
...
* initial commit
* added GPTNeoXForTokenClassification
* typo
* doc
fixed extra comma that turned into a tuple
* unifying variable names
fixing forward call
* classifier_dropout is in config
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
---------
Co-authored-by: Prof. Peter Schneider-Kamp <jps@ordbogen.com>
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
2023-04-27 11:08:26 -04:00
Nayeon Han
e28fff18b8
🌐 [i18n-KO] Translated multilingual.mdx to Korean ( #23008 )
...
docs: ko: `multilingual.mdx`
Co-authored-by: Hyeonseo Yun <0525_hhgus@naver.com>
Co-authored-by: Gabriel Yang <gabrielwithhappy@gmail.com>
Co-authored-by: Jungnerd <46880056+jungnerd@users.noreply.github.com>
Co-authored-by: Sohyun Sim <96299403+sim-so@users.noreply.github.com>
Co-authored-by: Wonhyeong Seo <wonhseo@kakao.com>
2023-04-27 08:06:12 -04:00
fxmarty
3042c63a95
Add methods to PreTrainedModel to use PyTorch's BetterTransformer ( #21259 )
...
* fix mess
* better documentation
* typo
* fix doc
* update
* add test
* fix test
* more tests
* Update src/transformers/modeling_utils.py
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* move to utils
* Apply suggestions from code review
Co-authored-by: Michael Benayoun <mickbenayoun@gmail.com>
* nit
---------
Co-authored-by: younesbelkada <younesbelkada@gmail.com>
Co-authored-by: Younes Belkada <49240599+younesbelkada@users.noreply.github.com>
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
Co-authored-by: Michael Benayoun <mickbenayoun@gmail.com>
2023-04-27 11:03:42 +02:00
Ritik Nandwal
20ac86c6f1
Add TensorFlow Wav2Vec2 for sequence classification ( #22073 )
...
* Add initial changes for TF wav2vec2 for sequence classification
* Add suggested changes
* Add serving and serving output methods
* Add serving_output implementation and fix layer_weights
* Add fixes
* Fixed test cases
* Fixing test and adding suggested changes
2023-04-26 13:35:30 +01:00
Hyeonseo Yun
4c2b4c4c3c
🌐 [i18n-KO] Translated token_classification.mdx to Korean ( #22945 )
...
* docs: ko: init: token_classification.mdx
* docs: ko: trans: tasks/token_classification.mdx
* docs: ko: revise: apply suggestions tasks/token_classification.mdx
right vocabulary, spell check, natural expression
Co-authored-by: Sohyun Sim <96299403+sim-so@users.noreply.github.com>
* docs: ko: revise: `Hub` to `허브` in tasks/token_classification.mdx
* docs: ko: revise: `example` in tasks/token_classification.mdx
Co-Authored-By: Gabriel Yang <gabrielwithhappy@gmail.com>
Co-Authored-By: Kihoon Son <75935546+KIHOON71@users.noreply.github.com>
Co-Authored-By: Sohyun Sim <96299403+sim-so@users.noreply.github.com>
Co-Authored-By: Nayeon Han <nayeon2.han@gmail.com>
Co-Authored-By: Wonhyeong Seo <wonhseo@kakao.com>
Co-Authored-By: Jungnerd <46880056+jungnerd@users.noreply.github.com>
* docs: ko: revise: ko expression in tasks/token_classification.mdx
Co-Authored-By: Gabriel Yang <gabrielwithhappy@gmail.com>
* Revert "docs: ko: revise: ko expression in tasks/token_classification.mdx"
This reverts commit 8efe28059b
2023-04-26 07:56:14 -04:00
Sohyun Sim
6dc2474727
🌐 [i18n-KO] Translated tasks/image_captioning.mdx to Korean ( #22943 )
...
docs: ko: tasks/image_captioning.mdx
Co-authored-by: Hyeonseo Yun <0525_hhgus@naver.com>
Co-authored-by: Jungnerd <46880056+jungnerd@users.noreply.github.com>
Co-authored-by: Wonhyeong Seo <wonhseo@kakao.com>
Co-authored-by: Gabriel Yang <gabrielwithhappy@gmail.com>
Co-authored-by: Nayeon Han <nayeon2.han@gmail.com>
Co-authored-by: Kihoon Son <75935546+kihoon71@users.noreply.github.com>
Co-authored-by: Hyeonseo Yun <0525yhs@gmail.com>
2023-04-26 07:54:58 -04:00
Daniel Levenson
4e1522d65a
Fix typo in mega.mdx ( #22998 )
...
MegaConfiig -> MegaConfig
2023-04-25 17:58:45 -04:00
Wonhyeong Seo
d95045717e
🌐 [i18n-KO] Translated serialization.mdx to Korean ( #22806 )
...
docs: ko: serialization.mdx
Co-authored-by: Sohyun Sim <96299403+sim-so@users.noreply.github.com>
Co-authored-by: Hyeonseo Yun <0525yhs@gmail.com>
2023-04-25 12:38:51 -04:00
Jari Van Melckebeke
81c1910c86
fixed small typo in code example ( #22982 )
...
fixed typo in code example
fixed a really small typo in the docs of single gpu inference
2023-04-25 08:56:21 -04:00
Nayeon Han
f0f5e28f82
🌐 [i18n-KO] Fixed tasks/masked_language_modeling.mdx ( #22965 )
...
fix: docs: missing newline before code block
2023-04-25 09:59:17 +02:00
Joao Gante
e4a97f82bf
Generate: assisted generation with sample (take 2) ( #22949 )
...
* temperature controls speed
2023-04-24 19:54:55 +01:00
Gabriel Yang
7701716efc
🌐 [i18n-KO] translate create_a_model doc to Korean ( #22754 )
...
docs: ko: translates create_a_model.mdx
Co-authored-by: Nayeon Han <nayeon2.han@gmail.com>
Co-authored-by: Hyeonseo Yun <0525_hhgus@naver.com>
Co-authored-by: Sohyun Sim <96299403+sim-so@users.noreply.github.com>
Co-authored-by: Jungnerd <46880056+jungnerd@users.noreply.github.com>
Co-authored-by: Wonhyeong Seo <wonhseo@kakao.com>
2023-04-24 13:02:19 -04:00
amyeroberts
8f20e61c85
Update feature selection in to_tf_dataset ( #21935 )
...
* Update feature selection
* Check compatibility with datasets version
* Checkout from datasets main
2023-04-24 17:34:30 +01:00
Matt
345a1371d8
Fix TF example in quicktour ( #22960 )
...
* Fix TF example in quicktour
* Fix model.fit() and the dataset section too
2023-04-24 17:25:13 +01:00
Nayeon Han
d6f1da6b71
🌐 [i18n-KO] Translated run_scripts.mdx to Korean ( #22793 )
...
docs: ko: `run_scripts` to Korean
Co-authored-by: Hyeonseo Yun <0525_hhgus@naver.com>
Co-authored-by: Gabriel Yang <gabrielwithhappy@gmail.com>
Co-authored-by: Sohyun Sim <96299403+sim-so@users.noreply.github.com>
Co-authored-by: Wonhyeong Seo <wonhseo@kakao.com>
Co-authored-by: Jungnerd <46880056+jungnerd@users.noreply.github.com>
2023-04-24 10:18:20 -04:00
Sohyun Sim
84097f6d38
🌐 [i18n-KO] Translated tasks/summarization.mdx to Korean ( #22783 )
...
docs: ko: tasks/summarization.mdx
Co-authored-by: Hyeonseo Yun <0525_hhgus@naver.com>
Co-authored-by: Jungnerd <46880056+jungnerd@users.noreply.github.com>
Co-authored-by: Wonhyeong Seo <wonhseo@kakao.com>
Co-authored-by: Nayeon Han <nayeon2.han@gmail.com>
Co-authored-by: Gabriel Yang <gabrielwithhappy@gmail.com>
Co-authored-by: Kihoon Son <75935546+kihoon71@users.noreply.github.com>
2023-04-24 09:03:02 -04:00
Nayeon Han
093be36f6c
🌐 [i18n-KO] Translated tasks/masked_language_modeling.mdx to Korean ( #22838 )
...
docs: ko: `tasks/masked_language_modeling.mdx` to Korean
Co-authored-by: Hyeonseo Yun <0525_hhgus@naver.com>
Co-authored-by: Gabriel Yang <gabrielwithhappy@gmail.com>
Co-authored-by: Sohyun Sim <96299403+sim-so@users.noreply.github.com>
Co-authored-by: Wonhyeong Seo <wonhseo@kakao.com>
Co-authored-by: Jungnerd <46880056+jungnerd@users.noreply.github.com>
2023-04-24 09:02:21 -04:00
Arthur
df017c3ccc
[CLAP] Doc nits ( #22957 )
...
clap nits
2023-04-24 14:00:29 +02:00
Hyeonseo Yun
137eb8e663
[i18n-KO] Translated accelerate.mdx to Korean ( #22830 )
...
* docs: ko: init: accelerate.mdx
* docs: ko: translated: accelerate.mdx
* docs: ko: revised: natural expression accelerate.mdx
Co-Authored-By: Gabriel Yang <gabrielwithhappy@gmail.com>
* docs: ko: revised: natural expression2 accelerate.mdx
Co-authored-by: Sohyun Sim <96299403+sim-so@users.noreply.github.com>
---------
Co-authored-by: Gabriel Yang <gabrielwithhappy@gmail.com>
Co-authored-by: Sohyun Sim <96299403+sim-so@users.noreply.github.com>
2023-04-24 07:49:05 -04:00
NielsRogge
3d3204c025
Add FocalNet ( #21532 )
...
Adds FocalNet by Microsoft to transformers
---------
Co-authored-by: Niels Rogge <nielsrogge@Nielss-MacBook-Pro.local>
Co-authored-by: alaradirik <alaradirik@gmail.com>
2023-04-23 20:03:05 +03:00
Connor Henderson
b950c38565
tests: Fix flaky test for NLLB-MoE ( #22880 )
...
* add test update and docs edits
* docs edit suggestion
2023-04-21 17:09:40 +01:00
fxmarty
3d852da2db
Expose AutoModelForMaskGeneration ( #22910 )
...
* expose
* style
* add dummy object
* amazed by the quality of transformers CI
2023-04-21 10:04:45 -04:00
Arthur
f143037789
Add automatic-mask-generation pipeline for Segment Anything Model (SAM) ( #22840 )
...
* cleanup
* updates
* more refactoring
* make style
* update inits
* support other inputs in base
* update based on review
Co-authored-by: Nicolas Patry <patry.nicolas@gmail.com>
* Update tests/pipelines/test_pipelines_automatic_mask_generation.py
Co-authored-by: Nicolas Patry <patry.nicolas@protonmail.com>
* update
* fixup
* TODO x and y to refactor, _h _w refactored here
* update docstring
* more nits
* style on these
* more doc fix
* rename variables
* update
* updates
* style
* update
* fix `_mask_to_rle_pytorch`
* styling
* fix ask to rle, wrong outputs
* add device arg
* update
* more updates, fix tets
* udpate
* update docstrings
* styling
* fixup
* add notebook on the docs
* update orginal sizes
* fix docstring
* updat condition on point_per-batch
* updates tests
* fix CI test
* extend is required, append does not work!
* fixup
* fix CI tests
* whit pixels left
* address doc comments
* fix doc
* slow pipeline tests
* update auto init
* add revision
* make fixup
* update p!ipoeline tag when calling tests
* alphabeitcal order in inits
* fix copies
* last style nits
* Apply suggestions from code review
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* reformat docstring
* more reformat
* address most of the comments
* Update src/transformers/pipelines/mask_generation.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* final refactor
* Update src/transformers/models/sam/image_processing_sam.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* fixup and fix slow tests
* revert
---------
Co-authored-by: Nicolas Patry <patry.nicolas@gmail.com>
Co-authored-by: Nicolas Patry <patry.nicolas@protonmail.com>
Co-authored-by: younesbelkada <younesbelkada@gmail.com>
Co-authored-by: Younes Belkada <49240599+younesbelkada@users.noreply.github.com>
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
2023-04-20 19:27:24 +02:00
fxmarty
4cfe328bae
Fix SAM example in documentation ( #22887 )
...
fix sam example
2023-04-20 12:22:42 +02:00
Younes Belkada
2da73f6302
[SAM] Correct arxiv link ( #22886 )
...
put correct link
2023-04-20 11:23:12 +02:00
Arthur
474bf508df
Add Segment Anything Model (SAM) ( #22654 )
...
* initial commit
* keys match
* update, fix conversion
* fixes, inference working
* fix
* more fixes
* more fixes
* clean up
* more clean up
* fix copies and add convext copied layer norm
* stash
* pretty big upfate
* cleaning
* more cleaning
* fixup stuffs
* fix copies
* fix iinit
* update test removing tokenizer
* nits
* add pretrained
* more nits
* remove tracking of pipeline
* few fixes
* update san and conversion script
* fix mask decoder and prompt encoder conversion
* fixes
* small update
* fix order
* fix
* fix image embeddings
* nites
* few fixes
* fix logits
* clean up
* fixes boxes inference
* v1 AMG
* clean up
* some clean up
* multi points support
* amg working
* fixup
* clean up
* readme
* update toctree
* fix type hint
* multiple fixes
* fixup
* fixes
* updates
* updates
* more tests
* few fixes
* change to `SamForMaskGeneration`
* doc
* fixup
* fix more tests
* multiple fixes
* fix CI tests
* refactor processor
* renamings
* draft the pipeline
* refactor
* fix tests
* fix test
* few cleanings
* fix test
* edit pipelien support chunking
* udate
* add slow tests
* fix nit
* fixup
* fix nit
* current chunk pipleine
* cast boxes in fp32
* nit
* current updates
* piepleine works
* fixup
* clean up config
* fix slow tests
* fix slow tests
* clean up
* update doc and pipeline
* adds more slow tests
* fix slow tests
* cleaning
* tests pass
* add docstring
* fix copies
* clean up
* support batch of images
* style
* dummy is needed, add tests
* fix slow tests
* fix CI
* update
* adds more tests
* fixes
* fixes
* fixup
* fixes
* few fixes
* filter
* few fixes
* some refactor
* touches finales
* fix
* style
* remove pipeline files
* fixes nits
* revert pipeline changes
* fix test
* fixup
* remove automodel for automatic mask generation
* fix failing torch tests
* update mdx
* revert removal of `MODEL_FOR_AUTOMATIC_MASK_GENERATION_MAPPING`
* update sam config based on review
Co-authored-by: amyeroberts <aeroberts4444@gmail.com>
Co-authored-by: sgugger <sylvain.gugger@gmail.com>
* update low_resolution_masks -> pred_masks
inti ln with layer_norm_eps
add_decomposed_rel_pos doc
forward doc of SamForMaskGeneration
* update processor docstring
* remove image processor import empty
* update for testing
* output vision hidden states + clean recomm
also test all iou values
* fixup
* fixup
* remove unused
* Update src/transformers/models/sam/modeling_sam.py
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* Update src/transformers/models/sam/image_processing_sam.py
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* nits
* fix
* fix CI tests and slow tests
* replace with Amy's processor
* clearer docstring
* add `SamVisionNeck`
* refactor - all CI tests should pass
* fix broken import on Gcolab
* few fixes here and there
* fix another bug
* fix more bugs
* update and merge
* correct ckpt
* address comments
* add tips
* revert
* fix docstring
* replace with `SamModel`
* make fixup
* add support for bathed images and batch ed points
* make fixup this time, really
* make fixup again and again
* few fixes here and there, this should be the touche finale
* Update docs/source/en/model_doc/sam.mdx
* fixup
* correct checkpoints
* correct name
* rm unneeded file
* add notebook
---------
Co-authored-by: younesbelkada <younesbelkada@gmail.com>
Co-authored-by: amyeroberts <aeroberts4444@gmail.com>
Co-authored-by: sgugger <sylvain.gugger@gmail.com>
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
Co-authored-by: Younes Belkada <49240599+younesbelkada@users.noreply.github.com>
2023-04-19 21:01:49 +02:00
Joao Gante
78cda46f17
Generate: Add assisted generation ( #22211 )
...
* working mvp
* remove breakpoint
* fix commit
* standardize outputs
* tmp commit
* tests almost ready
* tmp commit
* skip a few models
* Add streaming; Docs and examples
* document limitations
* PR commits
* Amy PR comments
2023-04-18 17:36:56 +01:00
Gabriel Yang
42288269c3
🌐 [i18n-KO] Fix anchor links for docs auto_tutorial, training ( #22796 )
...
docs: ko: fix anchor links for docs (auto_tutorial, training)
Co-authored-by: Hyeonseo Yun <0525_hhgus@naver.com>
Co-authored-by: Sohyun Sim <96299403+sim-so@users.noreply.github.com>
Co-authored-by: Na Yeon Han <nayeon2.han@gmail.com>
Co-authored-by: Wonhyeong Seo <wonhseo@kakao.com>
Co-authored-by: Jungnerd <46880056+jungnerd@users.noreply.github.com>
2023-04-18 09:11:30 -04:00
Sylvain Gugger
dacd34568d
Mark auto models as important ( #22815 )
...
* Mark auto models as important
* Annoying file with bad line endings
2023-04-17 15:33:01 -04:00
Wonhyeong Seo
4d2c52e830
🌐 [i18n-KO] Translated tasks/translation.mdx to Korean ( #22805 )
...
docs: ko: tasks/translation.mdx
2023-04-17 11:30:17 -04:00
Jungnerd
abbc96a214
[i18n-KO] fix: docs: ko: sagemaker anchors and _toctree.yml ( #22549 )
...
fix: docs: ko: sagemaker anchors and `_toctree.yml`
Co-authored-by: Hyeonseo Yun <0525_hhgus@naver.com>
Co-authored-by: Gabriel Yang <gabrielwithhappy@gmail.com>
Co-authored-by: Sohyun Sim <96299403+sim-so@users.noreply.github.com>
Co-authored-by: Na Yeon Han <nayeon2.han@gmail.com>
Co-authored-by: Wonhyeong Seo <wonhseo@kakao.com>
2023-04-17 07:41:52 -04:00
Na Yeon Han
18c894814e
🌐 [i18n-KO] Translated custom_models.mdx to Korean ( #22534 )
...
docs: ko: translated `custom_models.mdx`
Co-authored-by: Wonhyeong Seo <wonhseo@kakao.com>
Co-authored-by: Gabriel Yang <gabrielwithhappy@gmail.com>
Co-authored-by: Jungnerd <46880056+jungnerd@users.noreply.github.com>
2023-04-17 07:39:53 -04:00
Mayank Agarwal
daf53241d6
Fix word_ids hyperlink ( #22765 )
...
* Fix word_ids hyperlink
* Add suggested fix
2023-04-14 16:18:15 +01:00
Sohyun Sim
c8df3900c8
[WIP] 🌐 [i18n-KO] Translated tutorial/proprecssing.mdx to Korean ( #22578 )
...
* add ko preprocessing
* translate preprocessing.mdx to korean
* translate preprocessing.mdx
* Update preprocessing.mdx
Fixed the line 273 as below:
또한, 특징 추출기에 `sampling_rate` 인자를 추가하여 발생할 수 있는 조용한 오류(silent errors)를 더 잘 디버깅하는 것을 권장합니다.
* translate Image part
* translated preprocess.mdx
* Update docs/source/ko/preprocessing.mdx
Co-authored-by: Wonhyeong Seo <wonhseo@kakao.com>
* Update docs/source/ko/preprocessing.mdx
Co-authored-by: Wonhyeong Seo <wonhseo@kakao.com>
* Update docs/source/ko/preprocessing.mdx
Co-authored-by: Wonhyeong Seo <wonhseo@kakao.com>
* Update docs/source/ko/preprocessing.mdx
Co-authored-by: Wonhyeong Seo <wonhseo@kakao.com>
* Update docs/source/ko/preprocessing.mdx
Co-authored-by: Wonhyeong Seo <wonhseo@kakao.com>
* Update docs/source/ko/preprocessing.mdx
Co-authored-by: Wonhyeong Seo <wonhseo@kakao.com>
* Update docs/source/ko/preprocessing.mdx
Co-authored-by: Wonhyeong Seo <wonhseo@kakao.com>
* Update docs/source/ko/preprocessing.mdx
Co-authored-by: Wonhyeong Seo <wonhseo@kakao.com>
* Update docs/source/ko/preprocessing.mdx
* Update docs/source/ko/preprocessing.mdx
* Update docs/source/ko/preprocessing.mdx
* Update docs/source/ko/preprocessing.mdx
* Update docs/source/ko/preprocessing.mdx
* Update docs/source/ko/preprocessing.mdx
* fixed translation
---------
Co-authored-by: Wonhyeong Seo <wonhseo@kakao.com>
2023-04-14 07:26:44 -04:00
Hyeonseo Yun
bfb3925fcb
🌐 [i18n-KO] Translated sequence_classification.mdx to Korean ( #22655 )
...
* docs: ko: init: tasks/sequence_classification.mdx
* docs: ko: revised: change voca in tasks/sequence_classification.mdx
* docs: ko: revised: [RE] change voca in tasks/sequence_classification.mdx
* docs: ko: revised: spell check and sentence naturally in tasks/sequence_classification.mdx
* docs: ko: revised: spell check and consistent vocabulary in tasks/sequence_classification.mdx
* docs: ko: revised: Add full stop and change voca in tasks/sequence_classification.mdx
* docs: ko: revised: sync first section templates in tasks/sequence_classification.mdx
Co-authored-by: Wonhyeong Seo <wonhseo@kakao.com>
* fix: revert use of full-stops to colons
* colons are used to emphasize the code block that follows
* @0525hhgus @wonhyeongseo docs: ko: revised: sync second section templates in tasks/sequence_classification.mdx
Co-Authored-By: Wonhyeong Seo <wonhseo@kakao.com>
* docs: ko: revised: change 'train', 'finetuning' in tasks/sequence_classification.mdx
---------
Co-authored-by: Wonhyeong Seo <wonhseo@kakao.com>
2023-04-13 21:40:36 -04:00
Joao Gante
9dfd6a4baa
Generate: handle text conditioning with multimodal encoder-decoder models ( #22748 )
2023-04-13 19:51:13 +01:00
Gabriel Yang
4def2fe969
🌐 [i18n-KO] Translated training.mdx to Korean ( #22670 )
...
translate training doc to Korean
2023-04-13 11:04:47 -04:00
NielsRogge
8eb38f638d
[Pix2struct] Simplify generation ( #22527 )
...
* Add model to doc tests
* Remove generate and replace by prepare_inputs_for_generation
* More fixes
* Remove print statements
* Update integration tests
* Fix generate
* Remove model from auto mapping
* Use auto processor
* Fix integration tests
* Fix test
* Add inference code snippet
* Remove is_encoder_decoder
* Update docs
* Remove notebook link
2023-04-13 09:01:14 -04:00
ARKA1112
d87ef00c31
Modify pipeline_tutorial.mdx ( #22726 )
...
generator(model="openai/whisper-large") always returns error. As the error says the generator expects an input, just like the .flac file above. Even the generator object has no parameters called model. While there are parameters which can be passed to generator like 'batch_size' but to pass a model i believe the the parameter has to be passed while instantiating the pipeline and not as a parameter to the instance.
I believe the correct term should be:
generator = pipeline(model="openai/whisper-large", device=0)
2023-04-12 15:20:25 +01:00
Younes Belkada
370f0ca18c
[bnb] Let's make serialization of int8 models possible ( #22177 )
...
* make serialization of int8 models possible
* make fixup
* add docs
* add ability to push to hub and save pretrained
* fixes
* more addition
* more tests
* fix issues
* change variable
* clearer message
* adapt from suggestions
* few fixes
* remove unused function
* Update src/transformers/utils/quantization_config.py
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* address last comments
* last warning
* clarify doc
* protect import
* Update src/transformers/modeling_utils.py
* Apply suggestions from code review
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
---------
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
2023-04-12 08:01:18 -04:00
pioliverse
523ca4e016
add model resources for CPMAnt (new) ( #20906 )
...
* resolve conflicts
* rebase and make style
* test
* test
* test
* rebase and make style
* rebase and make style
* tests
* tests
* rewrite some functions
* rebase and make style
* fix load_tf_weights_in_cpmant
* reformat some unrelated files
* upgrade quality
* fix some bugs & docstring
* add models and tests
* solve conflicts
* resolve conflicts
* resolve conflicts
* resolve conflicts
* resolve conflicts
* tests
* resolve conflicts
* resolve conflicts
* fix load_tf_weights_in_cpmant
* reformat some unrelated files
* upgrade quality
* fix some bugs & docstring
* save resolution
* make style
* delete redefinition code
* reformat function
* reformat
* resolve conflicts
* resolve conflicts
* resolve conflicts
* resolve conflicts
* resolve conflicts
* tests
* resolve conflicts
* resolve conflicts
* fix load_tf_weights_in_cpmant
* reformat some unrelated files
* upgrade quality
* resolve conflicts
* resolve conflicts
* resolve conflicts
* resolve conflicts
* resolve conflicts
* fix load_tf_weights_in_cpmant
* reformat some unrelated files
* upgrade quality
* resolve conflicts
* make style
* fix bugs and refactor
* modify docstrings and make style
* unify import format in __init__.py
* fix import-altclp bug
* fix copies to update index.md
* fix unused config parameters
* fix unused config parameters
* fix unused config parameters
* update README_ja.md
* dummy commit for unit test
* fix attention mask
* add CPMAntTokenizer&-Fast to auto-mapping
* drop redundant changes in README_ko
* fix defaults in docstring
* fix use_cache and some docstring
* add missing args in tokenizer
* modify tester inheritance
* add is_jieba_available
* fix some bugs
* make style and fix-copies
* add doctests
* skip integration tests
* add is_jieba_available
* fix bugs in common tests
* adjust docstrings and make style
* add argument docstring
* adjust code to some specifications
* make style and fix-copies
* add fast tokenization test
* dummy commit for unit test
* dummy commit for unit test
* dummy commit for unit test
* normalize some comments and names
* Bert->CPMAnt
* camel names and drop redundant codes
* make style and fix-coies
* add CpmTokenizerFast _import_structure
* drop cpmanttokenizerfast in model_doc
* fix some problems
* fix CPMAnt tokenization for common test
* make style and fixup
* fix copies and fixup
* fix bugs in tokenization test
* dummy commit for connection failure in unittest
* fix copies
* drop trailing comma
* fix decorator in tests
* dummy commit for connection failure in unittest
---------
Co-authored-by: Gong Baitao <gongbaitao11@gmail.com>
2023-04-12 07:33:20 -04:00
Arthur
b76e6ebd44
remove wrong doc in readme ( #22723 )
2023-04-12 07:11:12 -04:00
Sylvain Gugger
28c19ab58d
Make it easier to develop without a dev install ( #22697 )
...
* Make it easier to develop without a dev install
* Remove ugly hack that doesn't work anyway
2023-04-11 08:41:53 -04:00
Sugawara
6daa9cb515
add GPTNeoXForSequenceClassification ( #22671 )
...
* add GPTNeoXForSequenceClassification
* move the labels to logits.device (ref: #22561 )
* fix
2023-04-10 11:52:23 -04:00
Kirill
14fc1a2467
Fix quantization docs typo ( #22666 )
2023-04-10 08:53:53 -04:00
