Yih-Dar
1be0145d6a
Skip some slow tests for doctesting in PRs (Circle)CI ( #24753 )
...
* fix
* fix
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-07-11 22:08:14 +02:00
Arthur
fb78769b9c
[MT5
] Fix CONFIG_MAPPING issue leading it to load umt5 class ( #24678 )
...
* update
* add umt5 to auto tokenizer mapping
* nits
* fixup
* fix failing torch test
2023-07-07 11:33:54 +09:00
amyeroberts
f3e96235a3
documentation_tests.txt - sort filenames alphabetically ( #24647 )
...
* Sort filenames alphabetically
* Add check for order
2023-07-04 17:06:05 +01:00
Arthur
799df10aef
[Umt5
] Add google's umt5 to transformers
( #24477 )
...
* add tokenization template
* update conversion script
* update modeling code
* update
* update convert checkpoint
* update modeling
* revert changes on convert script
* new conversion script for new format
* correct position bias
* cleaning a bit
* Credit co authors
Co-authored-by: agemagician
<ahmed.elnaggar@tum.de>
Co-authored-by: stefan-it
<>
* styling
* Add docq
* fix copies
* add co author
* Other Author
* Merge branch 'main' of https://github.com/huggingface/transformers into add-umt5
* add testing
* nit
* Update docs/source/en/model_doc/umt5.mdx
Co-authored-by: Stefan Schweter <stefan@schweter.it>
* fix t5
* actual fix?
* revert wrong changes
* remove
* update test
* more fixes
* revert some changes
* add SPIECE_UNDERLINE
* add a commone xample
* upfate
* fix copies
* revert changes on t5 conversion script
* revert bytefallback changes since there was no addition yet
* fixup
* fixup
* ingore umt5 cutom testing folder
* fix readmes
* revertT5 changes
* same outputs
* fixup
* update example
* Apply suggestions from code review
* style
* draft addition of all new files
* current update
* fix attention and stuff
* finish refactoring
* auto config
* fixup
* more nits
* add umt5 to init
* use md format
* Update README.md
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* revert changes on mt5
* revert mt4 changes
* update test
* more fixes
* add to mapping
* fix-copies
* fix copies
* foix retain grad
* fix some tests
* nits
* done
* Update src/transformers/models/umt5/modeling_umt5.py
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* Update docs/source/en/model_doc/umt5.md
* Update src/transformers/models/umt5/__init__.py
* Update docs/source/en/model_doc/umt5.md
Co-authored-by: Stefan Schweter <stefan@schweter.it>
* Update src/transformers/models/umt5/modeling_umt5.py
* update conversion script + use google checkpoints
* nits
* update test and modelling
* stash slow convert
* update fixupd
* don't change slow
---------
Co-authored-by: stefan-it <>
Co-authored-by: Stefan Schweter <stefan@schweter.it>
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
2023-07-03 07:38:21 +02:00
Yih-Dar
c817bc44e2
Check all objects are equally in the main __init__
file ( #24573 )
...
* fix
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-06-29 17:49:59 +02:00
Sanchit Gandhi
1c1c90756d
Add Musicgen ( #24109 )
...
* Add Audiocraft
* add cross attention
* style
* add for lm
* convert and verify
* introduce t5
* split configs
* load t5 + lm
* clean conversion
* copy from t5
* style
* start pattern provider
* make generation work
* style
* fix pos embs
* propagate shape changes
* propagate shape changes
* style
* delay pattern: pad tokens at end
* audiocraft -> musicgen
* fix inits
* add mdx
* style
* fix pad token in processor
* override generate and add todos
* add init to test
* undo pattern delay mask after gen
* remove cfg logits processor
* remove cfg logits processor
* remove logits processor in favour of mask
* clean pos embs
* make fix copies
* update readmes
* clean pos emb
* refactor encoder/decoder
* make fix copies
* update conversion
* fix config imports
* update config docs
* make style
* send pattern mask to device
* pattern mask with delay
* recover prompted audio tokens
* fix docstrings
* laydown test file
* pattern edge case
* remove t5 ref
* add processing class
* config refactor
* better pattern comment
* check if mask is not present
* check if mask is not present
* refactor to auto class
* remove encoder configs
* fix processor
* processor import
* start updating conversion
* start updating tests
* make style
* convert t5, encodec, lm
* convert as composite
* also convert processor
* run generate
* classifier free gen
* comments and clean up
* make style
* docs for logit proc
* docstring for uncond gen
* start lm tests
* work tests
* let the lm generate
* refactor: reshape inside forward
* undo greedy loop changes
* from_enc_dec -> from_sub_model
* fix input id shapes in docstrings
* Apply suggestions from code review
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
* undo generate changes
* from sub model config
* Update src/transformers/models/musicgen/modeling_musicgen.py
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
* make generate work again
* generate uncond -> get uncond inputs
* remove prefix allowed tokens fn
* better error message
* logit proc checks
* Apply suggestions from code review
Co-authored-by: Joao Gante <joaofranciscocardosogante@gmail.com>
* make decoder only tests work
* composite fast tests
* make style
* uncond generation
* feat extr padding
* make audio prompt work
* fix inputs docstrings
* unconditional inputs: dict -> model output
* clean up tests
* more clean up tests
* make style
* t5 encoder -> auto text encoder
* remove comments
* deal with frames
* fix auto text
* slow tests
* nice mdx
* remove can generate
* todo - hub id
* convert m/l
* make fix copies
* only import generation with torch
* ignore decoder from tests
* don't wrap uncond inputs
* make style
* cleaner uncond inputs
* add example to musicgen forward
* fix docs
* ignore MusicGen Model/ForConditionalGeneration in auto mapping
* add doc section to toctree
* add to doc tests
* add processor tests
* fix push to hub in conversion
* tips for decoder only loading
* Apply suggestions from code review
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* fix conversion for s / m / l checkpoints
* import stopping criteria from module
* remove from pipeline tests
* fix uncond docstring
* decode audio method
* fix docs
* org: sanchit-gandhi -> facebook
* fix max pos embeddings
* remove auto doc (not compatible with shapes)
* bump max pos emb
* make style
* fix doc
* fix config doc
* fix config doc
* ignore musicgen config from docstring
* make style
* fix config
* fix config for doctest
* consistent from_sub_models
* don't automap decoder
* fix mdx save audio file
* fix mdx save audio file
* processor batch decode for audio
* remove keys to ignore
* update doc md
* update generation config
* allow changes for default generation config
* update tests
* make style
* fix docstring for uncond
* fix processor test
* fix processor test
---------
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
Co-authored-by: Joao Gante <joaofranciscocardosogante@gmail.com>
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
2023-06-29 14:48:59 +01:00
Xiaoli Wang
239ace152b
Fix TypeError: Object of type int64 is not JSON serializable ( #24340 )
...
* Fix TypeError: Object of type int64 is not JSON serializable
* Convert numpy.float64 and numpy.int64 to float and int for json serialization
* Black reformatted examples/pytorch/token-classification/run_ner_no_trainer.py
* * make style
2023-06-27 12:15:49 +01:00
NielsRogge
868363abb9
Add InstructBLIP ( #23460 )
...
* Squash 88 commits
* Use markdown
* Remove mdx files due to bad rebase
* Fix modeling files due to bad rebase
* Fix style
* Update comment
* fix
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-06-26 11:23:57 +02:00
Yih-Dar
0f968ddaa3
Check auto mappings could be imported via from transformers
( #24400 )
...
* fix
* fix
* fix
* fix
* fix
* fix
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-06-21 17:31:57 +02:00
Sylvain Gugger
eb849f6604
Migrate doc files to Markdown. ( #24376 )
...
* Rename index.mdx to index.md
* With saved modifs
* Address review comment
* Treat all files
* .mdx -> .md
* Remove special char
* Update utils/tests_fetcher.py
Co-authored-by: Lysandre Debut <lysandre.debut@reseau.eseo.fr>
---------
Co-authored-by: Lysandre Debut <lysandre.debut@reseau.eseo.fr>
2023-06-20 18:07:47 -04:00
amyeroberts
bdfd57d1d1
Fix ImageGPT doc example ( #24317 )
...
* Fix ImageGPT doc example
* Update src/transformers/models/imagegpt/image_processing_imagegpt.py
* Fix types
2023-06-16 17:01:22 +01:00
Yih-Dar
7504be35ab
Fix check_config_attributes
: check all configuration classes ( #24231 )
...
* fix
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-06-15 11:39:20 +02:00
Matthijs Hollemans
0c3fdccf2f
[WIP] add EnCodec model ( #23655 )
...
* boilerplate stuff
* messing around with the feature extractor
* fix feature extractor
* unit tests for feature extractor
* rename speech to audio
* quick-and-dirty import of Meta's code
* import weights (sort of)
* cleaning up
* more cleaning up
* move encoder/decoder args into config
* cleanup model
* rename EnCodec -> Encodec
* RVQ parameters in config
* add slow test
* add lstm init and test_init
* Add save & load
* finish EncodecModel
* remove decoder_input_values as they are ont used anywhere (not removed from doc yet)
* fix test feature extraction model name
* Add better slow test
* Fix tests
* some fixup and cleaning
* Improve further
* cleaning up quantizer
* fix up conversion script
* test don't pass, _encode_fram does not work
* update tests with output per encode and decode
* more cleanup
* rename _codebook
* remove old config cruft
* ratios & hop_length
* use ModuleList instead of Sequential
* clean up resnet block
* update types
* update tests
* fixup
* quick cleanup
* fix padding
* more styl,ing
* add patrick feedback
* fix copies
* fixup
* fix lstm
* fix shape issues
* fixup
* rename conv layers
* fixup
* fix decoding
* small conv refactoring
* remove norm_params
* simplify conv layers
* rename conv layers
* stuff
* Clean up
* Add padding logic
use padding mask
small conv refactoring
remove norm_params
simplify conv layers
rename conv layers
stuff
add batched test
update
Clean up
merge and update for padding
fix padding
fixup
* clean up more
* clean up more
* More clean ups
* cleanup convolutions
* typo
* fix typos
* fixup
* build PR doc?
* start refactoring docstring
* fix don't pad when no strid and chunk
* update docstring
* update docstring
* nits
* update going to lunch
* update config and model
* fix broken testse (becaue of the config changes)
* fix scale computation
* fixu[
* only return dict if speciefied or if config returns it
* remove todos
* update defaults in config
* update conversion script
* fix doctest
* more docstring + fixup
* nits on batched_tests
* more nits
* Apply suggestions from code review
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
* update basxed on review
* fix update
* updaet tests
* Apply suggestions from code review
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* fixup
* add overlap and chunl_length_s
* cleanup feature extraction
* teste edge cases truncation and padding
* correct processor values
* update config encodec, nits
* fix tests
* fixup
* fix 24Hz test
* elle tests are green
* fix fixup
* Apply suggestions from code review
* revert readme changes
* fixup
* add example
* use facebook checkpoints
* fix typo
* no pipeline tests
* use slef.pad everywhere we can
* Apply suggestions from code review
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* update based on review
* update
* update mdx
* fix bug and tests
* fixup
* fix doctest
* remove comment
* more nits
* add more coverage for `test_truncation_and_padding`
* fixup
* add last test
* fix text
* nits
* Update tests/models/encodec/test_modeling_encodec.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* take care of the last comments
* typo
* fix test
* nits
* fixup
* Update src/transformers/models/encodec/feature_extraction_encodec.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
---------
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
Co-authored-by: arthur.zucker@gmail.com <arthur.zucker@gmail.com>
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
2023-06-14 18:57:23 +02:00
Yih-Dar
41a8fa4e14
Add the number of model
test failures to slack CI report ( #24207 )
...
fix
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-06-12 21:27:10 +02:00
Sylvain Gugger
89b00eef94
Fix expected value in tests of the test fetcher ( #24077 )
...
* Fix expected value in tests of the test fetcher
* Fix trigger for repo util tests
2023-06-07 11:38:56 -04:00
Yih-Dar
ff4c0fc7d2
Tiny fix for check_self_hosted_runner.py
( #24052 )
...
fix
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-06-06 18:17:41 +02:00
amyeroberts
a717e0318c
Add TimmBackbone model ( #22619 )
...
* Add test_backbone for convnext
* Add TimmBackbone model
* Add check for backbone type
* Tidying up - config checks
* Update convnextv2
* Tidy up
* Fix indices & clearer comment
* Exceptions for config checks
* Correclty update config for tests
* Safer imports
* Safer safer imports
* Fix where decorators go
* Update import logic and backbone tests
* More import fixes
* Fixup
* Only import all_models if torch available
* Fix kwarg updates in from_pretrained & main rebase
* Tidy up
* Add tests for AutoBackbone
* Tidy up
* Fix import error
* Fix up
* Install nattan in doc_test_job
* Revert back to setting self._out_xxx directly
* Bug fix - out_indices mapping from out_features
* Fix tests
* Dont accept output_loading_info for Timm models
* Set out_xxx and don't remap
* Use smaller checkpoint for test
* Don't remap timm indices - check out_indices based on stage names
* Skip test as it's n/a
* Apply suggestions from code review
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* Cleaner imports / spelling is hard
---------
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
2023-06-06 17:11:30 +01:00
Sylvain Gugger
b8935980a2
Modification of one text example file should trigger said test ( #24051 )
2023-06-06 12:02:56 -04:00
Sylvain Gugger
4aa13224a5
Update the update metadata job to use upload_folder ( #23917 )
2023-05-31 14:10:14 -04:00
Sylvain Gugger
3ff443a6d9
Re-enable squad test ( #23912 )
...
* Re-enable squad test
* [all-test]
* [all-test] Fix all test command
* Fix the all-test
2023-05-31 13:44:26 -04:00
Denisa Roberts
88f50a1e89
Add TensorFlow implementation of EfficientFormer ( #22620 )
...
* Add tf code for efficientformer
* Fix return dict bug - return last hidden state after last stage
* Fix corresponding return dict bug
* Override test tol
* Change default values of training to False
* Set training to default False X3
* Rm axis from ln
* Set init in dense projection
* Rm debug stuff
* Make style; all tests pass.
* Modify year to 2023
* Fix attention biases codes
* Update the shape list logic
* Add a batch norm eps config
* Remove extract comments in test files
* Add conditional attn and hidden states return for serving output
* Change channel dim checking logic
* Add exception for withteacher model in training mode
* Revert layer count for now
* Add layer count for conditional layer naming
* Transpose for conv happens only in main layer
* Make tests smaller
* Make style
* Update doc
* Rm from_pt
* Change to actual expect image class label
* Remove stray print in tests
* Update image processor test
* Remove the old serving output logic
* Make style
* Make style
* Complete test
2023-05-31 10:43:12 +01:00
Eli Simhayev
4b6a5a7caa
[Time-Series] Autoformer model ( #21891 )
...
* ran `transformers-cli add-new-model-like`
* added `AutoformerLayernorm` and `AutoformerSeriesDecomposition`
* added `decomposition_layer` in `init` and `moving_avg` to config
* added `AutoformerAutoCorrelation` to encoder & decoder
* removed caninical self attention `AutoformerAttention`
* added arguments in config and model tester. Init works! 😁
* WIP autoformer attention with autocorrlation
* fixed `attn_weights` size
* wip time_delay_agg_training
* fixing sizes and debug time_delay_agg_training
* aggregation in training works! 😁
* `top_k_delays` -> `top_k_delays_index` and added `contiguous()`
* wip time_delay_agg_inference
* finish time_delay_agg_inference 😎
* added resize to autocorrelation
* bug fix: added the length of the output signal to `irfft`
* `attention_mask = None` in the decoder
* fixed test: changed attention expected size, `test_attention_outputs` works!
* removed unnecessary code
* apply AutoformerLayernorm in final norm in enc & dec
* added series decomposition to the encoder
* added series decomp to decoder, with inputs
* added trend todos
* added autoformer to README
* added to index
* added autoformer.mdx
* remove scaling and init attention_mask in the decoder
* make style
* fix copies
* make fix-copies
* inital fix-copies
* fix from https://github.com/huggingface/transformers/pull/22076
* make style
* fix class names
* added trend
* added d_model and projection layers
* added `trend_projection` source, and decomp layer init
* added trend & seasonal init for decoder input
* AutoformerModel cannot be copied as it has the decomp layer too
* encoder can be copied from time series transformer
* fixed generation and made distrb. out more robust
* use context window to calculate decomposition
* use the context_window for decomposition
* use output_params helper
* clean up AutoformerAttention
* subsequences_length off by 1
* make fix copies
* fix test
* added init for nn.Conv1d
* fix IGNORE_NON_TESTED
* added model_doc
* fix ruff
* ignore tests
* remove dup
* fix SPECIAL_CASES_TO_ALLOW
* do not copy due to conv1d weight init
* remove unused imports
* added short summary
* added label_length and made the model non-autoregressive
* added params docs
* better doc for `factor`
* fix tests
* renamed `moving_avg` to `moving_average`
* renamed `factor` to `autocorrelation_factor`
* make style
* Update src/transformers/models/autoformer/configuration_autoformer.py
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
* Update src/transformers/models/autoformer/configuration_autoformer.py
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
* fix configurations
* fix integration tests
* Update src/transformers/models/autoformer/configuration_autoformer.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* fixing `lags_sequence` doc
* Revert "fixing `lags_sequence` doc"
This reverts commit 21e34911e3
.
* Update src/transformers/models/autoformer/modeling_autoformer.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Update src/transformers/models/autoformer/modeling_autoformer.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Update src/transformers/models/autoformer/modeling_autoformer.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Apply suggestions from code review
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Update src/transformers/models/autoformer/configuration_autoformer.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* model layers now take the config
* added `layer_norm_eps` to the config
* Update src/transformers/models/autoformer/modeling_autoformer.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* added `config.layer_norm_eps` to AutoformerLayernorm
* added `config.layer_norm_eps` to all layernorm layers
* Update src/transformers/models/autoformer/configuration_autoformer.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Update src/transformers/models/autoformer/configuration_autoformer.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Update src/transformers/models/autoformer/configuration_autoformer.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Update src/transformers/models/autoformer/configuration_autoformer.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* fix variable names
* added inital pretrained model
* added use_cache docstring
* doc strings for trend and use_cache
* fix order of args
* imports on one line
* fixed get_lagged_subsequences docs
* add docstring for create_network_inputs
* get rid of layer_norm_eps config
* add back layernorm
* update fixture location
* fix signature
* use AutoformerModelOutput dataclass
* fix pretrain config
* no need as default exists
* subclass ModelOutput
* remove layer_norm_eps config
* fix test_model_outputs_equivalence test
* test hidden_states_output
* make fix-copies
* Update src/transformers/models/autoformer/configuration_autoformer.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* removed unused attr
* Update tests/models/autoformer/test_modeling_autoformer.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Update src/transformers/models/autoformer/modeling_autoformer.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Update src/transformers/models/autoformer/modeling_autoformer.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Update src/transformers/models/autoformer/modeling_autoformer.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Update src/transformers/models/autoformer/modeling_autoformer.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Update src/transformers/models/autoformer/modeling_autoformer.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Update src/transformers/models/autoformer/modeling_autoformer.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* use AutoFormerDecoderOutput
* fix formatting
* fix formatting
---------
Co-authored-by: Kashif Rasul <kashif.rasul@gmail.com>
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
2023-05-30 10:23:32 +02:00
Sylvain Gugger
6e4bc67099
Revamp test selection for the example tests ( #23737 )
...
* Revamp test selection for the example tests
* Rename old XLA test and fake modif in run_glue
* Fixes
* Fake Trainer modif
* Remove fake modifs
2023-05-25 09:38:21 -04:00
Maria Khalusova
2eaaf17a0b
Export to ONNX doc refocused on using optimum, added tflite ( #23434 )
...
* doc refocused on using optimum, tflite
* minor updates to fix checks
* Apply suggestions from code review
Co-authored-by: regisss <15324346+regisss@users.noreply.github.com>
* TFLite to separate page, added links
* Removed the onnx list builder
* make style
* Update docs/source/en/serialization.mdx
Co-authored-by: regisss <15324346+regisss@users.noreply.github.com>
---------
Co-authored-by: regisss <15324346+regisss@users.noreply.github.com>
2023-05-24 08:13:23 -04:00
Sylvain Gugger
b7b81d9344
Clean up CUDA kernels ( #23455 )
2023-05-18 14:14:43 -04:00
Yih-Dar
ca3df9f0cf
Run doctest (in PRs) only when some doc example(s) are modified ( #23387 )
...
* fix
* fix
* update
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-05-16 23:29:02 +02:00
Sylvain Gugger
9cf4a8b456
Build with non Python files ( #23405 )
...
* Add a test of the built release
* Polish everything
* Trigger CI
2023-05-16 14:23:10 -04:00
Yih-Dar
569a97adb2
Revert "Only add files with modification outside doc blocks" ( #23371 )
...
fix
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-05-15 14:28:36 +02:00
Yih-Dar
a3975f94f3
Only add files with modification outside doc blocks ( #23327 )
...
* min. version for pytest
* fix
* fix
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-05-12 16:35:15 +02:00
Yih-Dar
6a6225beab
Fix doctest files fetch issue ( #23277 )
...
* fix
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-05-11 17:14:06 +02:00
Yih-Dar
9088fcae82
Bring back the PR Refactor doctests + add CI
to main
( #23271 )
...
* Revert "Revert "[Doctests] Refactor doctests + add CI" (#23245 )"
This reverts commit 69ee46243c
.
* try not expose HfDocTestParser
* move into testing_utils.py
* remove pytest install
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-05-10 22:00:48 +02:00
Sylvain Gugger
69ee46243c
Revert "[Doctests] Refactor doctests + add CI" ( #23245 )
...
Revert "[Doctests] Refactor doctests + add CI (#22987 )"
This reverts commit 627f44799a
.
2023-05-09 15:26:15 -04:00
Arthur
627f44799a
[Doctests] Refactor doctests + add CI ( #22987 )
...
* intiial commit
* new styling
* update
* just run doctest in CI
* remove more test for fast dev
* update
* update refs
* update path and fetch upstream
* update documentatyion trests
* typo
* parse pwd
* don't check for files that are in hidden folders
* just give paths relative to transformers
* update
* update
* update
* major refactoring
* make sure options is ok
* lest test that mdx is tested
* doctest glob
* nits
* update doctest nightly
* some cleaning
* run correct test on diff
* debug
* run on a single worker
* skip_cuda_test tampkate
* updates
* add rA and continue on failure
* test options
* parse `py` codeblock?
* we don't need to replace ignore results, don't remember whyu I put it
* cleanup
* more cleaning
* fix arg
* more cleaning
* clean an todo
* more pre-processing
* doctest-module has none so extra `- ` is needed
* remove logs
* nits
* doctest-modules ....
* oups
* let's use sugar
* make dataset go quiet
* add proper timeout
* nites
* spleling timeout
* update
* properly skip tests that have CUDSA
* proper skipping
* cleaning main and get tests to run
* remove make report?
* remove tee
* some updates
* tee was removed but is the full output still available?
* [all-test]
* only our tests
* don't touch tee in this PR
* no atee-sys
* proper sub
* monkey
* only replace call
* fix sub
* nits
* nits
* fix invalid syntax
* add skip cuda doctest env variable
* make sure all packages are installed
* move file
* update check repo
* revert changes
* nit
* finish cleanup
* fix re
* findall
* update don't test init files
* ignore pycache
* `-ignore-pycache` when running pytests
* try to fix the import missmatch error
* install dec
* pytest is required as doctest_utils imports things from it
* the only log issues were dataset, ignore results should work
* more cleaning
* Update .circleci/create_circleci_config.py
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* Apply suggestions from code review
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* [ydshieh] empty string if cuda is found
* [ydshieh] fix condition
* style
* [ydshieh] fix
* Add comment
* style
* style
* show failure
* trigger CI
---------
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
Co-authored-by: Yih-Dar <2521628+ydshieh@users.noreply.github.com>
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-05-09 20:34:48 +02:00
Alara Dirik
441658dd6c
Add focalnet backbone ( #23104 )
...
Adds FocalNet backbone to return features from all stages
2023-05-03 19:32:42 +03:00
Lingepumpe
5427250351
Avoid invalid escape sequences, use raw strings ( #22936 )
...
* Avoid invalid escape sequences, use raw strings
* Integrate PR feedback
2023-04-25 09:17:56 -04:00
Yih-Dar
60f9649653
Fix DeepSpeed
CI job link in Past CI ( #22967 )
...
* Fix job link
* fix artifact name logic
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-04-25 09:52:19 +02:00
Yih-Dar
5166c30e29
Fix a minor bug in CI slack report ( #22906 )
...
* fix
* fix
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-04-21 20:36:35 +02:00
Yih-Dar
3080fb714f
Fix Slack report for Nightly CI and Past CI ( #22901 )
...
fix
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-04-21 11:23:16 +02:00
Yih-Dar
435abb22cb
Fix counting in Slack report for some jobs ( #22913 )
...
fix
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-04-21 11:22:23 +02:00
Arthur
f143037789
Add automatic-mask-generation
pipeline for Segment Anything Model (SAM) ( #22840 )
...
* cleanup
* updates
* more refactoring
* make style
* update inits
* support other inputs in base
* update based on review
Co-authored-by: Nicolas Patry <patry.nicolas@gmail.com>
* Update tests/pipelines/test_pipelines_automatic_mask_generation.py
Co-authored-by: Nicolas Patry <patry.nicolas@protonmail.com>
* update
* fixup
* TODO x and y to refactor, _h _w refactored here
* update docstring
* more nits
* style on these
* more doc fix
* rename variables
* update
* updates
* style
* update
* fix `_mask_to_rle_pytorch`
* styling
* fix ask to rle, wrong outputs
* add device arg
* update
* more updates, fix tets
* udpate
* update docstrings
* styling
* fixup
* add notebook on the docs
* update orginal sizes
* fix docstring
* updat condition on point_per-batch
* updates tests
* fix CI test
* extend is required, append does not work!
* fixup
* fix CI tests
* whit pixels left
* address doc comments
* fix doc
* slow pipeline tests
* update auto init
* add revision
* make fixup
* update p!ipoeline tag when calling tests
* alphabeitcal order in inits
* fix copies
* last style nits
* Apply suggestions from code review
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* reformat docstring
* more reformat
* address most of the comments
* Update src/transformers/pipelines/mask_generation.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* final refactor
* Update src/transformers/models/sam/image_processing_sam.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* fixup and fix slow tests
* revert
---------
Co-authored-by: Nicolas Patry <patry.nicolas@gmail.com>
Co-authored-by: Nicolas Patry <patry.nicolas@protonmail.com>
Co-authored-by: younesbelkada <younesbelkada@gmail.com>
Co-authored-by: Younes Belkada <49240599+younesbelkada@users.noreply.github.com>
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
2023-04-20 19:27:24 +02:00
Arthur
474bf508df
Add Segment Anything Model (SAM) ( #22654 )
...
* initial commit
* keys match
* update, fix conversion
* fixes, inference working
* fix
* more fixes
* more fixes
* clean up
* more clean up
* fix copies and add convext copied layer norm
* stash
* pretty big upfate
* cleaning
* more cleaning
* fixup stuffs
* fix copies
* fix iinit
* update test removing tokenizer
* nits
* add pretrained
* more nits
* remove tracking of pipeline
* few fixes
* update san and conversion script
* fix mask decoder and prompt encoder conversion
* fixes
* small update
* fix order
* fix
* fix image embeddings
* nites
* few fixes
* fix logits
* clean up
* fixes boxes inference
* v1 AMG
* clean up
* some clean up
* multi points support
* amg working
* fixup
* clean up
* readme
* update toctree
* fix type hint
* multiple fixes
* fixup
* fixes
* updates
* updates
* more tests
* few fixes
* change to `SamForMaskGeneration`
* doc
* fixup
* fix more tests
* multiple fixes
* fix CI tests
* refactor processor
* renamings
* draft the pipeline
* refactor
* fix tests
* fix test
* few cleanings
* fix test
* edit pipelien support chunking
* udate
* add slow tests
* fix nit
* fixup
* fix nit
* current chunk pipleine
* cast boxes in fp32
* nit
* current updates
* piepleine works
* fixup
* clean up config
* fix slow tests
* fix slow tests
* clean up
* update doc and pipeline
* adds more slow tests
* fix slow tests
* cleaning
* tests pass
* add docstring
* fix copies
* clean up
* support batch of images
* style
* dummy is needed, add tests
* fix slow tests
* fix CI
* update
* adds more tests
* fixes
* fixes
* fixup
* fixes
* few fixes
* filter
* few fixes
* some refactor
* touches finales
* fix
* style
* remove pipeline files
* fixes nits
* revert pipeline changes
* fix test
* fixup
* remove automodel for automatic mask generation
* fix failing torch tests
* update mdx
* revert removal of `MODEL_FOR_AUTOMATIC_MASK_GENERATION_MAPPING`
* update sam config based on review
Co-authored-by: amyeroberts <aeroberts4444@gmail.com>
Co-authored-by: sgugger <sylvain.gugger@gmail.com>
* update low_resolution_masks -> pred_masks
inti ln with layer_norm_eps
add_decomposed_rel_pos doc
forward doc of SamForMaskGeneration
* update processor docstring
* remove image processor import empty
* update for testing
* output vision hidden states + clean recomm
also test all iou values
* fixup
* fixup
* remove unused
* Update src/transformers/models/sam/modeling_sam.py
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* Update src/transformers/models/sam/image_processing_sam.py
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* nits
* fix
* fix CI tests and slow tests
* replace with Amy's processor
* clearer docstring
* add `SamVisionNeck`
* refactor - all CI tests should pass
* fix broken import on Gcolab
* few fixes here and there
* fix another bug
* fix more bugs
* update and merge
* correct ckpt
* address comments
* add tips
* revert
* fix docstring
* replace with `SamModel`
* make fixup
* add support for bathed images and batch ed points
* make fixup this time, really
* make fixup again and again
* few fixes here and there, this should be the touche finale
* Update docs/source/en/model_doc/sam.mdx
* fixup
* correct checkpoints
* correct name
* rm unneeded file
* add notebook
---------
Co-authored-by: younesbelkada <younesbelkada@gmail.com>
Co-authored-by: amyeroberts <aeroberts4444@gmail.com>
Co-authored-by: sgugger <sylvain.gugger@gmail.com>
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
Co-authored-by: Younes Belkada <49240599+younesbelkada@users.noreply.github.com>
2023-04-19 21:01:49 +02:00
Yih-Dar
648bd5a8aa
Show diff between 2 CI runs on Slack reports ( #22798 )
...
fix
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-04-19 19:27:37 +02:00
Sylvain Gugger
dacd34568d
Mark auto models as important ( #22815 )
...
* Mark auto models as important
* Annoying file with bad line endings
2023-04-17 15:33:01 -04:00
NielsRogge
8eb38f638d
[Pix2struct] Simplify generation ( #22527 )
...
* Add model to doc tests
* Remove generate and replace by prepare_inputs_for_generation
* More fixes
* Remove print statements
* Update integration tests
* Fix generate
* Remove model from auto mapping
* Use auto processor
* Fix integration tests
* Fix test
* Add inference code snippet
* Remove is_encoder_decoder
* Update docs
* Remove notebook link
2023-04-13 09:01:14 -04:00
Elabonga Atuo
7ade6ef7d4
[Doctest] Add configuration_mvp.py ( #22735 )
...
* added configuration file for mvp model
* added configuration_mvp.py line to file
2023-04-13 08:19:18 +02:00
Elabonga Atuo
51007976ec
[Doctest] Add configuration_m2m_100.py ( #22733 )
...
m2m-100-config for doctest
2023-04-13 08:17:07 +02:00
Sylvain Gugger
28c19ab58d
Make it easier to develop without a dev install ( #22697 )
...
* Make it easier to develop without a dev install
* Remove ugly hack that doesn't work anyway
2023-04-11 08:41:53 -04:00
Yih-Dar
c7ec71baf5
Update tiny model summary file for recent models ( #22637 )
...
* Update tiny model summary file for recent models
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-04-06 22:52:59 +02:00
Yih-Dar
d0b83fe2e1
A script to add/update pipeline_model_mapping
systematically ( #22180 )
...
* Auto. add and update pipeline_model_mapping
* Fix style and quality
* Finalize (comments)
* fix
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-04-06 18:08:14 +02:00
Yih-Dar
2c22bc79c2
Make tiny model creation + pipeline testing more robust ( #22500 )
...
* Final Tiny things
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-04-06 17:45:55 +02:00