Pavel Iakubovskii
|
48461c0fe2
|
Make pipeline able to load processor (#32514)
* Refactor get_test_pipeline
* Fixup
* Fixing tests
* Add processor loading in tests
* Restructure processors loading
* Add processor to the pipeline
* Move model loading on tom of the test
* Update `get_test_pipeline`
* Fixup
* Add class-based flags for loading processors
* Change `is_pipeline_test_to_skip` signature
* Skip t5 failing test for slow tokenizer
* Fixup
* Fix copies for T5
* Fix typo
* Add try/except for tokenizer loading (kosmos-2 case)
* Fixup
* Llama not fails for long generation
* Revert processor pass in text-generation test
* Fix docs
* Switch back to json file for image processors and feature extractors
* Add processor type check
* Remove except for tokenizers
* Fix docstring
* Fix empty lists for tests
* Fixup
* Fix load check
* Ensure we have non-empty test cases
* Update src/transformers/pipelines/__init__.py
Co-authored-by: Lysandre Debut <hi@lysand.re>
* Update src/transformers/pipelines/base.py
Co-authored-by: Lysandre Debut <hi@lysand.re>
* Rework comment
* Better docs, add note about pipeline components
* Change warning to error raise
* Fixup
* Refine pipeline docs
---------
Co-authored-by: Lysandre Debut <hi@lysand.re>
|
2024-10-09 16:46:11 +01:00 |
|
Aritra Roy Gosthipaty
|
965e98dc54
|
[Port] TensorFlow implementation of Mistral (#29708)
* chore: initial commit
* chore: adding imports and inits
* chore: adding the causal and classification code
* chore: adding names to the layers
* chore: using single self attn layer
* chore: built the model and layers
* chore: start with testing
* chore: docstring change, transpose fix
* fix: rotary embedding
* chore: adding cache implementation
* remove unused torch
* chore: fixing the indexing issue
* make fix-copies
* Use modeling_tf_utils.keras
* make fixup
* chore: fixing tests
* chore: adding past key value logic
* chore: adding multi label classfication test
* fix: switching on the built parameters in the layers
* fixing repo consistency
* ruff formats
* style changes
* fix: tf and pt equivalence
* removing returns from docstrings
* fix docstrings
* fix docstrings
* removing todos
* fix copies
* fix docstring
* fix docstring
* chore: using easier rotate_half
* adding integration tests
* chore: addressing review related to rotary embedding layer
* review changes
* [run-slow] mistral
* skip: test save load after resize token embedding
* style
---------
Co-authored-by: Matt <rocketknight1@gmail.com>
|
2024-05-23 17:48:49 +01:00 |
|