transformers/docs/source
Arthur 8bd2b1e8c2
Add support for Pixtral (#33449)
* initial commit

* gloups

* updates

* work

* weights match

* nits

* nits

* updates to support the tokenizer :)

* updates

* Pixtral processor (#33454)

* rough outline

* Add in image break and end tokens

* Fix

* Udo some formatting changes

* Set patch_size default

* Fix

* Fix token expansion

* nit in conversion script

* Fix image token list creation

* done

* add expected results

* Process list of list of images (#33465)

* updates

* working image and processor

* this is the expected format

* some fixes

* push current updated

* working mult images!

* add a small integration test

* Uodate configuration docstring

* Formatting

* Config docstring fix

* simplify model test

* fixup modeling and etests

* Return BatchMixFeature in image processor

* fix some copies

* update

* nits

* Update model docstring

* Apply suggestions from code review

* Fix up

* updates

* revert modeling changes

* update

* update

* fix load safe

* addd liscence

* update

* use pixel_values as required by the model

* skip some tests and refactor

* Add pixtral image processing tests (#33476)

* Image processing tests

* Add processing tests

* woops

* defaults reflect pixtral image processor

* fixup post merge

* images -> pixel values

* oups sorry Mr docbuilder

* isort

* fix

* fix processor tests

* small fixes

* nit

* update

* last nits

* oups this was really breaking!

* nits

* is composition needs to be true

---------

Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
2024-09-14 12:28:39 +02:00
..
de Skip tests properly (#31308) 2024-06-26 21:59:08 +01:00
en Add support for Pixtral (#33449) 2024-09-14 12:28:39 +02:00
es Updated the custom_models.md changed cross_entropy code (#33118) 2024-08-26 13:15:43 +02:00
fr Add French version of run scripts tutorial (#31483) 2024-06-28 18:02:30 +02:00
hi More fixes for doctest (#30265) 2024-04-16 11:58:55 +02:00
it Updated the custom_models.md changed cross_entropy code (#33118) 2024-08-26 13:15:43 +02:00
ja Updated the custom_models.md changed cross_entropy code (#33118) 2024-08-26 13:15:43 +02:00
ko 🌐 [i18n-KO] Translated llm_optims.md to Korean (#32325) 2024-08-30 09:52:41 -07:00
ms Remove old TF port docs (#30426) 2024-04-23 16:06:20 +01:00
pt Updated the custom_models.md changed cross_entropy code (#33118) 2024-08-26 13:15:43 +02:00
te docs: fix broken link (#31370) 2024-06-12 11:33:00 +01:00
tr Translate index.md to Turkish (#27093) 2023-11-08 08:35:20 -05:00
zh Updated the custom_models.md changed cross_entropy code (#33118) 2024-08-26 13:15:43 +02:00
_config.py [#29174] ImportError Fix: Trainer with PyTorch requires accelerate>=0.20.1 Fix (#29888) 2024-04-08 14:21:16 +01:00