* Add a Dockerfile for PyTorch + ROCm based on official AMD released artifact
* Add a new artifact single-amdgpu testing on main
* Attempt to test the workflow without merging.
* Changed BERT to check if things are triggered
* Meet the dependencies graph on workflow
* Revert BERT changes
* Add check_runners_amdgpu to correctly mount and check availability
* Rename setup to setup_gpu for CUDA and add setup_amdgpu for AMD
* Fix all the needs.setup -> needs.setup_[gpu|amdgpu] dependencies
* Fix setup dependency graph to use check_runner_amdgpu
* Let's do the runner status check only on AMDGPU target
* Update the Dockerfile.amd to put ourselves in / rather than /var/lib
* Restore the whole setup for CUDA too.
* Let's redisable them
* Change BERT to trigger tests
* Restore BERT
* Add torchaudio with rocm 5.6 to AMD Dockerfile (#26050)
fix dockerfile
Co-authored-by: Felix Marty <felix@hf.co>
* Place AMD GPU tests in a separate workflow (correct branch) (#26105)
AMDGPU CI lives in an other workflow
* Fix invalid job name is dependencies.
* Remove tests multi-amdgpu for now.
* Use single-amdgpu
* Use --net=host for now.
* Remote host networking.
* Removed duplicated check_runners_amdgpu step
* Let's tag machine-types with mi210 for now.
* Machine type should be only mi210
* Remove unnecessary push.branches item
* Apply review suggestions moving from `x-amdgpu` to `x-gpu` introducing `amd-gpu` and `miXXX` labels.
* Remove amdgpu from step names.
* finalize
* delete
---------
Co-authored-by: fxmarty <9808326+fxmarty@users.noreply.github.com>
Co-authored-by: Felix Marty <felix@hf.co>
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
* a draft version
* v2 integration
* fix
* make it more generic and works for IA3
* add set adapter and multiple adapters support
* fixup
* adapt a bit
* oops
* oops
* oops
* adapt more
* fix
* add more refactor
* now works with model class
* change it to instance method as it causes issues with `jit`.
* add CR
* change method name
* add `add_adapter` method
* clean up
* Update src/transformers/adapters/peft_mixin.py
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
* add moe utils
* fixup
* Update src/transformers/adapters/peft_mixin.py
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
* adapt
* oops
* fixup
* add is_peft_available
* remove `requires_backend`
* trainer compatibility
* fixup + docstring
* more details
* trigger CI
* Apply suggestions from code review
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* Update src/transformers/modeling_utils.py
* fixup + is_main_process
* added `save_peft_format` in save_pretrained
* up
* fix nits here and there
* nits here and there.
* docs
* revert `encoding="utf-8"`
* comment
* added slow tests before the PEFT release.
* fixup and nits
* let's be on the safe zone
* added more comments
* v1 docs
* add remaining docs
* Apply suggestions from code review
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
* move to `lib_integrations`
* fixup
* this time fixup
* Apply suggestions from code review
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* address final comments
* refactor to use `token`
* add PEFT to DockerFile for slow tests.
* added pipeline support.
---------
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
* Limit Pydantic to V1 in dependencies
Pydantic is about to release V2 release which will break a lot of things. This change prevents `transformers` to be used with Pydantic V2 to avoid breaking things.
* more
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
* Use PyAV instead of Decord
* Get frame indices
* Fix number of frames
* Update src/transformers/models/videomae/image_processing_videomae.py
* Fix up
* Fix copies
* Update timesformer doctests
* Update docstrings
* Change the import of kenlm from github to pypi
* Change the import of kenlm from github to pypi in circleci config
* Fix code quality issues
* Fix isort issue, add kenlm in extras for audio
* Add kenlm to deps
* Add kenlm to deps
* Commit 'make fixup' changes
* Remove version from kenlm deps
* commit make fixup changes
* Remove manual installation of kenlm
* Remove manual installation of kenlm
* Remove manual installation of kenlm
* bnb minor modifications
- refactor documentation
- add troubleshooting README
- add PyPi library on DockerFile
* Apply suggestions from code review
Co-authored-by: Stas Bekman <stas00@users.noreply.github.com>
* Apply suggestions from code review
* Apply suggestions from code review
* Apply suggestions from code review
* put in one block
- put bash instructions in one block
* update readme
- refactor a bit hardware requirements
* change text a bit
* Apply suggestions from code review
Co-authored-by: Yih-Dar <2521628+ydshieh@users.noreply.github.com>
* apply suggestions
Co-authored-by: Yih-Dar <2521628+ydshieh@users.noreply.github.com>
* add link to paper
* Apply suggestions from code review
Co-authored-by: Stas Bekman <stas00@users.noreply.github.com>
* Update tests/mixed_int8/README.md
* Apply suggestions from code review
* refactor a bit
* add instructions Turing & Amperer
Co-authored-by: Stas Bekman <stas00@users.noreply.github.com>
* add A6000
* clarify a bit
* remove small part
* Update tests/mixed_int8/README.md
Co-authored-by: Stas Bekman <stas00@users.noreply.github.com>
Co-authored-by: Yih-Dar <2521628+ydshieh@users.noreply.github.com>
* first commit
* correct replace function
* add final changes
- works like charm!
- cannot implement tests yet
- tested
* clean up a bit
* add bitsandbytes dependencies
* working version
- added import function
- added bitsandbytes utils file
* small fix
* small fix
- fix import issue
* fix import issues
* Apply suggestions from code review
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* refactor a bit
- move bitsandbytes utils to utils
- change comments on functions
* reformat docstring
- reformat docstring on init_empty_weights_8bit
* Update src/transformers/__init__.py
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* revert bad formatting
* change to bitsandbytes
* refactor a bit
- remove init8bit since it is useless
* more refactoring
- fixed init empty weights issue
- added threshold param
* small hack to make it work
* Update src/transformers/modeling_utils.py
* Update src/transformers/modeling_utils.py
* revmoe the small hack
* modify utils file
* make style + refactor a bit
* create correctly device map
* add correct dtype for device map creation
* Apply suggestions from code review
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* apply suggestions
- remove with torch.grad
- do not rely on Python bool magic!
* add docstring
- add docstring for new kwargs
* add docstring
- comment `replace_8bit_linear` function
- fix weird formatting
* - added more documentation
- added new utility function for memory footprint tracking
- colab demo to add
* few modifs
- typo doc
- force cast into float16 when load_in_8bit is enabled
* added colab link
* add test architecture + docstring a bit
* refactor a bit testing class
* make style + refactor a bit
* enhance checks
- add more checks
- start writing saving test
* clean up a bit
* male style
* add more details on doc
* add more tests
- still needs to fix 2 tests
* replace by "or"
- could not fix it from GitHub GUI
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* refactor a bit testing code + add readme
* make style
* fix import issue
* Update src/transformers/modeling_utils.py
Co-authored-by: Michael Benayoun <mickbenayoun@gmail.com>
* add few comments
* add more doctring + make style
* more docstring
* raise error when loaded in 8bit
* make style
* add warning if loaded on CPU
* add small sanity check
* fix small comment
* add bitsandbytes on dockerfile
* Improve documentation
- improve documentation from comments
* add few comments
* slow tests pass on the VM but not on the CI VM
* Fix merge conflict
* make style
* another test should pass on a multi gpu setup
* fix bad import in testing file
* Fix slow tests
- remove dummy batches
- no more CUDA illegal memory errors
* odify dockerfile
* Update docs/source/en/main_classes/model.mdx
* Update Dockerfile
* Update model.mdx
* Update Dockerfile
* Apply suggestions from code review
* few modifications
- lm head can stay on disk/cpu
- change model name so that test pass
* change test value
- change test value to the correct output
- torch bmm changed to baddmm in bloom modeling when merging
* modify installation guidelines
* Apply suggestions from code review
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* Apply suggestions from code review
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* Apply suggestions from code review
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* replace `n`by `name`
* merge `load_in_8bit` and `low_cpu_mem_usage`
* first try - keep the lm head in full precision
* better check
- check the attribute `base_model_prefix` instead of computing the number of parameters
* added more tests
* Update src/transformers/utils/bitsandbytes.py
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* Merge branch 'integration-8bit' of https://github.com/younesbelkada/transformers into integration-8bit
* improve documentation
- fix typos for installation
- change title in the documentation
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
Co-authored-by: Michael Benayoun <mickbenayoun@gmail.com>
* Stricter pt-to-tf checks; Update docker image for related tests
* check all attributes in the output
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* [trainer / deepspeed] fix hyperparameter_search
* require optuna
* style
* oops
* add dep in the right place
* create deepspeed-testing dep group
* Trigger CI
* Add PT + TF automatic builds
* Apply suggestions from code review
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
* Wrap up
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
* Base move
* Examples reorganization
* Update references
* Put back test data
* Move conftest
* More fixes
* Move test data to test fixtures
* Update path
* Apply suggestions from code review
Co-authored-by: Lysandre Debut <lysandre@huggingface.co>
* Address review comments and clean
Co-authored-by: Lysandre Debut <lysandre@huggingface.co>
- Use cuda:10.2 image instead of 10.1 (to address version mismatch
warning with pytorch)
- Use devel version that is built on the runtime and includes headers
and development tools (was otherwise failing to build apex)
* Added transformers-pytorch-cpu and gpu Docker images
Signed-off-by: Morgan Funtowicz <morgan@huggingface.co>
* Added automatic jupyter launch for Docker image.
Signed-off-by: Morgan Funtowicz <morgan@huggingface.co>
* Move image from alpine to Ubuntu to align with NVidia container images.
Signed-off-by: Morgan Funtowicz <morgan@huggingface.co>
* Added TRANSFORMERS_VERSION argument to Dockerfile.
Signed-off-by: Morgan Funtowicz <morgan@huggingface.co>
* Added Pytorch-GPU based Docker image
Signed-off-by: Morgan Funtowicz <morgan@huggingface.co>
* Added Tensorflow images.
Signed-off-by: Morgan Funtowicz <morgan@huggingface.co>
* Use python 3.7 as Tensorflow doesnt provide 3.8 compatible wheel.
Signed-off-by: Morgan Funtowicz <morgan@huggingface.co>
* Remove double FROM instructions on transformers-pytorch-cpu image.
Signed-off-by: Morgan Funtowicz <morgan@huggingface.co>
* Added transformers-tensorflow-gpu Docker image.
Signed-off-by: Morgan Funtowicz <morgan@huggingface.co>
* use the correct ubuntu version for tensorflow-gpu
Signed-off-by: Morgan Funtowicz <morgan@huggingface.co>
* Added pipelines example notebook
Signed-off-by: Morgan Funtowicz <morgan@huggingface.co>
* Added transformers-cpu and transformers-gpu (including both PyTorch and TensorFlow) images.
Signed-off-by: Morgan Funtowicz <morgan@huggingface.co>
* Docker images doesnt start jupyter notebook by default.
Signed-off-by: Morgan Funtowicz <morgan@huggingface.co>
* Tokenizers notebook
Signed-off-by: Morgan Funtowicz <morgan@huggingface.co>
* Update images links
Signed-off-by: Morgan Funtowicz <morgan@huggingface.co>
* Update Docker images to python 3.7.6 and transformers 2.5.1
Signed-off-by: Morgan Funtowicz <morgan@huggingface.co>
* Added 02-transformers notebook.
Signed-off-by: Morgan Funtowicz <morgan@huggingface.co>
* Trying to realign 02-transformers notebook ?
Signed-off-by: Morgan Funtowicz <morgan@huggingface.co>
* Added Transformer image schema
* Some tweaks on tokenizers notebook
* Removed old notebooks.
Signed-off-by: Morgan Funtowicz <morgan@huggingface.co>
* Attempt to provide table of content for each notebooks
Signed-off-by: Morgan Funtowicz <morgan@huggingface.co>
* Second attempt.
Signed-off-by: Morgan Funtowicz <morgan@huggingface.co>
* Reintroduce transformer image.
Signed-off-by: Morgan Funtowicz <morgan@huggingface.co>
* Keep trying
Signed-off-by: Morgan Funtowicz <morgan@huggingface.co>
* It's going to fly !
Signed-off-by: Morgan Funtowicz <morgan@huggingface.co>
* Remaining of the Table of Content
Signed-off-by: Morgan Funtowicz <morgan@huggingface.co>
* Fix inlined elements for the table of content
Signed-off-by: Morgan Funtowicz <morgan@huggingface.co>
* Removed anaconda dependencies for Docker images.
Signed-off-by: Morgan Funtowicz <morgan@huggingface.co>
* Removing notebooks ToC
Signed-off-by: Morgan Funtowicz <morgan@huggingface.co>
* Added LABEL to each docker image.
Signed-off-by: Morgan Funtowicz <morgan@huggingface.co>
* Removed old Dockerfile
Signed-off-by: Morgan Funtowicz <morgan@huggingface.co>
* Directly use the context and include transformers from here.
Signed-off-by: Morgan Funtowicz <morgan@huggingface.co>
* Reduce overall size of compiled Docker images.
Signed-off-by: Morgan Funtowicz <morgan@huggingface.co>
* Install jupyter by default and use CMD for easier launching of the images.
Signed-off-by: Morgan Funtowicz <morgan@huggingface.co>
* Reduce number of layers in the images.
Signed-off-by: Morgan Funtowicz <morgan@huggingface.co>
* Added README.md for notebooks.
Signed-off-by: Morgan Funtowicz <morgan@huggingface.co>
* Fix notebooks link in README
Signed-off-by: Morgan Funtowicz <morgan@huggingface.co>
* Fix some wording issues.
Signed-off-by: Morgan Funtowicz <morgan@huggingface.co>
* Added blog notebooks too.
Signed-off-by: Morgan Funtowicz <morgan@huggingface.co>
* Addressing spelling errors in review comments.
Signed-off-by: Morgan Funtowicz <morgan@huggingface.co>
Co-authored-by: MOI Anthony <xn1t0x@gmail.com>
