transformers/tests
Sourab Mangrulkar a73b1d59a3
accelerate deepspeed and gradient accumulation integrate (#23236)
* mixed precision support via accelerate

* fix issues

* fix for the sharded ddp case

* fix flax and tf failing tests

* `refactor the place to create `Accelerator` object

* move ddp prep to accelerate

* fix 😅

* resolving comments

* move fsdp handling to accelerate

* fixex

* fix saving

* shift torch dynamo handling to accelerate

* shift deepspeed integration and save & load utils to accelerate

* fix accelerate launcher support

* oops

* fix 🐛

* save ckpt fix

* Trigger CI

* nasty 🐛 😅

* as deepspeed needs grad_acc fixes, transfer grad_acc to accelerate

* make tests happy

* quality 

* loss tracked needs to account for grad_acc

* fixing the deepspeed tests

* quality 

* 😅😅😅

* tests 😡

* quality 

* Trigger CI

* resolve comments and fix the issue with the previous merge from branch

* Trigger CI

* accelerate took over deepspeed integration

---------

Co-authored-by: Stas Bekman <stas@stason.org>
2023-05-31 15:16:22 +05:30
..
benchmark [Test refactor 1/5] Per-folder tests reorganization (#15725) 2022-02-23 15:46:28 -05:00
bettertransformer Add methods to PreTrainedModel to use PyTorch's BetterTransformer (#21259) 2023-04-27 11:03:42 +02:00
bitsandbytes 4-bit QLoRA via bitsandbytes (4-bit base model + LoRA) (#23479) 2023-05-24 12:52:45 +02:00
deepspeed accelerate deepspeed and gradient accumulation integrate (#23236) 2023-05-31 15:16:22 +05:30
extended [tests] switch to torchrun (#22712) 2023-04-12 08:25:45 -07:00
fixtures [WIP] add SpeechT5 model (#18922) 2023-02-03 12:43:46 -05:00
generation Better TF docstring types (#23477) 2023-05-24 13:52:52 +01:00
models Add TensorFlow implementation of EfficientFormer (#22620) 2023-05-31 10:43:12 +01:00
onnx Fix issue introduced in PR #23163 (#23363) 2023-05-15 11:38:44 +02:00
optimization Make schedulers picklable by making lr_lambda fns global (#21768) 2023-03-02 12:08:43 -05:00
pipelines [image-to-text pipeline] Add conditional text support + GIT (#23362) 2023-05-22 21:45:50 +02:00
repo_utils Revamp test selection for the example tests (#23737) 2023-05-25 09:38:21 -04:00
sagemaker Avoid invalid escape sequences, use raw strings (#22936) 2023-04-25 09:17:56 -04:00
tokenization Update quality tooling for formatting (#21480) 2023-02-06 18:10:56 -05:00
tools Fix image segmentation tool test (#23306) 2023-05-11 14:38:11 -04:00
trainer Paged Optimizer + Lion Optimizer for Trainer (#23217) 2023-05-24 12:53:28 +02:00
utils Fix the regex in get_imports to support multiline try blocks and excepts with specific exception types (#23725) 2023-05-24 15:40:19 -04:00
__init__.py GPU text generation: mMoved the encoded_prompt to correct device 2020-01-06 15:11:12 +01:00
test_backbone_common.py Add methods to update and verify out_features out_indices (#23031) 2023-05-04 10:15:06 +01:00
test_configuration_common.py Add RWKV-4 (#22797) 2023-05-09 13:04:10 -04:00
test_feature_extraction_common.py Prepare tests for hfh 0.14 (#22958) 2023-04-24 09:31:50 -04:00
test_image_processing_common.py Prepare tests for hfh 0.14 (#22958) 2023-04-24 09:31:50 -04:00
test_image_transforms.py Rescale image back if it was scaled during PIL conversion (#22458) 2023-03-30 11:29:11 +01:00
test_modeling_common.py [AutoModel] fix torch_dtype=auto in from_pretrained (#23379) 2023-05-16 10:21:42 -07:00
test_modeling_flax_common.py Fix bigbird random attention (#21023) 2023-04-27 13:52:28 -04:00
test_modeling_tf_common.py Overhaul TF serving signatures + dummy inputs (#23234) 2023-05-24 17:03:24 +01:00
test_pipeline_mixin.py Update tiny models and pipeline tests (#23446) 2023-05-18 17:29:04 +02:00
test_sequence_feature_extraction_common.py Apply ruff flake8-comprehensions (#21694) 2023-02-22 09:14:54 +01:00
test_tokenization_common.py chore: allow protobuf 3.20.3 requirement (#22759) 2023-05-10 20:22:56 +02:00