* fix(Wav2Vec2ForCTC): torch export
Resolves the issue described in #34022 by implementing the
masking of the hidden states using an elementwise multiplication
rather than indexing with assignment.
The torch.export functionality seems to mark the tensor as frozen
even though the update is legal.
This change is a workaround for now to allow the export of the
model as a FxGraph. Further investigation is required to find
the real solution in pytorch.
* [run-slow] hubert, unispeech, unispeech_sat, wav2vec2
* change cpu offload warning for fp8 quantization
* change cpu offload warning for fp4 quantization
* change cpu offload variable name for fp8 and fp4 quantization
Update 'trainer._get_eval_sampler()' to support 'group_by_length' argument
Trainer didn't support grouping by length for evaluation, which made evaluation slow with 'eval_batch_size'>1.
Updated 'trainer._get_eval_sampler()' method was based off of 'trainer._get_train_sampler()'.
* auto-gptq requirement is removed & model is changed & tokenizer pad token is assigned
* values func is changed with extensions & sequence key value bug is fixed
* map key value check is added in ExtensionsTree
* empty trimmed_ids bug is fixed
* tail_id IndexError is fixed
* empty trimmed_ids bug fix is updated for failed test
* too much specific case for specific tokenizer is removed
* input_ids check is updated
* require auto-gptq import is removed
* key error check is changed with empty list check
* empty input_ids check is added
* empty trimmed_ids fix is checked with numel function
* usage change comments are added
* test changes are commented
* comment style and quality bugs are fixed
* test comment style and quality bug is fixed
* Fix FSDP Initialization for resume training
* Added init_fsdp function to work with dummy values
* Fix FSDP initialization for resuming training
* Added CUDA decorator for tests
* Added torch_gpu decorator to FSDP tests
* Fixup for failing code quality tests
* add idefics
* conflicts after merging main
* enable tests but need to fix some
* fix tests
* no print
* fix/skip some slow tests
* continue not skip
* rebasing broken smth, this is the fix
* mistral qna start
* mixtral qna
* oops
* qwen2 qna
* qwen2moe qna
* add missing input embed methods
* add copied to all methods, can't directly from llama due to the prefix
* make top level copied from
* refactor: benchmarks
Based on a discussion with @LysandreJik & @ArthurZucker, the goal of
this PR is to improve transformers' benchmark system.
This is a WIP, for the moment the infrastructure required to make things
work is not ready. Will update the PR description when it is the case.
* feat: add db init in benchmarks CI
* fix: pg_config is missing in runner
* fix: add psql to the runner
* fix: connect info from env vars + PR comments
* refactor: set database as env var
* fix: invalid working directory
* fix: `commit_msg` -> `commit_message`
* fix: git marking checked out repo as unsafe
* feat: add logging
* fix: invalid device
* feat: update grafana dashboard for prod grafana
* feat: add `commit_id` to header table
* feat: commit latest version of dashboard
* feat: move measurements into json field
* feat: remove drop table migration queries
* fix: `torch.arrange` -> `torch.arange`
* fix: add missing `s` to `cache_position` positional argument
* fix: change model
* revert: `cache_positions` -> `cache_position`
* fix: set device for `StaticCache`
* fix: set `StaticCache` dtype
* feat: limit max cache len
* fix script
* raise error on failure!
* not try catch
* try to skip generate compilation
* update
* update docker image!
* update
* update again!@
* update
* updates
* ???
* ??
* use `torch.cuda.synchronize()`
* fix json
* nits
* fix
* fixed!
* f**k
* feat: add TTNT panels
* feat: add try except
---------
Co-authored-by: Arthur Zucker <arthur.zucker@gmail.com>
* Generate using exported model and enable gemma2-2b in ExecuTorch
* [run_slow] gemma, gemma2
* truncate expected output message
* Bump required torch version to support gemma2 export
* [run_slow] gemma, gemma2
---------
Co-authored-by: Guang Yang <guangyang@fb.com>
* Default synced_gpus to True when using FullyShardedDataParallel
Fixes#30228
Related:
* https://github.com/pytorch/pytorch/issues/100069
* https://github.com/pytorch/pytorch/issues/123962
Similar to DeepSpeed ZeRO Stage 3, when using FSDP with multiple GPUs and differently sized data per rank, the ranks reach different synchronization points at the same time, leading to deadlock
To avoid this, we can automatically set synced_gpus to True if we detect that a PreTrainedModel is being managed by FSDP using _is_fsdp_managed_module, which was added in 2.0.0 for torch.compile: https://github.com/pytorch/pytorch/blob/v2.0.0/torch/distributed/fsdp/_dynamo_utils.py
* Remove test file
* ruff formatting
* ruff format
* Update copyright year
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>
* Add test for FSDP-wrapped model generation
Before #33483, these tests would have hung for 10 minutes before crashing due to a timeout error
* Ruff format
* Move argparse import
* Remove barrier
I think this might cause more problems if one of the workers was killed
* Move import into function to decrease load time
https://github.com/huggingface/transformers/pull/33483#discussion_r1787972735
* Add test for accelerate and Trainer
https://github.com/huggingface/transformers/pull/33483#discussion_r1790309675
* Refactor imports
* Ruff format
* Use nullcontext
---------
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>
