transformers/tests/trainer
Mahdi Baghbanzadeh c61fcde910
Enhance DataCollatorForLanguageModeling with Configurable Token Replacement Probabilities (#35251)
* DataCollatorForLanguageModeling class was updated with new parameters that provides more control over the token masking and relacing

* DataCollatorForLanguageModeling class was updated with new parameters that provides more control over the token masking and relacing

* Addressed review comments, modified the docstring and made a test for the DataCollatorForLanguageModeling
2025-01-14 17:01:10 +00:00
..
__init__.py [Test refactor 1/5] Per-folder tests reorganization (#15725) 2022-02-23 15:46:28 -05:00
test_data_collator.py Enhance DataCollatorForLanguageModeling with Configurable Token Replacement Probabilities (#35251) 2025-01-14 17:01:10 +00:00
test_trainer_callback.py add a callback hook right before the optimizer step (#33444) 2024-09-13 10:43:45 +02:00
test_trainer_distributed.py CI: update to ROCm 6.0.2 and test MI300 (#30266) 2024-05-13 18:14:36 +02:00
test_trainer_fsdp.py Remove FSDP wrapping from sub-models. (#34452) 2024-11-15 23:00:03 +01:00
test_trainer_seq2seq.py Trainer - deprecate tokenizer for processing_class (#32385) 2024-10-02 14:08:46 +01:00
test_trainer_tpu.py [Test refactor 1/5] Per-folder tests reorganization (#15725) 2022-02-23 15:46:28 -05:00
test_trainer_utils.py Add strategy to store results in evaluation loop (#30267) 2024-04-17 12:42:27 +01:00
test_trainer.py [tests] make cuda-only tests device-agnostic (#35607) 2025-01-13 14:48:39 +01:00