transformers/tests
David LaPalomento b45cf0e90a
Guard against unset resolved_archive_file (#35628)
* archive_file may not be specified
When loading a pre-trained model from a gguf file, resolved_archive_file may not be set. Guard against that case in the safetensors availability check.

* Remap partial disk offload to cpu for GGUF files
GGUF files don't support disk offload so attempt to remap them to the CPU when device_map is auto. If device_map is anything else but None, raise a NotImplementedError.

* Don't remap auto device_map and raise RuntimeError
If device_map=auto and modules are selected for disk offload, don't attempt to map them to any other device. Raise a runtime error when a GGUF model is configured to map any modules to disk.

---------

Co-authored-by: Marc Sun <57196510+SunMarc@users.noreply.github.com>
2025-02-14 14:44:31 +01:00
..
agents use torch.testing.assertclose instead to get more details about error in cis (#35659) 2025-01-24 16:55:28 +01:00
bettertransformer use torch.testing.assertclose instead to get more details about error in cis (#35659) 2025-01-24 16:55:28 +01:00
deepspeed DeepSpeed github repo move sync (#36021) 2025-02-05 08:19:31 -08:00
extended [tests] skip tests for xpu (#33553) 2024-09-19 19:28:04 +01:00
fixtures
fsdp [tests] make cuda-only tests device-agnostic (#35607) 2025-01-13 14:48:39 +01:00
generation CI: avoid human error, automatically infer generative models (#33212) 2025-02-13 16:27:11 +01:00
models Revert qwen2 breaking changes related to attention refactor (#36162) 2025-02-14 13:44:14 +01:00
optimization Support constant lr with cooldown (#35453) 2025-02-10 13:21:55 +01:00
peft_integration use torch.testing.assertclose instead to get more details about error in cis (#35659) 2025-01-24 16:55:28 +01:00
pipelines docs: fix return type annotation of get_default_model_revision (#35982) 2025-02-13 11:59:15 +01:00
quantization Guard against unset resolved_archive_file (#35628) 2025-02-14 14:44:31 +01:00
repo_utils [Modular] skip modular checks based on diff (#36130) 2025-02-13 12:53:21 +00:00
sagemaker Trainer - deprecate tokenizer for processing_class (#32385) 2024-10-02 14:08:46 +01:00
tokenization Fix PretrainedTokenizerFast check => Fix PretrainedTokenizerFast Save (#35835) 2025-02-13 12:00:33 +01:00
tp Update-tp test (#35844) 2025-02-03 09:37:02 +01:00
trainer Move DataCollatorForMultipleChoice from the docs to the package (#34763) 2025-02-13 12:01:28 +01:00
utils Fix make_batched_videos and add tests (#36143) 2025-02-13 17:14:30 -05:00
__init__.py
test_backbone_common.py
test_configuration_common.py Load sub-configs from composite configs (#34410) 2024-11-05 11:34:01 +01:00
test_feature_extraction_common.py
test_image_processing_common.py Refactoring of ImageProcessorFast (#35069) 2025-02-04 17:52:31 -05:00
test_image_transforms.py
test_modeling_common.py CI: avoid human error, automatically infer generative models (#33212) 2025-02-13 16:27:11 +01:00
test_modeling_flax_common.py CI: avoid human error, automatically infer generative models (#33212) 2025-02-13 16:27:11 +01:00
test_modeling_tf_common.py 🚨All attention refactor🚨 (#35235) 2024-12-18 16:53:39 +01:00
test_pipeline_mixin.py Add image text to text pipeline (#34170) 2024-10-31 15:48:11 -04:00
test_processing_common.py Prepare processors for VideoLLMs (#36149) 2025-02-14 11:34:08 +01:00
test_sequence_feature_extraction_common.py
test_tokenization_common.py apply_chat_template: consistent behaviour for return_assistant_tokens_mask=True return_tensors=True (#35582) 2025-02-04 10:27:52 +01:00
test_training_args.py CI: fix test-save-trainer (#36191) 2025-02-14 10:20:56 +01:00