transformers/tests
amyeroberts 49e44b216b
Update feature extractor methods to enable type cast before normalize (#18499)
* Update methods to optionally rescale
This is necessary to allow for casting our images / videos to numpy arrays within the feature extractors' call. We want to do this to make sure the behaviour is as expected when flags like  are False. If some transformations aren't applied, then the output type can't be unexpected e.g. a list of PIL images instead of numpy arrays.

* Cast images to numpy arrays in call to enable consistent behaviour with different configs

* Remove accidental clip changes

* Update tests to reflect the scaling logic
We write a generic  function to handle rescaling of our arrays. In order for the API to be intuitive, we take some factor c and rescale the image values by that. This means, the rescaling done in normalize and to_numpy_array are now done with array * (1/255) instead of array / 255. This leads to small differences in the resulting image. When testing, this was in the order of 1e-8, and so deemed OK
2022-08-17 19:57:07 +01:00
..
benchmark [Test refactor 1/5] Per-folder tests reorganization (#15725) 2022-02-23 15:46:28 -05:00
deepspeed deprecate is_torch_bf16_available (#17738) 2022-06-20 08:40:11 -04:00
extended Update self-push workflow (#17177) 2022-05-13 16:28:00 +02:00
fixtures add a warning in SpmConverter for sentencepiece's model using the byte fallback feature (#16629) 2022-04-11 11:06:10 +02:00
generation Generate: validate model_kwargs (and catch typos in generate arguments) (#18261) 2022-08-12 14:53:51 +01:00
mixed_int8 [bnb] Minor modifications (#18631) 2022-08-17 00:48:10 +02:00
models Update BLOOM parameter counts (#18531) 2022-08-12 19:36:18 +02:00
onnx Fix Yolos ONNX export test (#18606) 2022-08-17 10:04:49 +02:00
optimization [Test refactor 1/5] Per-folder tests reorganization (#15725) 2022-02-23 15:46:28 -05:00
pipelines Use commit hash to look in cache instead of calling head (#18534) 2022-08-10 11:55:18 -04:00
sagemaker transformers-cli login => huggingface-cli login (#18490) 2022-08-06 09:42:55 +02:00
tokenization fix train_new_from_iterator in the case of byte-level tokenizers (#17549) 2022-06-08 15:30:41 +02:00
trainer small change (#18584) 2022-08-12 20:04:38 +02:00
utils Update feature extractor methods to enable type cast before normalize (#18499) 2022-08-17 19:57:07 +01:00
__init__.py GPU text generation: mMoved the encoded_prompt to correct device 2020-01-06 15:11:12 +01:00
test_configuration_common.py Use commit hash to look in cache instead of calling head (#18534) 2022-08-10 11:55:18 -04:00
test_feature_extraction_common.py Use new huggingface_hub tools for download models (#18438) 2022-08-05 10:12:40 -04:00
test_modeling_common.py Use new huggingface_hub tools for download models (#18438) 2022-08-05 10:12:40 -04:00
test_modeling_flax_common.py Load sharded pt to flax (#18419) 2022-08-12 09:48:10 +02:00
test_modeling_tf_common.py Use new huggingface_hub tools for download models (#18438) 2022-08-05 10:12:40 -04:00
test_sequence_feature_extraction_common.py Some tests misusing assertTrue for comparisons fix (#16771) 2022-04-19 14:44:08 +02:00
test_tokenization_common.py Use new huggingface_hub tools for download models (#18438) 2022-08-05 10:12:40 -04:00