Thibault Douzon 4e441e529c fix LayoutLMv3TokenizerFast subword label after 'Ġ' token (#21695) ... LayoutLMv3TokenizerFast produces empty 'Ġ' token with `offset_mapping = (0, 0)`. Next token is wrongly assumed to also be beginning of word and isn't correctly assigned `pad_token_label`. Modify test with text that produce 'Ġ' token. Remove copy check from LayoutLMv2TokenizerFast for `_batch_encode_plus`. solves issue: #19978		2023-04-03 10:32:36 -04:00
..
benchmark	[Test refactor 1/5] Per-folder tests reorganization (#15725)	2022-02-23 15:46:28 -05:00
deepspeed	[deepspeed] offload + non-cpuadam optimizer exception (#22043)	2023-03-09 08:12:57 -08:00
extended	Apply ruff flake8-comprehensions (#21694)	2023-02-22 09:14:54 +01:00
fixtures	[WIP] add SpeechT5 model (#18922)	2023-02-03 12:43:46 -05:00
generation	Generate: TextIteratorStreamer (streamer for gradio) (#22501)	2023-04-03 15:04:37 +01:00
mixed_int8	[bnb] fix bnb failing test (#22439)	2023-03-29 15:13:00 +02:00
models	fix LayoutLMv3TokenizerFast subword label after 'Ġ' token (#21695)	2023-04-03 10:32:36 -04:00
onnx	Time to Say Goodbye, torch 1.7 and 1.8 (#22291)	2023-03-21 19:22:01 +01:00
optimization	Make schedulers picklable by making lr_lambda fns global (#21768)	2023-03-02 12:08:43 -05:00
pipelines	Really fix quality due to ruff release	2023-03-22 20:56:22 -04:00
repo_utils	Test fetch v2 (#22367)	2023-03-31 16:18:43 -04:00
sagemaker	Apply ruff flake8-comprehensions (#21694)	2023-02-22 09:14:54 +01:00
tokenization	Update quality tooling for formatting (#21480)	2023-02-06 18:10:56 -05:00
trainer	🔥py38 + torch 2 🔥🔥🔥🚀 (#22204)	2023-03-16 22:59:23 +01:00
utils	Use real tokenizers if tiny version(s) creation has issue(s) (#22428)	2023-03-29 16:16:23 +02:00
__init__.py	GPU text generation: mMoved the encoded_prompt to correct device	2020-01-06 15:11:12 +01:00
test_configuration_common.py	Remove set_access_token usage + fail tests if FutureWarning (#22051)	2023-03-09 09:23:48 -05:00
test_feature_extraction_common.py	Remove set_access_token usage + fail tests if FutureWarning (#22051)	2023-03-09 09:23:48 -05:00
test_image_processing_common.py	Remove set_access_token usage + fail tests if FutureWarning (#22051)	2023-03-09 09:23:48 -05:00
test_image_transforms.py	Rescale image back if it was scaled during PIL conversion (#22458)	2023-03-30 11:29:11 +01:00
test_modeling_common.py	Making sure we can use safetensors to serialize all the time. (#22437)	2023-03-31 16:07:35 +02:00
test_modeling_flax_common.py	Remove set_access_token usage + fail tests if FutureWarning (#22051)	2023-03-09 09:23:48 -05:00
test_modeling_tf_common.py	Remove set_access_token usage + fail tests if FutureWarning (#22051)	2023-03-09 09:23:48 -05:00
test_pipeline_mixin.py	Automatically create/update tiny models (#22275)	2023-03-23 19:14:17 +01:00
test_sequence_feature_extraction_common.py	Apply ruff flake8-comprehensions (#21694)	2023-02-22 09:14:54 +01:00
test_tokenization_common.py	Fix llama tokenizer (#22402)	2023-04-03 09:07:32 -04:00