4 Commits

Author	SHA1	Message	Date
davidleonfdez	9f8bfe703c	Fix #16660 (tokenizers setters of ids of special tokens) (#16661) ... * Fix setters of *_token_id properties of SpecialTokensMixin * Test setters of common tokens ids * Move to a separate test checks of setters of tokens ids * Add independent test for ByT5 * Add Canine test * Test speech to text	2022-04-13 07:49:06 -04:00
SaulLu	be9474bd35	add a test checking the format of convert_tokens_to_string's output (#16540) ... * add new tests * add comment to overridden tests	2022-04-04 16:57:24 +02:00
Sylvain Gugger	4975002df5	Reorganize file utils (#16264) ... * Split file_utils in several submodules * Fixes * Add back more objects * More fixes * Who exactly decided to import that from there? * Second suggestion to code with code review * Revert wront move * Fix imports * Adapt all imports * Adapt all imports everywhere * Revert this import, will fix in a separate commit	2022-03-23 10:26:33 -04:00
Lysandre Debut	29c10a41d0	[Test refactor 1/5] Per-folder tests reorganization (#15725) ... * Per-folder tests reorganization Co-authored-by: sgugger <sylvain.gugger@gmail.com> Co-authored-by: Stas Bekman <stas@stason.org>	2022-02-23 15:46:28 -05:00