5 Commits

Author	SHA1	Message	Date
Anton Vlasjuk	71c1985069	Immutability for data collators (#30603) ... * immutability fix for seq2seq as well as immutability tests for the collators * ensure we don't act on none labels and formatting * remove tf/pt in respective tests as they are not required * more type error fixes tf/np * remove todo * apply suggestions from code review * formatting / style	2024-05-08 17:54:49 +01:00
Anton Vlasjuk	9112520b15	Fix seq2seq collator padding (#30556) ... * fix seq2seq data collator to respect the given padding strategy further added tests for the seq2seq data collator in the style of the `data_collator_for_token_classification` (pt, tf, np) * formatting and change bool equals "==" to "is" * add missed return types in tests * update numpy test as it can handle unequal shapes, not like pt or tf	2024-04-30 18:32:30 +01:00
Dean Wyatte	2f4cdd97f5	handle numpy inputs in whole word mask data collator (#22032)	2023-03-10 10:50:29 -05:00
Alexander Markov	610acc5ae9	Data collator for token classification pads labels column when receives pytorch tensors (#20244) ... * token cls data_collator pads labels column * remove walrus operator for code quality * remove redundat space * remove comment that was fixed * PR comments fix Co-authored-by: Alexander Markov <amarkov.me@gmail.com>	2022-11-16 12:18:46 -05:00
Lysandre Debut	29c10a41d0	[Test refactor 1/5] Per-folder tests reorganization (#15725) ... * Per-folder tests reorganization Co-authored-by: sgugger <sylvain.gugger@gmail.com> Co-authored-by: Stas Bekman <stas@stason.org>	2022-02-23 15:46:28 -05:00