théo gigant fed27ffc7e Adding FlaxNoRepeatNGramLogitsProcessor (#29677) ... * fix issue with logit processor in beam search in Flax * adding FlaxNoRepeatNGramLogitsProcessor class + unit test * style correction and code verification * add FlaxNoRepeatNGramLogitsProcessor to the test_processor_list and test_processor_list_jitted tests * fix an issue where ngrams are banned only if they appear ==1 time + update description of get_previous_ngrams * replace non-jit compatible masking of ngrams that are not yet generated with jittable version * Revert "fix issue with logit processor in beam search in Flax" This reverts commit 09b70d7e4d. * add FlaxNoRepeatNGramLogitsProcessor to _get_logits_processor * change the method of casting to boolean of banned tokens indices * fix code style * remove some useless operations + significantly faster computation of update indices using jax.lax.fori_loop * remove useless loop iterations * set some variables that were calculated and used multiple times * fix format		2024-04-02 11:39:33 +02:00
..
__init__.py	[Test refactor 1/5] Per-folder tests reorganization (#15725)	2022-02-23 15:46:28 -05:00
test_beam_constraints.py	Generate: move generation_*.py src files into generation/*.py (#20096)	2022-11-09 15:34:08 +00:00
test_beam_search.py	Time to Say Goodbye, torch 1.7 and 1.8 (#22291)	2023-03-21 19:22:01 +01:00
test_configuration_utils.py	Generate: get generation mode from the generation config instance 🧼 (#29441)	2024-03-06 11:18:35 +00:00
test_flax_logits_process.py	Adding FlaxNoRepeatNGramLogitsProcessor (#29677)	2024-04-02 11:39:33 +02:00
test_flax_utils.py	Add support for beam search's num_return_sequencs flag in flax (#23082)	2023-05-03 10:50:34 -04:00
test_framework_agnostic.py	Update all references to canonical models (#29001)	2024-02-16 08:16:58 +01:00
test_logits_process.py	Change in-place operations to out-of-place in LogitsProcessors (#29680)	2024-03-21 16:37:33 +00:00
test_stopping_criteria.py	Move eos_token_id to stopping criteria (#29459)	2024-03-27 12:18:10 +00:00
test_streamers.py	Update all references to canonical models (#29001)	2024-02-16 08:16:58 +01:00
test_tf_logits_process.py	Better TF docstring types (#23477)	2023-05-24 13:52:52 +01:00
test_tf_utils.py	v4.39 deprecations 🧼 (#29492)	2024-03-07 10:44:43 +00:00
test_utils.py	[generate] fix breaking change for patch (#29976)	2024-04-02 09:51:45 +02:00