bastrob
|
464d986b6c
|
Add missing Flaubert tokenizer tests (#30492)
* add flaubert tokenization test, enrich inheritance in FlaubertTokenizer.
* fix quality code ci
* ensure parameter consistency
* fix ci
* fix copyright year and flatten vocab list.
* fix style
|
2024-06-05 13:52:16 +02:00 |
|