riaz.somc/transformers
1
0
Fork 0
mirror of https://github.com/huggingface/transformers.git synced 2025-07-19 20:48:22 +06:00
Code Issues Actions 18 Packages Projects Releases Wiki Activity
18,411 Commits 876 Branches 227 Tags 2 GiB
1fcaad6df9
Commit Graph

2 Commits

Author SHA1 Message Date
Yih-Dar
1fcaad6df9
Use lru_cache for tokenization tests (#36818) 
* fix

* fix

* fix

* fix

---------

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2025-03-28 15:09:35 +01:00
bastrob
464d986b6c
Add missing Flaubert tokenizer tests (#30492) 
* add flaubert tokenization test, enrich inheritance in FlaubertTokenizer.

* fix quality code ci

* ensure parameter consistency

* fix ci

* fix copyright year and flatten vocab list.

* fix style
 2024-06-05 13:52:16 +02:00