transformers/utils
NielsRogge ace74d16bd
Add Nougat (#25942)
* Add conversion script

* Add NougatImageProcessor

* Add crop margin

* More improvements

* Add docs, READMEs

* Remove print statements

* Include model_max_length

* Add NougatTokenizerFast

* Fix imports

* Improve postprocessing

* Improve image processor

* Fix image processor

* Improve normalize method

* More improvements

* More improvements

* Add processor, improve docs

* Simplify fast tokenizer

* Remove test file

* Fix docstrings

* Use NougatProcessor in conversion script

* Add is_levensthein_available

* Add tokenizer tests

* More improvements

* Use numpy instead of opencv

* Add is_cv2_available

* Fix cv2_available

* Add is_nltk_available

* Add image processor tests, improve crop_margin

* Add integration tests

* Improve integration test

* Use do_rescale instead of hacks, thanks Amy

* Remove random_padding

* Address comments

* Address more comments

* Add import

* Address more comments

* Address more comments

* Address comment

* Address comment

* Set max_model_input_sizes

* Add tests

* Add requires_backends

* Add Nougat to exotic tests

* Use to_pil_image

* Address comment regarding nltk

* Add NLTK

* Improve variable names, integration test

* Add test

* refactor, document, and test regexes

* remove named capture groups, add comments

* format

* add non-markdown fixed tokenization

* format

* correct flakyness of args parse

* add regex comments

* test functionalities for crop_image, align long axis and expected output

* add regex tests

* remove cv2 dependency

* test crop_margin equality between cv2 and python

* refactor table regexes to markdown

add newline

* change print to log, improve doc

* fix high count tables correction

* address PR comments: naming, linting, asserts

* Address comments

* Add copied from

* Update conversion script

* Update conversion script to convert both small and base versions

* Add inference example

* Add more info

* Fix style

* Add require annotators to test

* Define all keyword arguments explicitly

* Move cv2 annotator

* Add tokenizer init method

* Transfer checkpoints

* Add reference to Donut

* Address comments

* Skip test

* Remove cv2 method

* Add copied from statements

* Use cached_property

* Fix docstring

* Add file to not doctested

---------

Co-authored-by: Pablo Montalvo <pablo.montalvo.leroux@gmail.com>
2023-09-26 07:06:04 +02:00
..
test_module AutoImageProcessor (#20111) 2022-11-08 19:54:41 +00:00
tf_ops Check TF ops for ONNX compliance (#10025) 2021-02-15 07:55:10 -05:00
add_pipeline_model_mapping_to_test.py A script to add/update pipeline_model_mapping systematically (#22180) 2023-04-06 18:08:14 +02:00
check_build.py Clean up CUDA kernels (#23455) 2023-05-18 14:14:43 -04:00
check_config_attributes.py add VITS model (#24085) 2023-09-01 10:50:06 +01:00
check_config_docstrings.py [Check] Fix config docstring (#26222) 2023-09-18 19:58:01 +02:00
check_copies.py Doc checks (#25408) 2023-08-10 10:53:22 +02:00
check_doc_toc.py Doc checks (#25408) 2023-08-10 10:53:22 +02:00
check_doctest_list.py Run doctest for new files (#25588) 2023-08-21 11:08:38 +02:00
check_dummies.py Doc checks (#25408) 2023-08-10 10:53:22 +02:00
check_inits.py Doc checks (#25408) 2023-08-10 10:53:22 +02:00
check_model_tester.py Add a new script to check model testers' config (#22063) 2023-03-13 19:11:19 +01:00
check_repo.py Add ViTMatte (#25843) 2023-09-19 10:56:10 -03:00
check_self_hosted_runner.py Tiny fix for check_self_hosted_runner.py (#24052) 2023-06-06 18:17:41 +02:00
check_table.py [docs] removed MaskFormerSwin and TimmBackbone from the table on index.md (#26347) 2023-09-25 09:41:59 -04:00
check_task_guides.py More utils doc (#25457) 2023-08-17 07:58:35 +02:00
check_tf_ops.py Check TF ops for ONNX compliance (#10025) 2021-02-15 07:55:10 -05:00
create_dummy_models.py Make bark could have tiny model (#25290) 2023-08-04 15:13:14 +02:00
custom_init_isort.py More utils doc (#25457) 2023-08-17 07:58:35 +02:00
download_glue_data.py Raise exceptions instead of asserts (#13907) 2021-10-07 12:44:23 +05:30
extract_warnings.py Make Slack CI reporting stronger (#21823) 2023-02-28 17:12:44 +01:00
get_ci_error_statistics.py Show diff between 2 CI runs on Slack reports (#22798) 2023-04-19 19:27:37 +02:00
get_github_job_time.py Make Slack CI reporting stronger (#21823) 2023-02-28 17:12:44 +01:00
get_modified_files.py exclude deleted files in the fixup script (#21436) 2023-02-03 12:57:02 -05:00
get_previous_daily_ci.py Fix a minor bug in CI slack report (#22906) 2023-04-21 20:36:35 +02:00
get_test_info.py Add an utility file to get information from test files (#21856) 2023-03-01 17:53:29 +01:00
not_doctested.txt Add Nougat (#25942) 2023-09-26 07:06:04 +02:00
notification_service_doc_tests.py Migrate doc files to Markdown. (#24376) 2023-06-20 18:07:47 -04:00
notification_service.py Add the number of model test failures to slack CI report (#24207) 2023-06-12 21:27:10 +02:00
past_ci_versions.py (Re-)Enable Nightly + Past CI (#22393) 2023-03-30 21:06:35 +02:00
print_env.py Print more library versions in CI (#17384) 2022-06-02 10:24:16 +02:00
release.py More utils doc (#25457) 2023-08-17 07:58:35 +02:00
slow_documentation_tests.txt moved ctrl to Salesforce/ctrl (#26183) 2023-09-18 13:52:43 +02:00
sort_auto_mappings.py More utils doc (#25457) 2023-08-17 07:58:35 +02:00
tests_fetcher.py Tests: detect lines removed from "utils/not_doctested.txt" and doctest ALL generation files (#25763) 2023-08-29 16:15:05 +01:00
update_metadata.py Add image to image pipeline (#25393) 2023-09-22 19:53:55 +03:00
update_tiny_models.py Update tiny model summary file for recent models (#22637) 2023-04-06 22:52:59 +02:00