Xiaoli Wang
239ace152b
Fix TypeError: Object of type int64 is not JSON serializable ( #24340 )
...
* Fix TypeError: Object of type int64 is not JSON serializable
* Convert numpy.float64 and numpy.int64 to float and int for json serialization
* Black reformatted examples/pytorch/token-classification/run_ner_no_trainer.py
* * make style
2023-06-27 12:15:49 +01:00
Sylvain Gugger
eb849f6604
Migrate doc files to Markdown. ( #24376 )
...
* Rename index.mdx to index.md
* With saved modifs
* Address review comment
* Treat all files
* .mdx -> .md
* Remove special char
* Update utils/tests_fetcher.py
Co-authored-by: Lysandre Debut <lysandre.debut@reseau.eseo.fr>
---------
Co-authored-by: Lysandre Debut <lysandre.debut@reseau.eseo.fr>
2023-06-20 18:07:47 -04:00
amyeroberts
a717e0318c
Add TimmBackbone model ( #22619 )
...
* Add test_backbone for convnext
* Add TimmBackbone model
* Add check for backbone type
* Tidying up - config checks
* Update convnextv2
* Tidy up
* Fix indices & clearer comment
* Exceptions for config checks
* Correclty update config for tests
* Safer imports
* Safer safer imports
* Fix where decorators go
* Update import logic and backbone tests
* More import fixes
* Fixup
* Only import all_models if torch available
* Fix kwarg updates in from_pretrained & main rebase
* Tidy up
* Add tests for AutoBackbone
* Tidy up
* Fix import error
* Fix up
* Install nattan in doc_test_job
* Revert back to setting self._out_xxx directly
* Bug fix - out_indices mapping from out_features
* Fix tests
* Dont accept output_loading_info for Timm models
* Set out_xxx and don't remap
* Use smaller checkpoint for test
* Don't remap timm indices - check out_indices based on stage names
* Skip test as it's n/a
* Apply suggestions from code review
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* Cleaner imports / spelling is hard
---------
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
2023-06-06 17:11:30 +01:00
Arthur
474bf508df
Add Segment Anything Model (SAM) ( #22654 )
...
* initial commit
* keys match
* update, fix conversion
* fixes, inference working
* fix
* more fixes
* more fixes
* clean up
* more clean up
* fix copies and add convext copied layer norm
* stash
* pretty big upfate
* cleaning
* more cleaning
* fixup stuffs
* fix copies
* fix iinit
* update test removing tokenizer
* nits
* add pretrained
* more nits
* remove tracking of pipeline
* few fixes
* update san and conversion script
* fix mask decoder and prompt encoder conversion
* fixes
* small update
* fix order
* fix
* fix image embeddings
* nites
* few fixes
* fix logits
* clean up
* fixes boxes inference
* v1 AMG
* clean up
* some clean up
* multi points support
* amg working
* fixup
* clean up
* readme
* update toctree
* fix type hint
* multiple fixes
* fixup
* fixes
* updates
* updates
* more tests
* few fixes
* change to `SamForMaskGeneration`
* doc
* fixup
* fix more tests
* multiple fixes
* fix CI tests
* refactor processor
* renamings
* draft the pipeline
* refactor
* fix tests
* fix test
* few cleanings
* fix test
* edit pipelien support chunking
* udate
* add slow tests
* fix nit
* fixup
* fix nit
* current chunk pipleine
* cast boxes in fp32
* nit
* current updates
* piepleine works
* fixup
* clean up config
* fix slow tests
* fix slow tests
* clean up
* update doc and pipeline
* adds more slow tests
* fix slow tests
* cleaning
* tests pass
* add docstring
* fix copies
* clean up
* support batch of images
* style
* dummy is needed, add tests
* fix slow tests
* fix CI
* update
* adds more tests
* fixes
* fixes
* fixup
* fixes
* few fixes
* filter
* few fixes
* some refactor
* touches finales
* fix
* style
* remove pipeline files
* fixes nits
* revert pipeline changes
* fix test
* fixup
* remove automodel for automatic mask generation
* fix failing torch tests
* update mdx
* revert removal of `MODEL_FOR_AUTOMATIC_MASK_GENERATION_MAPPING`
* update sam config based on review
Co-authored-by: amyeroberts <aeroberts4444@gmail.com>
Co-authored-by: sgugger <sylvain.gugger@gmail.com>
* update low_resolution_masks -> pred_masks
inti ln with layer_norm_eps
add_decomposed_rel_pos doc
forward doc of SamForMaskGeneration
* update processor docstring
* remove image processor import empty
* update for testing
* output vision hidden states + clean recomm
also test all iou values
* fixup
* fixup
* remove unused
* Update src/transformers/models/sam/modeling_sam.py
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* Update src/transformers/models/sam/image_processing_sam.py
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* nits
* fix
* fix CI tests and slow tests
* replace with Amy's processor
* clearer docstring
* add `SamVisionNeck`
* refactor - all CI tests should pass
* fix broken import on Gcolab
* few fixes here and there
* fix another bug
* fix more bugs
* update and merge
* correct ckpt
* address comments
* add tips
* revert
* fix docstring
* replace with `SamModel`
* make fixup
* add support for bathed images and batch ed points
* make fixup this time, really
* make fixup again and again
* few fixes here and there, this should be the touche finale
* Update docs/source/en/model_doc/sam.mdx
* fixup
* correct checkpoints
* correct name
* rm unneeded file
* add notebook
---------
Co-authored-by: younesbelkada <younesbelkada@gmail.com>
Co-authored-by: amyeroberts <aeroberts4444@gmail.com>
Co-authored-by: sgugger <sylvain.gugger@gmail.com>
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
Co-authored-by: Younes Belkada <49240599+younesbelkada@users.noreply.github.com>
2023-04-19 21:01:49 +02:00
Aaron Gokaslan
5e8c8eb5ba
Apply ruff flake8-comprehensions ( #21694 )
2023-02-22 09:14:54 +01:00
Connor Henderson
0f96c26de6
refactor: Make direct_transformers_import util ( #21652 )
...
* refactor: Make direct_import util
* edit direct import fn
* add docstring
* make import function specific to transformers only
* edit doc string
2023-02-16 11:32:32 -05:00
Sylvain Gugger
9d1116e995
Update deprecated load_module ( #21651 )
2023-02-15 15:57:24 -05:00
Sylvain Gugger
6f79d26442
Update quality tooling for formatting ( #21480 )
...
* Result of black 23.1
* Update target to Python 3.7
* Switch flake8 to ruff
* Configure isort
* Configure isort
* Apply isort with line limit
* Put the right black version
* adapt black in check copies
* Fix copies
2023-02-06 18:10:56 -05:00
Arthur
d7b66d9b44
update template ( #20885 )
...
* update template
* replace redme entries
* make style
2023-01-04 10:15:45 +01:00
Younes Belkada
cab7799f7b
Add japanese translation of template ( #20870 )
...
* add japanese translation of template
* fix japanese translation
- fix special cases
- fix typos
- manually translate special cases
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>
2022-12-23 14:39:42 +01:00
Sourab Mangrulkar
4a4cd6cd02
having new model entries in Hindi for Hindi README ( #20869 )
2022-12-23 12:00:48 +05:30
Sourab Mangrulkar
73ec12eafb
README in Hindi 🇮🇳 ( #20097 )
...
* Created README_hd.md
A Hindi Translation for README
* updated check_copies.py
Added the Proper info for Hindi Translation of README File !
* updated README_hd.md
Fixed some translation issues !
* Update README_hd.md
* Update README_hd.md
* Update README_hd.md
* fixing 🐛 for `make fix-copies`
* run `make fix-copies`
* `make fix-copies` 😅
Co-authored-by: Akshit Gulyan <103456810+AkshitGulyan@users.noreply.github.com>
2022-12-06 01:04:40 +05:30
NielsRogge
6dc884abc8
[Maskformer] Add MaskFormerSwin backbone ( #20344 )
...
* First draft
* Fix backwards compatibility
* More fixes
* More fixes
* Make backbone more general
* Improve backbone
* Improve test
* Fix config checkpoint
* Address comments
* Use model_type
* Address more comments
* Fix special model names
* Remove MaskFormerSwinModel and MaskFormerSwinPreTrainedModel from main init
* Fix typo
* Update backbone
* Apply suggestion
Co-authored-by: Niels Rogge <nielsrogge@Nielss-MacBook-Pro.local>
2022-11-28 20:33:49 +01:00
Ikko Ashimine
38e5b71abb
Add Japanese translated README ( #19945 )
...
* Add japanese translated README.md
* Add README_ja.md link
* Add japanese transkate to check_copies.py
* Add guide to Japanese README.md
* Update README_ja.md
Co-authored-by: Younes Belkada <49240599+younesbelkada@users.noreply.github.com>
* Update utils/check_copies.py
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
Co-authored-by: Younes Belkada <49240599+younesbelkada@users.noreply.github.com>
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
2022-11-01 09:18:08 -04:00
Rak Alexey
1f7e40d04f
Improve check copies ( #19829 )
...
* print first diff line intead of first code part line
* fix style
2022-10-24 11:24:18 -04:00
Oussamaosman02
0b7b4c60c6
Adding the README_es.md and reference to it in the others files readme ( #19427 )
...
* Adding the README_es.md and reference to it in the others files readme
* Updating the check_copies.py
* Updating README_es.md
* Updating chec_copies
2022-10-11 12:56:25 -04:00
NielsRogge
2ab790e82d
Add Donut ( #18488 )
...
* First draft
* Improve script
* Update script
* Make conversion work
* Add final_layer_norm attribute to Swin's config
* Add DonutProcessor
* Convert more models
* Improve feature extractor and convert base models
* Fix bug
* Improve integration tests
* Improve integration tests and add model to README
* Add doc test
* Add feature extractor to docs
* Fix integration tests
* Remove register_buffer
* Fix toctree and add missing attribute
* Add DonutSwin
* Make conversion script work
* Improve conversion script
* Address comment
* Fix bug
* Fix another bug
* Remove deprecated method from docs
* Make Swin and Swinv2 untouched
* Fix code examples
* Fix processor
* Update model_type to donut-swin
* Add feature extractor tests, add token2json method, improve feature extractor
* Fix failing tests, remove integration test
* Add do_thumbnail for consistency
* Improve code examples
* Add code example for document parsing
* Add DonutSwin to MODEL_NAMES_MAPPING
* Add model to appropriate place in toctree
* Update namespace to appropriate organization
Co-authored-by: Niels Rogge <nielsrogge@Nielss-MacBook-Pro.local>
2022-08-12 16:40:58 +02:00
Sylvain Gugger
aeeab1ffd0
Fix template for new models in README ( #18182 )
2022-07-18 16:01:51 +02:00
Sylvain Gugger
7c046c5c22
Update localized READMES when template is filled. ( #18062 )
2022-07-08 11:08:52 -04:00
Patrick von Platen
7f14839f55
[Wav2Vec2Conformer] Official release ( #17709 )
...
* [Wav2Vec2Conformer] Official release
* remove from not-in-readme
2022-06-15 18:34:15 +02:00
Sylvain Gugger
048dd73bba
Check list of models in the main README and sort it ( #17517 )
...
* Script for README
* Fix copies
* Complete error message
2022-06-02 08:10:08 -04:00
Sylvain Gugger
bd908e9bb1
Fix README localizer script ( #17407 )
2022-05-25 07:23:40 -04:00
Sylvain Gugger
afe5d42d8d
Black preview ( #17217 )
...
* Black preview
* Fixup too!
* Fix check copies
* Use the same version as the CI
* Bump black
2022-05-12 16:25:55 -04:00
Sylvain Gugger
b9a768b3ff
Enable doc in Spanish ( #16518 )
...
* Reorganize doc for multilingual support
* Fix style
* Style
* Toc trees
* Adapt templates
2022-04-04 10:25:46 -04:00
Sylvain Gugger
473709fc76
Use doc builder styler ( #16412 )
...
* Config update
* Use doc-builder styler
* Cleanup
* Adapt import
* We need it there too!
2022-03-28 07:45:18 -04:00
Sylvain Gugger
3a0f1684c3
Fix readme links and add CI check ( #16392 )
...
* Fix doc links in README
* Fix name
* Fix links in READMEs and doc index
* Error if there is something wrong so the CI knows
2022-03-24 11:59:09 -04:00
Lysandre Debut
eca77f4719
Updates the default branch from master to main ( #16326 )
...
* Updates the default branch from master to main
* Links from `master` to `main`
* Typo
* Update examples/flax/README.md
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
2022-03-23 03:46:59 -04:00
Lysandre Debut
7732d0fe7a
Upgrade black to version ~=22.0 ( #15565 )
...
* Upgrade black to version ~=22.0
* Check copies
* Fix code
2022-02-09 09:28:57 -05:00
Sylvain Gugger
1144d336b6
Copies and docstring styling ( #15202 )
...
* Style docstrings when making/checking copies
* Polish
2022-01-18 09:16:55 -05:00
Sylvain Gugger
4df7d05a87
Doc new front ( #14590 )
...
* Convert PretrainedConfig doc to Markdown
* Use syntax
* Add necessary doc files (#14496 )
* Doc fixes (#14499 )
* Fixes for the new front
* Convert DETR file for table
* Title is needed
* Simplify a bit
* Even simpler
* Remove imports
* Fix typo in toctree (#14516 )
* Fix checkpoints badge
* Update versions.yml format (#14517 )
* Doc new front github actions (#14512 )
* Doc new front github actions
* Fix docstring
* Fix feature extraction utils import (#14515 )
* Address Julien's comments
* Push to doc-builder
* Ready for merge
* Remove old build and deploy
* Doc misc fixes (#14583 )
* Rm versions.yml from doc
* Fix converting.rst
* Rm pretrained_models from toctree
* Fix index links (#14567 )
* Fix links in README
* Localized READMEs
* Fix copy script
* Fix find doc script
* Update README_ko.md
Co-authored-by: Julien Chaumond <julien@huggingface.co>
Co-authored-by: Julien Chaumond <julien@huggingface.co>
* Adapt build command to new CLI tools (#14578 )
* Fix typo
* Fix doc interlinks (#14589 )
* Convert PretrainedConfig doc to Markdown
* Use syntax
* Rm pattern <[a-z]+(.html).*>
* Rm huggingface.co/transformers/master
* Rm .html
* Rm .html from index.mdx
* Rm .html from model_summary.rst
* Update index.mdx rm html
* Update remove .html
* Fix inner doc links
* Fix interlink in preprocssing.rst
* Update pr_checks
Co-authored-by: Sylvain Gugger <sylvain.gugger@gmail.com>
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* Convert PretrainedConfig doc to Markdown
* Use syntax
* Add necessary doc files (#14496 )
* Doc fixes (#14499 )
* Fixes for the new front
* Convert DETR file for table
* Title is needed
* Simplify a bit
* Even simpler
* Remove imports
* Fix checkpoints badge
* Fix typo in toctree (#14516 )
* Update versions.yml format (#14517 )
* Doc new front github actions (#14512 )
* Doc new front github actions
* Fix docstring
* Fix feature extraction utils import (#14515 )
* Address Julien's comments
* Push to doc-builder
* Ready for merge
* Remove old build and deploy
* Doc misc fixes (#14583 )
* Rm versions.yml from doc
* Fix converting.rst
* Rm pretrained_models from toctree
* Fix index links (#14567 )
* Fix links in README
* Localized READMEs
* Fix copy script
* Fix find doc script
* Update README_ko.md
Co-authored-by: Julien Chaumond <julien@huggingface.co>
Co-authored-by: Julien Chaumond <julien@huggingface.co>
* Adapt build command to new CLI tools (#14578 )
* Fix typo
* Fix doc interlinks (#14589 )
* Convert PretrainedConfig doc to Markdown
* Use syntax
* Rm pattern <[a-z]+(.html).*>
* Rm huggingface.co/transformers/master
* Rm .html
* Rm .html from index.mdx
* Rm .html from model_summary.rst
* Update index.mdx rm html
* Update remove .html
* Fix inner doc links
* Fix interlink in preprocssing.rst
* Update pr_checks
Co-authored-by: Sylvain Gugger <sylvain.gugger@gmail.com>
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* Styling
Co-authored-by: Mishig Davaadorj <mishig.davaadorj@coloradocollege.edu>
Co-authored-by: Lysandre Debut <lysandre@huggingface.co>
Co-authored-by: Julien Chaumond <julien@huggingface.co>
2021-12-01 14:13:02 -05:00
Yeoun Yi
9f53f049c6
Translate README.md to Korean ( #14015 )
...
* Create README_ko.md
* Update README.md
* Update README_zh-hans.md
* Update README_zh-hant.md
* Update README_ko.md
* Update check_copies.py
* Update README_ko.md
* typo
* match with readme_ko
2021-10-22 07:42:31 -04:00
Li-Huai (Allan) Lin
7d84c3a488
Enable readme link synchronization ( #13785 )
...
* Enable readme link synchronization
* Style
* Reuse regex pattern
* Apply suggestions
* Update
2021-09-29 11:18:59 -04:00
Kamal Raj
78807d86eb
[FLAX] Question Answering Example ( #13649 )
...
* flax qa example
* Updated README: Added Large model
* added utils_qa.py FULL_COPIES
* Updates:
1. Copyright Year updated
2. added dtype arg
3. passing seed and dtype to load model
4. Check eval flag before running eval
* updated README
* updated code comment
2021-09-21 18:34:48 +05:30
Sylvain Gugger
88dbbfb2d6
Fix make fix-copies with type annotations ( #13586 )
2021-09-16 11:55:37 -04:00
Li-Huai (Allan) Lin
18447c206d
Enable automated model list copying for localized READMEs ( #13465 )
...
* Complete basic mechanism
* Save
* Complete everything
* Style & Quality
* Update READMEs
* Add testing
* Fix README.md format
* Apply suggestions
* Fix format
* Update utils/check_copies.py
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
2021-09-08 08:03:35 -04:00
Sylvain Gugger
57461ac0b4
Add possibility to maintain full copies of files ( #12312 )
2021-06-28 10:02:53 -04:00
Patrick von Platen
cebb96f53a
Add more subsections to main doc ( #11758 )
...
* add headers to main doc
* Apply suggestions from code review
* update
* upload
2021-05-18 14:38:56 +01:00
Patrick von Platen
32dbb2d954
make style ( #11442 )
2021-04-26 13:50:34 +02:00
Sylvain Gugger
40b049c701
Check copies blackify ( #10775 )
...
* Apply black before checking copies
* Fix for class methods
* Deal with lonely brackets
* Remove debug and add forward changes
* Separate copies and fix test
* Add black as a test dependency
2021-03-17 18:11:20 -04:00
Sylvain Gugger
801ff969ce
Refactor checkpoint name in BERT and MobileBERT ( #10424 )
...
* Refactor checkpoint name in BERT and MobileBERT
* Add option to check copies
* Add QuestionAnswering
* Add last models
* Make black happy
2021-03-03 11:21:17 -05:00
Sylvain Gugger
04fd783cc5
Check copies match full class/function names ( #10030 )
2021-02-08 04:58:25 -05:00
Lysandre Debut
2ae7388eee
Check table as independent script ( #8976 )
2020-12-07 19:55:12 -05:00
Julien Chaumond
9ad6194318
Tweak wording + Add badge w/ number of models on the hub ( #8914 )
...
* Add badge w/ number of models on the hub
* try to apease @sgugger 😇
* not sure what this `c` was about [ci skip]
* Fix script and move stuff around
* Fix doc styling error
Co-authored-by: Sylvain Gugger <sylvain.gugger@gmail.com>
2020-12-03 10:56:55 -05:00
Sylvain Gugger
c0df963ee1
Make the big table creation/check platform independent ( #8856 )
2020-12-01 11:45:57 -05:00
Sylvain Gugger
75f8100fc7
Add a direct link to the big table ( #8850 )
2020-11-30 10:29:23 -05:00
Sylvain Gugger
4821ea5aeb
Big model table ( #8774 )
...
* First draft
* Styling
* With all changes staged
* Update docs/source/index.rst
Co-authored-by: Julien Chaumond <chaumond@gmail.com>
* Styling
Co-authored-by: Julien Chaumond <chaumond@gmail.com>
2020-11-25 12:02:15 -05:00
Julien Plu
91a67b7506
Use LF instead of os.linesep ( #8491 )
2020-11-12 13:52:40 -05:00
Santiago Castro
969859d5f6
Fix doc errors and typos across the board ( #8139 )
...
* Fix doc errors and typos across the board
* Fix a typo
* Fix the CI
* Fix more typos
* Fix CI
* More fixes
* Fix CI
* More fixes
* More fixes
2020-10-29 10:33:33 -04:00
Stas Bekman
a5a8eeb772
fix DeprecationWarning ( #7834 )
...
in `tests/test_utils_check_copies.py` I was getting intermittently:
```
utils/check_copies.py:52
/mnt/nvme1/code/transformers-comet/utils/check_copies.py:52: DeprecationWarning: invalid escape sequence \s
while line_index < len(lines) and re.search(f"^{indent}(class|def)\s+{name}", lines[line_index]) is None:
```
So this should fix it.
2020-10-15 16:21:09 -04:00
Sylvain Gugger
a3cea6a8cc
Better links for models in READMED and doc index ( #7680 )
2020-10-09 11:17:16 -04:00
