transformers/docs/source/en
Jinho Park 17fdd35481
Add BROS (#23190)
* add Bros boilerplate

* copy and pasted modeling_bros.py from official Bros repo

* update copyright of bros files

* copy tokenization_bros.py from official repo and update import path

* copy tokenization_bros_fast.py from official repo and update import path

* copy configuration_bros.py from official repo and update import path

* remove trailing period in copyright line

* copy and paste bros/__init__.py from official repo

* save formatting

* remove unused unnecessary pe_type argument - using only crel type

* resolve import issue

* remove unused model classes

* remove unnecessary tests

* remove unused classes

* fix original code's bug - layer_module's argument order

* clean up modeling auto

* add bbox to prepare_config_and_inputs

* set temporary value to hidden_size (32 is too low because of the of the
Bros' positional embedding)

* remove decoder test, update create_and_check* input arguemnts

* add missing variable to model tests

* do make fixup

* update bros.mdx

* add boilerate plate for no_head inference test

* update BROS_PRETRAINED_MODEL_ARCHIVE_LIST (add naver-clova-ocr prefix)

* add prepare_bros_batch_inputs function

* update modeling_common to add bbox inputs in Bros Model Test

* remove unnecessary model inference

* add test case

* add model_doc

* add test case for token_classification

* apply fixup

* update modeling code

* update BrosForTokenClassification loss calculation logic

* revert logits preprocessing logic to make sure logits have original shape

* - update class name

* - add BrosSpadeOutput
- update BrosConfig arguments

* add boilerate plate for no_head inference test

* add prepare_bros_batch_inputs function

* add test case

* add test case for token_classification

* update modeling code

* update BrosForTokenClassification loss calculation logic

* revert logits preprocessing logic to make sure logits have original shape

* apply masking on the fly

* add BrosSpadeForTokenLinking

* update class name
put docstring to the beginning of the file

* separate the logits calculation logic and loss calculation logic

* update logic for loss calculation so that logits shape doesn't change
when return

* update typo

* update prepare_config_and_inputs

* update dummy node initialization

* update last_hidden_states getting logic to consider when return_dict is False

* update box first token mask param

* bugfix: remove random attention mask generation

* update keys to ignore on load missing

* run make style and quality

* apply make style and quality of other codes

* update box_first_token_mask to bool type

* update index.md

* apply make style and quality

* apply make fix-copies

* pass check_repo

* update bros model doc

* docstring bugfix fix

* add checkpoint for doc, tokenizer for doc

* Update README.md

* Update docs/source/en/model_doc/bros.md

Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>

* Update bros.md

* Update src/transformers/__init__.py

Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>

* Update docs/source/en/model_doc/bros.md

Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>

* apply suggestions from code review

* apply suggestions from code review

* revert test_processor_markuplm.py

* Update test_processor_markuplm.py

* apply suggestions from code review

* apply suggestions from code review

* apply suggestions from code review

* update BrosSpadeELForTokenClassification head name to entity linker

* add doc string for config params

* update class, var names to more explicit and apply suggestions from code review

* remove unnecessary keys to ignore

* update relation extractor to be initialized with config

* add bros processor

* apply make style and quality

* update bros.md

* remove bros tokenizer, add bros processor that wraps bert tokenizer

* revert change

* apply make fix-copies

* update processor code, update itc -> initial token, stc -> subsequent token

* add type hint

* remove unnecessary condition branches in embedding forward

* fix auto tokenizer fail

* update docstring for each classes

* update bbox input dimension as standard 2 points and convert them to 4
points in forward pass

* update bros docs

* apply suggestions from code review : update Bros -> BROS in bros.md

* 1. box prefix var -> bbox
2. update variable names to be more explicit

* replace einsum with torch matmul

* apply style and quality

* remove unused argument

* remove unused arguments

* update docstrings

* apply suggestions from code review: add BrosBboxEmbeddings, replace
einsum with classical matrix operations

* revert einsum update

* update bros processor

* apply suggestions from code review

* add conversion script for bros

* Apply suggestions from code review

* fix readme

* apply fix-copies

---------

Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
2023-09-14 18:02:37 +01:00
..
internal Generate: add missing logits processors docs (#25653) 2023-08-25 11:56:17 +01:00
main_classes Overhaul Conversation class and prompt templating (#25323) 2023-09-14 15:10:34 +01:00
model_doc Add BROS (#23190) 2023-09-14 18:02:37 +01:00
tasks Add BROS (#23190) 2023-09-14 18:02:37 +01:00
_config.py Enable doc in Spanish (#16518) 2022-04-04 10:25:46 -04:00
_toctree.yml Add BROS (#23190) 2023-09-14 18:02:37 +01:00
accelerate.md Fix typos (#25936) 2023-09-04 11:15:12 +01:00
add_new_model.md Fix typos (#25936) 2023-09-04 11:15:12 +01:00
add_new_pipeline.md Migrate doc files to Markdown. (#24376) 2023-06-20 18:07:47 -04:00
add_tensorflow_model.md Fix typos (#25936) 2023-09-04 11:15:12 +01:00
attention.md Migrate doc files to Markdown. (#24376) 2023-06-20 18:07:47 -04:00
autoclass_tutorial.md Update autoclass_tutorial.md (#25929) 2023-09-04 11:16:49 +01:00
benchmarks.md Migrate doc files to Markdown. (#24376) 2023-06-20 18:07:47 -04:00
bertology.md Migrate doc files to Markdown. (#24376) 2023-06-20 18:07:47 -04:00
big_models.md Fix typos (#25936) 2023-09-04 11:15:12 +01:00
chat_templating.md Overhaul Conversation class and prompt templating (#25323) 2023-09-14 15:10:34 +01:00
community.md Update community.md (#25928) 2023-09-04 11:16:34 +01:00
contributing.md Enable doc in Spanish (#16518) 2022-04-04 10:25:46 -04:00
create_a_model.md Fix typos (#25936) 2023-09-04 11:15:12 +01:00
custom_models.md Fix typos (#25936) 2023-09-04 11:15:12 +01:00
custom_tools.md [doc] Always call it Agents for consistency (#25958) 2023-09-05 12:27:20 +01:00
debugging.md Migrate doc files to Markdown. (#24376) 2023-06-20 18:07:47 -04:00
fast_tokenizers.md Migrate doc files to Markdown. (#24376) 2023-06-20 18:07:47 -04:00
generation_strategies.md Generate: legacy mode is only triggered when generation_config is untouched (#25962) 2023-09-12 12:08:17 +01:00
glossary.md Fix typos (#25936) 2023-09-04 11:15:12 +01:00
hpo_train.md enable optuna multi-objectives feature (#25969) 2023-09-12 18:01:22 +01:00
index.md Add BROS (#23190) 2023-09-14 18:02:37 +01:00
installation.md Migrate doc files to Markdown. (#24376) 2023-06-20 18:07:47 -04:00
llm_tutorial.md Fix typos (#25936) 2023-09-04 11:15:12 +01:00
model_memory_anatomy.md Fix typos (#25936) 2023-09-04 11:15:12 +01:00
model_sharing.md Fix typos (#25936) 2023-09-04 11:15:12 +01:00
model_summary.md Migrate doc files to Markdown. (#24376) 2023-06-20 18:07:47 -04:00
multilingual.md Fix typo in example code (#25583) 2023-08-18 07:58:59 +02:00
notebooks.md Enable doc in Spanish (#16518) 2022-04-04 10:25:46 -04:00
pad_truncation.md Migrate doc files to Markdown. (#24376) 2023-06-20 18:07:47 -04:00
peft.md [PEFT] Peft integration alternative design (#25077) 2023-08-18 19:08:03 +02:00
perf_hardware.md 🌐 [i18n-KO] Translated perf_hardware.md to Korean (#24966) 2023-07-25 07:44:24 -04:00
perf_infer_cpu.md Migrate doc files to Markdown. (#24376) 2023-06-20 18:07:47 -04:00
perf_infer_gpu_many.md [Docs / BetterTransformer ] Added more details about flash attention + SDPA (#25265) 2023-08-18 10:32:28 +02:00
perf_infer_gpu_one.md [Docs] More clarifications on BT + FA (#25823) 2023-08-29 13:52:25 +02:00
perf_infer_special.md Migrate doc files to Markdown. (#24376) 2023-06-20 18:07:47 -04:00
perf_torch_compile.md Fix rendering for torch.compile() docs (#25432) 2023-08-10 13:25:00 +02:00
perf_train_cpu_many.md Migrate doc files to Markdown. (#24376) 2023-06-20 18:07:47 -04:00
perf_train_cpu.md Migrate doc files to Markdown. (#24376) 2023-06-20 18:07:47 -04:00
perf_train_gpu_many.md deprecate sharded_ddp training argument (#24825) 2023-07-17 06:57:42 -04:00
perf_train_gpu_one.md Modify efficient GPU training doc with now-available adamw_bnb_8bit optimizer (#25807) 2023-08-31 10:55:10 +01:00
perf_train_special.md Migrate doc files to Markdown. (#24376) 2023-06-20 18:07:47 -04:00
perf_train_tpu_tf.md Migrate doc files to Markdown. (#24376) 2023-06-20 18:07:47 -04:00
perf_train_tpu.md Migrate doc files to Markdown. (#24376) 2023-06-20 18:07:47 -04:00
performance.md [docs] Performance docs tidy up, part 1 (#23963) 2023-07-24 08:57:24 -04:00
perplexity.md Migrate doc files to Markdown. (#24376) 2023-06-20 18:07:47 -04:00
philosophy.md Migrate doc files to Markdown. (#24376) 2023-06-20 18:07:47 -04:00
pipeline_tutorial.md Support loading base64 images in pipelines (#25633) 2023-08-29 19:24:24 +01:00
pipeline_webserver.md Suggestions on Pipeline_webserver (#25570) 2023-08-18 10:17:44 +02:00
pr_checks.md Document check copies (#25291) 2023-08-04 14:56:29 +02:00
preprocessing.md Removal of deprecated vision methods and specify deprecation versions (#24570) 2023-06-29 15:09:51 +01:00
quicktour.md [TYPO] fix typo/format in quicktour.md (#25519) 2023-08-16 08:03:23 +02:00
run_scripts.md Migrate doc files to Markdown. (#24376) 2023-06-20 18:07:47 -04:00
sagemaker.md Migrate doc files to Markdown. (#24376) 2023-06-20 18:07:47 -04:00
serialization.md Migrate doc files to Markdown. (#24376) 2023-06-20 18:07:47 -04:00
task_summary.md Fix doctest (#25031) 2023-07-25 22:10:06 +02:00
tasks_explained.md Migrate doc files to Markdown. (#24376) 2023-06-20 18:07:47 -04:00
testing.md fix wrong path in some doc (#25658) 2023-08-23 08:34:30 +02:00
tf_xla.md Migrate doc files to Markdown. (#24376) 2023-06-20 18:07:47 -04:00
tflite.md Migrate doc files to Markdown. (#24376) 2023-06-20 18:07:47 -04:00
tokenizer_summary.md Fix typo: Roberta -> RoBERTa (#25302) 2023-08-03 14:17:30 -07:00
torchscript.md Migrate doc files to Markdown. (#24376) 2023-06-20 18:07:47 -04:00
training.md Migrate doc files to Markdown. (#24376) 2023-06-20 18:07:47 -04:00
transformers_agents.md [doc] Always call it Agents for consistency (#25958) 2023-09-05 12:27:20 +01:00
troubleshooting.md Migrate doc files to Markdown. (#24376) 2023-06-20 18:07:47 -04:00