riaz.somc/transformers
1
0
Fork 0
mirror of https://github.com/huggingface/transformers.git synced 2025-07-29 17:22:25 +06:00
Code Issues Actions 22 Packages Projects Releases Wiki Activity
22454ae492
transformers/docs/source/model_doc
History
Li-Huai (Allan) Lin 22454ae492
Add REALM (#13292) 
* REALM initial commit

* Retriever OK (Update new_gelu).

* Encoder prediction score OK

* Encoder pretrained model OK

* Update retriever comments

* Update docs, tests, and imports

* Prune unused models

* Make embedder as a module `RealmEmbedder`

* Add RealmRetrieverOutput

* Update tokenization

* Pass all tests in test_modeling_realm.py

* Prune RealmModel

* Update docs

* Add training test.

* Remove completed TODO

* Style & Quality

* Prune `RealmModel`

* Fixup

* Changes:
1. Remove RealmTokenizerFast
2. Update docstrings
3. Add a method to RealmTokenizer to handle candidates tokenization.

* Fix up

* Style

* Add tokenization tests

* Update `from_pretrained` tests

* Apply suggestions

* Style & Quality

* Copy BERT model

* Fix comment to avoid docstring copying

* Make RealmBertModel private

* Fix bug

* Style

* Basic QA

* Save

* Complete reader logits

* Add searcher

* Complete searcher & reader

* Move block records init to constructor

* Fix training bug

* Add some outputs to RealmReader

* Add finetuned checkpoint variable names parsing

* Fix bug

* Update REALM config

* Add RealmForOpenQA

* Update convert_tfrecord logits

* Fix bugs

* Complete imports

* Update docs

* Update naming

* Add brute-force searcher

* Pass realm model tests

* Style

* Exclude RealmReader from common tests

* Fix

* Fix

* convert docs

* up

* up

* more make style

* up

* upload

* up

* Fix

* Update src/transformers/__init__.py

* adapt testing

* change modeling code

* fix test

* up

* up

* up

* correct more

* make retriever work

* update

* make style

* finish main structure

* Resolve merge conflict

* Make everything work

* Style

* Fixup

* Fixup

* Update training test

* fix retriever

* remove hardcoded path

* Fix

* Fix modeling test

* Update model links

* Initial retrieval test

* Fix modeling test

* Complete retrieval tests

* Fix

* style

* Fix tests

* Fix docstring example

* Minor fix of retrieval test

* Update license headers and docs

* Apply suggestions from code review

* Style

* Apply suggestions from code review

* Add an example to RealmEmbedder

* Fix

Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
2022-01-18 07:24:13 -05:00
..
albert.mdx Convert model files from rst to mdx (#14865) 2021-12-22 03:27:30 -05:00
auto.mdx Add TFVisionEncoderDecoderModel (#14148) 2022-01-10 13:30:14 -05:00
bart.mdx Doc styler examples (#14953) 2021-12-27 19:07:46 -05:00
barthez.mdx Convert model files from rst to mdx (#14865) 2021-12-22 03:27:30 -05:00
bartpho.mdx Doc styler examples (#14953) 2021-12-27 19:07:46 -05:00
beit.mdx Convert model files from rst to mdx (#14865) 2021-12-22 03:27:30 -05:00
bert-generation.mdx Map model_type and doc pages names (#14944) 2022-01-03 05:08:55 -05:00
bert-japanese.mdx Map model_type and doc pages names (#14944) 2022-01-03 05:08:55 -05:00
bert.mdx Convert rst to mdx bert (#14806) 2021-12-17 11:13:34 -05:00
bertweet.mdx Doc styler examples (#14953) 2021-12-27 19:07:46 -05:00
big_bird.mdx Map model_type and doc pages names (#14944) 2022-01-03 05:08:55 -05:00
bigbird_pegasus.mdx Convert model files from rst to mdx (#14865) 2021-12-22 03:27:30 -05:00
blenderbot-small.mdx Map model_type and doc pages names (#14944) 2022-01-03 05:08:55 -05:00
blenderbot.mdx Map model_type and doc pages names (#14944) 2022-01-03 05:08:55 -05:00
bort.mdx Convert model files from rst to mdx (#14865) 2021-12-22 03:27:30 -05:00
byt5.mdx Doc styler examples (#14953) 2021-12-27 19:07:46 -05:00
camembert.mdx Convert model files from rst to mdx (#14865) 2021-12-22 03:27:30 -05:00
canine.mdx Doc styler examples (#14953) 2021-12-27 19:07:46 -05:00
clip.mdx Doc styler examples (#14953) 2021-12-27 19:07:46 -05:00
convbert.mdx Convert model files from rst to mdx (#14865) 2021-12-22 03:27:30 -05:00
cpm.mdx Convert model files from rst to mdx (#14865) 2021-12-22 03:27:30 -05:00
ctrl.mdx Convert model files from rst to mdx (#14865) 2021-12-22 03:27:30 -05:00
deberta-v2.mdx Map model_type and doc pages names (#14944) 2022-01-03 05:08:55 -05:00
deberta.mdx Convert model files from rst to mdx (#14865) 2021-12-22 03:27:30 -05:00
deit.mdx Convert model files from rst to mdx (#14865) 2021-12-22 03:27:30 -05:00
detr.mdx Doc new front (#14590) 2021-12-01 14:13:02 -05:00
dialogpt.mdx Convert model files from rst to mdx (#14865) 2021-12-22 03:27:30 -05:00
distilbert.mdx Convert model files from rst to mdx (#14865) 2021-12-22 03:27:30 -05:00
dpr.mdx Convert model files from rst to mdx (#14865) 2021-12-22 03:27:30 -05:00
electra.mdx Add ElectraForCausalLM -> Enable Electra encoder-decoder model (#14729) 2021-12-27 12:37:52 +01:00
encoder-decoder.mdx Map model_type and doc pages names (#14944) 2022-01-03 05:08:55 -05:00
flaubert.mdx Convert model files from rst to mdx (#14865) 2021-12-22 03:27:30 -05:00
fnet.mdx Convert model files from rst to mdx (#14865) 2021-12-22 03:27:30 -05:00
fsmt.mdx Convert model files from rst to mdx (#14865) 2021-12-22 03:27:30 -05:00
funnel.mdx Convert model files from rst to mdx (#14865) 2021-12-22 03:27:30 -05:00
gpt_neo.mdx Doc styler examples (#14953) 2021-12-27 19:07:46 -05:00
gpt2.mdx Convert model files from rst to mdx (#14865) 2021-12-22 03:27:30 -05:00
gptj.mdx Doc styler examples (#14953) 2021-12-27 19:07:46 -05:00
herbert.mdx Doc styler examples (#14953) 2021-12-27 19:07:46 -05:00
hubert.mdx Convert model files from rst to mdx (#14865) 2021-12-22 03:27:30 -05:00
ibert.mdx Convert model files from rst to mdx (#14865) 2021-12-22 03:27:30 -05:00
imagegpt.mdx Map model_type and doc pages names (#14944) 2022-01-03 05:08:55 -05:00
layoutlm.mdx Doc styler examples (#14953) 2021-12-27 19:07:46 -05:00
layoutlmv2.mdx Doc styler examples (#14953) 2021-12-27 19:07:46 -05:00
layoutxlm.mdx Doc styler examples (#14953) 2021-12-27 19:07:46 -05:00
led.mdx Convert model files from rst to mdx (#14865) 2021-12-22 03:27:30 -05:00
longformer.mdx Doc styler examples (#14953) 2021-12-27 19:07:46 -05:00
luke.mdx Doc styler examples (#14953) 2021-12-27 19:07:46 -05:00
lxmert.mdx Convert model files from rst to mdx (#14865) 2021-12-22 03:27:30 -05:00
m2m_100.mdx Doc styler examples (#14953) 2021-12-27 19:07:46 -05:00
marian.mdx Doc styler examples (#14953) 2021-12-27 19:07:46 -05:00
mbart.mdx Doc styler examples (#14953) 2021-12-27 19:07:46 -05:00
megatron_gpt2.mdx Convert rst files (#14888) 2021-12-22 16:14:35 -05:00
megatron-bert.mdx Map model_type and doc pages names (#14944) 2022-01-03 05:08:55 -05:00
mluke.mdx Doc styler examples (#14953) 2021-12-27 19:07:46 -05:00
mobilebert.mdx Convert rst files (#14888) 2021-12-22 16:14:35 -05:00
mpnet.mdx Convert rst files (#14888) 2021-12-22 16:14:35 -05:00
mt5.mdx Convert rst files (#14888) 2021-12-22 16:14:35 -05:00
nystromformer.mdx Add Nystromformer (#14659) 2022-01-11 14:25:49 +01:00
openai-gpt.mdx Map model_type and doc pages names (#14944) 2022-01-03 05:08:55 -05:00
pegasus.mdx Doc styler examples (#14953) 2021-12-27 19:07:46 -05:00
perceiver.mdx Fix Perceiver docs (#14879) 2021-12-22 14:18:03 +01:00
phobert.mdx Convert rst files (#14888) 2021-12-22 16:14:35 -05:00
prophetnet.mdx Convert rst files (#14888) 2021-12-22 16:14:35 -05:00
qdqbert.mdx Doc styler examples (#14953) 2021-12-27 19:07:46 -05:00
rag.mdx Convert rst files (#14888) 2021-12-22 16:14:35 -05:00
realm.mdx Add REALM (#13292) 2022-01-18 07:24:13 -05:00
reformer.mdx Doc styler examples (#14953) 2021-12-27 19:07:46 -05:00
rembert.mdx Convert rst files (#14888) 2021-12-22 16:14:35 -05:00
retribert.mdx Convert rst files (#14888) 2021-12-22 16:14:35 -05:00
roberta.mdx Convert rst files (#14888) 2021-12-22 16:14:35 -05:00
roformer.mdx Add Flax RoFormer (#15005) 2022-01-04 13:23:10 +01:00
segformer.mdx [doc] consistent True/False/None default format (#14951) 2021-12-27 14:31:40 -08:00
sew-d.mdx Map model_type and doc pages names (#14944) 2022-01-03 05:08:55 -05:00
sew.mdx Convert rst files (#14888) 2021-12-22 16:14:35 -05:00
speech_to_text_2.mdx Map model_type and doc pages names (#14944) 2022-01-03 05:08:55 -05:00
speech_to_text.mdx Doc styler examples (#14953) 2021-12-27 19:07:46 -05:00
speech-encoder-decoder.mdx Map model_type and doc pages names (#14944) 2022-01-03 05:08:55 -05:00
splinter.mdx Convert rst files (#14888) 2021-12-22 16:14:35 -05:00
squeezebert.mdx Convert rst files (#14888) 2021-12-22 16:14:35 -05:00
t5.mdx Doc styler examples (#14953) 2021-12-27 19:07:46 -05:00
t5v1.1.mdx Doc styler examples (#14953) 2021-12-27 19:07:46 -05:00
tapas.mdx Doc styler examples (#14953) 2021-12-27 19:07:46 -05:00
transfo-xl.mdx Map model_type and doc pages names (#14944) 2022-01-03 05:08:55 -05:00
trocr.mdx support the trocr small models (#14893) 2022-01-10 09:28:03 -05:00
unispeech-sat.mdx Map model_type and doc pages names (#14944) 2022-01-03 05:08:55 -05:00
unispeech.mdx Convert rst files (#14888) 2021-12-22 16:14:35 -05:00
vision-encoder-decoder.mdx Add TFVisionEncoderDecoderModel (#14148) 2022-01-10 13:30:14 -05:00
vision-text-dual-encoder.mdx Map model_type and doc pages names (#14944) 2022-01-03 05:08:55 -05:00
visual_bert.mdx Doc styler examples (#14953) 2021-12-27 19:07:46 -05:00
vit.mdx Convert rst files (#14888) 2021-12-22 16:14:35 -05:00
wav2vec2_phoneme.mdx Wav2Vec2 meets phonemes (#14353) 2021-12-17 19:56:44 +01:00
wav2vec2.mdx Convert rst files (#14888) 2021-12-22 16:14:35 -05:00
wavlm.mdx Convert rst files (#14888) 2021-12-22 16:14:35 -05:00
xlm-prophetnet.mdx Map model_type and doc pages names (#14944) 2022-01-03 05:08:55 -05:00
xlm-roberta.mdx Map model_type and doc pages names (#14944) 2022-01-03 05:08:55 -05:00
xlm.mdx Convert rst files (#14888) 2021-12-22 16:14:35 -05:00
xlnet.mdx Convert rst files (#14888) 2021-12-22 16:14:35 -05:00
xls_r.mdx Convert rst files (#14888) 2021-12-22 16:14:35 -05:00
xlsr_wav2vec2.mdx Convert rst files (#14888) 2021-12-22 16:14:35 -05:00