transformers/docs/source
Nicolas Patry 543d0549f8
Adding new parameter to generate: max_time. (#9846)
* [WIP] Adding new parameter to `generate`:  `max_time`.

Generation by tokens number is sometimes a bit clunky because we don't
know how many tokens are good enough or even how many tokens are in
the payload (for pipelines users for instance). This leads to hard
to understand behavior.

This PR proposes a new argument `max_time` which is a float of seconds
for the allowed time for `generate` to run on.
Ideally combinations of `max_tokens=None`, `max_time=2` could be used to
generate as many tokens as possible within time budget.

NB: Another possible approach consists of passing a callback to `generate`
  putting the caller in charge of the actual decision of when to stop
  generating tokens. It opens the door to 'which args should we pass'
  to this callback. It's hard to imagine other use-cases for this
  early stopping behavior than time (that are not already covered by
  parameters of generate)

* Revamp with StoppingCriteria

* Removing deprecated mentions.

* Forgot arguments to stopping criteria.

* Readding max_length it's not just used as a stopping criteria.

* Default value for `stopping_criteria`.

* Address @patrickvonplaten comments.

- More docstrings
- Actual doc
- Include in global namespace
- Remove TF work.

* Put back `max_length` (deprecation different PR).

* Doc quality.

* Fixing old behavior without `stopping_criteria` but with `max_length`.

Making sure we don't break that in the future.

* Adding more tests for possible inconsistencies between

`max_length` and `stopping_criteria`.

* Fixing the torch imports.
2021-03-12 10:11:50 +01:00
..
_static Fix version control with anchors (#10595) 2021-03-08 10:19:22 -05:00
imgs [Templates] Add template "call-for-model" markdown and "call-for-big-bird" markdown (#9921) 2021-02-05 15:47:54 +03:00
internal Adding new parameter to generate: max_time. (#9846) 2021-03-12 10:11:50 +01:00
main_classes Document Trainer limitation on custom models (#10635) 2021-03-10 14:58:22 -05:00
model_doc Fix broken link (#10656) 2021-03-11 14:29:02 -05:00
add_new_model.rst Add new model docs (#9667) 2021-02-01 17:55:10 +03:00
benchmarks.rst Make doc styler detect lists on rst (#9488) 2021-01-11 08:53:41 -05:00
bertology.rst Fix documentation links always pointing to master. (#9217) 2021-01-05 06:18:48 -05:00
community.md Add new community notebook - Blenderbot (#10126) 2021-02-11 12:53:40 +03:00
conf.py feat(docs): navigate with left/right arrow keys (#10481) 2021-03-03 11:17:12 -05:00
contributing.md Update installation page and add contributing to the doc (#5084) 2020-06-17 14:01:10 -04:00
converting_tensorflow_models.rst Fix broken links in the converting tf ckpt document (#9791) 2021-01-26 03:37:57 -05:00
custom_datasets.rst Fix None in add_token_positions - issue #10210 (#10374) 2021-02-25 09:18:33 -07:00
examples.md per_device instead of per_gpu/error thrown when argument unknown (#4618) 2020-05-27 11:36:55 -04:00
favicon.ico Adding usage examples for common tasks (#2850) 2020-02-25 13:48:24 -05:00
glossary.rst Adds terms to Glossary (#10443) 2021-02-28 08:27:54 -05:00
index.rst [XLSR-Wav2Vec2] Add multi-lingual Wav2Vec2 models (#10648) 2021-03-11 17:44:18 +03:00
installation.md offline mode for firewalled envs (#10407) 2021-03-05 17:27:48 -08:00
migration.md Copyright (#8970) 2020-12-07 18:36:34 -05:00
model_sharing.rst [doc] nested markup is invalid in rst (#9898) 2021-01-30 09:59:19 -05:00
model_summary.rst ConvBERT Model (#9717) 2021-01-27 03:20:09 -05:00
multilingual.rst Fix documentation links always pointing to master. (#9217) 2021-01-05 06:18:48 -05:00
notebooks.md Update notebooks (#3620) 2020-04-06 14:32:39 -04:00
perplexity.rst Copyright (#8970) 2020-12-07 18:36:34 -05:00
philosophy.rst Minor documentation revisions from copyediting (#9266) 2020-12-23 10:15:49 -05:00
preprocessing.rst Minor documentation revisions from copyediting (#9266) 2020-12-23 10:15:49 -05:00
pretrained_models.rst Add m2m100 (#10236) 2021-03-06 22:14:16 +05:30
quicktour.rst Minor documentation revisions from copyediting (#9266) 2020-12-23 10:15:49 -05:00
serialization.rst Copyright (#8970) 2020-12-07 18:36:34 -05:00
task_summary.rst Remove unused variable in example for Q&A (#10392) 2021-02-25 09:18:47 -05:00
testing.rst Fix documentation links always pointing to master. (#9217) 2021-01-05 06:18:48 -05:00
tokenizer_summary.rst Minor documentation revisions from copyediting (#9266) 2020-12-23 10:15:49 -05:00
training.rst [trainer] deepspeed integration (#9211) 2021-01-12 19:05:18 -08:00