diff --git a/docs/source/en/generation_strategies.mdx b/docs/source/en/generation_strategies.mdx index 00ee9221fe6..ced19762f07 100644 --- a/docs/source/en/generation_strategies.mdx +++ b/docs/source/en/generation_strategies.mdx @@ -145,6 +145,12 @@ The `generate()` supports streaming, through its `streamer` input. The `streamer from a class that has the following methods: `put()` and `end()`. Internally, `put()` is used to push new tokens and `end()` is used to flag the end of text generation. + + +The API for the streamer classes is still under development and may change in the future. + + + In practice, you can craft your own streaming class for all sorts of purposes! We also have basic streaming classes ready for you to use. For example, you can use the [`TextStreamer`] class to stream the output of `generate()` into your screen, one word at a time: diff --git a/src/transformers/generation/streamers.py b/src/transformers/generation/streamers.py index 4006256f839..979405bc136 100644 --- a/src/transformers/generation/streamers.py +++ b/src/transformers/generation/streamers.py @@ -39,6 +39,12 @@ class TextStreamer(BaseStreamer): """ Simple text streamer that prints the token(s) to stdout as soon as entire words are formed. + + + The API for the streamer classes is still under development and may change in the future. + + + Parameters: tokenizer (`AutoTokenizer`): The tokenized used to decode the tokens. @@ -128,6 +134,12 @@ class TextIteratorStreamer(TextStreamer): useful for applications that benefit from acessing the generated text in a non-blocking way (e.g. in an interactive Gradio demo). + + + The API for the streamer classes is still under development and may change in the future. + + + Parameters: tokenizer (`AutoTokenizer`): The tokenized used to decode the tokens.