mirror of
https://github.com/huggingface/transformers.git
synced 2025-07-14 10:08:29 +06:00
e642c78908
* initial commit for pipeline implementation Addition of input processing and history concatenation * Conversation pipeline tested and working for single & multiple conversation inputs * Added docstrings for dialogue pipeline * Addition of dialogue pipeline integration tests * Delete test_t5.py * Fixed max code length * Updated styling * Fixed test broken by formatting tools * Removed unused import * Added unit test for DialoguePipeline * Fixed Tensorflow compatibility * Fixed multi-framework support using framework flag * - Fixed docstring - Added `min_length_for_response` as an initialization parameter - Renamed `*args` to `conversations`, `conversations` being a `Conversation` or a `List[Conversation]` - Updated truncation to truncate entire segments of conversations, instead of cutting in the middle of a user/bot input * - renamed pipeline name from dialogue to conversational - removed hardcoded default value of 1000 and use config.max_length instead - added `append_response` and `set_history` method to the Conversation class to avoid direct fields mutation - fixed bug in history truncation method * - Updated ConversationalPipeline to accept only active conversations (otherwise a ValueError is raised) * - Simplified input tensor conversion * - Updated attention_mask value for Tensorflow compatibility * - Updated last dialogue reference to conversational & fixed integration tests * Fixed conflict with master * Updates following review comments * Updated formatting * Added Conversation and ConversationalPipeline to the library __init__, addition of docstrings for Conversation, added both to the docs * Update src/transformers/pipelines.py Updated docsting following review Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com> Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
81 lines
2.4 KiB
ReStructuredText
81 lines
2.4 KiB
ReStructuredText
Pipelines
|
|
----------------------------------------------------
|
|
|
|
The pipelines are a great and easy way to use models for inference. These pipelines are objects that abstract most
|
|
of the complex code from the library, offering a simple API dedicated to several tasks, including Named Entity
|
|
Recognition, Masked Language Modeling, Sentiment Analysis, Feature Extraction and Question Answering.
|
|
|
|
There are two categories of pipeline abstractions to be aware about:
|
|
|
|
- The :func:`~transformers.pipeline` which is the most powerful object encapsulating all other pipelines
|
|
- The other task-specific pipelines, such as :class:`~transformers.TokenClassificationPipeline`
|
|
or :class:`~transformers.QuestionAnsweringPipeline`
|
|
|
|
The pipeline abstraction
|
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
The `pipeline` abstraction is a wrapper around all the other available pipelines. It is instantiated as any
|
|
other pipeline but requires an additional argument which is the `task`.
|
|
|
|
.. autofunction:: transformers.pipeline
|
|
|
|
|
|
The task specific pipelines
|
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
Parent class: Pipeline
|
|
=========================================
|
|
|
|
.. autoclass:: transformers.Pipeline
|
|
:members: predict, transform, save_pretrained
|
|
|
|
TokenClassificationPipeline
|
|
==========================================
|
|
|
|
.. autoclass:: transformers.TokenClassificationPipeline
|
|
|
|
NerPipeline
|
|
==========================================
|
|
|
|
This class is an alias of the :class:`~transformers.TokenClassificationPipeline` defined above. Please refer to that pipeline for
|
|
documentation and usage examples.
|
|
|
|
FillMaskPipeline
|
|
==========================================
|
|
|
|
.. autoclass:: transformers.FillMaskPipeline
|
|
|
|
FeatureExtractionPipeline
|
|
==========================================
|
|
|
|
.. autoclass:: transformers.FeatureExtractionPipeline
|
|
|
|
TextClassificationPipeline
|
|
==========================================
|
|
|
|
.. autoclass:: transformers.TextClassificationPipeline
|
|
|
|
QuestionAnsweringPipeline
|
|
==========================================
|
|
|
|
.. autoclass:: transformers.QuestionAnsweringPipeline
|
|
|
|
|
|
SummarizationPipeline
|
|
==========================================
|
|
|
|
.. autoclass:: transformers.SummarizationPipeline
|
|
|
|
|
|
TextGenerationPipeline
|
|
==========================================
|
|
|
|
.. autoclass:: transformers.TextGenerationPipeline
|
|
|
|
|
|
ConversationalPipeline
|
|
==========================================
|
|
|
|
.. autoclass:: transformers.Conversation
|
|
|
|
.. autoclass:: transformers.ConversationalPipeline |