riaz.somc/transformers
1
0
Fork 0
mirror of https://github.com/huggingface/transformers.git synced 2025-08-01 18:51:14 +06:00
Code Issues Actions 3 Packages Projects Releases Wiki Activity

Clarify stride option (#22684)

Browse Source 
* Clarify stride option

* formatting
This commit is contained in:
Luc CAILLIAU 2023-04-11 15:06:54 +02:00 committed by GitHub
parent 0224aaf67f
commit 06b05d4575
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 2 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

3
src/transformers/pipelines/token_classification.py
View File

@ -68,7 +68,8 @@ class AggregationStrategy(ExplicitEnum):
same entity together in the predictions or not. same entity together in the predictions or not.
stride (`int`, *optional*): stride (`int`, *optional*):
If stride is provided, the pipeline is applied on all the text. The text is split into chunks of size If stride is provided, the pipeline is applied on all the text. The text is split into chunks of size
model_max_length. Works only with fast tokenizers and `aggregation_strategy` different from `NONE`. model_max_length. Works only with fast tokenizers and `aggregation_strategy` different from `NONE`. The
value of this argument defines the number of overlapping tokens between chunks.
aggregation_strategy (`str`, *optional*, defaults to `"none"`): aggregation_strategy (`str`, *optional*, defaults to `"none"`):
The strategy to fuse (or not) tokens based on the model prediction. The strategy to fuse (or not) tokens based on the model prediction.