mirror of
https://github.com/huggingface/transformers.git
synced 2025-07-31 02:02:21 +06:00
cd927a4736
* let's go! * initial implementation of token-level timestamps * only return a single timestamp per token * remove token probabilities * fix return type * fix doc comment * strip special tokens * rename * revert to not stripping special tokens * only support models that have alignment_heads * add integration test * consistently name it token-level timestamps * small DTW tweak * initial support for ASR pipeline * fix pipeline doc comments * resolve token timestamps in pipeline with chunking * change warning when no final timestamp is found * return word-level timestamps * fixup * fix bug that skipped final word in each chunk * fix failing unit tests * merge punctuations into the words * also return word tokens * also return token indices * add (failing) unit test for combine_tokens_into_words * make combine_tokens_into_words private * restore OpenAI's punctuation rules * add pipeline tests * make requested changes * PR review changes * fix failing pipeline test * small stuff from PR * only return words and their timestamps, not segments * move alignment_heads into generation config * forgot to set alignment_heads in pipeline tests * tiny comment fix * grr |
||
|---|---|---|
| .. | ||
| benchmark | ||
| bettertransformer | ||
| bnb | ||
| deepspeed | ||
| extended | ||
| fixtures | ||
| generation | ||
| models | ||
| onnx | ||
| optimization | ||
| pipelines | ||
| repo_utils | ||
| sagemaker | ||
| tokenization | ||
| tools | ||
| trainer | ||
| utils | ||
| __init__.py | ||
| test_backbone_common.py | ||
| test_configuration_common.py | ||
| test_configuration_utils.py | ||
| test_feature_extraction_common.py | ||
| test_feature_extraction_utils.py | ||
| test_image_processing_common.py | ||
| test_image_processing_utils.py | ||
| test_image_transforms.py | ||
| test_modeling_common.py | ||
| test_modeling_flax_common.py | ||
| test_modeling_flax_utils.py | ||
| test_modeling_tf_common.py | ||
| test_modeling_tf_utils.py | ||
| test_modeling_utils.py | ||
| test_pipeline_mixin.py | ||
| test_sequence_feature_extraction_common.py | ||
| test_tokenization_common.py | ||
| test_tokenization_utils.py | ||