mirror of
https://github.com/huggingface/transformers.git
synced 2025-08-03 03:31:05 +06:00
2ac5b9325e
* Initial commit of PatchTST model classes Co-authored-by: Phanwadee Sinthong <phsinthong@gmail.com> Co-authored-by: Nam Nguyen <namctin@gmail.com> Co-authored-by: Vijay Ekambaram <vijaykr.e@gmail.com> Co-authored-by: Ngoc Diep Do <55230119+diepi@users.noreply.github.com> Co-authored-by: Wesley Gifford <79663411+wgifford@users.noreply.github.com> * Add PatchTSTForPretraining * update to include classification Co-authored-by: Phanwadee Sinthong <phsinthong@gmail.com> Co-authored-by: Nam Nguyen <namctin@gmail.com> Co-authored-by: Vijay Ekambaram <vijaykr.e@gmail.com> Co-authored-by: Ngoc Diep Do <55230119+diepi@users.noreply.github.com> Co-authored-by: Wesley Gifford <79663411+wgifford@users.noreply.github.com> * clean up auto files * Add PatchTSTForPrediction * Fix relative import * Replace original PatchTSTEncoder with ChannelAttentionPatchTSTEncoder * temporary adding absolute path + add PatchTSTForForecasting class * Update base PatchTSTModel + Unittest * Update ForecastHead to use the config class * edit cv_random_masking, add mask to model output * Update configuration_patchtst.py * add masked_loss to the pretraining * add PatchEmbeddings * Update configuration_patchtst.py * edit loss which considers mask in the pretraining * remove patch_last option * Add commits from internal repo * Update ForecastHead * Add model weight initilization + unittest * Update PatchTST unittest to use local import * PatchTST integration tests for pretraining and prediction * Added PatchTSTForRegression + update unittest to include label generation * Revert unrelated model test file * Combine similar output classes * update PredictionHead * Update configuration_patchtst.py * Add Revin * small edit to PatchTSTModelOutputWithNoAttention * Update modeling_patchtst.py * Updating integration test for forecasting * Fix unittest after class structure changed * docstring updates * change input_size to num_input_channels * more formatting * Remove some unused params * Add a comment for pretrained models * add channel_attention option add channel_attention option and remove unused positional encoders. * Update PatchTST models to use HF's MultiHeadAttention module * Update paper + github urls * Fix hidden_state return value * Update integration test to use PatchTSTForForecasting * Adding dataclass decorator for model output classes * Run fixup script * Rename model repos for integration test * edit argument explanation * change individual option to shared_projection * style * Rename integration test + import cleanup * Fix outpu_hidden_states return value * removed unused mode * added std, mean and nops scaler * add initial distributional loss for predition * fix typo in docs * add generate function * formatting * add num_parallel_samples * Fix a typo * copy weighted_average function, edit PredictionHead * edit PredictionHead * add distribution head to forecasting * formatting * Add generate function for forecasting * Add generate function to prediction task * formatting * use argsort * add past_observed_mask ordering * fix arguments * docs * add back test_model_outputs_equivalence test * formatting * cleanup * formatting * use ACT2CLS * formatting * fix add_start_docstrings decorator * add distribution head and generate function to regression task add distribution head and generate function to regression task. Also made add PatchTSTForForecastingOutput, PatchTSTForRegressionOutput. * add distribution head and generate function to regression task add distribution head and generate function to regression task. Also made add PatchTSTForForecastingOutput, PatchTSTForRegressionOutput. * fix typos * add forecast_masking * fixed tests * use set_seed * fix doc test * formatting * Update docs/source/en/model_doc/patchtst.md Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com> * better var names * rename PatchTSTTranspose * fix argument names and docs string * remove compute_num_patches and unused class * remove assert * renamed to PatchTSTMasking * use num_labels for classification * use num_labels * use default num_labels from super class * move model_type after docstring * renamed PatchTSTForMaskPretraining * bs -> batch_size * more review fixes * use hidden_state * rename encoder layer and block class * remove commented seed_number * edit docstring * Add docstring * formatting * use past_observed_mask * doc suggestion * make fix-copies * use Args: * add docstring * add docstring * change some variable names and add PatchTST before some class names * formatting * fix argument types * fix tests * change x variable to patch_input * format * formatting * fix-copies * Update tests/models/patchtst/test_modeling_patchtst.py Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com> * move loss to forward * Update src/transformers/models/patchtst/modeling_patchtst.py Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com> * Update src/transformers/models/patchtst/modeling_patchtst.py Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com> * Update src/transformers/models/patchtst/modeling_patchtst.py Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com> * Update src/transformers/models/patchtst/modeling_patchtst.py Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com> * Update src/transformers/models/patchtst/modeling_patchtst.py Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com> * formatting * fix a bug when pre_norm is set to True * output_hidden_states is set to False as default * set pre_norm=True as default * format docstring * format * output_hidden_states is None by default * add missing docs * better var names * docstring: remove default to False in output_hidden_states * change labels name to target_values in regression task * format * fix tests * change to forecast_mask_ratios and random_mask_ratio * change mask names * change future_values to target_values param in the prediction class * remove nn.Sequential and make PatchTSTBatchNorm class * black * fix argument name for prediction * add output_attentions option * add output_attentions to PatchTSTEncoder * formatting * Add attention output option to all classes * Remove PatchTSTEncoderBlock * create PatchTSTEmbedding class * use config in PatchTSTPatchify * Use config in PatchTSTMasking class * add channel_attn_weights * Add PatchTSTScaler class * add output_attentions arg to test function * format * Update doc with image patchtst.md * fix-copies * rename Forecast <-> Prediction * change name of a few parameters to match with PatchTSMixer. * Remove *ForForecasting class to match with other time series models. * make style * Remove PatchTSTForForecasting in the test * remove PatchTSTForForecastingOutput class * change test_forecast_head to test_prediction_head * style * fix docs * fix tests * change num_labels to num_targets * Remove PatchTSTTranspose * remove arguments in PatchTSTMeanScaler * remove arguments in PatchTSTStdScaler * add config as an argument to all the scaler classes * reformat * Add norm_eps for batchnorm and layernorm * reformat. * reformat * edit docstring * update docstring * change variable name pooling to pooling_type * fix output_hidden_states as tuple * fix bug when calling PatchTSTBatchNorm * change stride to patch_stride * create PatchTSTPositionalEncoding class and restructure the PatchTSTEncoder * formatting * initialize scalers with configs * edit output_hidden_states * style * fix forecast_mask_patches doc string --------- Co-authored-by: Gift Sinthong <gift.sinthong@ibm.com> Co-authored-by: Nam Nguyen <namctin@gmail.com> Co-authored-by: Vijay Ekambaram <vijaykr.e@gmail.com> Co-authored-by: Ngoc Diep Do <55230119+diepi@users.noreply.github.com> Co-authored-by: Wesley Gifford <79663411+wgifford@users.noreply.github.com> Co-authored-by: Wesley M. Gifford <wmgifford@us.ibm.com> Co-authored-by: nnguyen <nnguyen@us.ibm.com> Co-authored-by: Ngoc Diep Do <diiepy@gmail.com> Co-authored-by: Kashif Rasul <kashif.rasul@gmail.com> Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com> Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com> |
||
|---|---|---|
| .. | ||
| albert | ||
| align | ||
| altclip | ||
| audio_spectrogram_transformer | ||
| auto | ||
| autoformer | ||
| bark | ||
| bart | ||
| barthez | ||
| bartpho | ||
| beit | ||
| bert | ||
| bert_generation | ||
| bert_japanese | ||
| bertweet | ||
| big_bird | ||
| bigbird_pegasus | ||
| biogpt | ||
| bit | ||
| blenderbot | ||
| blenderbot_small | ||
| blip | ||
| blip_2 | ||
| bloom | ||
| bridgetower | ||
| bros | ||
| byt5 | ||
| camembert | ||
| canine | ||
| chinese_clip | ||
| clap | ||
| clip | ||
| clipseg | ||
| clvp | ||
| code_llama | ||
| codegen | ||
| conditional_detr | ||
| convbert | ||
| convnext | ||
| convnextv2 | ||
| cpm | ||
| cpmant | ||
| ctrl | ||
| cvt | ||
| data2vec | ||
| deberta | ||
| deberta_v2 | ||
| decision_transformer | ||
| deformable_detr | ||
| deit | ||
| deta | ||
| detr | ||
| dinat | ||
| dinov2 | ||
| distilbert | ||
| dit | ||
| donut | ||
| dpr | ||
| dpt | ||
| efficientformer | ||
| efficientnet | ||
| electra | ||
| encodec | ||
| encoder_decoder | ||
| ernie | ||
| ernie_m | ||
| esm | ||
| falcon | ||
| flaubert | ||
| flava | ||
| fnet | ||
| focalnet | ||
| fsmt | ||
| funnel | ||
| fuyu | ||
| git | ||
| glpn | ||
| gpt_bigcode | ||
| gpt_neo | ||
| gpt_neox | ||
| gpt_neox_japanese | ||
| gpt_sw3 | ||
| gpt2 | ||
| gptj | ||
| gptsan_japanese | ||
| graphormer | ||
| groupvit | ||
| herbert | ||
| hubert | ||
| ibert | ||
| idefics | ||
| imagegpt | ||
| informer | ||
| instructblip | ||
| jukebox | ||
| kosmos2 | ||
| layoutlm | ||
| layoutlmv2 | ||
| layoutlmv3 | ||
| layoutxlm | ||
| led | ||
| levit | ||
| lilt | ||
| llama | ||
| longformer | ||
| longt5 | ||
| luke | ||
| lxmert | ||
| m2m_100 | ||
| marian | ||
| markuplm | ||
| mask2former | ||
| maskformer | ||
| mbart | ||
| mbart50 | ||
| mega | ||
| megatron_bert | ||
| megatron_gpt2 | ||
| mgp_str | ||
| mistral | ||
| mluke | ||
| mobilebert | ||
| mobilenet_v1 | ||
| mobilenet_v2 | ||
| mobilevit | ||
| mobilevitv2 | ||
| mpnet | ||
| mpt | ||
| mra | ||
| mt5 | ||
| musicgen | ||
| mvp | ||
| nat | ||
| nezha | ||
| nllb | ||
| nllb_moe | ||
| nougat | ||
| nystromformer | ||
| oneformer | ||
| openai | ||
| opt | ||
| owlv2 | ||
| owlvit | ||
| patchtst | ||
| pegasus | ||
| pegasus_x | ||
| perceiver | ||
| persimmon | ||
| phi | ||
| phobert | ||
| pix2struct | ||
| plbart | ||
| poolformer | ||
| pop2piano | ||
| prophetnet | ||
| pvt | ||
| qdqbert | ||
| rag | ||
| realm | ||
| reformer | ||
| regnet | ||
| rembert | ||
| resnet | ||
| roberta | ||
| roberta_prelayernorm | ||
| roc_bert | ||
| roformer | ||
| rwkv | ||
| sam | ||
| seamless_m4t | ||
| segformer | ||
| sew | ||
| sew_d | ||
| speech_encoder_decoder | ||
| speech_to_text | ||
| speech_to_text_2 | ||
| speecht5 | ||
| splinter | ||
| squeezebert | ||
| swiftformer | ||
| swin | ||
| swin2sr | ||
| swinv2 | ||
| switch_transformers | ||
| t5 | ||
| table_transformer | ||
| tapas | ||
| time_series_transformer | ||
| timesformer | ||
| timm_backbone | ||
| transfo_xl | ||
| trocr | ||
| tvlt | ||
| umt5 | ||
| unispeech | ||
| unispeech_sat | ||
| upernet | ||
| videomae | ||
| vilt | ||
| vision_encoder_decoder | ||
| vision_text_dual_encoder | ||
| visual_bert | ||
| vit | ||
| vit_hybrid | ||
| vit_mae | ||
| vit_msn | ||
| vitdet | ||
| vitmatte | ||
| vits | ||
| vivit | ||
| wav2vec2 | ||
| wav2vec2_conformer | ||
| wav2vec2_phoneme | ||
| wav2vec2_with_lm | ||
| wavlm | ||
| whisper | ||
| x_clip | ||
| xglm | ||
| xlm | ||
| xlm_prophetnet | ||
| xlm_roberta | ||
| xlm_roberta_xl | ||
| xlnet | ||
| xmod | ||
| yolos | ||
| yoso | ||
| __init__.py | ||