The code of the paper "A Closer Look at Transformers for Time Series Forecasting: Understanding Why They Work and Where They Struggle".
The code was extended from the time series models library. All the transfromers compared in the paper are using the implementation from this library except specific extentions described in the paper.
Please download the benchmark datasets according to the instructions in time series models library.
The simulated datasets can be generated by the following two files in data_provider folder: sim_dependent_data.py and sim_independent_data.py.
requirement.txt includes packages we installed in the virtual environment for running all experiments.
run.py is for training and testing models and is inherited from time series models library with some additional configurations.
est_mi.py is for estimating Mutual information Scores of trained models. Please note that est_mi.py will require large GPU memory for datasets with a large number of features. est_mi_slow.py is for such cases that needs to fit into a small GPU memory.
Please find example scripts of running these files in the script folder. Specific configurations for extentions described in the paper:
`--no_skip`: if it's True, skip connection in attention layer will be removed. Only iTransformer used this configuration in the experiments included in the paper.
`--fuse_decoder`: Add a fuse layer to enable capturing interaction between variables in the decoder. True or False. Only iTransformer used this configuration in the experiments included in the paper.
`--decoder_type`: when `--fuse_decoder` set to True , this can be set to 'conv2d' or 'MLP' to specify the type of an additional fuse layer in the decoder.
`--no_zero_norm`: if it's True, zero-score normalization will be removed. Crossformer, iTransformer, PatchTST, and TimeXer used this configuration in the experiments included in the paper.
@inproceedings{
chen2025a,
title={A Closer Look at Transformers for Time Series Forecasting: Understanding Why They Work and Where They Struggle},
author={Yu Chen and Nathalia C{\'e}spedes and Payam Barnaghi},
booktitle={Forty-second International Conference on Machine Learning},
year={2025},
url={https://openreview.net/forum?id=kHEVCfES4Q}
}
Please feel free to contact me by this email yc14600@gmail.com if you have any questions.