Welcome to your guide on implementing the Informer model, a breakthrough architecture designed for long-sequence time-series forecasting. The Informer has been recognized as the Best Paper at AAAI21 and leverages ProbSparse Attention to improve efficiency in processing large datasets. In this blog, we’ll walk you through the steps to set up and run the model, ensuring you can easily apply this powerful approach to your own datasets.

Requirements

Before diving into the implementation, ensure you have the following dependencies installed:

• Python 3.6
• matplotlib == 3.1.1
• numpy == 1.19.4
• pandas == 0.25.1
• scikit_learn == 0.21.3
• torch == 1.8.0

You can quickly install the dependencies using the command:

pip install -r requirements.txt

Setting Up the Data

The ETT dataset used in the Informer model can be downloaded from the repo ETDataset. Make sure to place all required data files into the data/ETT folder. Additionally, you can find the ECL data and Weather data through the links below:

• Google Drive
• BaiduPan, password: 6gan

Steps to Reproduce Results

Now that your environment is set up, follow these steps to easily reproduce the results:

1. Initialize the Docker image using: make init.
2. Download the datasets using: make dataset.
3. Run each script in the scripts directory using: make run_module module=bash ETTh1.sh.
4. Alternatively, run all the scripts at once:

for file in $(ls scripts); do make run_module module=bash scripts/$file; done

Using Informer for Forecasting

Here are the commands to train and test the model using the ProbSparse self-attention on different datasets:

# ETTh1
python -u main_informer.py --model informer --data ETTh1 --attn prob --freq h

# ETTh2
python -u main_informer.py --model informer --data ETTh2 --attn prob --freq h

# ETTm1
python -u main_informer.py --model informer --data ETTm1 --attn prob --freq t

Understanding the Code with an Analogy

Think of the Informer architecture like a well-organized library. Each book (data point) is categorized to easily find relevant information. The library recommends books based on popularity (ProbSparse Attention), ensuring you spend less time sifting through shelves (data) filled with content that is less relevant (lazy queries). Instead, you quickly retrieve the most frequently borrowed items (active queries), allowing for efficient access to knowledge (accurate forecasting).

Troubleshooting

In case you encounter errors, here are some troubleshooting tips:

• If you experience a runtime error like RuntimeError: The size of tensor a (98) must match the size of tensor b (96) at non-singleton dimension 1, ensure that your PyTorch version is compatible. You might need to adjust the Conv1d settings in modelsembed.py to align with your current version.
• Check that all dataset files are correctly placed in the data/ETT directory.

For more insights, updates, or to collaborate on AI development projects, stay connected with fxis.ai.

Final Thoughts

At fxis.ai, we believe that such advancements are crucial for the future of AI, as they enable more comprehensive and effective solutions. Our team is continually exploring new methodologies to push the envelope in artificial intelligence, ensuring that our clients benefit from the latest technological innovations.