TSAD: Architecture Design for Time Series Forecasting

While Transformers have revolutionized the field of time series forecasting, they still largely depend on manually designed components—applying uniform types of attention, normalization, and activation functions across diverse contexts. What if, instead, we could automatically determine the most suitable architecture for each unique dataset? This paper introduces TSAD (Time Series Architecture Design), an end-to-end neural architecture search (NAS) framework tailored for time series Transformers. TSAD simultaneously optimizes multiple architectural dimensions—including attention mechanisms, normalization layers, activation functions, and embedding operations—through a differentiable architecture search strategy. This approach enables the discovery of customized model configurations that consistently outperform handcrafted baseline models. Our empirical analysis reveals a surprising and counterintuitive phenomenon: architectures discovered via NAS, despite their high complexity, often suffer from significant performance deterioration. In contrast, simpler and more structurally transparent designs tend to deliver stronger and more stable generalization. Our work not only extends the frontier of automated model design for sequential data but also provides critical insights that may reshape how neural architecture search is approached in time series modeling.

• Publication year

  2025

• Venue

  2025 International Conference on Big Data Applications, Mechatronics Engineering and Automation (BDAMEA)

• Publication date

  2025-12-12

• Fields of study

  Not labeled

• Identifiers
  DOI 10.1109/BDAMEA68159.2025.11406682
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Neural Architecture Search for global multi-step Forecasting of Energy Production Time Series

  2025cited by this paper
• Long-term Forecasting with TiDE: Time-series Dense Encoder

  2023cited by this paper
• TransNAS-TSAD: harnessing transformers for multi-objective neural architecture search in time series anomaly detection

  2023cited by this paper
• iTransformer: Inverted Transformers Are Effective for Time Series Forecasting

  2023cited by this paper
• Crossformer: Transformer Utilizing Cross-Dimension Dependency for Multivariate Time Series Forecasting

  2023cited by this paper
• Revisiting Long-term Time Series Forecasting: An Investigation on Linear Mapping

  2023cited by this paper
• FEDformer: Frequency Enhanced Decomposed Transformer for Long-term Series Forecasting

  2022cited by this paper
• Transformers in Time Series: A Survey

  2022cited by this paper
• Are Transformers Effective for Time Series Forecasting?

  2022cited by this paper
• TimesNet: Temporal 2D-Variation Modeling for General Time Series Analysis

  2022cited by this paper
• Neural Architecture Search for Transformers: A Survey

  2022cited by this paper
• Non-stationary Transformers: Exploring the Stationarity in Time Series Forecasting

  2022cited by this paper
• A Time Series is Worth 64 Words: Long-term Forecasting with Transformers

  2022cited by this paper
• Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecasting

  2021cited by this paper
• SCINet: Time Series Modeling and Forecasting with Sample Convolution and Interaction

  2021cited by this paper
• Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting

  2020cited by this paper
• GLU Variants Improve Transformer

  2020cited by this paper
• Searching for a Robust Neural Architecture in Four GPU Hours

  2019cited by this paper
• PC-DARTS: Partial Channel Connections for Memory-Efficient Architecture Search

  2019cited by this paper
• Efficient Neural Architecture Search via Parameter Sharing

  2018cited by this paper
• An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling

  2018cited by this paper
• DARTS: Differentiable Architecture Search

  2018cited by this paper
• Learning Transferable Architectures for Scalable Image Recognition

  2017cited by this paper
• Attention is All you Need

  2017cited by this paper
• Effective Approaches to Attention-based Neural Machine Translation

  2015cited by this paper
• Neural Machine Translation by Jointly Learning to Align and Translate

  2014cited by this paper
• Novel neural modulators.

  2003cited by this paper

• No citing papers are available for this paper.