TIME-MOE: Billion-Scale Time Series Foundation Model with Mixture-of-Experts | by Nikos Kafritsas | Oct, 2024

And open-source as well!

The Mixture-of-Experts (MOE) architecture has surged in popularity with the rise of large language models (LLMs).

As time-series models adopt cutting-edge techniques, Mixture-of-Experts has naturally found its place in the time-series foundation space.

This article discusses Time-MOE, a time-series foundation model that uses MOE to improve forecasting accuracy while reducing computational costs. Key contributions include:

1. Time-300B Dataset: The largest open time-series dataset, with 300 billion time points across 9 domains, and a scalable data-cleaning pipeline.
2. Scaling Laws for Time Series: Insights into how scaling laws affect large time-series models.
3. Time-MOE architecture: A family of open-source time-series models leveraging MOE to enhance performance.

Let’s get started

✅ Find the hands-on project for Time-MOE in the AI Projects folder, along with other cool projects!

Time-MOE is a 2.4B parameter open-source time-series foundation model using Mixture-of-Experts (MOE) for zero-shot forecasting