Time-series forecasting is becoming a lost art. It’s the engine behind: • Retail • Insurance • Energy pricing • Fitness tracking (and more) Retail alone is a $5 trillion industry in the US! And yet, the tricks of the trade are locked away. Here’s what you should know: (View Tweet)
EDA The best way to start any ML problem. Here are some ways to analyze time series data: • Seasonality (monthly, daily, etc.) • Trends (big for proper evaluation) • Autocorrelation • Diff. between older and newer series • Tons of raw samples (View Tweet)
Don’t forget to overlay other variables than just historical factors, though. Often what appears to be a random fluctuation has an obvious cause. Price shifts, advertising campaigns, etc. (View Tweet)
Benchmarking Is an MAE of 96.05 good? What about a MAPE of 36%? There’s no way to know unless you create a dumb benchmark. Predicting the mean is decent, but keep in mind that even a naive forecaster could outperform that usually. (View Tweet)
Try things that better consider the domain, such as a rolling mean, predicting the value seen in the last day of available data, etc. Then, use the performance of that baseline to benchmark your models down the line. (View Tweet)
Domain expertise If available, ALWAYS engage with someone that understands the underlying operation and dynamics at play from the business side. There can be spikes caused by collection issues, strikes, out of stocks, you name it. A 5 minute convo can save you 50 hours. (View Tweet)
Model evaluation Any answer other than roll-forward/expanding window cross-validation is wrong for 99.999% of problems. Do NOT do any sort of random split on your samples that doesn’t ensure future values aren’t included. That’s a death sentence for production. (View Tweet)
And don’t forget evaluation metrics. MAE and RMSE are fine. MAPE/WAPE have their place. But if you aren’t using scaled metrics like RMSSE and MASE, you probably aren’t capturing your model performance very well. (View Tweet)
Even better: deep dive the underlying business operations so that you can directly convert your forecast accuracy into $$$ uplift for your business. (View Tweet)
Now, we're gonna get into the fun stuff: modeling techniques! “State-of-the-art” largely depends on business use-case. I’ll give something for everyone here. (View Tweet)
Statistical models In low-data environments, it’s hard to beat good ol’ ARIMA and ETS. But that’s not the end of the story. There have been some innovations in the form of the Theta model, STR, and many others (View Tweet)
LightGBM/XGBoost If your company has any decent amount of data, you probably won’t beat this. Combine rolling/lag features, grouped aggregations, and all of the external variables you can get your hands on and there’s a chance you can slap one model onto your whole enterprise. (View Tweet)
LSTMs/GRUs/Transformers If you have a MASSIVE amount of data and spend enough time tuning the model, you can beat LightGBM/XGBoost sometimes. The same feature engineering techniques work, but keep in mind that these models are much more finicky about the scale of the data. (View Tweet)
Combining predictions Time-series data changes its behavior all the time, so ensembling multiple models usually gives much more robust results. Once you’ve found your best single model, try all of the other ones on the previous list and ensemble some of them for diversity! (View Tweet)
Hierarchical Forecasting is also incredibly powerful for making your forecasts more coherent. Not only will it make your forecasts at different levels agree with each other, but it can also give a nice accuracy boost! (View Tweet)
Production In addition to the common concerns of data/model drift, you have to actually apply your models. Forecasts are usually just an intermediate output into something like: • Labor modeling • Causal analysis • Ad targeting …etc So how do you optimize for that? (View Tweet)
Topic TL;DR Domain-specific EDA Domain-specific benchmarking Stakeholder relationships for anomalies Model evaluation Pick your model from data volume Combining predictions Applying your models in production (View Tweet)
If you want to learn these topics in detail, join the second cohort of my course. Last run had over 100 engaged students! https://t.co/y0DHX1MQH7 (View Tweet)
Don’t take my word for it. Thank you! (View Tweet)