You always have choices. Should I order cheese in my burger? Should I choose the cheese or cream knafa? There are also additional choices, not immediately obvious from the menu. There might be ice cream on the menu, but you could have that with the knafa, rather than as an either/or situation. Then there are things totally off the menu, that you didn’t even know about. Maybe there’s a secret menu, that you’ll have to know about to ask for. When it comes to researching how to improve our forecasts at Turnleaf Analytics, it is all about finding those choices, some of which we might already know about. In other instances, these choices could be something totally new.
In our case, it boils down to two things, finding new data and ways of improving the modelling. Modelling is always some sort of convenient approximation for the world. Are there ways we can make the model closer to reality? At every step in the forecasting pipeline, we make specific choices in how we treat the data, that can impact the final outcome. All of these steps in the pipeline are subject to continual research to see if we can make improvements.
First, we need to think about how we treat missing data in a time series, then how to deal with outliers etc. There are many different algorithms to solve these problems each with their own specific tradeoffs. Are there things we know about the data, which we can model separately? One of these examples can be seasonality. Even a simple plot can show that inflation very often exhibits seasonality. Can we think of ways to model seasonality in an appropriate way? We have looked at many algorithms to deal with seasonality and have also created our own proprietary algorithm. Our MARCOS platform includes many of these proprietary algorithms to preprocess the data, as well as many regression models, to allow clients to create their own economic forecasts, for whatever indicator they want.
Feature engineering is one way to add considerable value to the model, but it requires a lot of time and domain expertise. Hence, you need to make a choice of where to focus this energy. You don’t have enough bandwidth to do detailed feature engineering everywhere. One example of this is creating high frequency indices, from existing data we’ve collected, to track consumer prices for a number of sectors like food etc. However, there are also many other areas where we’ve applied feature engineering to improve the model. The dataset is limited (limited y points, and lots of x variables), which means we cannot simply throw lots of (irrelevant) data at a data hungry model and let it “feature engineer” for us (eg. this happened with computer vision as an example).
On the data side, there can well be places where we know we have missing data, and it will take time to find the source for that. In other instances, there might be data missing that we have no idea about. When you miss an inflation print, this can help identify what data is missing for the model, at least from a short term forecasting perspective. However, in other instances, there might be data we don’t know is missing. It’ll take an element of lateral thinking to find. Just because a dataset sounds cool, it doesn’t necessarily mean it’ll be useful as well. Additional research is needed to see whether it really does help to improve the forecasting performance of the model. The sources of data can vary between countries, as can the amount of data we can collect, particularly in EM. We forecast inflation across 36 countries both in DM and EM, so the continual searching for new and innovative data sources is a big challenge. We have a data scout continually scouring for new data to improve the model.
Whilst I’ve given this example related to financial research (and ordering at a restaurant), it’s just as applicable in many other areas like in a career. Whatever choice you make, there’s likely to be many more choices you could make, that you still don’t know about.
The key point: not making a choice is not an option! And yes, I made the choice to illustrate the post with a photo of something totally different..