When Big Data Isn’t Enough: Solving the long-range forecasting problem in supervised learning

DOI

10.2991/smont-19.2019.51How to use a DOI?

Keywords

forecasting; supervised learning; neural networks; data mining; age-period-cohort models

Abstract

In a world where big data is everywhere, no one has big data relative to the economic cycle. Data volume needs to be thought of along two dimensions. (1) How many accounts / transactions / data fields do we have? (2) How much time history do we have? Few, if any, big data sets include history covering one economic cycle (back to 2005) or two economic cycles (back to 1998). Therefore, unstructured learning algorithms will be unable to distinguish between long-term macroeconomic drivers and point-in-time variations across accounts or transactions. This is the colinearity problem that is well known in consumer lending. This paper presents a solution to the colinearity problem in the context of applying neural networks to modeling consumer behavior. An initial model is built using methods that are specifically tuned to capture long-term drivers of performance. That model is treated as given information to a neural network that then learns potentially highly non-linear dynamics relative to the given knowledge. This approach of incorporating given knowledge into a supervised learning algorithm solves a deep but rarely recognized problem. In short, almost all models created via supervised learning will not give correct long-range forecasts when long-term drivers are present without corresponding data covering multiple cycles for those drivers. Thus, the need for such a solution is great and the solution provided here is sufficiently general that it can apply to a broad range of applications where both high frequency and low frequency drivers are present in the data.)

Copyright

© 2019, the Authors. Published by Atlantis Press.

Open Access

This is an open access article distributed under the CC BY-NC license (http://creativecommons.org/licenses/by-nc/4.0/).

Download article (PDF)

TY  - CONF
AU  - Joseph L. Breeden
AU  - Eugenia Leonova
PY  - 2019/04
DA  - 2019/04
TI  - When Big Data Isn’t Enough: Solving the long-range forecasting problem in supervised learning
BT  - Proceedings of the 2019 International Conference on Modeling, Simulation, Optimization and Numerical Techniques (SMONT 2019)
PB  - Atlantis Press
SP  - 229
EP  - 232
SN  - 1951-6851
UR  - https://doi.org/10.2991/smont-19.2019.51
DO  - 10.2991/smont-19.2019.51
ID  - Breeden2019/04
ER  -