How to trust forecast data.
Curiosity – what will the future be like?
Few things have interested humans since they began walking upright as much as the question, "What will the future be like, what will it look like around the next bend in the river, or when will the sun move north again in winter?" The drive is a mixture of curiosity and the evolutionary need to be able to act in a planned manner in order to survive. Curiosity makes the synapses in the brain thicker in childhood, they network more intensively in adults, and even brain mass continues to grow in old age if people remain curious. The curiosity to look into the future and also to develop a suitable methodology for it has two interesting aspects. A tactical one, which ensures short-term survival, and a strategic evolutionary one, through the emergence of more powerful brains.
Without being able to look into the future as accurately as possible, the world would come to a standstill. Curiosity became a necessity; nothing works without forecasts. Not only electricity and traffic networks would collapse. The human brain alone can no longer penetrate today's wealth of data and complexity. An interesting evolutionary step took place, which would probably have fascinated Darwin: The machine extension of the human brain, "electron brains", equipped with artificial intelligence.
From prophecy to prognosis.
The desire or the necessity to take a look into the future fueled the search for a suitable methodology to do so. At first, the most practical solution seemed to be to contact the gods through oracles, priests, medicine men, etc., to ask them about the future and whether it was favorable. In addition to the more obscure methods, such as the visceral examination, there were elements that could have a certain predictive power: Celestial constellations or bird flight, which regularly preceded events.
The way from prophecy, divination or fortune-telling to non-trivial prognosis, was an astonishingly long one. Even the rationally gifted ancient Greeks did not manage it yet. It was not until mankind entered the most innovative era of its existence, the Renaissance, that the realization dawned that the future could hardly depend on the flight of birds. The world was to be made calculable and predictable. Mathematicians such as Gerolamo Cardano, with important insights into the calculation of probability, or Johannes Keppler, with the exact calculation of the orbits of the planets, set about calculating and measuring the world.
Forecasting and trust – the foundation of trust.
Trust plays a central role in a person's life. His experiences in the first year of life determine how he will basically approach other people in later life: Distrustful or trusting. Trust in social life, the prediction of how a counterpart might behave, is more an emotional than an analytical matter. If critical decisions are made on the basis of forecasts, the issue of trust becomes a central question. The foundation of trust is formed by the data basis, model selection and methodology. Together with the traceability of the forecast values, as well as the regular accuracy of forecast to actual data, trust is created in the forecasting process.
Forecasts based on the HAKOM TSM and the 3D Trust.
For the calculation of trustworthy forecasts, trustworthy basic technologies are necessary and this over the entire processing chain. This concerns the development of the hypothesis, the data measurement and transmission, the preprocessing up to the determination of the results. Time series play a central role in the entire forecasting process.
Data preparation with the HAKOM TSM Time Series Manager.
The foundation of the trust is the data base, which must be representative as well as correct and reliable. Before modeling, time series that will be included in the model have to be brought into a processable form, i.e. outliers have to be removed, missing data have to be methodically correctly added or marked, the time series have to be brought to the same time grid and, last but not least, time zones have to be correctly considered. This requires, especially in the area of Big Data, a high performance time series technology, such as the HAKOM TSM.
Forecast Model construction, analysis and selection & the "time travel".
Once the time series have been appropriately prepared, modeling can begin. Subsequently, different forecasting techniques can also be tested on the basis of this model. To train the models, forecasts are calculated for historical data for values that also lie in the past. In order to be able to compare forecast results, it is necessary to be able to reconstruct input data of the different models at a later point in time. This is the only way to check whether a modified model and/or forecasting technique would actually produce better results under the same input variables. This methodology could also be interpreted as "time travel": A historical condition is reconstructed in a time series.
The HAKOM 3D Trust Approach.
To set up a trustworthy forecasting process, the HAKOM TSM Time Series Manager with the 3D Timescale Support provides 3 time dimensions:
- Dimension 1: Base time axis.
- Dimension 2: Audit axis.
- Dimension 3: Notation axis.
The first dimension chronologically records the data to be observed. In the second dimension, the audit axis logs the input data and forecasts, setting the server timestamp for the forecast, which cannot be manipulated. A "state" of a time series is created. Since input data and forecast data are separated in time, they are kept in sync via this server timestamp to conveniently navigate through the data sets and reconstruct when they were created. The third dimension, the notation axis, allows different "states" of time series to be assigned to a version, for example to be able to assign the results and their bases to a specific point in time.
A technology that creates trust needs these three time dimensions – trust, the HAKOM TSM technology – transparent and traceable.
HAKOM TSM – not only for forecasts.
Time series are not only a forecasting topic. Among other things, they play an essential role in the field of business analytics, Business Intelligence (BI). In general, the development of computational methods without a forecasting approach, e.g. in optimization processes, requires these three time axes in order to ultimately obtain trustworthy results and trust.
And in all mentioned cases there is a focus on the visualization of data. With the TSM Visuals, the Grafana or PowerBI Connector for the TSM, HAKOM offers strong solutions for the graphical visualization of results and the emergence of trust.
HAKOM TSM – speed on board.
Many time series technologies rely on one time axis, one dimension. If these technologies go into operational use after the modeling phase, a second dimension is missing as an audit axis to log input data and calculation results. The lack of traceability of computational results is a knockout criterion for the trust. In addition, a versioning requirement arises in each model with the time, which creates the need for the third dimension – the notation axis. A subsequent technical addition of further time axes leads to a loss of computing speed. This is not practical, especially for Big Data. High performance through the entire workflow with the HAKOM TSM Time Series Manager: Three dimensions on-board right from the start.
Trust is the essential factor for any time series technology "... and don't forget: Life is a time series!"
