Problems encountered in producing forecasts are (1) limitations to using economic data, (2) data measurement error and bias, (3) limitations of historical estimates, (4) the use of ex-post risk and return measures, (5) non-repeating data patterns, (6) failing to account for conditioning information, (7) misinterpreting of correlations, (8) psychological bias, and (9) model uncertainty
- Data time lags – The time lag with which economic data are collected, processed, and disseminated can impede their use because data that are not timely may be of little value in assessing current conditions.
- Data Revisions – Sometimes these revisions are substantial, which may give rise to significantly different inferences.
- Changes in Data Definitions and Methodology Over time – Analysts should also be aware that suppliers of economic and financial indexes periodically re-base these indexes, meaning that the specific period used as the base of the index is changed
- Measurement Errors and Biases:
- Transcription Errors – These are errors in gathering and recording data.
- Survivorship Bias – This bias arises when a data series reflects only entities that survived to the end of the period.
- Appraisal Bias – For certain assets without liquid public markets, notably but not only real estate, appraisal data are used in lieu of transaction data. Appraised values tend to be less volatile than market-determined values.
- Limitations of historical estimates:
- Regime changes cause nonstationary data – Changes in technological, political, legal, and regulatory environments; disruptions such as wars and other calamities; and changes in policy stances can all alter risk–return relationships. Called regime changes.
- Nonstationarity – different parts of a data series reflect different underlying statistical properties
- Longer time periods reduce this limitation
- Regime changes cause nonstationary data – Changes in technological, political, legal, and regulatory environments; disruptions such as wars and other calamities; and changes in policy stances can all alter risk–return relationships. Called regime changes.
- Using ex-post data to calculate ex ante risk/return – Looking backward, we are likely to underestimate ex ante risk and overestimate ex ante anticipated returns. The key point is that high ex post returns that reflect fears of adverse events that did not materialize provide a poor estimate of ex ante expected returns.
- Data mining – This arises from repeatedly searching a dataset until a statistically significant pattern emerges.
- Time Period Bias – This relates to results that are period specific. The time period selected can alter results.
- Fail to account for conditioning information – The analyst should not ignore relevant information or analysis in formulating expectations. Unconditional forecasts, which dilute this information by averaging over environments, can lead to misperception of prospective risk and return. Thus, analysts should account for current conditions in their forecasts
- Misinterpretations of correlations – Correlations do not prove causation
- Psychological Biases :
- Anchoring bias is the tendency to give disproportionate weight to the first information received or first number envisioned, which is then adjusted.
- Status quo bias reflects the tendency for forecasts to perpetuate recent observations—that is, to avoid making changes and preserve the status quo, and/or to accept a default option.
- Confirmation bias is the tendency to seek and overweight evidence or information that confirms one’s existing or preferred beliefs and to discount evidence that contradicts those beliefs.
- Overconfidence bias is unwarranted confidence in one’s own intuitive reasoning, judgment, knowledge, and/or ability.
- Prudence bias reflects the tendency to temper forecasts so that they do not appear extreme or the tendency to be overly cautious in forecasting.
- Availability bias is the tendency to be overly influenced by events that have left a strong impression and/or for which it is easy to recall an example.
- Model Uncertainty – pertains to whether a selected model is structurally and/or conceptually correct
- Parameter Uncertainty – Arises because a quantitative model’s parameters are invariably estimated with error.
- Input Uncertainty – Occurs if inputs are incorrect