Guest blog: Forecasting is impossible

Another great guest blog from the Value Perspective team at Schroders.  They point out the futility of forecasting, a lesson we humans are pretty bad at learning.

A twice-yearly report from the Office for Budgetary Responsibility appears to acknowledge the futility of trying to forecast the future – though that apparently has not prevented its experts from trying.

 

Charged with making long-term weather forecasts during the Second World War, Kenneth Arrow – still the youngest economist ever to win the Nobel Prize – quickly concluded he may as well be making random guesses and so asked his US Air Force superiors to relieve him of the task.

 

“The commanding general is well aware that the forecasts are no good,” came back the reply. “However, he needs them for planning purposes.”

 

Here on The Value Perspective, we find ourselves wondering if one of the commanding general’s descendants is now working at the Office for Budgetary Responsibility (OBR), whose twice-yearly Forecast Evaluation Report is produced with the most honourable of intentions – and is jam-packed with caveats relating to the reliability or otherwise of the predictions contained therein.

 

 

Regular visitors to The Value Perspective – and very possibly casual ones too – will be well aware of our strong belief the future is uncertain and therefore impossible to predict.

 

Over at the OBR, which was created in 2010 “to provide independent and authoritative analysis of the UK public finances”, they appear to hold a similar view about the future – it just does not seem to stop them trying to predict it.

Take the latest report, published last month, which kicks off the executive summary by explaining:

 

“Twice a year at the OBR, we provide a detailed central forecast for the economy and the public finances. These provide a transparent benchmark against which to judge the significance of new economic and fiscal data and against which to estimate and explain the likely impact of policy decisions.”

 

All perfectly commendable, then, and indeed precisely the sort of thing you would expect from such a body – except that this air of great authority is somewhat spoilt by the lines that immediately follow:

 

“But since the future can never be known with precision, all such ‘point’ forecasts are necessarily surrounded by uncertainty – the likelihood that any given one will turn out to be accurate in all respects is negligible.”

 

The OBR argues, again quite laudably, it is important to publish the detail of its forecasts for the principal reasons of transparency, accountability and self-discipline before going into great detail about its forecasting models (which receive a clean bill of health) and the lessons it has learnt.

 

Before long, however, we are back to learning where it has got things wrong or, as the OBR would prefer to put it, ‘got things different’.

 

 

As the report notes: “We describe the arithmetic divergence between our central forecasts and the subsequent outturns as ‘differences’ rather than ‘errors’, because in many cases it would have been impossible to avoid them given the information available when the forecast was made. Where we do find genuine errors, which could (and should) have been corrected if we had spotted them, they are described as such.”

 

“Errors of this sort are inevitable from time to time in a highly disaggregated forecasting exercise like ours,” it adds, which is beautifully worded – and engagingly honest too, given if it is essentially conceding right from the off that the job the report’s authors are trying to do is essentially an impossible one.

 

In that context, the paragraphs considering the, ahem, “differences” with the OBR’s three previous forecasts is eye-catching.

 

That is because, when reviewing the reports from March and November 2016 and March 2017 in the context of leaving the European Union, it is the one written before the June 2016 referendum that has proved the most accurate.

 

That this was completed when the least data on the ramifications of Brexit was available tells us something – although perhaps only that, when it comes to forecasting, it is better to be lucky than good.

 

 

The OBR take on this is characteristically ‘glass half-full’, with the report chirping:

“Overall, the slowing in growth took a little longer to emerge than we expected as households and businesses took time to adjust their spending. And, of course, many non-Brexit related forecast judgements, such as the strength of the global economy and movements in commodity prices, will also have affected the path of the UK economy.

“Nevertheless, on the current vintage of data, our early assessments of the immediate impact of the Brexit vote have fared reasonably well.”

 

Well, OK – up to a point, perhaps. What we really take from this and other paragraphs of the report that relentlessly accentuate the positives, here on The Value Perspective, however, is that the global economy is a chaotic system.

That is to say, there are so many forces at work and their interactions are so complex, that extremely small variations in the strength of the forces and the way they interact can produce hugely different outcomes.

 

What is more, as we discussed in Possible outcomes, the global economy is a ‘level two’ chaotic system – in other words, it reacts to predictions about it and, as a result, can never be predicted accurately.

 

Forecasting the public finances is clearly a very difficult thing to do and the OBR has a very tough job on its hands.

 

As we say, it is doing that job for all the right reasons but, when it touches on markets and economics and the potential impacts, it becomes impossible. Aside from the imperfect data available, even if you happen to get all your initial forecasts right, market reaction means they will end up wrong – sorry, different.

 

Elsewhere in its report, the OBR says: “In judging our own performance – and in assessing the relative performance of different forecasters – it is important to remember that the current outturn data represent a relatively early draft of economic history. The stories we have told in previous reports often need to be updated after subsequent data revisions.”

 

 

The use of the word ‘stories’ there is particularly instructive – not just for the unintentional nod towards the dangers of narrative fallacy but also as a reminder that, at a time when pretty much every financial services company has been publishing predictions as to how the coming year will pan out, investors might find it helpful mentally to pencil in at the start of any of these the words ‘Once upon a time …’

 

Of more practical worth, we would argue, here on The Value Perspective, would be to focus instead on the one factor that has time and again been shown – on average and over the longer term – to be the biggest driver of whether or not you make money as an investor.

 

That is the price you pay for an asset – that is to say, its valuation.

 

Past performance is not a guide to future performance and may not be repeated.