It’s that time of year when we all gather to give thanks for our blessings, overindulge in various foodstuffs and, of course, go shopping.
Shopmas is also the time of the year when I call out the National Retail Federation for their bogus retail forecasts; they return the favor by calling me a Grinch. Sales forecasts drawn from the NRF’s annual survey have been consistently bad – rarely even coming close to actual holiday retail sales numbers as reported by the Census Bureau – and the methodology is unreliable.
Our annual dance looks something like this:
• An NRF survey of shoppers asks them what they plan to spend this holiday season and what they spent last year.
• The net difference between those two numbers gets shared by the media as the gain (or loss) for the upcoming holiday shopping season.
• Headline writers will get it totally wrong, as much of the media reports the surveys as if they were hard sales data.
• The forecasts inevitably are proven wrong.
Almost no one recalls this when the actual sales data comes out in January.
The NRF did make a minor adjustment to their approach this year, now forecasting a range for holiday sales gains. And so:
The National Retail Federation recently announced that it expects holiday retail sales in November and December – excluding automobiles, gasoline and restaurants – to increase between 3.6 percent and 4 percent for a total of $678.75 billion to $682 billion, up from $655.8 billion last year.
Predicting a range instead of a single specific number, while not an improvement on the underlying methodology, does increase the likelihood that the NRF’s prediction will be correct. It is much easier to hit a range of almost half a percent than to nail an exact number.
Well played, NRF. You are a worthy opponent.
As it turns out, the way the NRF forecasts seasonal sales results is pretty awful. Relying on a survey of shoppers instead of using real sales data is deeply problematic.
Why? Well, there are quite a few reasons:
People’s memories are surprisingly unreliable. The way to learn what you spent last year during the holidays is to look at your bank accounts and credit card statements. Guessing that number is simply untrustworthy.
• As individuals, we really do not know what we are going to do in the future. People are simply terrible at predicting their own behaviors. This has been repeatedly validated by academic literature.
• When we survey people, we learn how they are feeling at that very moment – and little else. Trying to forecast future behavior this way is an inherently problematic methodology.
Incidentally, this also helps to explain why “things” tend to not make us all that much happier. People often expect that a shiny new house/car/boat/gadget/bauble will increase their total life satisfaction; instead, their predictions fail, because they eventually become used to it.
The last way we know that this survey methodology is unreliable is to look at its track record. It has been uniformly awful.
Note it is not just this one retail trade group that relies on a flawed methodology, but others as well. Even Gallup, which should know better, does the same thing.
Why make a big deal about this? Well, facts matter, especially to investors. Relying on overly optimistic (or excessively pessimistic) forecasts spells trouble for people who put capital at risk.
We can make an educated guess as to what sales data for 2017 will look like, but not much more than that. Despite what you might have read, we barely have any idea of what holiday sales will be like.
Happy Shopmas, everybody. Grinch out.
Barry Ritholtz is a Bloomberg View columnist.
