Note: We first published a valuation based market forecast in September of 2010. At that time we used only the Shiller PE data to generate our forecast, and our analysis suggested investors should expect under 5% per year after inflation over the subsequent 10 year horizon. Over the 40 months since we have introduced several new metrics and applied much more comprehensive methods to derive our forecast estimates. Still, our estimates are far from perfect.
From a statistical standpoint, the use of overlapping periods substantially impairs the statistical significance of our estimates. This is unavoidable, as our sample only extends back to 1926, which gives us under 90 years to work with, and our research suggests that secular mean reversion exerts its strongest influence on a periodicity somewhere between 15 and 20 years. As a result, our true sample size is somewhere between 3 and 4, which is not very high.
Aside from statistical challenges, our analysis potentially faces issues related to changes in the way accounting identities have been calculated through time, changes to the geographic composition of earnings, and myriad other factors. In response to questions raised by thoughtful analysts in recent research, we have integrated new earnings series from Bloomberg and S&P into our Cyclically Adjusted PE calculation. Primarily, the new series adjust earnings for changes to GAAP rules in 2001 related to corporate write-downs. Each of the series has merit, so we took the step of averaging the three series. I’m sure bulls and bears alike will find this method intensely unsatisfying; we certainly hope so, as the best compromises have this precise character.
It’s worth noting that the new earnings series do not alter the final regression forecast model results because our multiple regression model rejects the Shiller PE as statistically insignificant to the forecast. That is, it is highly correlated with, but less significant than, other series like market cap/GNP and q ratio. This has been the case from the beginning of this article series, so it isn’t due to the new earnings data. Nevertheless we include regression parameters and r-squared estimates for all of the modified Shiller PEs in the matrixes as usual.
The bottom line is that, despite statistical and accounting challenges, our indicators have proved to be of fairly consistent value in identifying periods of over and under-valuation in U.S. markets over the full 90 year period, notwithstanding the last two decades. We admit that the two decades since 1994 seem like strange anomalies relative to the other seven decades; history will eventually prove whether this anomaly relates to a structural change in the calculation of the underlying valuation metrics, a regime shift in the range of possible long-term returns, or an increase in the ambient slope of drift.
We all must acknowledge that the current globally coordinated monetary experiment truly has no precedent in modern history. For this reason the range of potential outcomes is much wider than it might otherwise be. Things could persist for much longer, and reach never before seen extremes (in both directions, mind you!) before it’s over.
Lastly, I am struggling to reconcile a conundrum I identified very early in the development of this model. Namely, the fact that the simple regression of real total returns with reinvested dividends carries very different implications than the suite of other indicators we have tested. I am troubled by the theoretical veracity of incorporating dividend reinvestment for extrapolation purposes, because the vast majority of dividends are NOT reinvested, but rather are paid out, and represent a material source of total income in the economy. However, the trend fit is surprisingly tight, and I can’t say with conviction that the trend fit isn’t valid for those investors who are reinvesting dividends. It is a puzzle.
Above all, any analysis that relies on the past to offer guidance about the future makes the strong assumption that the future will in fact resemble the past. We have no guarantee that this will be the case. Many optimistic analysts assert that the invention of central banking, global communications and trade, robotics, 3D printing, Paul Krugman, or any number of ‘game changers’ that have evolved over the past few decades renders comparisons with our past misguided. Surely we won’t make the mistakes of our ancestors; there will be no more war, no misguided political decisions, no shortsighted thinking, no natural disasters, no panics or conflicts or excesses which derail our arc toward ever-increasing prosperity.
In case there is any ambiguity, we do not espouse this Polyanna-esque view. So long as markets, economies and politics are dominated by human judgement, the future is likely to resemble the past in most important respects.
—————————————————————————
We endorse the decisive evidence that markets and economies are complex, dynamic systems which are not reducible to linear cause-effect analysis over short or intermediate time frames. However, the future is likely to rhyme with the past. Thus, we believe there is substantial value in applying simple statistical models to discover average estimates of what the future may hold over meaningful investment horizons (10+ years), while acknowledging the wide range of possibilities that exist around these averages.
To be crystal clear, the commentary below makes no assertions whatsoever about whether markets will carry on higher from current levels. Expensive markets can get much more expensive in the intermediate term, and investors need look no further back than the late 2000s for just such an example. However, the historical implications of investing in expensive markets is that, at some point in the future, perhaps years from now, the market has a very high probability of trading back below current prices; perhaps far below. More importantly, investors must recognize that buying stocks at very expensive valuations will necessarily lead to future returns over the subsequent 10 – 20 years that are far below average.
Again, we are not making a forecast of market returns over the next several months; in fact, markets could go substantially higher from here. However, over the next 10 to 15 years, markets are very likely to revert to average valuations, which are much lower than current levels. This study will demonstrate that investors should expect 6.5% real returns to stocks only during those very rare occasions when the stock market passes through ‘fair value’ on its way to becoming very cheap, or very expensive. At all other periods, there is a better estimate of future returns than the long-term average, and this study endeavours to quantify that estimate.
The profit margin picture is critically important. Jeremy Grantham recently stated, “Profit margins are probably the most mean-reverting series in finance, and if profit margins do not mean-revert, then something has gone badly wrong with capitalism. If high profits do not attract competition, there is something wrong with the system and it is not functioning properly.” On this basis, we can expect profit margins to begin to revert to more normalized ratios over coming months. If so, stocks may face a future where multiples to corporate earnings are contracting at the same time that the growth in earnings is also contracting. This double feedback mechanism may partially explain why our statistical model predicts such low real returns in coming years. Caveat Emptor.
Modeling Across Many Horizons
Process
Forecasting Expected Returns

Chart 2. 15-Year Forecast Returns vs. 15-Year Actual Future Returns