The Gap Between Theory and Practice
Photo by Kelly Sikkema on Unsplash
Mean-variance optimisation is mathematically rigorous and intellectually compelling. In practice, it has a notorious reputation among professional investors for producing portfolios that are extreme, unstable, and practically unusable. The problem is not with the mathematics — it is with the nature of the inputs the mathematics requires.
To construct an efficient frontier portfolio, the optimiser needs expected returns, variances, and covariances for every asset in the investment universe. For a portfolio of 50 assets, that is 50 expected returns, 50 variances, and 1,225 pairwise covariances. All of these must be estimated from historical data — and all of them are noisy.
The Estimation Error Problem
Photo by Markus Spiske on Unsplash
Expected return estimates are the most treacherous input. Stock returns are highly variable: the standard error of a mean annual return estimated from 20 years of monthly data is large enough that the estimates for most assets are statistically indistinguishable from each other. In other words, we cannot reliably know in advance which assets will have higher expected returns.
The optimiser does not know this. It treats the estimated expected returns as if they were precise and acts on even tiny differences between them. If the estimated expected return for an emerging market bond is 8.1% and for a developed market bond is 7.9%, the optimiser may recommend a 60% allocation to the emerging market bond — a precise, confident response to what is statistical noise.
Richard Michaud described this problem with clarity in his landmark 1989 paper in the Financial Analysts Journal, "The Markowitz Optimization Enigma: Is Optimized Optimal?" Michaud showed that optimisation tends to concentrate the portfolio in the assets with the most estimation error — exactly the assets about which the investor is most uncertain. The "optimal" portfolio is, in effect, an error maximiser.
What Optimised Portfolios Look Like in Practice
Photo by Anne Nygård on Unsplash
Without constraints, a raw mean-variance optimiser typically produces portfolios that would strike any reasonable investor as absurd: 80% in a single small-cap factor, zero allocation to major asset classes, extreme long-short positions. Professional practitioners universally apply ad hoc constraints — no single asset above 10%, no short positions, minimum allocations to major regions — that override the pure optimisation output. The constraints transform the problem into a more plausible portfolio, but they also acknowledge that the unconstrained output was not fit for purpose.
Even with constraints, the optimised portfolio can change dramatically with small changes in inputs. A shift in one expected return estimate of 0.5% may flip the recommended allocation between two asset classes by 30 percentage points. This instability means that regular rebalancing of a mean-variance-optimised portfolio generates high turnover and transaction costs, eroding the theoretical benefits.
The Black-Litterman Fix
Photo by Mazin Omron on Unsplash
In 1992, Fischer Black and Robert Litterman at Goldman Sachs proposed a partial remedy. Rather than starting from unconstrained historical estimates, their model begins from the market portfolio — which is observable and can be argued to represent the market's collective view of expected returns — and adjusts away from it only to the extent that the investor has specific, explicit views about individual assets or markets.
The Black-Litterman approach dramatically reduces the sensitivity to input estimation errors. Starting from the market portfolio as a neutral baseline means the optimiser only moves away from that baseline when the investor has conviction, and the magnitude of the move is proportional to the investor's expressed confidence in the view. The result is portfolios that are more diversified, more stable, and more intuitive — at the cost of relying less purely on statistical optimisation.
The Black-Litterman model is now widely used in institutional asset management, though even it cannot solve the fundamental problem that expected returns are fundamentally unknowable with precision.