Would you believe someone who claims knowledge of how to transform lead into gold, and yet he is not rich? Enter the perplexing world of financial academia, the modern-day “alchemists”
According to the just-published 2016 Rich List of the World’s Top-Earning Hedge Fund Managers by Institutional Investor’s Alpha magazine, eight of the top ten earners fall into the “quant” category, and half of the 25 richest of the year are quants. The firms listed include the likes of Renaissance Technologies, D.E. Shaw, Two Sigma, Millennium, Citadel and Schonfeld, none of which engage in “smart beta” or factor-based investments. None of these firms apply the theories published by Economics Nobel laureates such as James Tobin, Eugene Fama, Robert Shiller and dozens of others. Instead, these firms rely on a combination of mathematics and computational technology.
THE FOUNDATIONS OF FACTOR INVESTING
Consider the Capital Asset Pricing model (CAPM), the bedrock of financial economics. It was developed by William Sharpe (also a Nobel laureate) and others back in the 1960s. Almost six decades later, students around the world are still taught that the return of a security is a linear function of the risk-free rate and the “risk premium.” That’s it — your formula to riches! Forget about balance sheet information, news, sentiment, flows, frictions, … — everything can be condensed in a simple formula that can be estimated with 18th century mathematics. Since its publication in the prestigious Journal of Finance, this formula has been explained in every finance textbook, and is the foundation of the so-called “factor investing” in general and “smart beta” in particular.
Could things really be that simple? Financial markets are incredibly complex systems, where millions of individuals interact with each other exchanging information asynchronously and asymmetrically. Thousands of papers published in the Social Science Research Network (SSRN) claim to validate these theories empirically. Of course, all this evidence is based on statistical backtests, which are known to be easily manipulated.
Real scientists do not validate a theory through historical simulations. They conduct experiments. Here is how it works in the experimental sciences, in this order:
- A large number of forecasts are made (forward simulations).
- Measurements take place.
- Forecasting errors are evaluated.
However, the evidence presented in SSRN often follows a different order:
- Measurements take place.
- A large number of backward simulations are made.
- Backtesting errors are evaluated, and one model is selected.
Backtesting errors are not forecasting errors, because the backtest can be modified conveniently until some algebraic expression overfits the data. This is called selection bias, which the American Statistical Association has declared to be highly misleading:
Running multiple tests on the same data set at the same stage of an analysis increases the chance of obtaining at least one invalid result. Selecting the one “significant” result from a multiplicity of parallel tests poses a grave risk of an incorrect conclusion. Failure to disclose the full extent of tests and their results in such a case would be highly misleading.
That’s right — the statistical approaches used to validate financial theories are considered inappropriate, if not fraudulent, by the main professional organization for statisticians. This has led the current President of the American Finance Association to acknowledge that most claimed research findings in financial economics are likely false. The Econometric Society is shamefully silent on this regard, ignoring the “multiple testing” problem, as if it didn’t exist. To this day, as far as we are aware, no major Econometrics textbook mentions the dangers of backtest overfitting.
Have you ever wondered why empirical papers in finance are rarely published in statistical journals? One reason is that most empirical finance papers would have never been published in true statistical journals, because their validation methods typically do not account for multiple testing effects. Instead, they are published in econometrics journals, most of which are not refereed by mathematicians or statisticians.
WHERE IS THE SCIENTIFIC EVIDENCE?
First, we should distinguish between academic discussion and scientific evidence. For centuries, some of the most brilliant minds in history argued philosophical and theological theories in highly formalized terms. For example, after becoming the seventeenth century’s pre-eminent scientist, Sir Isaac Newton devoted many years of his life to the pursuit of Alchemy, and to prove that the world would end in the year 2060, based on the academic analysis of the scriptures (he also predicted 2012, but alas, here we are). Newton’s academic theories about the end of the world are not rigorous, experimentally validated scientific theories, but neither are CAPM or smart beta, since they have not been subjected to rigorous, inside-out experimental analysis.
Second, science does not accept the argument from authority (argumentum ad verecundiam). Newton’s success with gravity does not mean that he could turn lead into gold, even though he was convinced he knew how. Likewise, Nobel prize winners’ claims that CAPM can turn your copper dimes into gold have not been supported by the rich body of experimental evidence that one would expect for such claims.
So where is the scientific evidence? Why do financial academics call peer-review to accepting a paper without ever conducting out-of-sample replication? Why insist that students pile on debt to learn theories that have not been rigorously tested? Why are investors lured into products that are not likely to perform as backtested? One possibility is that businesses are using financial academia as a marketing tool to extract fees, and financial academics are all too happy to abide, because these products are the only (non-scientific) validation they can get. Some of those Nobelists are paid to join the boards of those funds, where their remuneration is a function of the assets under management, not the investment performance.
Many of the most successful investors use modern mathematics and computer science, not the relatively simple math used by many financial academics. In fact, it is well known that these investors avoid hiring students or academics with a finance background. That’s right, studying finance would exclude you from working at some of the leading investment firms.
A SOLUTION TO THE CRISIS IN FINANCIAL RESEARCH
In the year 1911, Albert Einstein predicted that light from another star should be bent by the Sun’s gravity. Few people paid much attention to that claim, even though the math behind it satisfied mathematicians of the stature of David Hilbert. Everything changed when Arthur Eddington presented experimental evidence from the solar eclipse of May 29, 1919. Once again, a beautiful theory means nothing unless experimental data confirms it. Gathering that data can take years, but that is how science works.
In the same spirit, perhaps we should require financial academics to commit a portion of their salaries (say 10%) to the validation of their papers. Those funds would be entrusted to the University’s endowment management company, which would invest them according to the proposed theory. Investing real money leaves an undeniable, certifiable and auditable trail, like the decades-long track record of the quant investors mentioned earlier. The Nobel prizes in physics, chemistry or medicine are awarded to exceptional individuals who presented conclusive and reproducible experimental data. The equivalent in Economics is to require the next laureate to present brokerage account statements for say $100 million, perhaps more. If a physicist was paid a dime for every time his theory is validated, he would be a billionaire. Unlike physicists, economists could actually make a lot of money by conducting financial experiments, if only their theories were correct.
If financial academics did that, they would provide some measure of credibility for their axiomatic theories and dogmatic beliefs. Until then, their backtests will continue to be misused to dupe unsuspecting savers into investments that have never performed. Because if their theories are actually correct, where are the billionaire financial academics?