Figure 1 Histograms of 5-year Returns from Seven Asset Prices

Random Walk Part 2 – Does Any Asset Price Fit the Bell Curve?

Originally Published September 25, 2017 in Advisor Perspectives

This is the second of my four-part empirical research into the fallacy of the random walk paradigm in investments. Part 1focused on the failure of the random walk to depict the Dow Jones Industrial Average. In this article, I expand the study to include six diverse asset classes including large-caps (the S&P500), small-caps (the Russell 2000), emerging markets, gold, the dollar and the 10-year Treasury bond. Asset prices do not walk in tiny steps along an orderly bell curve, but often take giant leaps leaving chaotic turbulence behind.

What are the common features among seven widely diverse asset classes? Why are asset prices so hard to pin down by random walk or other analytical and descriptive models? What are the investment implications if asset returns do not fit the bell curves?

Part 3 will deal with its flaws in defining investment risk. Part 4 offers a new reward and risk framework alternative to the random walk paradigm. The new framework yields new insights on how to logically beat the S&P 500 total return.

Empirical evidence against the random walk model

In Part 1, I presented price return histograms of the Dow Jones Industrial Average from 1900 to 2016. In this paper, I extend the empirical research to six other assets. Detailed results are in the Appendix. Figure 1 summarizes the five-year return histograms (light blue bars) of seven asset classes – the DJIA from Part 1 and six other assets from the Appendix. No return histogram (light blue bar) has any resemblance to its corresponding random walk probability density functions PDF (dark blue curve). All random walk PDFs are unimodal (with a single central mean) with matching wings on both sides. All empirical distributions have multiple peaks with no central symmetry.

Common themes among different asset classes

1. None of the return histograms of the seven asset classes follows the random walk bell curve for periods longer than one day. The longer the return periods, the less bell-shaped they become. Real world prices do not follow a random walk.

2. Even for the one-day returns, all histograms show asymmetric fat tails. Random walk theorists have no explanation for them and treat them as anomalous statistical outliers.

3. All return distributions of time horizons beyond one year do not have a single mean and a definable variance. Random walk's mean-variance paradigm does not reflect reality.

4. Random walk's bell curve underestimates the probabilities of both large losses and gains beyond ± one standard deviation. This has dire consequences in risk management.

5. It is standard practice to scale returns and volatilities in different time horizons by the number of trading days and by the square root of trading days, respectively. Data show that neither return nor the volatility follows such scaling rules.

Why are asset prices so elusive for the theoreticians to capture?

1. Random walk is not the only model that fails to explain price behaviors. Power-lawsgame theoryagent-based modelsbehavioral finance and adaptive market hypothesis are all incomplete theories at best. The rational beliefs equilibria model appears to have the potential for solving the asset-pricing puzzle but it is still too early to tell.

2. Why are prices so hard to pin down? The culprits are the agents involved. Unlike mindless pollens and particles that blindly obey physical laws, humans write their own rules, adapt to their own mistakes and adjust their actions to new encounters with a mix of unscripted rational and emotional responses.

3. Investors appraise prices from the perspective of an imagined future shaped by their past memories and current value judgments. These mental chain reactions transform their decision-making processes into highly complex systems. Our imagined future today can in turn create a new future that could then alter the current reality. The feedback loops continue in real time and exacerbate the already complex systems.

4. To model asset prices properly, economists must track all these nonlinear, multifaceted and interactive dynamics. By comparison, modeling the physical world is a trivial task.

What are the implications for investors?

1. There are many types of randomness described by different analytical probability density functions (PDFs). The bell curve is one of the most well behaved kinds. Figure 1 exhibits a wild form of randomness that does not fit any known PDF. Frank Knight called them "radical uncertainties". Donald Rumsfeld named them "unknown unknowns". Nassim Nicholas Taleb referred to them as "Black Swans".

2. Because asset prices do not follow the bell curve, it comes as no surprise why many random-walk based doctrines stopped working when prices plunged beyond one standard deviation. Markowitz's mean-covariance matrix, Sharpe's beta, Fama's factors and Black-Scholes' volatility neutrality all broke down during market crashes.

3. Because asset prices are radically random, investors should be skeptical of all market forecasts. Figure 2 shows the 10-year return histogram of the S&P 500 from 1928 to 2016. If you think that forecasting returns on a widely disperse bell curve is challenging, then mining for predictive patterns in the erratic randomness in figure 2 is a fool's errand because the probabilities of a +200% gain and a -20% loss are nearly the same.

4. Additionally, statistical flaws are common in many long-term forecasts. For example, analysts use overlapping data to compute the CAPE-based regression lines (here) and researchers gauge secular market cycles with sample sizes of less than a dozen (here). These forecasters not only try to predict something that is statistically unpredictable, they do so with incorrect math.

Concluding remarks

We live our lives every day without forecasting when the next big earthquakes will hit our hometowns. We mitigate quake risks by upgrading building codes and buying insurance.

Likewise, because asset prices exhibit radical randomness, predicting future returns is futile. Investors should focus on managing risk. To manage investment risk properly, however, we must first identify what risk truly is. Unfortunately, the academics view risk as volatility through a distorted random walk lens. In Part 3, I will point out the conceptual flaws and operational pitfalls of viewing volatility as risk. Mistaking risk as volatility has dire financial consequences for investors.

In Part 4, I will present a new framework for defining reward and risk as an alternative to the random walk paradigm. The new reward-risk framework offers investors a probabilistic path for beating the S&P500 total return – a blasphemy in the view of the efficient market orthodoxy. We win not by developing models with superior forecasting ability, but by beating Mr. Market at his own game. Stay tuned for details.

Theodore Wong graduated from MIT with a BSEE and MSEE degree and earned an MBA degree from Temple University. He served as general manager in several Fortune-500 companies that produced infrared sensors for satellite and military applications. After selling the hi-tech company that he started with a private equity firm, he launched TTSW Advisory, a consulting firm offering clients investment research services. For almost four decades, Ted has developed a true passion for studying the financial markets. He applies engineering design principles and statistical tools to achieve absolute investment returns by actively managing risk in both up and down markets. He can be reached at

Appendix: Returns of six Asset Classes – Actual Histograms vs. Random Walk PDFs

Section (A) shows linear and logarithmic histograms of short-horizon returns of three equity assets (the S&P500, the Russell 2000 and emerging markets). Section (B) shows linear and logarithmic histograms of short-horizon returns of three alternative assets (gold, the dollar and the 10-year Treasury bond). Section © shows linear and logarithmic histograms of long-horizon returns of the S&P500, the Russell 2000 and emerging markets. Section (D) shows linear and logarithmic histograms of long-horizon returns of gold, the dollar and the 10-year Treasury bond. Data sources are FREDYahoo Finance and MetaStock.

(A) Short-horizon returns of large-caps, small-caps and the emerging markets

Figure A1 shows linear histograms of the returns in short-term horizons – daily, weekly, monthly and quarterly (top to bottom) across different equity styles – the S&P500 (big-caps), the Russell 2000 (small-caps) and emerging markets (left to right). Big-caps and small-caps (left and center) bear some resemblances to the random walk probability density functions (PDFs). However, for the emerging markets (right), fat tails are visible even in monthly and quarterly returns. Gaps between the data and the PDFs near the peaks are noticeable among all three assets.

Figure A2 displays the same data in logarithmic histograms. All log histograms exhibit fat tails on both sides of the peaks. The random walk PDFs fail to account for the high probabilities at both return extremes across all three assets.

(B) Short-horizon returns of gold, the dollar and the U.S. Treasury bond

Figure A3 shows three different alternatives – gold, the dollar and the 10-year Treasury bond (left to right) in linear histograms of short-horizon returns – daily, weekly, monthly and quarterly (top to bottom). Deviations from the random walk PDFs grow with increasing time horizons. Gold and the dollar have wider spreads than the random walk PDFs while the 10-year Treasury has tall spikes near the peaks.

Figure A4 displays the same data in a log scale. All histograms exhibit higher probabilities at both extremes than those predicted by the PDFs. Gold and bond are asymmetric and skewed to the right relative to the PDFs.

(C ) Long-horizon returns of large-caps, small-caps and the emerging markets

Figure A5 shows linear histograms of long-horizon returns – one year, five years and ten year (top to bottom) across three equity indices – the S&P500, the Russell 2000 and emerging markets (left to right). All charts show multiple peaks and wildly different spreads.

Figure A6 shows the same data in a log scale. Beyond one year, the data distributions show no central tendency and they are skewed to the right. Actual returns offer much better upside opportunities than what random walk predicts.

(D) Long-horizon returns of gold, the dollar and the U.S. Treasury bond

Figure A7 shows linear histograms of different long-horizon returns – one year, five years and ten year (top to bottom) across three alternatives – gold, the dollar and the 10-year Treasury bond (left to right). None of these plots resembles the random-walk bell curve.

Figure A8 shows log histograms of the same data in Figure A7. Most of the fat tails are skewed to the right versus the perfectly symmetric random walk PDFs.


Modeling Cyclical Markets – Part 2

Originally Published November 7, 2016 in Advisor Perspectives

In Part 1 of this series, I presented Primary-ID, a rules- and evidence-based model that identifies major price movements, which are traditionally called cyclical bull and bear markets. This article debuts Secondary-ID, a complementary model that objectively defines minor price movements, which are traditionally called rallies and corrections within bull and bear markets.

The traditional definitions of market cycles

Market analysts define market cycles by the magnitude of price movements. Sequential up and down price movements in excess of 20% are called primary cycles. Price advances more than 20% are called bull markets. Declines worse than -20% are called bear markets. Sequential up and down price movements within 20% are called secondary cycles. Price retracements milder than -20% are called corrections. Advances shy of 20% are called rallies. Talking heads at CNBC frequently use this financial vernacular.

But has anyone bothered to ask how factual these fancy terms and lofty labels really are?

Experts also measure market cycles by their durations. They reported that since 1929, there have been 25 bull markets with gains over 20% with a median duration of 13.1 months, and 25 bear markets with losses worse than 20% with a median duration of 8.3 months.

But is "median duration" the proper statistical yardstick to measure stock market cycle lengths?

Fact-checking the 20% thresholds

Before presenting Secondary-ID, let’s pause to fact-check these two market cycle yardsticks. The ±20% primary cycle rules-of-thumb have little practical use in guiding investment decisions. If we wait for a +20% confirmation before entering the market, we would have missed the bulk of the upside. Conversely, if we wait for an official kick-off of a new cyclical bear market, our portfolios would have shrunk by -20%. The ±20% thresholds may be of interests to stock market historians, but offer no real benefit to investors.

Besides being impractical, the ±20% demarcations are also baseless. This falsehood can be demonstrated by examining the historical evidence. Figures 1A and 1B show the daily closing prices of the S&P 500 from 1928 to 2015. The green bars in Figure 1A are price advances from an interim low to an interim high of over +5%. The red bars in Figure 1B are price retracements from an interim high to an interim low worse than -5%. Price movements less than ±5% are ignored as noise. There were a total of 198 advances and 166 retracements in 88 years. From the figures, it's not obvious why ±20% were picked as the thresholds for bull and bear markets. The distributions of green and red bars show no unique feature near the ±20% markers.


To determine how indistinct the ±20% markers are in the distributions, I plot the same daily data in histograms as shown in Figures 2A and 2B. The probabilities of occurrence are displayed on the vertical axes for each price change in percent on the horizontal axes. For example, Figure 2A shows that a +20% rally has a 3% chance of occurring; and Figure 2B shows that a -20% retreat has near a 3.5% chance. There is no discontinuity either at +20% in Figure 2A that separates bull markets from rallies, nor at -20% in Figure 2B that differentiates bear markets from corrections.


There are, however, two distinct distribution patterns in both up and down markets. Figure 2A shows an exponential drop in the probability distribution with increasing rally sizes from +10% to +40%. Above +45%, the histogram is flat. Figure 2B shows a similar exponential decline in the probability distribution with increasing retracements from -5% to -40%. Beyond -45%, the histogram is again flat. The reasons behind the exponential declines in the distributions and the two-tier histogram pattern are beyond the scope of this paper. It's clear, however, that there is no distinct inflection point at ±20% from Figures 2A and 2B. In fact, it would be more statistically correct to use the ±45% as the thresholds for bull and bear markets. But such large thresholds for primary cycles would be worthless for investors.

Figures 2A and 2B also expose one other fallacy. It's often believed that price support and resistance levels are set by the Fibonacci ratios. One doesn't have to read scientific dissertations using advanced mathematical proofs to dispel the Fibonacci myth. A quick glance at Figure 2A or 2B would turn any Fibonacci faithful into a skeptic. If price tops and bottoms are set by the Fibonacci ratios, we would have found such footprints at ±23.6%, ±38.2%, ±50.0%, ±61.8%, or ±100%. No Fibonacci pivot points can be found in 88 years of daily S&P 500 data.

Fact-checking the market duration yardstick

I now turn to the second cyclical market yardstick-cycle duration. It's been reported that since 1929, the median duration for bull markets is 13.1 months and the median duration for bear markets is 8.1 months. The same report also notes that the spread in bull market durations spans from 0.8 to 149.8 months; and the dispersion among bear market durations extents from 1.9 to 21 months. When the data is so widely scattered, the notion of median is meaningless. Let me explain why with the following charts.

Figures 3A and 3B show duration histograms for all rallies and retreats, respectively. The vertical axes are the probabilities of occurrence for each duration shown on the horizontal axes. The notions of median and average are only useful when the distributions have a central tendency. When the frequency distributions are skewed to the extent seen in Figure 3A or both are skewed and dispersed like in Figure 3B, median durations cited in those reports are meaningless.


Figures 3A and 3B also expose one other myth. We often hear market gurus warning us that the bull (or bear) market is about to end because it's getting old. Chair Yellen was right when she said that economic expansions don't die of old age. Cyclical markets don't follow an actuarial table. They can live on indefinitely until they get hit by external shocks. Positive shocks (pleasant surprises) end bear markets and negative shocks (abrupt panics) kill bull markets. These black swans follow Taleb distributions in which average and median are not mathematically defined. In my concluding remarks I further expand on the origin of cyclical markets.

Many Wall Street beliefs and practices are just glorified folklores decorated with Greek symbols and pseudo-scientific notations to puff up their legitimacy. Many widely followed technical and market-timing indicators are nothing but glamorized traditions and legends. Their theoretical underpinnings must be carefully examined and their claims must be empirically verified. It's unwise to put ones' hard earned money at risk by blindly following any strategy without fact-checking it first, no matter how well accepted and widely followed it may be.

Envisioning cyclical markets through a calculus lens

Now that I have shown how absurd these two yardsticks are in gauging market cycles, I would like to return to the subject at hand – modeling cyclical markets. The methodology is as follows: First, start with a metric that is fundamentally sound. The Super Bowl indicator, for example, is an indicator with no fundamental basis. Next, transform the metric into a quasi range-bound indicator. Then devise a set of rational rules using the indicator to formulate a hypothesis. High correlations without causations are not enough. Causations must be grounded in logical principles such as economics, behavioral finance, fractal geometry, chaos theory, game theory, number theory, etc. Finally, a hypothesis must be empirically validated with adequate samples to be qualified as a model.

Let me illustrate my modeling approach with Primary-ID. The Shiller CAPE (cyclically adjusted price-earnings ratio) is a fundamentally sound metric. But when the CAPE is used in its original scalar form, it is prone to calibration error because it's not range-bound. To transform the scalar CAPE into a range-bound indicator, I compute the year-over-year rate-of-change of the CAPE (e.g. YoY-ROC % CAPE). A set of logically sound buy-and-sell rules is devised to activate the indicator into actionable signals. After the hypothesis is validated empirically over a time period with multiple bull and bear cycles, Primary-ID is finally qualified as a model.

This modeling approach can be elucidated with a calculus analogy. The scalar Shiller CAPE is analogous to "distance." The vector indicator YoY-ROC % CAPE is analogous to "velocity." When "velocity" is measured in the infinitesimal limit, it's equivalent to the "first derivative" in calculus. In other words, Primary-ID is similar to taking the first derivative of the CAPE. There are, however, some differences between the YoY-ROC % CAPE indicator and calculus. First, a derivative is an instantaneous rate-of-change of a continuous function. The YoY-ROC % CAPE indicator is not instantaneous, but with a finite time interval of one year. Also, the YoY-ROC % CAPE indicator is not a continuous function, but is based on a discrete monthly time series – the CAPE. Finally, a common inflection point of a derivative is the zero crossing, but the signal crossing of Primary-ID is at -13%.

Secondary-ID – a model for minor market movements

I now present a model called Secondary-ID. If Primary-ID is akin to "velocity" or the first derivative of the CAPE and is designed to detect major price movements in the stock market, then Secondary-ID is analogous to "acceleration/deceleration" or the second derivative of the CAPE and is designed to sense minor price movements. Secondary-ID is a second-order vector because it derives its signals from the month-over-month rate-of-change (MoM-ROC %) of the year-over-year rate-of-change (YoY-ROC %) in the Shiller CAPE metric.

Figures 4A to 4D show the S&P 500, the Shiller CAPE, Primary-ID signals and Secondary-ID signals, respectively. The indicator of Primary-ID (Figure 4C) is identical to that of Secondary-ID (Figure 4D), namely, the YoY-ROC % CAPE. But their signals differ. The signals in Figures 4C and 4D are color-coded – bullish signals are green and bearish signals are red. The details of the buy and sell rules for Primary-ID were described in Part 1. The bullish and bearish rules for Secondary-ID are presented below.


Bullish signals are triggered by a rising YoY-ROC % CAPE indicator or when the indicator is above 0%. For bearish signals, the indicator must be both falling and below 0%. "Rising" is defined as a positive month-over-month rate-of-change (MoM-ROC %) in the ROC % CAPE indicator; and "falling", a negative MoM-ROC %. Because it is a second-order vector, Secondary-ID issues more signals than Primary-ID. It's noteworthy that the buy and sell signals of Secondary-ID often lead those of Primary-ID. The ability to detect acceleration and deceleration makes Secondary-ID more sensitive to changes than Primary-ID that detects only velocity.

For ease of visual examination, Figures 5A shows the S&P 500 color-coded with Secondary-ID signals. Figure 5B is the same as Figure 4D describing how those signals are triggered by the buy and sell rules. Since 1880, Secondary-ID has called 26 of the 28 recessions (a 93% success rate). The two misses were in 1926 and 1945, both were mild recessions. Secondary-ID turned bearish in 1917, 1941, 1962, 1966 and 1977 but no recessions followed. However, these bearish calls were followed by major and/or minor price retracements. If Mr. Market makes a wrong recession call and the S&P 500 plummets, it's pointless to argue with him and watching our portfolios tank. Secondary-ID is designed to detect accelerations and decelerations in market appraisal by the mass. Changes in appraisal often precede changes in market prices, regardless of whether those appraisals lead to actual economic expansions or recessions.


Secondary-ID not only meets my five criteria for robust model design (simplicity, sound rationale, rule-based clarity, sufficient sample size, and relevant data), it has one more exceptional merit – no overfitting. In the development of Secondary-ID, there is no in-sample training involved and no optimization done on any adjustable parameter. Secondary-ID has only two possible parameters to adjust. The first one is the time-interval for the second-order rising and falling vector. Instead of looking for an optimum time interval, I choose the smallest time increment in a monthly data series – one month. One month in a monthly time series is the closest parallel to the infinitesimal limit on a continuous function. The second possible adjustable parameter is the signal crossing. I select zero crossing as the signal trigger because zero is the natural center of an oscillator. The values selected for these two parameters are the most logical choices and therefore no optimization is warranted. Because no parameters are adjusted, there's no need for in-sample training. Hence Secondary-ID is not liable to overfitting.

Performance comparison: Secondary-ID, Primary-ID and the S&P 500

The buy and sell rules of Secondary-ID presented above are translated into executable trading instructions as follows: When the YoY-ROC CAPE is rising (i.e. a positive MoM-ROC %), buy the S&P 500 (e.g. SPY) at the close in the following month. When the YoY-ROC CAPE is below 0% and falling (i.e. a negative MoM-ROC %), sell the S&P 500 at the close in the following month and use the proceeds to buy U.S. Treasury bond (e.g. TLT). The return while holding the S&P 500 is the total return with dividends reinvested. The return while holding the bond is the sum of both bond coupon and bond price changes caused by interest rate movements.

Figures 6A shows the S&P 500 total return index and the total return of the U.S. Treasury bond. In 116 years, return on stocks is nearly twice that of bonds. But in the last three decades, bond prices have risen dramatically thanks to a steady decline in inflation since 1980 and the protracted easy monetary policies since 1995. Figures 6B shows the cumulative total returns of Primary-ID, Secondary-ID and the S&P 500 on a $1 investment made in January 1900. The S&P 500 has a total return of 9.7% with a maximum drawdown of -83%. By comparison, Primary-ID has a hypothetical compound annual growth rate (CAGR) of 10.4% with a maximum drawdown of -50% and trades once every five years on average. The performance stats on Primary-ID are slightly different from that shown in Figure 5B in Part 1 because Figure 6B is updated from July to August 2016.

Secondary-ID delivers a hypothetical CAGR of 12.8% with a -36% maximum drawdown and trades once every two years on average. Note that Primary-ID and Secondary-ID are working in parallel to avoid most if not all bear markets. Secondary-ID offers an extra performance edge by minimizing the exposure to bull market corrections and by participating in selected bear market rallies.


I now apply the same buy and sell rules in the recent 16 years to see how the model would have performed in a shorter but more recent sub-period. This is not an out-of-sample test since there's no in-sample training. Rather, it's a performance consistency check with a much shorter and more recent period. Figures 7A shows the total return of the S&P 500 and the U.S. Treasury bond price index from 2000 to August 2016. The return on bonds in this period is higher than that of the S&P 500. Record easy monetary policies since 2003 and large-scale asset purchases by global central banks since 2010 pumped up bond prices. Two severe back-to-back recessions dragged down the stock market. Figures 7B shows the cumulative total returns of Primary-ID, Secondary-ID and the S&P 500 on a $1 investment made in January 2000.


Since 2000, the total return index of the S&P 500 has returned 4.3% compounded with a maximum drawdown of -51%. By comparison, Primary-ID has a CAGR of 7.7% with a maximum drawdown of -23% and trades once every five years on average. Again, the performance stats on Primary-ID shown in Figure 7B are slightly different from that shown in Figure 5B in Part 1 because Figure 7B is updated to August 2016. Secondary-ID delivers a hypothetical CAGR of 10.5% with a maximum drawdown of only -16% and trades once every 1.4 years on average. The performance edge in return and risk of Secondary-ID over both Primary-ID and the S&P 500 total return index is remarkable. The consistency in performance gaps in both the entire 116-year period and in the most recent 16-year sub-period lends credence to Secondary-ID.

Theoretical support for both cyclical market models

The traditional concepts of "primary cycles" and "secondary cycles" rely on amplitude and periodicity yardsticks to track market cycles in the past and to predict market cycles in the future. Primary-ID and Secondary-ID do not deal with primary or secondary market cycles. Their focus is on cyclical markets – major and minor price movements. All market movements are driven by changes in investors' collective market appraisals. The Shiller CAPE is selected as the core metric because it is a value assessment gauge-based fundamental indicator – appraising the inflation adjusted S&P 500 price relative to its rolling 10-year average earnings. Although the scalar-CAPE is prone to overshoot and valuations misinterpretation, the first- and second-order vectors of the CAPE are not. Primary-ID and Secondary-ID sense both major changes and minor shifts in investors' collective market appraisal that often precede market price action.

Like Primary-ID, Secondary-ID also finds support from many of the behavioral economics principles. First, prospect theory shows that a -10% loss hurts investors twice as much as the pleasure a +10% gain brings. Such reward-risk disparities are recognized by the asymmetrical buy and sell rules in both models. Second, both models use vector-based indicators. This is supported by the findings of Daniel Kahneman and Amos Tversky that investors are sensitive to the relative changes (vectors) in their wealth much more so than to the absolute levels of their wealth (scalars). Finally, the second-order vector in Secondary-ID is equivalent to the second derivative of the concave and convex value function described by the two distinguished behavioral economists in 1979.

Concluding remarks – cyclical markets vs. market cycles

I developed rules- and evidence-based models to assess cyclical markets and not market cycles. The traditional notion of market cycles is defined with a prescribed set of pseudo-scientific attributes such as amplitude and periodicity that are neither substantiated by historical evidence nor grounded in statistics. Cyclical markets, on the other hand, are the outcomes of random external shocks imposing big tidal waves and small ripples on a steadily rising economic growth trend. Cyclical markets cannot be explained or predicted using the traditional cycle concepts because past cyclical patterns are the outcomes of non-Gaussian randomness. Let me illustrate with a simple but instructive narrative.

Cyclical markets can be visualized with a simple exercise. Draw an ascending line on a graph paper with the y-axis in a logarithmic scale and the x-axis in a linear time scale. The slope of the line is the long-term compound growth rate of the U.S. economy. Next, disrupt this steadily rising trendline with sharp ruptures of various amplitudes at erratic time intervals. These abrupt ruptures represent man-made crises (e.g., recessions) or natural calamities (e.g., earthquakes). Amplify these shocks with overshoots in both up and down directions to emulate the cascade-feedback loops driven by the herding mentality of human psychology. You now have a proxy of the S&P 500 historical price chart.

This descriptive model of cyclical markets explains why conventional market cycle yardsticks – the ±20% thresholds and median durations will never work. Unpredictable shocks will not adhere to a prescribed set of amplitude or duration. Non-Gaussian randomness cannot be captured by the mathematical formulae defining average and median. The conceptual framework of market cycles is flawed and that's why it fails to explain cyclical markets.

Looking from the perspective of Primary-ID and Secondary-ID, cyclical bull markets can last as long as the CAPE velocity is positive and/or accelerating. Cyclical bear markets can last as long as the CAPE velocity is negative and/or decelerating. Stock market movements are not constrained by the ±20% thresholds or cycle life-expectancy stats. Primary-ID detects the velocity of the stock market valuation assessment by all stock market participants that drives bull or bear markets. Secondary-ID senses subtle accelerations and decelerations in the same collective market valuation assessment. These second-order waves manifest themselves in stock market rallies and corrections. It doesn't matter whether the market is down less than -20%, labeled by experts as a correction, or plunges by worse than -20%, which is called a cyclical bear market, Primary-ID and Secondary-ID capture the price movements just the same.

Does synergy exist between Primary-ID and Secondary-ID? Would the sum of the two offer performance greater than those of the parts? A composite index of the two models enables the use of leverage and short strategies that pave the way for more advanced portfolio engineering and risk management tactics. Do these more complex strategies add value? For answers, please stay tuned for Part 3.

Theodore Wong graduated from MIT with a BSEE and MSEE degree and earned an MBA degree from Temple University. He served as general manager in several Fortune-500 companies that produced infrared sensors for satellite and military applications. After selling the hi-tech company that he started with a private equity firm, he launched TTSW Advisory, a consulting firm offering clients investment research services. For almost four decades, Ted has developed a true passion for studying the financial markets. He applies engineering design principles and statistical tools to achieve absolute investment returns by actively managing risk in both up and down markets. He can be reached at



A Market Valuation Gauge That Works

Originally Published March 15, 2016 in Advisor Perspectives

In my previous article, I examined many popular metrics that all show that U.S. equities have been overvalued for over 20 years. The conventional explanation is that the overvaluation and its unusually long duration is a statistical outlier. But those aberrations were observed in only 15% of the data population (20 out of 134 years) and are unlikely to be statistical outliers. The root cause is not yet known. Until the anomaly is better understood, naively equating the lack of mean reversion with overvaluations will lead to misguided valuations and ill-advised investment strategies.

A decade ago, I began searching for a valuation indicator that is immune to possible mean-reversion malfunction. The challenge proved to be much more difficult than anticipated. I ultimately had to abandon my search and developed my own valuations gauge, the total return oscillator (TR-Osc) and present it here.

Oscillatory gauge

Mean reversion is the underpinning of all valuations metrics. The basic concept of valuations relies on the notion that value oscillates between an upper bound (overvalued) and an lower bound (undervalued) around a median (fair-valued). How do you calibrate a gauge that has an unbounded output or with a drifing median that confuses mean reversion? A functioning valuations gauge should resemble a pseudo sine-wave oscillator with quasi-periodicity.

Although the cyclically adjusted price-to-earnings ratio (CAPE) oscillated around a stable geometric mean of 14 from 1880 to 1994, its mean has risen to 26.2 since 1995 (Figure 1A) – a telltale sign of mean reversion malfunction. By contrast, my TR-Osc has been bounded by well-defined upper and lower demarcations for over a century. The mean of TR-Osc measured from 1875 to 1994 is almost identical to the value computed over the last 20 years (Figure 1B). After reaching either extreme, TR-Osc always reverts toward its long-term historical mean.

From 1880 to 1950, TR-Osc and CAPE were almost in sync. After 1955, the two indicators began to diverge. Although both the CAPE and TR-Osc detected the dot-com bubble in 2000 (red squares), only the TR-Osc warned us about the 1987 Black Monday crash (red circle). After the 2000 peak, CAPE stayed elevated and came down only once in mid-2009 to touch its historical mean at 14. The TR-Osc, however, dropped to its lower bound in January 2003 (green arrow) getting ready for the six-year bull market from 2003 to 2008. The TR-Osc did it again after the housing bubble when it dipped below the lower bound of 0% in February 2009, just in time to reenter the market at the start of a seven-year bull market from 2009 to present.

In 2008, the TR-Osc reached a minor summit (red triangle) while CAPE exhibited no peak at all. Both TR-Osc and CAPE indicate that the meltdown in global financial markets did not stem from an overvalued equity market. I will expand on this later when I discuss the real estate sector.


Common deficiencies in all contrarian indicators

There are two common deficiencies shared by all contrarian indicators including all traditional valuations models. First, their signals are often premature because the market can stay overvalued or undervalued for years. Greenspan's 1996 irrational exuberance speech alluded to an overvalued market but it was four years too early. From 1996 to the dot-com peak in 2000, the S&P500 surged 87% and the NASDAQ 288%.

The second deficiency of all contrarian indicators is that the market can reverse direction without hitting either extreme at all. The CAPE, for example, was not undervalued in 2002 or 2009. Value investors would have missed out on huge gains of 90% and 180% from the two spectacular bull markets in the 2000s.

The dual gauges of the TR-Osc: scalar and vector

Before I explain how the TR-Osc overcomes these two deficiencies, let me first describe the TR-Osc. The TR-Osc captures what investors in the aggregate earn by investing in the S&P 500. That is the sum of two components – the first from price changes and the second from dividend yields. Price return is the trailing five-year compound annual growth rate (CAGR). Dividend yield is the annual return from the dividends investors received. The look-back period doesn't necessarily have to be five years. All rolling periods from 2 to 20 years can do the job. In addition, both real (inflation-adjusted) and nominal TR-Osc's work equally well because inflation usually does not change much over a five-year period.

The TR-Osc overcomes the two deficiencies by having two orthogonal triggers, a scalar marker and a vector sensor. The oscillatory and mean-reverting attributes of the TR-Osc allow overvaluation and undervaluation markers to be clearly defined (Figure 2). When the TR-Osc was near the upper bound (the 20% overvalued marker), the S&P 500 often peaked. When the TR-Osc was near the lower bound (the 0% undervalued marker), the market soon bottomed. But in 2008, the TR-Osc only reached 12% and the market was not overvalued. Investors had no warning from the valuation marker to avert the impending subprime meltdown. Valuation markers (scalar) alone are not enough. The TR-Osc needs a second trigger, a motion sensor (vector) that tracks the up or down direction of valuations.


Let me illustrate how the scalar and vector triggers work in concert and how buy/sell signals are executed. When the TR-Osc is rising (an up-vector) or drops below the lower bound at 0% (an undervalued marker), a bullish market stance is issued. When the TR-Osc is falling (a down vector) but stays above 0% (not undervalued), or when it exceeds the upper bound at 20% (an overvalued marker), a bearish alarm is sounded. The asymmetry in the buy/sell rules stems from prospect theory, which contends that losses have more emotional impact to people than an equivalent amount of gains.

When a bullish signal is issued, buy the S&P 500 (e.g. SPY). When a bearish alarm is sounded, sell the S&P 500. After exiting the stock market, park the proceeds in 10-year Treasury bonds. The return while holding the S&P 500 is the total return with dividends reinvested. The return while holding U.S. Treasury bonds is the geometric sum of both bond yields and bond price percentage changes caused by interest rate changes.

The performance data presented in this article assume that all buy and sell signals issued at the end of the month were executed at the close in the following month. When the TR-Osc signals were executed closer to the issuance dates, both return and risk performances were slightly better.

TR-Osc performance stats

Figure 3 shows two hypothetical cumulative returns from 1880 to 2015 – the TR-Osc with the buy/sell rules stated above and the S&P 500 total return. Over 135 years, the TR-Osc has a 190 basis point CAGR edge over the buy-and-hold benchmark with less than half of the drawdown risk.

The TR-Osc traded infrequently – less than one round trip a year on average. The TR-Osc is an insurance policy that protects investors against catastrophic market losses while preserving their long-term capital gain tax benefits.


Let's take a closer look at the TR-Osc signals in two more recent time windows. Since 1950, there have been 10 recessions. Figure 4A shows that the TR-Osc kept investors out of the market in all 10 of them. Figure 4B shows that the latest TR-Osc bearish call was issued in September 2015. The TR-Osc sidestepped the recent stock market turmoil and has kept investors' money safe in Treasury bonds.


Table 1 shows performance stats for various sets of bull and bear market cycles. TR-Osc beats the S&P500 total return in CAGR, maximum drawdown, and volatility. The consistency in outperforming the S&P500 in returns and in risk over different sets of full bull/bear cycles demonstrates the robustness of TR-Osc.


TR-Osc has universal applicability

Like the CAPE, the TR-Osc’s efficacy is not limited to the S&P 500. It can also measure valuations in overseas markets (developed and emerging), hard assets and currencies. For example, Figure 5 shows three alternative spaces – raw materials (Figure 5A), oil and gas (Figure 5B) and real estate (Figure 5C) (data source: Professor Kenneth French). This universal applicability of the TR-Osc also enables intermarket synergies. Recall in Figure 2 that the stock market was not overvalued in 2008 according to both the CAPE and the S&P 500 TR-Osc. Note that the real estate TR-Osc correctly detected the housing bubble (red square in Figure 5C). When the systemic risk spread to the stock market, the S&P 500 TR-Osc vector sensed the danger and turned bearish.


Figures 6A to 6C shows that the TR-Osc improves both the return and drawdown in two distinctively different spaces – precious metals (data source: Professor Kenneth French), the Canadian dollar and the Australian dollar (data source: FRED). Prices in precious metals fluctuate widely at rapid speeds while foreign currencies crawl in narrow ranges at a snaillike pace. It's remarkable that the TR-Osc works equally well across drastically different investment classes. How does the TR-Osc help a diverse group of characters with different personalities perform better?


The analytics of TR-Osc

You may say that TR-Osc is just a five-year rolling total return. But what breathes new life into an otherwise ordinary formula is the analytics behind the TR-Osc. The adaptability of buy and sell rules is the reason behind the TR-Osc's universal applicability. As indicated previously, the TR-Osc has two triggers: valuation markers (scalar) and valuation directional sensor (vector). How did I pick the values for these triggers? The vector is obvious – up is bullish and down is bearish – but how do I select the valuation markers?

In Figures 2, 5 and 6, the middle blue line is the mean. The upper blue lines are the overvaluation markers and the lower blue lines, undervaluation markers. The upper blue lines are M standard deviations above the mean and the lower blue lines, N standard deviations below the mean. Each time series has a unique personality. For example, the means of most currencies are near 0% while the mean of the S&P 500 is near 9%. More volatile investments like precious metals, oil and gas would have larger standard deviations than the serene currency space. The values of M and N are selected to match the personality of each underlying investment. The general range for both M and N is between 1 and 2.

A common flaw in the design of engineering or investments systems is over-fitting. I have developedfive criteria to minimize this bad practice. The five criteria are simplicity, sound rationale, rule-based clarity, sufficient sample size, and economic cycle stability. The TR-Osc not only meets all of these criteria but offers one additional merit – universal applicability. It works not only on the S&P 500, but on overseas markets and across a diverse set of alternative investments.

Theoretical support for TR-Osc

Traditional valuation metrics rely on fundamentals, which often experience paradigm shifts across secular cycles. Fundamental factors can be influenced by generational changes – technological advances, demographic waves, socioeconomic evolutions, structural shifts, political reforms or wars. Therefore the means in many of the traditional valuation metrics can drift when the prevailing fundamentals change.

The TR-Osc downplays the importance of the external fundamental factors and focuses primarily on the internal instinct of the investors. Investors' value perception has two behavioral anchors. The first anchor drives investors toward the greed/fear emotional extremes. For example, when the S&P 500 delivers a five-year compound annual return in excess of the 20%, euphoria tends to reach a steady state and investors become increasingly risk adverse. When their returns get stuck at 0% five years in a row, investors are in total despair and the market soon capitulates. Both greed and fear extremes can be quantified by the TR-Osc's over- and undervaluation markers.

The second behavioral anchor is the tendency of herding with the crowd. When neither greed nor fear is at extreme levels, investors have a behavioral bias toward crowd-herding. Once a trend is established in either up or down direction, more investors will jump onboard the momentum train and price momentum will solidify into sustainable trends. The collective movement of the masses is tracked by the TR-Osc's vector sensor.

Concluding remarks

Unlike fundamental factors which can be altered by paradigm shifts over long arcs of time, human behaviors which are hardwired into our brains have not changed for thousands of years. The efficient market hypothesis assumes that markets are made up of a large number of rational investors efficiently digesting all relevant information to maximize their wealth. Behavioral finance theory suggests that investors are often driven by the inherent cognitive psychology of people whose decisions are often irrational and their actions exhibit behavioral biases. Perhaps the aberration (the malfunctioned mean reversion) observed in many of the traditional valuations ratios suggests that investors are not 100% homo economicus beings after all. More often than not, investors behave irrationally when they are besieged by emotions.

The TR-Osc captures the essence of both traditional finance and behavioral economics by reading investors' value perception from both the rational and the emotional wirings of their brains. It elucidates many valuable but abstract concepts from both schools into quantitative, objective and actionable investment strategies. As long as humans continue to use their dual-process brains (see also Dr. Daniel Kahneman) in decision making, TR-Osc will likely endure as a calibrated valuation gauge until humans evolve into the next stage.

The TR-Osc asserts that the current stock market is not overvalued. Instead, since mid-2015, its vector has been reverting towards its stable historical mean.

Theodore Wong graduated from MIT with a BSEE and MSEE degree. He served as general manager in several Fortune-500 companies that produced infrared sensors for satellite and military applications. After selling the hi-tech company that he started with a private equity firm, he launched TTSW Advisory, a consulting firm offering clients investment research services. For over three decades, Ted has developed a true passion in the financial markets. He applies engineering statistical tools to achieve absolute investment returns by actively managing risk in both up and down markets. He can be reached at