This post was originally published on my GoodeValue.com blog on 3/18/2009. Due to blog moves it was not correctly moved to this blog so I have reposted it. 

The Plight of the Ordinary Investor

A few finance professors measured the performance of all investors in Taiwan over a period of years by looking at the data direct from the exchange. They found that individual investors and traders underperformed the Taiwanese stock market by an average of 3.8 percentage points per year. Institutions outperformed by an average of 1.5 percentage points. While comprehensive data such as these are not available in the United States, smaller surveys have consistently shown that individual investors underperform the broad market, whether they buy individual stocks or mutual funds or hedge funds. Individual investors are often like kids suffering from shiny ball syndrome: whatever stock is hot or in the news catches their attention and they buy it; they trade too much generating lots of commissions, sub-par returns, and big tax bills. Such ‘investing’ is the opposite of rational investing.

This article will show you the way to avoid becoming just another losing individual investor (not that only individual investors do poorly—Alfred Cowles’ pioneering research 70 years ago showed that both stock newsletters’ and insurance companies’ stocks picks underperformed the market as a whole). This is the definitive guide not about how to invest, but about how to learn to invest.

Certainty and Overconfidence

I recently asked readers of my blog on Motley Fool CAPS whether they were good or bad drivers. Driving skill should be only loosely correlated with interest in investing (the very young are horrid drivers and are unlikely to be avid investors, while the very old are poor drivers and are unlikely to be reading blogs), so while this is not a representative sample I had no doubt that my little poll would replicate more scientific polls. What I found was unsurprising to me: out of 159 responses, 46% rated themselves as “above average drivers”, 44% rated themselves as “average drivers”, and only 10% rated themselves as “below average drivers.” In driving as well as in other endeavors, people believe that they are smarter, more attractive, and better than they are in reality. I decided to replicate this study by asking my blog readers about their attractiveness: while 39% of 102 respondents thought they were in the most attractive third of the population, only 22% said they were in the least attractive third of the population. What is amazing is that people can be overconfident about anything: in a third poll I asked the simple and nonsensical question “Quack?” to which 58% of 66 respondents replied “Quack!!!!!!!!!!!” while only 42% responded “Quack?”.

Most people are overconfident most of the time (this is sometimes known as the Lake Wobegon Effect). This has advantages. Those who are not overconfident never dare to dream and to try to do unlikely things. Thus they never accomplish as much as the overconfident do. To misquote George Bernard Shaw, “The diffident man adapts himself to the world: the overconfident one persists in trying to adapt the world to himself. Therefore all progress depends on the overconfident man.” If people were not overconfident they would apply to far fewer jobs, they would have fewer dates, there would be fewer world records and less greatness.

But in certain areas our human tendency towards overconfidence acts as a tether rather than as a pair of wings. One of these situations is investing. People are generally overconfident in their abilities as investors, just as they are in other aspects of their lives. Professor Terrence Odean has theorized that investors become overconfident because they attribute their successes to skill while forgetting about or attributing to bad luck their failures. One of the key findings in the psychological study of overconfidence is that the worst performers are the most overconfident; they have the worst ‘calibration’, or understanding of their own skill. This miscalibration prevents underperformers from correcting their errors. The first step in fixing a problem is to realize that a problem exists.

Improving Investment Performance First Requires Measuring It

How much do you weigh? While people may not like this question, most people can answer it accurately. Anyone who cares about their weight or fitness can probably recite a reasonably accurate history of how their weight has fluctuated over the last couple years. If you care about your health and fitness more than about your weight, you can probably recite other statistics, such as how quickly you can run a mile, how much you can bench-press, and what your resting heart rate is. This makes perfect sense: if an athlete did not measure her progress, how could she know if she were training correctly?

People purport to care about their investment returns. In reality, few really care. If they did care, they would put forth the effort to accurately track their investment performance. Back when I was a more normal investor, holding several mutual funds and maybe 15 stocks, I set forth to measure my investment performance. It took me about 20 hours to enter all my transactions into the free website Icarra.com. The result? I found that my grand efforts netted me maybe 2% better performance than the S&P 500 per year. That was not very much considering how much effort I was putting into managing my portfolio! I was also not confident that I could continue to outperform the market. Seeing my performance forced me to question whether I was putting too much effort into investing (about 20 hours per week or more following companies, reading books, and researching better ways to invest). That time was definitely hurting my performance in graduate school.

Just as an athlete would not fail to measure her performance, an investor must not fail to measure hers. Otherwise an investor will not know whether she is doing well or poorly. Icarra is free and easy to use. There are other ways to track performance, but most of them fail utterly. To accurately track performance you need to account for cash that is sitting in your brokerage account and you need to account for deposits and withdrawals from your brokerage account. Remember the Beardstown Ladies, an extremely profitable investment club from the mid-1990s? They “beat the market” and became famous. The only problem was that they failed to properly account for deposits they made to their brokerage account, such that they counted those deposits as profits. The Beardstown Ladies actually under-performed the S&P 500 by 1.9% per year from 1983 to 1997.

If you wish to see my performance, take a look at my long term longs portfolio on Icarra; the goal of this portfolio is just to offer market-like returns. See also my Roth Quant portfolio, which was traded using a quantitative trading strategy and is currently all in cash. On the other hand, I am a professional stock day-trader. I do not track these sorts of trades (90% of which are intra-day) online and I do not compare them to the market. There would be no reason for these trades to correlate with the stock market as I go both long and short; I track them in an Excel file and measure my success by total profit, profit margin, and other such metrics.

Once you start using Icarra (or a similar program / service), make sure you look at your Sharpe Ratio and Sortino Ratio (I prefer the Sortino). They are measurements of volatility-weighted performance relative to your benchmark (Icarra uses the T-bill return as the benchmark). Large positive numbers are good! Volatile portfolios are riskier than non-volatile portfolios, so these numbers give you a good idea of risk-adjusted portfolio performance.

I beseech thee now, go out and track thy performance!

The Data do not Lie, and if You Listen Closely You May Hear Their Glorious Song!

“How do I invest? How do I Trade?” These are questions I receive fairly often, as a fairly popular blogger (a couple hundred visitors per day at GoodeValue.com despite my infrequency of blogging) and a Top Fool on Motley Fool CAPS. The answer I give changes over time with my mood and as my opinions change. But the core of any good answer to that question is this: do what works. So be empirical! See what works in investing and then do it! Of course, such advice is almost too vague to be useful. But I find that most investors are not empirical at all. They use their gut, they go with hunches, they make decisions based only on a few pieces of evidence.

A simple empirical question, “what kinds of stocks outperform the market?” has a few different answers. Simply by asking the question we avoid the pitfalls of many investors who buy and sell randomly; as an example, my real estate agent bought shares in Arch Coal “because [he] saw so many trains going by, full of coal.” One variation of this question is as follows: What has been the best-performing class of stocks over the last 85 years? The answer is small value stocks. By far the best performers are microcap value stocks. They are highly volatile, risky, and illiquid. (See the great data behind the linked academic articles at Prof. Ken French’s website.) Of course, the articles I link are not the ‘truth’; it pays to check out contrary viewpoints and different data sets. For academic research on investing, search on SSRN; why not try searching a term such as ‘Motley Fool’ to start?

My arguments in favor of empiricism do not come at the expense of theory; I believe that any investing or trading strategy should have good theoretical underpinnings as well. Hence, we must ask ourselves: Why do value stocks outperform? Ken French and Eugene Fama, authors of the above-linked articles, argue that they are riskier. Higher returns are compensation for higher risk. Others, such as Joseph Piotroski, argue that the value premium does not reflect risk but rather a market failure. I believe that consistently illogical human thinking results in value stocks being consistently priced too low. (I thus fall into the behavioral finance camp.)

Why Do Things Happen? The Importance of Theory

Every year I see articles in the Wall Street Journal and other reputable financial news rags about the Superbowl indicator of the stock market. Supposedly if an NFC team wins, the stock market will go up that year. If an AFC team wins, the stock market will go down. Of course, there is no reasonable way the Superbowl could affect the stock market. Thus any investor who follows this indicator would be doing something very stupid.

Not every stock market indicator is as foolish and as theoretically vacuous as the Superbowl indicator, though; not all dumb theories are as easy to debunk. Take some popular technical indicators such as Gann Angles and Elliott Wave Principle, both of which posit that stock market prices follow certain mathematical rules. Neither of these technical analysis theories have behind them a plausible theory of why they should work. The hypothesis that Elliott Waves follow fundamental mathematical relationships found in nature is a non-explanation. You can find lots of things in nature that obey interesting mathematical rules (like the Fibonacci series), but it is impossible to predict a priori what will or will not follow such a series. Thus saying that something follows some mathematical rule describes but does not explain it.

Any theory should be testable and thus falsifiable (it should make predictions that can be proven wrong), and to quote one technical analyst, “The Elliott Wave Principle, as popularly practiced, is not a legitimate theory, but a story, and a compelling one that is eloquently told by Robert Prechter. The account is especially persuasive because EWP has the seemingly remarkable ability to fit any segment of market history down to its most minute fluctuations. I contend this is made possible by the method’s loosely defined rules and the ability to postulate a large number of nested waves of varying magnitude. This gives the Elliott analyst the same freedom and flexibility that allowed pre-Copernican astronomers to explain all observed planet movements even though their underlying theory of an Earth-centered universe was wrong.” (quote from Wikipedia article linked above.)

A person can legitimately criticize the vacuous theory behind a strategy while believing the strategy still works. Such is the case with me and some basic technical analysis chart patterns like head and shoulders and cup and handle patterns. Whatever theory originally attempted to explain them matters little to me. My theory is that they work because they are very popular and people believe them. They become a sort of self-fulfilling prophecy. Professor Carol Osler’s research has found that some basic chart patterns do work in foreign exchange markets. Conversely, one can find a theory behind a strategy to be plausible while finding that the strategy itself does not work. Such is the case with Dow Theory. Yet while the theory seems plausible (at least to me), a test of William Peter Hamilton’s early 20^th century stock market prognostications using the theory was mixed, with Dow Theory underperforming the stock market in real terms but outperforming on a volatility-adjusted basis (it had a higher Sharpe ratio).

One last note on theory: the Dogs of the Dow is one of the most over-hyped, over-used useless investing strategies, with little theory behind it. The Motley Fool used to have a couple portfolios based on the Dogs of the Dow and they had even less theory backing them up.

What is the Null Hypothesis? What is the Default Investment?

In scientific endeavors we try to prove the new by showing that the data do not support the current way of thinking (the ‘null hypothesis’). In investing we need such a null hypothesis to form the default investment so that people do not invest randomly and risk losing lots of money (I have previously given thought to what the default investment should be). I will derive the default investment and then I will show why so few other investments make sense when compared to it.

Imagine that novice investors picked investments more or less randomly (which is pretty close to the truth). George invests in a corporate bond fund, Mary invests in biotech stocks, Clarissa invests in blue chips, and Thomas invests in sovereign debt. Add in a few million more neophyte investors and on average, these investors are buying the market of all investable assets. On average, these investors (through mutual funds and individually) own a portion of almost all the US corporate bonds, Treasury bonds , foreign corporate bonds, foreign government sovereign bonds, all United States and foreign stocks (except for the smallest foreign stocks), and much investment real estate in developed countries. This group as a whole will do as well as the market as a whole (all bonds + investment real estate + stocks). Most of these investors are poorly diversified: stock investors often own just a couple stocks; mutual fund investors are often not diversified across asset classes, many holding almost all stocks, some holding all bonds. Because these investors have so little diversification, they will tend to do either much better or much worse than the market. The variance in outcomes will be high. Even worse, the high variance in outcomes will likely lead those with poor outcomes to avoid risky assets and those with good outcomes to over-trade, believing they are investing geniuses rather than just lucky. The default investment should simply be what the average individual investor already holds: a wide variety of stocks, bonds, and real estate from throughout the world. This can easily be done with the purchase of a few low-fee index funds.

The Market is Efficient and It Does Not Care if You Agree

When most people hear an explanation of the efficient market hypothesis (EMH) they instinctively reject the concept. “What?! Of course if you are smart and do good research you can beat the market. Just look at Peter Lynch / Warren Buffet / me!” (Of course, I could reply, “Look at how mutual fund managers generally underperform the market and how there is little evidence they exhibit skill and how after taxes active mutual fund and hedge fund investors generally lose to index fund investors.”) Try to hold back your instinctive dislike of such a concept as I explain why the market is efficient even if the EMH is not completely correct.

There are a large number of very smart people doing a lot of work to try to determine the correct prices for stocks and bonds. These people are highly motivated and often make lots of money. If they find an undervalued stock or bond they buy it, pushing up the price and profiting from the increase in value. If they find an overvalued stock they short sell it (or if there is an overvalued bond they buy credit default swaps on it) and cause its price to decrease a little. The more undervalued / overvalued a stock or bond is, the more investors will flock to trade it and the quicker it will return to a reasonable estimation of its fair value. This does not mean that stocks and bonds will always be accurately priced. It does mean that it is very rare for a security to be very inaccurately priced.

Take a look at how the market deals with news. When Roche recently upped its bid for Genentech (DNA) from $93 to $95, the stock price did not increase. It had already increased by about $2 per share a few minutes after a Dow Jones Newswires report came out days earlier saying that there were rumors that Roche would raise its bid. Similarly, when an analyst increases his price target or rating for a stock it will tend to jump immediately. Price moves based on news are almost instantaneous and are not very profitable even for the hedge funds with computer programs that trade based on news stories milliseconds after they hit the newswires.

Even “soft news” such as blog posts can quickly affect the price of a stock. When I put out a scathing blog post on H2Diesel (now New Generation Biofuels, NGBF) back when it was traded on the dreadful OTC BB (which is not a very efficient market and is overpriced on average because of difficulties in short selling), within five days the stock price had dropped 40% (see chart: my blog post came out on 10/12/2007 and it was syndicated on Seeking Alpha on 10/15/2007). (I should note that unfortunately I did not short sell this stock even though I could have done so; my capital was tied up elsewhere.)

In a fairly efficient market where many smart investors fight over tiny scraps of information and do thorough research to try to gain an upper hand and outperform the market, an investor cannot profit from publicly known information unless other investors fail to appreciate the importance of such information. What this means is that any investor who thinks that they can beat the market with some stock pick is betting that every other current buyer and seller of that stock does not understand something fundamental about that company. Any decision to pick stocks, actively trade, or time the market is a big bet that everyone else is wrong. Investors should not make such hubristic decisions lightly. There are three things that might validly convince an investor to try to beat the market: know something no one else knows, know something everyone else knows but does not understand, or know something that no one else can act upon.

Know What No One Else Knows

The easiest way to profit from this is to acquire inside information. It worked for Ivan Boesky (his story is told in the excellent book Den of Thieves) and it can work for you! This is obviously illegal. A legal way to do this is to make investments based on what insiders are doing (Does that work and does that make sense? Remember, these are the questions you should always ask yourself. Yes and yes, according to reams of research and a good book, Investment Intelligence from Insider Trading by Nejut Seyhun). Another way is to do what analysts are supposed to do: talk to suppliers, industry contacts, customers, management, and check out stores so that you understand the company better than anyone other than the company’s management. While this can work, it is an impossibly large amount of work for any non-professional, and even the best stock analysts do not have enviable track records.

An example of this would have been how I knew before almost all other investors about the acute shortage of acetonitrile—I am married to a chemist and have friends working at many large chemical and pharmaceutical companies; this gives me access to lots of chemical-industry information, and as long as it is not specific to a particular company, trading on it would not be illegal. If I felt that the information was actionable, I could easily have bought stock in Sigma-Aldrich (SIAL), one of the largest distributors of acetonitrile (which is a by-product of the manufacture of certain plastics, most often used in cars), long before anyone heard of the problem. Unfortunately, the shortage is not actionable because it does not benefit Sigma-Aldrich or any other company and most companies affected by it can cope and are thus not short candidates.

Another way to know what no one else (or at least very few people) know is to concentrate on the smallest and least-followed companies. I made plenty of money over the last year and a half by doing in-depth research of sketchy penny stock companies and then short selling their stocks. As an example, take a look at my early blog posts on Continental Fuels (now CNFU.ob, formerly CFUL.ob). The company’s capital structure and market capitalization was so obscured by complicated convertible stock deals that I doubted any of the investors understood just how badly overvalued the company was. (This also falls under the rubrics of “know something everyone else knows but does not understand”–because the investors in such companies are usually idiots–and “know something that no one else can act upon”—because it is very difficult to short sell OTC BB stocks).

Know Something Everyone Else Knows but No One Understands

This is the investing method that everyone thinks they can do but almost none can. A great example of this is the story of John McQuown (who later helped to develop the first index fund while at Wells Fargo; this story and others of the rise of modern finance are described in Peter Bernstein’s excellent book Capital Ideas). A mechanical engineer, he borrowed $1600 against his Chevy convertible in 1957 to buy 400 shares of Texas Instruments (TXN). He believed that the invention of the transistor would revolutionize all electronics and TI would benefit. He was right. He liquidated his investment two years later for $180,000 (112 times his original investment!). Unfortunately he let it get to his head and in three more years he had lost all the money he had made in that trade.

I myself have made some small profits by understanding earlier than many how bad the housing crash was going to be. I used my understanding of the situation to profitably short sell some banks that had huge construction loans outstanding. Others saw the housing collapse and credit crunch earlier or more clearly and made fortunes (John Paulson comes to mind). A few years ago an insightful investor could have made a killing by understanding how powerful Google’s (GOOG) technology was (and how incompetent Yahoo and Microsoft would be in response) or how good Steve Jobs’ ideas and designs would be at Apple (AAPL).

One problem with this way of beating the market is that it requires the investor to be smarter / wiser than almost everyone else. The investor also has to be lucky as to timing. It was a no-brainer that internet stocks were overvalued in 1997. Plenty of people went broke shorting them for the next two years. Calling the top required luck as well as skill. Relying upon luck is a very poor strategy. Even ignoring the question of timing, it is very hard to be consistently wiser than every other investor. Many who try make a few good calls and then make a ton of very bad calls (like John McQuown above).

The other problem with this strategy is that everybody understands the big themes. Everyone knew Google would be wildly successful when it went public. Its P/E was in the stratosphere. The harder question was whether the stock already priced in the growth the company would achieve. Everyone knew that computers would change everything in the early 1980s. That was true. Microsoft (MSFT) and Apple (AAPL) would have made stellar investments. Yet that did no good for investors in public companies that went bankrupt such as Miniscribe, Syquest, Tandy, Wang Laboratories (which was one of the hottest stocks of the time and had 27,000 employees), and many others. The same thing happened during the Tronix boom of the late 50s, the railroad booms in the 19^th century, and the dot-com boom of the late 1990s: while huge changes occurred and many got rich off them, many more investors lost money investing in the companies that failed. For every Microsoft there is an Iomega. For every Cisco (CSCO) there are a Nortel and a Lucent (ALU) and a Motorola (MOT).

 

Know Something That No One Else Can Act Upon

Perhaps the best example of knowing something that no one else can act upon is one of my best day-trading strategies. I look for low-priced stocks that go up 40% or more in under a minute. I then short sell them. No one but a few other day traders will touch these stocks for a few reasons: they are illiquid, these patterns play out very quickly (within a minute or two), and very few traders have the right tools to identify such stocks. I blogged about a recent such stock that gained 1000% in minutes and lost it almost as quickly. Remember my criteria for a good investment or trading strategy above? Any good strategy should be empirically supported (as this strategy is: I have made a couple dozen such trades and they have been highly net profitable and over 80% of individual trades have been profitable) and theoretically valid. My theory for why this trading strategy works is simple: the only reason for such huge moves in these stocks is that a trader accidentally places a larger order than desired (say, buying 10,000 shares instead of 1,000) or accidentally places a market order instead of a limit order.

Another example of knowing something that no one else can act upon is the use of arbitrage strategies last October and November and even now. The market meltdown killed arbitrage traders, who are often highly leveraged. This made possible plenty of easy trades such as arbitraging KV Therapeutics (motto: “Oops, we might have contaminated the drugs we’ve been selling! Sorry.”) by going long KV-A and short KV-B (I did this after the B shares doubled and the A shares did not move). In Mueller Water, going long MWA-B short MWA could have netted an arbitrageur an unleveraged 30% back in November (the company recently converted all the B shares into A shares). A similar arbitrage in Blockbuster (BBI & BBI-B) would have been very profitable. The key to profit from this type of strategy is to find a market where the normal buyers cannot buy for some reason and then look for bargains. Many believe that mortgage-backed securities (MBS) offer such opportunities now; even good ones are trading at very low levels. Of course, there are also many MBS that are worth $0.

Interim Conclusions: Data & Theory Good … but Problems Lurk

I have hopefully shown why any investment philosophy should be both proven by historical data and theoretically sound. But in the messy, uncertain world of human behavior we cannot just observe that something happened in the past and know that the same thing will happen in the future. Unlike with chemistry, where mixing potassium metal and water will always lead to fun things, the observations and actions of investors will change how the stock market acts over time. George Soros has coined the term “reflexivity” to refer to this. Soros said, “The theory holds, in the most general terms, that the way philosophy and natural science have taught us to look at the world is basically inappropriate when we are considering events which have thinking participants.”

Besides the problem of reflexivity there are also statistical problems and practical problems to those seeking to ‘beat the market.’ This article grows long, however, and my wrists grow weak. A guide to the problems facing any practitioner of evidence-based investing and examples of such problems will be coming in Parts 2 & 3 of this series.

Disclaimer: No positions in any stocks mentioned. This blog has a terms of use that is incorporated by reference into this post; you can find all my disclaimers and disclosures there as well..