The Most Dangerous Trap For Stock Pickers (And 9 Ways Top Investors Avoid It)

In the career of all investors there comes a time where you'll get excited about an investment.  If you’ve done your homework (if not, check out the 15 favorite books of hedge fund analysts), then you already know the maxim that emotions and investing shouldn't mix. Facts, logic, hard work, and careful due diligence need to take precedence in order to outperform the vast body of investment professionals that constitute Mr. Market. 

That being said, with your research done and any emotional bias carefully taken out of the equation, you can start to feel comfortable with your investment decisions… right?

Wrong. Once graduated from the rookie leagues of day traders that “trade on instinct/gut” serious investors face a new set of behavioral gotchas. These aren’t emotions per se, but consistent and systematic flaws in human decision-making that thwart our best attempts at being truly logical. These so called cognitive biases pose a significant impediment to making good investment decisions.  For value investors, with their focus on amassing tremendous detail and due diligence on single stocks, one insidious bias stands above all others: confirmation bias.

Confirmation bias, also called confirmatory bias or myside bias, is the tendency to search for, interpret, favor, and recall information in a way that confirms one's preexisting beliefs or hypotheses. (Wikipedia, 8/2017)

While the effect has recently been brought into the popular imagination by books like Michael Lewis' The Undoing Project, it has been well known in the world of value investing since the days of Ben Graham.  Spending hundreds of hours researching a stock anchors analysts to their work, leading them to systematically and subconsciously ignore downsides, risks, and other warning signs that would otherwise discredit their sunk work and render it useless.

Given this, more work does not necessarily equal better investment performance. Looking at data from SumZero, the world’s largest buy-side investment research platform, we can see this effect in action.  We analyzed over 10,000 ideas submitted by thousands of analysts since 2008, and came to a surprising conclusion.

Performance of investment ideas that are above or below average word count out of all ideas on SumZero.

The average word count of these investment  ideas is about 1700 words. If we segment these ideas into distinct buckets, we can see that ideas with a higher than average word count return almost 10% less than ideas with a below average word count. Thus, lengthier reports don’t generate greater returns than much shorter reports; in fact they perform worse!

For instance, this December 2016 long idea on AMD was only 1500 words long and returned 425% to date, while this  June 2016 short idea on GM lost almost 25% despite almost 3000 words of analysis.  Granted, we are speaking in averages, and there are many ideas that break this mold i.e. profitable verbose ideas and unprofitable concise ones. 

Investment research based on the inertia of sunk work can lead to bad investment decisions.  If you find that you're gaining conviction in an idea as you spend more time looking at it, question whether or not this is because of confirmation bias.

As Warren Buffett famously said: "It is better to be approximately right than precisely wrong." 

Based on SumZeros data, our analysis, and the well-researched information bias (the fallacy that more information always leads to better decisions), it is reasonable to infer that a longer "better researched" investment reports will not outperform a shorter one.

The best investors are not just aware of confirmation bias; they also have systematic ways of dealing with it as part of their investment process.

To get insight into best practices for avoiding confirmation bias, I spoke with Jeff Meyers, founder of Cobia Capital Management, Henrik Heffermehl, portfolio manager at Park Lane, and John Rolfe, founder of Argand Capital Advisors, the three top ranked investors (out of over ten thousand professional analysts on SumZero).  Below their wisdom is aggregated into 9 rules for avoiding confirmation bias.

1. Focus on the potential for loss instead of the potential reward
Meyers: It is difficult to make the decision to sell a large position that you have done many hours of work on and have already imagined its profits in your head.  I do guarantee though that it is much more painful to do nothing and then lose lots of money on that position.

Heffermehl: When I write out my initial investment thesis, I always include a pre-mortem: if something is going to go wrong looking back on this case, what is it likely to be? I try to see being wrong from time to time as a natural part of doing business, there should be as little ego as possible involved. That way it will hurt less when you're wrong.

2. Abide strictly to checklists and processes

Heffermehl: Writing the thesis down is always an absolute requirement. If I still like the idea after writing it up, I'll go through a proprietary checklist (60 or so items, not all apply to all companies). These items cover various points on capital structure, management, moats, valuation and catalysts. If the margin of safety is inadequate I end up doing nothing and will just monitor the company in case the valuation becomes more favorable at some point or a problem I have with some other issue gets resolved. 

Having a structured process helps to reduce errors, but will not eliminate them. Since errors can't realistically be eliminated, I focus on damage control by always looking for a large margin of safety in terms of valuation, that can allow me to be wrong about an investment without incurring large losses. The flipside of this is that I'll probably miss out on lots of things. I accept that. Better safe than sorry.

Rolfe: Having a “true north” for what drives investment decisions is the only way to be successful in this business over the long-term. It ensures that you can reasonably narrow down the thousands of potential investment opportunities (i.e. publicly listed equities, in my case) to a manageable number that are worth pursuing. Without this discipline, you can easily waste hours or days chasing down an opportunity that you will never get comfortable with, based solely on a well-crafted pitch that somebody makes to you that falls outside your investment wheelhouse. 

So, for instance, as the current bull market has gotten long in the tooth I have found that a lot of self-proclaimed “value” investors have been finding it difficult to find attractive investment and, as a result, have been stretching what I would consider the traditional precepts of value investing by pitching and/or investing in unprofitable SaaS-related software businesses.

3. Have someone else double check your work.

Heffermehl: I have to convince a two-person board to ok every investment. This additional hurdle is very useful - I think it reduces unforced errors. It's harder to fool yourself if you have to discuss your hypothesis with someone first. And their perspectives and questions also forces you to reexamine your assumptions an extra time. This might not work as well for others, but it works really well at our firm.

Rolfe: For the last nine years I’ve been running my partnership as a one-man shop. So, the issue of confirmation bias is particularly relevant as there is nobody present on a day-to-day basis to serve as a sanity check, point out these biases, and help me avoid them. I have to be extra careful with process to counteract this.

4. Be a student of the opposing view. The internet makes this easy.
Rolfe: If a prospect seems promising, next steps include research on counter-arguments to my initial thesis. Buyside-driven websites like SumZero, are great resources to get access to contrary opinions for both long and short sale opportunities. In addition, the emergence over the last few years of many activist short sellers has been a huge plus for those looking for contrary opinions to their long prospects.

Recently, I spent some time looking at going long UK-based support and construction services company called Carillion plc.  As I dug in a bit more on the business I found a well-reasoned short seller’s report on the business. As a result, I tabled my work on the business pending a deeper dive on the alleged issues. Several weeks later, the company announced a massive write-down, a huge profit shortfall, and a significant management reorganization. The stock dropped by roughly 70%.

Heffermehl: Apart from the people I work with and our respective networks, which is the most valuable source of variant views and extra color for me, I think communities like SumZero are great places to seek out views that differ from my own. People don't hold back there if they think you're wrong. I'll look at reports from analysts on both the buy and sell side who are negative or short, and try to contact them in person if possible to hear the counter-thesis to my own. Starting from the assumption that somebody knows more than you do is healthy.

While researching Tesco back in 2014. Tesco's moats were eroding, but I had somehow convinced myself otherwise and I mistook what was in fact secular margin pressure for temporary weakness. Had I been more diligent in seeking out the basis for the short case, I might have avoided that mistake. I started suspecting that something was wrong with my thinking after a sudden strategy change, management change-up and an unexpected profit warning. That made me reconsider my assumptions about Tesco's cost advantage and sell our Tesco holdings at a fairly small loss. Things got a lot worse for Tesco shortly after, so admitting my mistake early probably saved us a lot of money.

5. Be data driven; don’t rely on "feel".

Meyers: In order to be a successful investor, it is extremely important to be intellectually honest.  If you deceive yourself when putting on a position, things will not work out the way you envision them.  This intellectual honesty is critical once you already have a position on, as well.  If data points arise that contradict your thesis and you downplay or simply ignore them, then your thesis is compromised and you will not get the result you are looking for.  

Rolfe: Broadly speaking, my approach to minimizing the risk of confirmation bias has been to develop agnostic quantitative systems over the years to measure the relative attractiveness of both portfolio investments and prospects in an unbiased manner. The primary tool for doing this is effectively a large linear program (simplex algorithm) that ranks both investments and prospects based on three primary factors: 1) size of the discrepancy between trading value and intrinsic value, 2) strength and immediacy of any underlying (potential) catalyst, and 3) quality of the underlying business. Effectively, I keep a real-time ranking of all investments and prospects with respect to their probability-weighted discount to intrinsic value. This keeps me honest with respect to not only whether a given idea merits inclusion in the portfolio, but how to appropriately size positions relative to one another as prices move.

6. Don’t set artificial deadlines; they are not a good reason to invest.

Heffermehl: I don't mind spending a lot of time on an investment and then setting it aside for a while (or permanently). I don't set deadlines for completing work on a name. Procrastination can be constructive for the investment process, I think.

7. Talk to management.

Meyers: A key example is one of our larger, long-term positions, Silicom Ltd (SILC). We first came upon Silicom in late 2008 as many small technology stocks were getting crushed by the decline in the overall stock market.  We liked the underlying demand driver for Silicom which was that its customers’ markets were all growing 10-15%.  This meant that Silicom itself should grow at a minimum baseline of 10-15%.  But finally and indeed most importantly, we had extensive conversations with management and came away impressed with their strategy and their character.  Both of these factors are critical to us.  

Rolfe: I won’t typically put a core position into the portfolio without speaking with management of the company. Often, a management call will be towards the end up my diligence process as I don’t typically want to speak with a member of senior management before being fully up-to-speed on the company. It is often during management calls that I will hit a stumbling block on an investment that, prior to the call, seemed promising. Oftentimes, the red flag will arise because management is unable to articulate a compelling approach to (or even an understanding of) capital allocation. Other times, management will confirm prior causes for concern that arose during the pre-call diligence process.

For instance, I recently spent time looking at a UK-based company called Utilitywise plc. The company has a new management team in place, is highly regarded by its customers, and appeared to have an extremely cheap valuation relative to its growth prospects. However, there was a material unfavorable discrepancy between reported earnings and cash flow as a result of the necessity for management to estimate long-term contract profitability at the time of signing. Management’s estimates in this regard appeared to be grounded in solid historical precedent, however, during my conversations with management it was disclosed that all UK companies are currently subject to a new revenue recognition standard (IFRS 15), and that Utilitywise expects to disclose potential changes to its accounting under this standard within the next few months.  I elected to hold off on further work until the company made a definitive disclosure with respect to IFRS impact on the P&L.

8. Treat work as something useful for later, even if the answer ends up being “no” right now.

Heffermehl: I don't view a broken initial hypothesis as a waste of time. The work might come in handy later in a different context. So in that sense, nearly all ideas are worth pursuing to a point.

9. Treat "grassroots" research with care.

Rolfe: The tendency for confirmation bias is often particularly acute with investors that rely on a lot of grass-roots diligence (i.e. making customer calls, competitor calls, etc.) to drive their investment decisions. Grass roots diligence suffers from two primary issues. The first is that the volume of calls and/or contacts is rarely large enough to be statistically significant. Thus, investment decisions can get made based on a sample that isn’t truly representative. Second, customers who are willing to respond to investor inquiries often aren’t representative of the overall customer base. Those willing to speak often skew towards more favorable assessments of the business in question, as those with negative experiences may no longer be customers. Earlier in my career I relied more heavily on these sorts of qualitative markers to drive investment decision, but found that they often served merely to confirm preconceived notions about the relative attractiveness of a given potential investment. Over time, I’ve moved away from qualitative assessments towards a heavier reliance on the numbers themselves, which if you’re focused on cash flow (as opposed to GAAP earnings) tend not to lie.

Meyers: We have found that often our most successful investments are those for which we do the most primary fieldwork and run down as many leads and information sources as possible.  In certain sectors primary research is a cornerstone of our diligence process.

One particularly good example of this is our most profitable short of 2014.  The company was a recent IPO that billed itself as a high-growth Software-as-a-Service (SaaS) offering for large enterprises.  We quickly found a sales demonstration event for potential customers and attended to see and interact with the product firsthand.  What we realized was that the product was barely Software, and was actually a laborious and highly manual Services process.

We also leveraged a personal relationship with an employee of a very large business that could be a buyer of the company’s product and spoke to someone in the department that would use it.  This employee explained very matter-of-factly that the company used a homegrown system that worked extremely well, and as far as she knew most other companies within the same market used similar systems as well.

With a potential customer having expressed a total lack of need for the product, we then sought out the opinions of existing customers.  We looked through several years of press releases and case studies and reached out to every customer the company had named.  In our experience, cold introduction requests to customers mainly lead to discussions in two cases – when the customer is very satisfied and wants to laud a product, or when a customer hates a product and wants an opportunity to vent.  In this case, all our calls fell into the latter bucket.  Time after time we heard stories of lacking functionality, incomplete integrations, terrible customer service, and extreme dissatisfaction.  Amazingly, many of these customers who had just been announced in recent years had already rolled off the product or were actively searching for alternatives.

Having identified customer dissatisfaction, we turned our focus to employee views of the company.  Sites like Glassdoor can be helpful to glean some opinions, but only after you push aside opinions and complaints and focus on more substantive comments.  In this case, employees commented constantly about a sole focus on growth, and a lack of manpower and training in the customer service organization.  These employees viewed the “growth at all costs” mentality as leading to disgruntled customers who were left to fend for themselves shortly after implementation.

Whereas the company billed itself as the one and only provider of this type of product, we identified no less than three direct private competitors all of whom boasted very impressive client rosters.  Management of all three firms were happy to speak with us, and discussed at length how their competitor, in the lead up to its public offering, threw all its weight behind new customer sign ups, offering one-time concessions and diverting focus from its implementation and service functions.  All three competitors told us that they had seen an uptick in leads coming from customers of the company who were unhappy with subsequent price increases and subpar service.

While conducting all this research, we also spoke to and met with management of the company on numerous occasions.  We found them to be highly promotional, not particularly credible, and unable to justify or define numerous basic claims made in their public marketing materials.  Perhaps the biggest red flag was just how evasive management could be – following one group meeting at an investor conference, we wrote in our notes, “[CFO] is truly unbelievable… he was unable or unwilling to answer the majority of questions posed by me or any of the other people in the meeting.”

The combination of all these factors convinced us that this company was poised for a spectacular fall.  Indeed after shorting the stock, the company went on to miss earnings and revise guidance several times.  We believe that as analysts and investors finally started to do their own diligence and come to some of the same conclusions that we did, the company started to suffer serious multiple contraction, which ultimately brought our gain on the investment to over 65%.