Startup Success Requires The Drive For Data

Editor's note: Jeff Ma is the founder of tenXer, a startup that provides tools to help people better understand their performance, progress, and productivity at work. Previously, he helped start GolfSpan.com, CircleLending, and Citizen Sports. Follow him on Twitter.

The first thing you learn when you start playing blackjack is something called basic strategy - a decision matrix that tells you the correct play for every hand based on your cards and the dealer's exposed card. Data and statistics will tell you that basic strategy is always the optimal strategy. In fact, following it perfectly reduces the casino's edge over the average player from 3 percent to 0.5 percent.

Yet many people choose to ignore basic strategy, instead relying on hunches and, in the process, falling for cognitive biases. Specifically, in a research study, UCLA Professor Bruce Carlin found that most players are far too conservative, favoring inactivity over activity especially as it pertains to "expected regret."

The classic example of this is when a player has 16 and the dealer is showing a seven. Basic strategy dictates that the player should hit that hand (take an additional card). Yet many people are afraid to do it because if they get a six, seven, eight, nine, ten, jack, queen, or king, they will lose immediately. That's an eight in thirteen chance (62 percent) of losing immediately. Yet if they do nothing, their chances of winning are even worse.

The player in this case is falling for omission bias – a cognitive bias where we favor inactivity over activity especially as it pertains to potentially doing harm. Yes, there is a natural allure to standing pat on that 16 and hoping that the dealer busts, but it is the wrong decision.

Blackjack is littered with these difficult decisions, and when money – and therefore emotion – are involved (just like as they are when starting a company), it is hard to avoid making bad decisions. One of the first lessons that I've taken from my days at the table to my work in startups is the importance of being data driven, and that starts with first collecting good data.

In this lean startup era, learning is the most important thing you can do when you launch your product, and without the proper tools in place to capture data, you cannot learn.

At my current company, tenXer, we have played around with a combination of different tools, including KissMetrics, Google Analytics, and a homegrown solution. Regardless of what you use, it is paramount that you have something in place from day one that you can rely on to give you accurate data.

Of course not all situations in business are as cut and dried analytically as blackjack. But that is why in the startup world we have to be resourceful. A couple months after starting tenXer, well before launch, we wanted to test some messaging. Specifically we were concerned that words like "productivity" and "analyze" might not be very consumer friendly.

So we took a page out of the fake landing page playbook and tested different messaging in Facebook and Google ads that pointed to our LaunchRock.  With this effort, we were able to capture real leads while capturing data that messages like "be more productive" and "analyze your work" converted, as well as more consumer-friendly terms, such as "improve" and "be better at your job."

Another challenge we face at tenXer, is prioritization of the services that we connect to.  Currently we connect to services like Gmail, Calendar, Twitter, Jira, GitHub, Phabricator, and Pivotal Tracker. But how do we decide what service to connect to next? This is a difficult decision because there are a lot of criteria involved: How good is their API? How many users do they have? How unique is their offering? How valuable is their data? etc.  In order to make a more qualitative decision a bit more quantitative, we isolate these specific criteria and rate each on a scale of 1-5. While this is still somewhat of a subjective measure it helps us avoid favoritism toward a certain service.

Finally, the most important decision a startup has to make is around prioritization of features. With limited resources it is impossible to do everything, so understanding what features your customers really want is paramount. But sometimes that can be difficult, because asking a point blank question of your users can be very leading and can yield poor data.

So to solve this issue we leaned on technology. We sent out the typical Survey Monkey to our users where we asked them to prioritize different proposed features, but we also gave them open text fields to comment on things they liked and wanted to see in tenXer. We then used a proprietary sentiment analysis tool, developed primarily for use on Twitter, to identify different themes in their responses. Some common themes emerged, and fortunately those themes mimicked what we were seeing in their more structured survey responses. This gave us the confidence to move ahead on some specific product features.

While the decision to be data driven seems obvious enough, so does the notion of hitting a 16 against a dealer's seven. Yet many times both are ignored. And when this happens the player seldom wins.