• July 20, 2016

Analytics Gone Wrong: The Consequences of Bad Metrics

Make sure your metrics are driving insights instead of costly diversions.  

Big data and analytics strategies can drive game-changing success. But most big data and analytics projects fail. Why? Some analysts say enterprises fail to ask the right questions—or any questions at all. But successful analytics results are all but impossible to achieve if you can’t identify the questions you’re expecting the data to answer.

Another driver of analytics underachievement: far too many enterprises measure the wrong things. It’s true that the process of measuring itself can lead to positive results. Track and plot something, and you’ll notice its ups and downs and learn what drives the ebbs and flows. But measuring the wrong things can lead to costly diversions. For instance, we might measure the number of hits a website racks up instead of how many conversions to customers it makes.

In the Brookings Institute white paper, “Solving Journalism’s Hidden Problem: Terrible Analytics,” author Tom Rosenstiel makes exactly this point. He argues that web analytics processes used in publishing are a mess. They not only offer too little useful information, but they also mostly measure the wrong things, delivering data that is both false and illusory.

Down the Traffic Rabbit Hole

Rosenstiel cites the Holy Grail of digital journalism: unique visitors. But Rosenstiel says this is a useless metric. It’s deceiving because the number does not represent individual people. Instead, it represents individual devices. The same person accessing the same site from a phone, tablet, and laptop registers as three unique visitors. If that person cleans out their cookies, the count starts all over again. As a result, publishers and newsrooms have been making editorial decisions based on faulty measurements.

To tackle the problem, Rosenstiel and the American Press Institute, where he is executive director, launched a project to develop a more useful analytics system. They created a set of tags denoting specific journalistic characteristics, such as what a story is about, story type, relevance, inclusion of audio and video components, and the style of writing. In addition, temporary tags were created so that editors could test any hypotheses they might devise.

The findings were surprising. Contrary to the prevailing wisdom of shrinking attention spans, participants discovered that their readers like to read long stories, even on their mobile phones. Major enterprise stories, or journalism based on original in-depth reporting, also scored high among readers.

Course Correction

To drive unique visitor numbers, websites often use “click bait,” or provocative headlines, feel-good lists, and sexy photos to attract users. But these tactics may be hazardous to digital media health. Traffic magnets such as Mashable recently suffered layoffs, while Buzzfeed missed its earnings marks, and Vice Media suffered a sharp drop in traffic.

In this era of data abundance, it’s easy to lose sight of what really matters, whether your enterprise is in the publishing industry or any other. To keep your data projects on track, start with the right questions: What are you tracking and why? What is your data telling you? What are you missing? Starting with the right questions can keep you from measuring the wrong things and making costly diversions based on faulty data.

Like this story? Learn more about data as a strategic asset. And discover how HPE can help manage your data and analytics.