Why context in Analytics is so important

I'm not fan of certain dashboards and some data visualisation techniques that focus in one metric (in principle there's nothing wrong in it) but fail to give the context data that allows to explain what it means.

Classic every day example is having a dashboard with Conversion Rate performance without showing context data that explains its behaviour, like traffic source or device consumption.

Or showing Conversion Rate without total sales.

During the sad events this week in Paris I've seen another example that you might find simple, but it's good to call out.

This tweet was on my timeline:

If you had seen this picture (let me highlight: only the picture) without any context data could easily get to some conclusions just too quickly:

  • "US doesn't really care much about this hashtag"
  • "Asia, South Africa and Africa are not concerned at all for this hashtag"

Is this analysis correct? Not necessarily.

To make such assumptions we need to dig deeper and keep in mind some key variables that would affect the metric "hashtag use" visualised in a map.

These 2 variables are:

  1. Hour of day & Timezone
  2. Twitter penetration rate by country

Of course there are more variables, but for our analysis we need to consider these 2 at least.

About the first one, it's pretty straightforward: We need to know what time this picture was taken to understand the timezone.

A later picture of the same Hashtag shows a different story contradicting our initial analysis: 

Actually Twitter's visualisation is a motion chart that allows us to see the evolution of the hashtag, that solves for this important variable.

On the second variable to use for our analysis we can choose whatever we want: % of population with twitter account, active users by country... you get the idea. Maybe a country like China is not showing up in the map not because they don't care, just because they don't user twitter at all. Using some quickly searched data from peerreach.com we can see that US has a similar penetration rate than UK (approx. 12%) for example, and Saudi Arabia has 3X the penetration than US. 

To walk the extra mile you could go and find out population by country to figure out some total numbers. This data takes 10 seconds to get from Wikipedia:

This way we can quickly figure out that twitter has around 32 Million users in the States and roughly 8 million in UK (population is 62 Million). We don't need accurate data, just some context to shed light on the question "how many users twitter has in a given country?".

With this context, which takes few minutes to find, it's still difficult to make an accurate analysis, but it's easier to make an educated guess. Now we understand better the support to the hashtag across the planet.

Keep in mind context data in your next dashboard.

Jan 10th 2015 - Update

In this article from the New York Times I  found today an awesome example of using context data to explain a KPI. See how they do it.

Three-quarters of workers report replying to email within an hour or less of receiving it, according to a recent survey of 503 employees at workplaces in the United States.
— KOSTADIN KUSHLEV and ELIZABETH W. DUNN (NYTimes.com)

The main metric: "Three quarters of workers reply to email within an hour" is contextualised by:

  • The source of this data point (survey)
  • The sample size of this survey (503) 
  • The geographical location of the survey sample (United States)

That's exactly what we mean by using context in our analysis. Again journalist prove themselves to be great communicators of data.