Nov 23, 2014

Drawbacks of Using Time Metrics to Measure Blogs

When it comes to blogging, we all know that CONTENT is king. We also understand that social interactions and readers engagement play a primary role for making the blog successful.

So far, so good.

But then it's time to analyse data and make decisions. And that's where we often fail.

We usually take a web analytics tool like Google Analytics, install basic tracking code on pages, and analyze the blog like any other website. We look at most common metrics and take them as standard references to evaluate future performance. But we forget about the unique features that differentiate blogs from other digital properties: content consumption and social interactions.

This post will help you understand one of the most misused metrics to measure blogs performance: I am talking about time on page and time on site. Most bloggers don't understand what time metrics actually measure. So, first of all I will try to explain how they are calculated in a typical web analytics tool (it might be different from what you think!).

I will then discuss some of the drawbacks of using time metrics to measure blog performance and finally suggest a couple of more solid KPI's to better measure content engagement.

After reading this post, I am sure you will start looking at time metrics with a bit more critical thinking than before. And perhaps shift your blog analytics focus to other more powerful metrics.

Let's go!

How time metrics are calculated?

Let's clarify here how Google Analytics calculate time on page and session duration:
The time on page for a particular page is calculated by subtracting the initial view time for a particular page, from the initial view time for the subsequent page.
From the above definition we get that, to calculate time on a specific page, it's necessary to have a subsequent page too.

But not always actually.

To get a more accurate measure, Google Analytics also uses the "engagement hits" concept.
An engagement hit is a hit that results from an event that does not have the opt_noninteraction parameter applied. 

Let's provide a 3 simple scenarios to understand calculation.

1st Scenario: A Visit with More than One Pageview (and no engagement hits on the last page)

In the case of a multiple pageviews visit without any engagement hits on the last page, then the session duration metric is calculated as follows:

Session duration = Time of the first hit on the last page - time of the first hit on the first page

Here is a visual example of a 3 pageviews visit:

Time Calculation Google Analytics

So, when there is more than one pageview, Google Analytics measure the time between the first hit on the last page (11.10) and the first hit on the first page (11:00) to calculate session duration.

To calculate time on page it's the same process: time of the first hit on the subsecuent page minus time of the first hit on the previous page.

But what about Page 3 which is not followed by any other page (visitor exits the site on page 3)? Here is where we have trouble measuring time. Since there is no pageview after Page 3 (and no engagement hits), Google Analytics can't figure out the time spent on page. The user actually stayed a total of 15 min on Page 3, but Google Analytics can't calculate it. So, time on Page 3 is unknown and will be reported as 0 in Google Analytics reports.

Does it make sense so far? Let's move to second scenario.

2nd Scenario: A Visit with Only One Pageview (and no engagement hits)

If there is only one page viewed in a visit and no other engagement hit on the same page, then session duration is equal to zero, so time on page. Google Analytics does not find a subsequent page, so it can't figure out how much time the user actually spent on the page. Content reports (for that specific visit) will say "00:00:00".

THIS IS A COMMON SCENARIO FOR BLOGS, where often users land there just to check a speficic post (maybe the latest one published), and leave the blog without visiting other pages.

More on this later.

3rd Scenario: A Visit with Engagement Hits

In our last scenario, we assume that there is an engagement hit on the last page visited. Then the session duration metric is calculated as follows:

Session duration = Time of the last engagement hit on the last page - time first hit on the first page

Let's rearrange our previous 3 pageviews example with an engagement hit on Page 3:

Time Calculation with Engagement Hit Google Analytics

In this scenario, we can get a more accurate measure of Time on Page 3 as well as of Session Duration, since there is an engagement hit happening at 11.20 AM. Let say for example you have configured an event (with no opt_noninteraction parameter applied) firing when the user scrolls 100% of the page. Google Analytics can take this piece of data and use it to better calculate time metrics.

Got the the logic behind calculation? I hope this makes time metrics interpretation a bit more clearer.

Drawbacks of using time metrics

When comes to measure blogs, it's tempting to use time metrics as main KPI's or even set them as Goals in Google Analytics (e.g. count a Goal when user session reach 120 sec).

But now that we have clarified how time metrics are calculated, you should have no problem understanding their impact on blog analytics. Especially if you decide to configure them as Goals in Google Analytics.

So, what's wrong with using time metrics to measure your blog performance? Below I list which are in my opinion the 4 main drawbacks.

1. The Way they are Calculated (subsequent page rule)

What about those users who just visit one page and leave the site? This is quite a common behaviour for blog readers. They land to your blog to read your latest post (perhaps from an email subscription) and exit from the same page without interacting with any element.

No subsequent page and no engagement hits. One pageview, and time on page unknown (actually 0:00:00). Then what? Did they really spend no time on page or they actually read the post?

Google Analytics can't calculate time on page for a single pageview visit. Neither session duration. And this would also skew average time measures calculated on multiple sessions: my next drawback in this list.

2. They Average Potentially Very Different Values

Google Analytics offer average measures for various metrics, included time. While average values can be useful to understand trends in blog traffic, you should also be aware that any outlying value (good or bad) can badly skew your average time metrics.

I am particularly referring to relying on Avg. Session Duration metric as main KPI to measure your blog performance. Setting Avg. Session Duration as a Goal in Google Analytics, and analyse it at an aggregate level (no segmentation), could lead you to wrong decisions.

Average metrics can be dramatically skewed. However, they could be a good starting point for your analysis, if you find some way to "clean" them first. A good idea could be applying segments to the data in order to eliminate outliers and get more accurate data. For example analysing Avg. Session Duration for each individual traffic source.

3. They should measure engagement but we dont really know if it is good or bad engagement 

We generally consider more time spent on the site/blog as a sign of success. More time = higher visitor engagement.

But time metrics do not actually say anything about engagement!

The user spent more that 10 min on my blog and visited 5 pages. Is it good?? Maybe. Or maybe not. What if the user landed to my site from an organic search and looked all over the blog to find something without success. He wasted 10 min of his time navigating through my site and eventually left it disappointed. Wouldn't be better to call it bad engagement?

With this doubt in mind, I would not rely exclusively on time metrics to measure engagement. So, be cautious of setting time metrics as goals in Google Analytics and base your analysis only on that.

Nevertheless, there are cases where you can plan in advance what is good or bad engagement for you. It depends on your site business model. For support sites for example, you want visitors to find information quickly, then a short visit can be considered successful. On the other hand, if you have an advertising business and sell ad space on your site, you might want visitors to see as many pages as possible since you're paid by impressions.

4. They can be Biased by your Web Analytics Tool Implementation

As I mentioned on the previous section. Google Analytics uses also the concept of "engagement hits" to calculate time. This is certainly a great feature that can make time measurement more accurate in many cases, but it can also biase your data if wrongly applied. 

Depending on whether a particular user interaction (e.g. click to open a submenu) is tracked as an event (and opt_noninteraction applied) or not, time metrics can be dramaticaly biased. User behaviour is actually the same, but data can say two different stories.

Bottom line: make sure you know your implementation and think of potential consequences on metrics before configuring your web analytics tool.

More solid metrics to measure your blog performance

I am not going in detail on this section, since you can find a detailed list of KPI to use for measuring blogs, in my previous post.

To mention a few, you can measure your blog performance with more solid metrics such as:

  • # of subscriptions to your blog
  • # of social actions divided by # of posts published
  • # of comments divided by # of posts published
  • # of downloads (e.g. a post available to download in pdf format)
  • % of scrolling on posts
  • Depth of pages
  • Length of visit
  • Video play (completion rate)

The most important thing is to choose metrics that have a direct connection with your ultimate blog/website objectives. To make sure your metrics are solid enough, try to answer these questions:

> Do the metrics you chose, support your goals?

> Can you take actions on your blog using those metrics?

If your answer is YES, your metrics are solid enough to build a measurement plan from it. The metrics I mentioned in the list above, are good examples of great KPIs for blogs, because they focus on the essential aspects of blogging: content engagement and social interaction. 

In conclusion, you need to be careful and thorough when utilizing time metrics to make decisions. You must be aware of how they are calculated, and which are their limitations when it comes to understand user behaviour. Having clear in mind their limitations, time metrics can still be a good starting point to perform a web analysis. And perhaps, with some advanced segmentation you can get interesting insights.

What's your experience with time metrics? Were you aware of the logic behind their calculation? Do you agree with the drawbacks I discussed on this post?

Feel free to comment, criticize, share it!


No comments:

Post a Comment

Note: Only a member of this blog may post a comment.