An analysis of 4,000 reviews of on-demand services identifies performance, functionality and content as the three largest pain points in VOD.

Last week we held our VUIX 2015 conference – an event dedicated to online video product development, user interfaces and user experience. Co-hosted by Brightcove, leading broadcasters, service providers, operators and technology suppliers from around the world all met at Kensington Roof Gardens in London (joined by a flamingo or two!) to share knowledge and insights.

There were some great speakers on the day from companies like Hopster, the on-demand service targeted at 2-6 year olds, ITV, which will be launching its new ITV Hub in late November, and BBC Worldwide, where Gulliver Smithers, VP of Product, shared some thoughts on how the organisation approaches the creation and build of VOD services across platforms.

I was also invited on stage to give a presentation on some analysis that I’ve recently completed on the relationship between user experiences and user expectation – or the two UXs for short.

For the benefit of those who were unable to attend (you missed out!) I’ve condensed my findings into the article below. Here were some of my key takeouts:

• Performance related factors, including buffering, lag or faulty functionality, are by far the most common complaint of on-demand players on tablet devices;
• Advertisements are not necessarily detrimental to the viewing experience – only 5% of users take issue with them;
• Less than 3% of viewers cite pricing as a negative aspect of an online video service;
• An impressive content line up, good looking UI and “cool” functionality are some of best things about VOD services;
• Netflix is by far the most talked about on-demand product in the world on Twitter as of September 2015.

#1. The Mission

My primary objective in this research was to investigate whether the user experience in VOD services is matching up with user’s expectations. Some of the questions I was trying to answer included “are viewers getting what they want out of these products?”, “what do they like and dislike?” and “what functionality is missing that can turn a good on-demand player into a great one?”

If I found that the expectation was not being met my aim was to try and figure out why and, if possible, provide some practical ways in which service providers can become better at satisfying their customer’s UX needs.

Finally I wanted to identify the VOD business which (at the moment) is the best at fulfilling its user expectations and share some insight on how it reached this point.

So how did I do all of this?

#2. The Methodology

I used VOD Professional’s Video User Interface Library to help me conduct my analysis. We first built the VUIL back in 2012 to help broadcasters, service providers and content owners quickly analyse video interfaces from all over the world, across devices, and have been updating it ever since.

At the time of writing we’re tracking over 450 products on platforms like web, mobile, smart TV and games console. In total we’ve taken over 17,500 screenshots and produced 1,000+ benchmarks scores where we’re looking for 35 functions that commonly appear in VOD services.

We’ve also collected over 1.2 million user reviews from iTunes and the Google Play store and it was a sample of these that I looked at to determine user sentiment towards on-demand products.

Over a period of six months I examined 4,200 unique reviews of tablet VOD applications to try and make sense of the relationship between the two UXs. These comments were studied painstakingly by eye; there was no screen-scraping, I didn’t generate any word clouds. I locked myself away for a few days and just got on with it.

My sample included fourteen different UK and US VOD services, which included a mix of SVOD, AVOD, TVOD business models, and covered the two largest operating systems – Android and iOS.

I broke the idea of user expectation into nine different categories, which included things like content, functionality and performance, and would award either a negative or positive point to a service each time a user referenced one of these areas as either a good or bad thing about the fourteen products.

The results from this research were fascinating.

#3.1. 70% of Users Give Negative Feedback

This was actually lower than we expected. In total around 3,400 of the comments featured negative feedback versus around 1,400 which featured positive remarks.

It’s important to note that I don’t believe this number represents the feelings of all VOD users. It’s fair to assume that people inclined to comment will be those who have a complaint to make and that the positive commenters out there simply don’t have the motivation to leave a review – they’re too busy enjoying the content!

But what this number does indicate, and this becomes even more apparent when you start reading the reviews themselves, is that many users seem to regard the comments sections within app stores as a customer helpline or feedback form. It’s clear that many users believe that by posting grievances in a public forum they can somehow instigate change within a service.

#3.2. Performance Issues are the Biggest Complaint, followed by Missing Functionality

The most common complaint, by far, was about performance-related issues in VOD services which included buffering, lag and faulty functionality. In total around 47% of all negative comments made some reference to bad video delivery.

Users expect services to just work, regardless of device or location. This expectation has been partly informed by using the internet as a whole – customers are used to the idea of quality content across verticals, such as music, gaming and the news, being instantly delivered them. If they don’t receive this from one service, they are savvy enough to find it elsewhere.

Second to performance issues, missing functionality accounted for 12% of criticisms. Some of the most commonly asked for features included:

• Picture in Picture – this was recently introduced by HBO Now as part of Apple’s latest iOS 9 update. Also very popular in sport services;
• Download to Device ­- customers want the ability to take content with them to places where there isn’t Wi-Fi or a stable mobile internet connection. Amazon Prime Instant Video, Hopster and BBC iPlayer all offer this functionality;
• Enhanced Social Features – in particular the ability to comment on videos within a service and to share what you’re watching across social networks.

If you’d like the full list of functionality I put together as asked for by users, tweet me @consultVodkr

#3.3. Advertisements are not Necessarily Detrimental to the Viewing Experience

According to my data, advertisements do not affect the user experience as much as we previously thought. Less than 10% of those using services which featured ads, either dynamically inserted or as part of a live stream, found issue with them.

In fact, a small percentage (around 0.6%) even defended the placement of commercial breaks, recognising that their presence is what keeps an ad-supported service in business.

Advertisements, and their future in VOD services, was a hot topic among many of the speakers and attendees at VUIX 2015. There seemed to be a general consensus that whilst the consumer mentality towards ads is beginning to change, assisted by the rise of ad-blockers and ‘pay-to-remove’ services like HBO NOW and YouTube Red, they aren’t going anywhere soon.

It was agreed however that service providers must continue to experiment with ways in which they are delivered to improve the user experience. For example reducing the length of commercials, producing better ad content and improving targeting technology were all cited as useful next steps.

#3.4. Customers Love Content! And Good Looking UIs, Cool Functionality and Stable Video Delivery

So what do customers like about VOD services? Good question. The quality and / or quantity of video libraries was the common commendation across the 6 month sample, with 13% of users considering it a positive aspect of a video service.

This makes sense. As more and more VOD players enter the market, programming has become the key differentiator. Service providers and broadcasters of all sizes are looking to bolster their offerings with original or exclusive video to stand out from the crowd and many are beginning to look into introducing more robust recommendation / discovery technologies with which to surface their assets.

Following content, stable video delivery (6.93%), “cool” functionality (5.10%) and good looking user interfaces (2.64%) are some of the more positive things about VOD services.

#3.5. Pricing is Not That Big an Issue for Users

The cost of accessing a service hardly registered as a problem for users. Only 95 of the 3,431 negative reviews cited it as an issue which was less than 3% of the total sample. In fact when pricing was mentioned, it seemed that the majority of commenters were not upset that it was too high but in fact that it was there at all!

#3.6. Final Results

You can see the complete results of what customers like and dislike about VOD services in the graph below. Please feel free to send through any comments, questions or thoughts to me @consultVodkr

#4. Recommendations

So what can we learn from all this information? What are some of the practical ways in which VOD service providers can improve how they satisfy their customers UX needs? I identified three different areas.

#4.1. Infrastructure

Creating and maintaining a technological infrastructure to power a VOD service is of course paramount, but I use the term in this context to describe efficient communication channels with users. One thing that became very clear to me was the amount of entitlement that viewers have when watching content online: they expect the services to just work and can get quite vicious in their feedback if met with a negative experience.

This chimes with a study released by Conviva earlier this year which found that 75% of viewers abandon poor video experiences within four minutes and are then less likely to return.

This is why it is essential that easy-to-use feedback channels are in place to give customers a voice during the development process or, more importantly, if a problem arises. Traditionally, broadcasters haven’t needed to employ expert customer service departments since if a video delivery issue occurred, it was the technology (be it the TV set, aerial or set-top box) that often got the blame.

But in a market when an increasing number of broadcasters are establishing direct relationships with their customers, these communication channels are necessary for the ongoing improvement of a service.

#4.2. Knowledge

The VOD industry is still relatively young and changing all the time – only last week, for example, both Disney and YouTube announced new D2C subscription services which will launch later this year. Because of this it’s important for service providers to educate themselves on the evolving viewing habits of their customers in a world where content can be watched on any screen, at any time.

Deploying analytics solutions, like the ones offered by comScore, TVbeat and Brightcove in its VideoCloud platform, can help to gain a deep understanding on what customers are watching and how they’re interacting with a service. This knowledge can in turn be used to increase engagement, reduce churn and identify opportunities for further growth.

#4.3. Monetisation

Experimenting with monetisation strategies – distinct from core product pricing – can also be a consideration. A number of major players in the last year have made changes or additions to how they package their service in order to stand out from the crowd.

Take Dish, a satellite pay-TV provider in the US, for example. In January it launched a direct-to-consumer proposition called Sling TV to cater for cord-cutters, or those unwilling to take out a cable subscription. I spoke with Roger Lynch, CEO of Sling TV, at IBC last month and he told me that the decision to start a standalone subscription service was in direct response to the current pay-TV model in the US “being unsustainable”.

It’s clearly working – since launching ten months ago, Sling TV has netted around 250,000 paying subscribers. As the VOD market becomes progressively more competitive, service providers can potentially offer a better user experience by giving viewers different options to pay for their content.

#5. Netflix is King … Still

Finally I also conducted some research into the service which is best at fulfilling its customers’ expectations – Netflix.

The streaming service continues to dominate the SVOD market in the territories it has launched in. Whilst domestically growth slowed last quarter, its global expansion continues unabated, with 2.74 million international subscriptions made in Q3 against a forecast of 2.4 million. Amphere Analysis expects that Netflix will have 130 million subscribers within the next five years.

How is it producing such growth? Of course its impressive content library, which is increasingly becoming more focused on originals, and a robust product line up across devices both have something to do with it. Its subscription payment model is also a factor, as it gives users the ability to avoid advertisements and pay one simple monthly fee for their entertainment needs.

But I think it also has a lot to do with Netflix’s brand.

Here at VOD Pro we track how on-demand services are being talked about on Twitter in real-time. Our queries run 24 hours a day, 7 days a week and have been collecting data since May 2014. In total we’ve captured and stored around 150 million tweets.

Last month we were looking for 38 different terms, things like ‘BBC iPlayer’ or ‘NBC’, which in total generated over 9.7 million tweets. Here’s a graph of what percentage some of the terms made up of the total amount collected:

As you can see there is quite a percentage difference between the UK and US services, but this makes sense – Twitter’s user base stateside is larger than over here in Britain. But how many of the tweets did Netflix account for?

54%. Or, to be specific, 5,271,665 unique tweets. Check out the graph below.

Over half of the 10 million tweets we captured from around the world in September related to Netflix. And this is actually an increase of 30% from 7 months ago.

What’s even more interesting is when we begin digging into the sentiment behind these tweets. Services like BBC iPlayer are spoken about primarily in functional terms – such as discussing when a particular show will become available – but conversations surrounding Netflix are massively more emotional. Viewers are saying things like “I just want my bed and Netflix!” or “Our kids are going to get the ‘Netflix and chill’ talk instead of the ‘birds and the bees’“. This use of this type of language makes Netflix seem almost like a friend, a buddy, rather than an online video service.

The reason for this emotional connection can in part be explained by the cool persona that Netflix has built for itself. By offering binge-worthy content that everyone is talking about, shows like Orange is the New Black or Daredevil, the company is framing itself as a proposition that customers cannot culturally afford to miss out on.

Conclusion

So does the user experience in VOD services match the user expectation? I came away from my research thinking that yes it does, just about, but there’s still a lot of work that can be done.

By looking at my data you could argue that, for mobile at least, VOD services are lacking when it comes to cool functionality or strong video delivery performance. But you can’t ignore that people are still using them. And they’re using them a lot!

The uptake of VOD is growing every year – forecasts from Mintel suggest that UK streaming subscription revenues will rocket from £437 million to £1.17 billion between 2014 and 2019 – and it shows no signs of slowing down. The problem that service providers now face comes in the form of increased competition. Major media and technology companies are investing heavily on building online video presences whilst a large number of smaller services targeting particular, niche audiences are also entering the competition for eyeballs on screens.

We know from conversations with broadcasters that the questions they are continually asking themselves are “how do I stand out from the crowd?”, “what can I do that’s different?” and “how do I innovate?”

These are all great questions which hopefully this article has gone some way to answering. But here’s one thing we know for sure – viewers remain in the driving seat, and are redefining the rules for how, when and what they watch.

What do you think? Share your thoughts with us @consultVodkr