Friday, 27 September 2013

Learning about Netflix through their UI

Netflix famously invest huge money and effort in their recommendation algorithm, but they say less about their efforts on the overall user experience.  Here we take a look and see what it tells us.

The prime position in the desktop browser UI is given over to "Recently Watched", showing the last thing you watched. This works great for TV series or films you are part way through as you can quickly get back to playing the next episode. For Android tablet this has been renamed "Continue Watching" and shows two entries. In the Smart TV version the list is much longer and includes movies I've watched to the end, which seems a waste of space as I'm very unlikely to watch them again anytime soon. The prime position for "Recently Watched" and the small size of the list on Android and desktop tells us that most people on Netflix watch just one or two series at a time.

One entry for "Recently Watched" implies people tend to watch one series at a time. 

Next to "Recently watched" is "My List".  This is the almost universal "watchlist" feature - a list of shows and movies you want to watch that you've picked yourself - , but is a surprisingly recent addition to Netflix.  Items aren't automatically removed once you've watched them, which is a shame.

Then we reach "Popular on Netflix". I find this very interesting as it says that overall popularity is more important than algorithmic recommendation or social recommendation. The list scrolls horizontally, but on the desktop the scrolling method is clunky and slow to operate. It looks like there's some attempt to exclude items from here that you already watch - for example my "Popular on Netflix" list does not include "Breaking Bad", I presume because it's in my "Recently watched".

The top position on "Popular on Netflix" is "Real Steel", which was recently the subject of a (rare) Netflix marketing email.  While "Real Steel" did reasonably well at the box office, I'd be surprised if it would gain that level of popularity by itself.

Next up is "Top Picks for you".  If you just read the tech press this is what you'd think Netflix is all about - algorithmic recommendations.  But there's something odd going on.  First thing I land on in my top picks is "Goemon", which Netflix estimates  I would give 2.5 stars.  In Netflix's rating scheme 3 stars means I would like it. So Netflix's top picks for me includes a movie it doesn't think I'll like.  And it isn't because there aren't alternatives that it has more confidence that I'd like: the next section is "Random Picks", but it includes several items with higher estimated scores.  I can only presume that alongside the estimated rating, Netflix also calculates some kind of estimate of how likely you are to play a show.

Netflix is happy to recommend me a film it doesn't think I'll like.

As I said, "Random Picks" is up next.  Before the social recommendations in "Friends' Favourites".  So despite the overwhelming excitement in Tech-land about social recommendation Netflix thinks social recommendation is less useful than random choice. This matches what we at Tank Top TV have found when we've surveyed users: most of us don't trust our friend's recommendations!

Next we have a few rows based on my "taste preferences" - for me crime dramas and exciting foreign films.  Then a few rows of "Because you watched X".  These are lists of programmes that are popular amongst people who watched that programme.  For me, one of them is for "Breaking Bad", but everyone has watched Breaking Bad so the selection is somewhat eclectic.  Universally popular choices are not useful when making personalised recommendations.

Breaking Bad fans like all kinds of weird sh*t.

Below these are "Watch it Again", "New Releases" & "Recently Added", followed by a bunch of genre-based rows.  Often Netflix's rivals give new releases prominence, but on Netflix they're pretty much lost. I presume because the new things on Netflix aren't usually that new.  The extremely cheap price you pay for Netflix means (in the UK at least), that movies and shows that are new to Netflix are often (with a few notable exceptions) at least a year or two old.

Just to be clear, we have absolutely no relationship of any kind with Netflix. In fact we don't think Reed Hastings likes us.