Problem
The endless scroll
Problem
The ‘endless scroll’ of not being able to decide what to watch, and the inaccuracy of Netflix’s recommendations.
Objective
Create a feature that deepens user engagement by giving Netflix more data to work with to minimize the cold-start problem and improves personalization.
Research and insights
Research shows that people tend to watch Netflix on TV
I didn’t have a clear vision at the outset of the project, and needed further insight into Netflix users’ viewing habits. To get a better idea of who and what to design for, I did secondary research, user interviews, and surveys.
According to a 2018 study by Netflix, most consumers still use their television to watch online content. To confirm my secondary research, I ran a survey of my own with 20 participants, with 48% saying they prefer to use streaming platforms on TV, 25% on desktop, 21% on mobile, and 6% on tablet.
Early insights about how Netflix personalizes content
I broke my secondary research into four parts: 1) people’s video consumption habits, 2) the video-on-demand market, 3) competitive research, and 4) Netflix’s recommendation and personalization system, with the focus on the fourth.
According to a recent Netflix research study on trends in personalization, Netflix personalizes using:
- Ordering and ranking of videos horizontally
- Selection and placement of rows
- Personalized thumbnail images
- Reaching out to members directly about new or upcoming titles
But as Justin Basilico, Research/Engineering Director at Netflix, says, personalization is hard:
Varied interests
Every person is unique, and people don’t usually stick to one type of content.
Feedback loops
There is a risk of over-personalization without novelty, freshness, exploration, and diversity.
Context dependent
Users want to see different kinds of content depending on their circumstances.
Cold start
Users don’t necessarily open Netflix knowing what they want to watch.
Primary research shows that people generally have a hard time choosing what to watch
I gathered opinions and suggestions from 20 users of Netflix and other video-on-demand platforms through a mix of informal interviews and survey responses. Participants ranged in age from 23 to 47, with an average age of 29, and all but two were based in the US.
“They have access to shows I can’t get elsewhere.”
“We use Netflix to watch ‘trendy shows’ as conversation starters with family and friends.”
To gauge whether choice was a real problem among users, I asked participants, on a scale of 1 to 10, do you struggle with picking what to watch?
Understanding common user pain points
“I’m usually in the middle of a TV show, so I don’t have trouble choosing.”
“We had so much trouble last time we were trying to figure out what to watch.”
“Yeah, but it’s different with different people, no?”
“I don’t know... I always have trouble.”
“The content choices presented to me are random and feel like they’re based on other people’s tastes... Netflix just doesn’t seem to understand my personal tastes.”
“The recommendations never work for me. Not only are they inaccurate, but when I’m just testing a show out, Netflix thinks ‘because you love this’ and shows me a bunch of content within that realm.”
Define
Defining the flow of a new recommendation tool
Choice paralysis seemed to be a problem for many people. Based on the interviews and survey responses, I drafted a user journey map for a persona who is frustrated with a Netflix experience that doesn’t feel personalized, and who chooses to engage with a recommendation feature that only appears after a certain amount of browsing time.
Using this journey map, I drafted preliminary ideas for a new feature that gave users another touchpoint for browsing. Given that most users prefer to watch Netflix on TV, I designed for tvOS:
Reframing the problem
A minor setback meant reframing the problem
Before diving into this further, I had to step back. My initial concern was that creating what is essentially a more fleshed-out onboarding process would risk stripping user agency and lead to over-personalization. This quiz-like feature lets users input preferences based on three attributes:
- Time: movie or TV show?
- Mood: lighthearted? Dramatic?
- Genres: based on mood
Having the results screen show genres related to the mood, the ability to ‘refresh’ if the results weren’t accurate enough, and recommendations similar to the main one on screen would only make the problem of choice worse.
While my initial focus was on the user and giving them ways to interact, I realized that a user who has been browsing for a while ultimately just wants to be recommended content without extra effort. tvOS also has limited navigation, which would make this layout ineffective.
Guiding question
What does Netflix need to provide users with more accurate recommendations?
This re-evaluation raised new questions: How might Netflix improve its recommendations? What data is needed to give users more accurate recommendations based on their interests? How might Netflix provide an alternate touchpoint for better exploration?
Solution
Introducing Netflix Rapidfire
Under lockdown, time can feel both precious and endless. Netflix Rapidfire gives users back the time they spend endlessly browsing by speeding up the decision. By answering a few rapid-fire questions, users help Netflix understand their preferences and recommend new content to explore.
If one of the biggest complaints about Netflix is the inaccuracy of its recommendations, when personalization is one of the pillars of the company, what might Netflix need to strengthen it? Involving user feedback gives users the sense that Netflix is clearly using their data in service of them, and adds an extra level of personalization and a new avenue for exploration.
Testing and iterations
Refining the quiz through A/B testing
Testing was essential, as designing for TV proved challenging for someone more experienced with desktop and mobile interfaces. I split-tested each aspect of the design.
The feature was meant to minimize choice by asking only two questions: 1) What do you have time for? 2) What are you in the mood for? But when I tested the first iteration with three users, too few questions proved unsatisfying.
“If this was meant to give me more tailored content, then the quiz felt too short and vague to be satisfying.”
What happens when users pick “no preference” for TV or movie, and “surprise me” for mood? Netflix needs to ask more questions to give relevant recommendations. Going back to the ‘how might we’ statements, the most valuable information for Netflix is what the user likes and dislikes. With some users ignoring the thumbs up/down rating altogether, a gamified rating feature would give Netflix instant, relevant feedback.
Adapting to familiarity rather than switching the UI
After the short quiz, users would see a single list of personalized content. I originally wanted to change the UI to emulate a TV guide, so users could browse the list much like channel surfing on live TV. I made two versions and split-tested them to see which users preferred:
Three of the four participants liked having a playlist of options to browse. One didn’t know what to do with the small arrow on the right at first, and another disliked not knowing how far right they could go or how many recommendations Netflix would show.
“I immediately knew what to do with the interface showing the thriller movies because it’s so familiar.”
The layout had to mimic what users already know:
Measuring success for the first rollout through user feedback
Rolling out a feature at Netflix’s scale is not simple. To measure the success (or failure) of this feature, I added a simple call to action to rate it at the end of the flow, after the user finishes watching. Not only would this provide feedback on Netflix Rapidfire, but it would also keep optimizing Netflix’s recommendations for the user.
Final design
Netflix Rapidfire on tvOS
Takeaways
What I learned
Go back to the key problem
A critical part of this process was stepping back and returning to the key problem: inaccurate recommendations. When Netflix can’t make browsing feel personal on its own, it may benefit from, and occasionally need, its users’ help.
Test, test, and test some more
As an avid Netflix user, it was easy to fall into the trap of designing from my own experience. One of the most important lessons from this project was to test, test, and test some more, and keep iterating on the findings.
Next steps
If I took this further, I would dig into the data for insights into Netflix’s browsing experience. Recommendation systems are complex to design for, and there’s a reason personalization still takes so much effort for a company as large as Netflix. The feature could also keep evolving, for example by saving a previous Rapidfire search or letting users change preferences in profile settings.
