How to make a great product better, from the Lingumi product playbook
Just recently, a colleague came to me with a dropping conversion rate graph, a gut feeling, and a ready to build idea of a solution that included some costly changes to the product…It’s an easy enough mistake to make, and one that you may well have faced.
Improving great products is hard, and ironing out problems like a dropping conversion rate needs quick learning and a continued obsession with the real problems that users experience. Sometimes that means we need to slow down to understand the real problem before jumping into expensive changes.
I want to share how great teams can improve their products by solving problems for customers better, and only build changes that matter by continually focusing on rapid learning and iteration of solutions. I hope that this is useful for anyone managing, building, or working in product teams.
At Lingumi, we’re building a platform to deliver the world’s best teaching experience to pre-school children, all around the world. To do this, we’ve set up our teams to own customer problems that align with our product vision, and we give them the autonomy to solve them in the best ways possible.
However, that’s only one part of the story. We also encourage our people to spot when something isn’t working well so that they can transform potential issues into valuable opportunities. We continually make observations of how parents and children use Lingumi which means that we can then explore and deliver improved outcomes very quickly and cheaply.
Below I’ve outlined one of the ways that we do this. The process has many parts, but in this piece, I’ll cover how we:
- Set the team up for success
- Identify the opportunity
- Slow down to speed up
- Talk to real users as a team to find value
- Uncover and organise the insights together
- Create solutions to the biggest ‘How Might We’
- Start with one solution and test quickly!
- Evaluate the results, iterate, and go again
- Ship it, observe, and iterate again to create even more value
Set the team up for success
Making this kind of process a repeatable success starts with encouraging a “beginner’s mind,” a concept from Zen Buddhism known as shoshin. When starting with a problem to solve, we let go of any assumptions and biases that may exist in our team and allow ourselves to be open to learning new ideas.
“In the beginner’s mind there are many possibilities, in the expert’s mind there are few.” Shunryu Suzuki
We combine this with a promise to ourselves to have “no fear;” no fear in what we might discover, no fear in trying something new, no fear in getting it wrong. At Lingumi, we have embraced the “2-way door” philosophy of decision-making, something that Jeff Bezos describes as a changeable or reversible decision. This approach truly empowers our teams to be brave, knowing that if things go wrong there’s a way to reverse the outcome.
And finally, when considering potential solutions, we think big! Starting big allows us to narrow down if we need to and opens us up to considering ideas that we might not typically explore.
Identify the opportunity
There are several ways we might realise that something is creating unexpected results. Often, these initial observations are from data, where we start to see a spike of unusual activity or feedback from our continuous interviewing of users, or our Customer Success team.
Regardless of which source is the canary in the goldmine, it is prudent to cross-check the other to see if there is any correlation between them. Doing this helps give body to the problem, and allows the team to start to understand the context of what is happening.
The mistake my colleague made was to jump in and define the problem simply as “The conversion rate between Lesson 1 and Lesson 2 in China drops too quickly.” While this is true, and it is what we are trying to solve, it misses out a huge amount of context as it’s simply stating an observation rather than understanding the real issue at hand.
A better starting point in this instance would be to consider the problem from our users’ perspective, and we get that by talking to parents.
Slow down to speed up
Our data and feedback might be directing us to a particular screen, action, or flow in the product, but the actual source of our problem may be much further upstream.
In our example, it’s apparent that something is going wrong after the first lesson, and it is here we would start our investigation. However, with our beginner’s mind, we won’t assume that is where our users encounter the problem.
Taking a step back at this point allows us to question our users’ context and understanding in more detail. And the only reliable way we can gain this understanding is by talking to a number of our users.
We will want to understand what they are experiencing and how that relates to the mental model they would have built around Lingumi.
Explore the user’s mental model
A mental model is an explanation or understanding that you have in your mind to help you interpret the world. There are many forces at play when someone signs up for Lingumi that shapes their expectations of what they will find and how they will act.
For instance, we create ‘like live’ teacher-led classes that help children learn asynchronously, as we know this delivers better learning outcomes at a more affordable price. However, this approach is different from other forms of pre-school education, and a parent might expect either a live teacher or just a simple set of games their children can play. Alternatively, our own messaging in social channels or paid advertising might not be setting clear expectations or driving home the advantages of ‘like live’.
Put together, these differences create a mismatch between what our users expect and what they experience, and that might mean they, in our example, decide to not continue after experiencing the first lesson.
Do they see the value?
Combined with this, users must see and feel real value from the product.
“… what’s really going on is not that the user struggles to get through a workflow, or gets frustrated by a bug or the performance of an operation. What’s really going on is that the user doesn’t see why they should use the app in the first place. They don’t see the value.” Marty Cagan, SVPG.
Whatever the promise was, whatever they were expecting to happen, we haven’t made that core value super clear to them, and so they leave.
Talk to real users as a team to find value
When we talk to our users, we involve the whole squad. It allows the team to level-up their understanding together and encourages each other to capture, discuss, and challenge observations.
Deciding how to talk to people is dependent on what is being investigated by the squad. Typically when we are searching to understand a problem seen in data or feedback, we default to either face-to-face or phone interviews, or ethnographic study (if possible).
How we find people to talk to and how we conduct user interviews across the world, in different languages, and with children(!) is for another post.
As a team, we watch the conversation as one of us conducts the interview. We gather our post-its or add to our Miro board. One observation or quote or question per note, written to be easily read. This part of the process is one place where the beginner’s mind is invaluable; the team listens as if it is their first time experiencing this problem, and are unafraid to write down what might be even the most innocuous observation.
Uncover and organise the insights together
Once the interviews are complete the squad quickly, and without discussion, collect similar notes together. These notes are organised under a ‘What we heard’ column on the board.
From this loose collection, we pull out any themes we see and arrange them in the ‘What it means’ column. It is at this point we will have a clearer understanding of the real problem we are trying to solve.
Once we have the themes grouped they are labelled with a question that encapsulates the observations and insights of the theme so that they can easily be understood or prioritised. These questions act as provocations, and we use the ‘How Might We?’ (HMW) framing from design thinking to help us imagine an optimistic and possible solution from the problems we’ve seen our users experience.
The last step in this process is to decide which How Might We? questions we want to tackle. This prioritisation is either conducted by silent dot voting, or by the Product Manager who often takes the role of Decider in the squads. We move the chosen questions to the ‘What next?’ column and pick the one or two that we want to try and answer next.
We use a few different frameworks to help organise our thinking around solutions, and the above is one effective way of moving forward. Each squad also keeps an up-to-date Opportunity Solution Tree that allows us to explore new ideas for the same problems, and that can also be a valuable source of inspiration when needed.
Sidenote: It’s never* about the price
Remember ‘value’ and how users make decisions to use a product based on whether or not they perceive real value from it? They may say in interviews that something was too expensive for them, but what that means is that they have not received value from the product and the team needs to understand why to be successful.
It’s easy to test this as well by substituting an incredibly low price in a prototype and encourage users to ‘spend’ their own money. If they still wouldn’t buy our product or service when priced in pennies, then we quickly understand something else is at play.
Alternatively, we know our users understand the value they receive if (in prototypes) we can raise the price 10x and they are still interested! That is one signal that shows we have captured the essence of a great solution to their problems.
*OK, sometimes it might be.
Create solutions to the biggest ‘How Might We’
Time to break out and get drawing. Those familiar with the book Sprint! will already have some great exercises to use to help be creative with solution thinking.
Before we start that process, we try to quickly fill any gaps in the team’s knowledge by talking to experts or doing more desk research, but more often than not we can start thinking of how we might answer the problem at hand.
Whatever way we get to an idea, the focus of that idea must be the problem we are trying to solve. Every team member must remember this, as this is how the team explores a similar space and makes it possible to compare and consider each idea in the same way. This focus on the problem space also means we will have more impact.
Start with one solution and test quickly!
This process may appear long and slow, but that is not the case. It surprises me how quickly we can make an observation and be ready with several solutions to explore — it’s usually within a day.
Once there are some solutions to consider, do not waste time debating which test to explore first. We quickly pick one and set about exploring how we might bring the idea to life in a way that is as lean as possible. Sometimes that is a quick sketch where we run through the flow several times with team members so that we can iterate fast. Other times it’s a paper or lo-fi design that we can take to users to gather quick feedback and validation.
Sometimes it is necessary to learn from or prove an idea that relies on more abstract or technically advanced concepts. The key here is to consider how to create a situation that feels real enough to a user to get valuable feedback. That may mean writing a narrative that drops them into the middle of the problem or context. It may be that faking interactions with hidden team members elsewhere standing in for a complex AI.
Whichever way we test ideas, the essential point is to validate the core solution. We remove everything else that hinders this; we have even removed everything except the one thing we are testing so that we can be confident in what we observe.
When we talk about speed in this part of the process, we are talking about ‘speed to learning’. How fast can the team learn whether or not they are focusing in the right place? How quickly can the team iterate and maximise the value of the idea? Reducing the time to learning is an essential part of improving a great product, and it also sets the rhythm for the team to help bring the product vision to life sooner.
Evaluate the results, iterate, and go again
Testing for value does not need design. It does not need to be pretty because we want to know if someone understands the core idea, we want to know if our users need it, and we want to check that it solves their main problem.
We spend time understanding what we see when users use our prototypes. Small iterations are nearly always necessary, and sometimes complete overhauls are required. But testing early, often, and fast allows us to drop an idea if we can’t prove the value of it, and it lets us do this with very little time or money cost.
This step is crucial in making sure we spend our time and effort on ideas that make the product better by solving real problems for our users. By slowing down here, we speed up our progress, and our users benefit more from it.
Ship it, observe, and iterate again to create even more value
When we see positive results from user research, once we know our idea has been iterated upon and proven to a degree, once we are confident enough, then we will ship it to the product and observe.
At Lingumi, we encourage our teams to be bold and make decisions without fear, knowing that we can always roll-back a change with ease.
But a change is only the start, and teams can improve upon most ideas once a larger population begins to use them. So we continue to observe, refine, and question, and over time improve what we create to help make our great product even more fantastic.
Wrap up
There is no ‘one’ way of doing this, and what I have explained here is only a part of our playbook for improving the Lingumi product experience.
However, we start with the belief that everything we do is a prototype, and that everything we do should be focused on our users’ problems and validated through that lens. Every action we take should move us closer to having a positive impact on childrens’ pre-school education.
