Hello learners!
Welcome to the 21st lesson of the series “30 Days of PM” by Crework! As we go deeper into the solution space, we also need to understand how we can assess the solutions we are implementing. Starting with that theme, today we will talk about A/B Testing.
But before we start, make sure you are subscribed to us. In case you are not, subscribe to us right now and complete the challenge.
What is A/B Testing?
A/B testing, or ‘split-testing,’ is a method of testing variations of components on web pages, app interfaces, ads, or emails. Designers or marketers start with a goal of, say, earning more sign-ups and test two versions with similar audiences. If there’s a clear winner, they adopt that version.
What are the benefits of A/B testing?
A/B testing injects a healthy dose of reality into your products. It’s all too easy for product designers, developers, and marketers to fall in love with their own creations and A/B testing shows them what users really think.
For example, when designing a website, how can you determine the optimal location for each button? Or which graphics maximize sign-ups? Or what version of your navigation bar do users prefer? Only A/B testing can tell for certain.
Here are some benefits of A/B testing your digital product:
Optimize performance
Reveal hidden trends and user behaviors
Incorporate user feedback
Testing takes you right to the heart of users’ desires for your product. More often than not, you and your users share a goal: you want to help them solve a problem, enjoy an experience, make a purchase, or get help.
By testing and optimizing each stage of their journey, you can speed up their travels and both get more of what they want. Plus, A/B testing is non-invasive and doesn’t disrupt their experience because most of it occurs without their active knowledge.
After repeated tests, the product slowly evolves into something that’s intuitive, useful, and which people love.
When can you use A/B Tests?
Here are some common examples of A/B testing goals for several verticals:
E-commerce site: Increase purchases
Consumer app: Increase time spent in-app
Online publication: Increase sign-up conversion rates
Business app: Increase revenue-per-user
Advertising: Increase clicks and purchases
Email marketing: Increase opens and click-through rates
Now, to achieve these goals, the product or marketing team might want to test different variation of components like:
Layout
Navigation
Buttons
Text
CTA
Social proof
Pop-ups
Announcement bars
Widgets
Badges
Images
Videos
Links
These elements can be tested in terms of their size, placement, color, or even the timing and circumstances of when they appear.
How can you perform an A/B Test?
Step 1: Define your goal
Step 2: Look at your current data
Step 3: Come up with a hypothesis
Step 4: Perform A/B Test and analyze the results
Step 5: Repeat with something else
Now, let’s look at an example:
Let’s say you are working at an e-commerce company and you want to increase the clicks on the buy button. Now, as a PM, you are supposed to come up with an idea.
Step 1: Increase the clicks on the buy button on product pages
Step 2: You go through the data and find that current click rate on product page is 20% but the buy button is too small for the user to notice.
Step 3: You come up with a hypothesis “Making the CTA button more prominent will drive 20 percent more clicks”
Step 4: You increase the size of the buy button only for 20% of your users and let it be the same for 80% of the users. Then you observe the numbers over the next 1 week.
After 1 week, you found that for the 20% users, the click rate was 21%, while it was the same 20% for the users with the same button.
Although the click rate is higher, but the change is only 5%, way below our hypothesis, so we won’t implement the change.
Step 5: Implement any learnings and continue to test. Each time, return to your conversion funnel and use A/B testing to identify and remove blockages to users’ progress. In doing so, you’ll increase the utility of site or app.
Day 21 - Completed ✅
Congratulations on completing the 21st lesson of the series. 🥳
Now, you know what to do. Share your learnings with the world and be accountable.
If you have any feedback, please share it with us. We would love to have your suggestions and improve our future content for your better learning.
If you want to join a community of product aspirants where we discuss product management topics and case studies, you can join it here: Join Community!
References: