We've talked about using analytics to amplify and to scale an existing product market fit. Today, we have with us Eric Qi Dong, who was recently with about.me, Eric ran the product management and the analytics program there. And Eric is a Darden alumnus, I will mention. Eric, thanks for joining us. Thank you for having me. You have a rich background in this area. Can you tell us a little bit about what it's like to approach problems that you think are amenable to resolution with analytics? Yeah, definitely. I think when it comes to solving the data analytics problem, you need to take some step by step approach. First, you need to understand what is the question. Oftentimes, asking questions like, what is purpose of asking for the data? In what format do you want that data? And it can help clarify the question. And secondly, you can look at the question to see if you have the data to solve that problem. What do you do if you don't have the data to solve the problem? Well, I think that you need to think about where to get the data. It could be from building a third-party integration with external product or working with engineering in terms of building a new solution, where when we put your project management hats on, you need to understand how do you prioritize this task with this project against as you're running, is it more important? Can I get the resource to do that? Yep, that's great. And do you have any examples that you think fit some of those patterns that you could tell us about? Yeah, absolutely. So, I can talk about my experiences when I was at about.me. Great. At about.me, we helped people create personal pages for different purposes. So, one of the biggest projects I've running as a data analytics need was to help increase the user activation conversion rates. Now, because that was the goal of the project, so everything was around that goal. So including engineering, including user experience redesign, including product marketing, but all of the data sides, what I really need to ask myself is what kind of data questions people are asking? What kind of data do we need? So, it really comes down to two sides. One is understanding the existing user base, understanding the existing user data in order to make your recommendations in terms of user experience redesign. The other part is to monitor the web analytics funnel, and understand where can we do a better job with the A/B testing in order to reach that goal. So along with that two directions, what I was really focusing on was working with designers, making recommendations of existing user data, as well as designing A/B testings, monitoring the key metrics and making recommendations on how to increase that by running those testings. And how do you keep all that organized? Did you have a particular kind of cadence or set of cycles who would use to do this? Yeah, absolutely. So, we're focusing on starting from end to end, starting from having building this hypothesis, using some user survey to validate the hypothesis, and then comes down to wireframing, and after we have a good wireframe, we build a working prototype, where we can do internal testing as well as user testing. That's where my recommendations of existing user data really comes into play when I can recommend the user experience design with existing data. And then it comes down to before we decided to reset 100 percent to production, we really want to know how the new feature, how the new process may play. So, we run it in the A/B testing matter, meaning that we open like a small percentage of the production traffic to the new feature, which is new onboarding process and compare the key metrics with the existing one if that really helps increase. And then, we did some A/B testing with that. So before we decided to push that 100 percent to brush, we already have the confidence, where the data increase could be. So that really helps us to have the confidence to push that 100 percent production and to celebrate with a lot more customer communication and even PR releases. And how did it work out? What kind of outcomes did you get? Yeah, so we're very excited. After we pushed production, letting it run for more than a week, we're happy to see that the goal of the project, the increase of user activation conversion jumped by more than 130 percent. Hundred and thirty percent? Hundred and thirty percent, more than that. So that was a big win to team and it's a collaborative effort of not only data analytics, but also a redesign of the user experience, engineering, product marketing,.all of teams were involved in that and happily celebrating that moment. Eric, one thing we like to do is talk about three tips. What are your three tips for product managers that want to use these analytics-driven techniques to make their products better? Absolutely. So, when I was at about.me, my job was really focusing on using data to grow those key metrics. So, my recommendations are going to focus on that side. So, I think number one, you need to understand growing the data, growing the metrics, it's not just one person's effort. Sometimes, you need to involve different teams, different cross-functional teams in order to not only to be on the same page but also using different efforts to go over the data. Number two, you can't underestimate small wins. When you're on A/B testing, you may only have 5 percent increase, but all those small wins will finally accumulate to a big way. I think number three is that you can't grow your product based on a bad product market fit or a bad product. So examples being, you can't really have a 100 percent increase of your product just by running A/B testing, a lot of times, it has to be involved with redesigning the user experience, focusing more on the product market fit, doing a better job of product marketing, those efforts. So, these are my three recommendations. Great practical advice. Thanks, Eric, for joining us. Thank you.
