\r\n
This post will cover at a high level how SoloLearn uses data to make decisions. We can imagine the content of this article would be both familiar and helpful for those working on products such as ours, or who struggle with the same sorts of problems -- namely, building a product from scratch, iterating on it, or changing it in ways that make the lives of its users easier by helping them discover and learn what they are looking for and making their experience the most engaging and immersive that it can be.
\r\nBy: Gagik Chakhoyan, Data Science Team Lead
\r\nIn this series of blog posts we will cover how SoloLearn uses data to make business decisions.
\r\nWe can imagine the content of this article would be both familiar and helpful for those working on products such as ours, or who struggle with the same sorts of problems -- namely, building a product from scratch, iterating on it, or changing it in ways that make the lives of its users easier by helping them discover and learn what they are looking for and making their experience the most engaging and immersive that it can be.
\r\nIn this first post, we will begin with a high-level introduction to how we approach data and subsequent posts will cover more specific topics and include technical details.
\r\nSoloLearn is the world’s largest mobile coding community. It is not only where millions of learners come for programming lessons, but also where these learners make friends, communicate with peers, challenge each other and support each other. In this way, SoloLearn is both an edtech as well as a social application. This duality is great for the learner, but often a challenge for the data team.
\r\nTypically, the data and product teams in a SAAS business are focused on finding and perfecting the experience of the “core action” for the user -- in other words, the action that correlates best with retention and ongoing use of the product. The converse is that if the user does not perform this action, they are likely to leave the application. In products designed for single-use cases, which is common in the edtech industry, the core action is obvious -- complete the lesson. If we start to add use cases, as SoloLearn does, core action identification becomes more challenging because of the sheer volume of data.
\r\nFor example, Sololearn provides a Code Playground where learners can test their knowledge at any time, even after only a couple of lessons in a course. Additionally, learners can become friends with peers, and use SoloLearn’s social feed to post activities or other interesting events. On SoloLearn, the user learns to code, writes code, and is engaged in social activities, etc. -- so as you can see, finding the user’s core action becomes much more challenging.
\r\nIf you are a data scientist working in a dynamic environment such as the one we enjoy at Sololearn, it is very easy to focus on all of the numbers and lose sight of the fact that each data point represents a specific and unique human with their own motivations that is hoping SoloLearn solves their problems or meets their needs. This fact is something that we remind ourselves of as often as possible: each Sololearner is a unique individual, in a certain period of their life journey, having found Sololearn with personal expectations of how they will use the app and what value it will provide for them. However, no product can have a solution for all possible unique users and their needs, so the answer for how to provide an exceptional experience for all users lies in data aggregation.
\r\nBehavioural patterns always occur among a pool of users. Your job as a data scientist is to identify what those patterns are and what are the metrics that summarize them in the cleanest way possible. Aggregation of the data is a sacrifice you must make every day, since keeping things simple and keeping as much of the information about all users are in constant conflict. The decision of which metrics to track is crucial for every product; these decisions should be made carefully, and in close collaboration between product and marketing teams.
\r\nA good rule of thumb in choosing key metric (a.k.a. North Star Metric) is to start with your product strategy. There is a saying that goes “what you measure, you optimize”. It is a wise idea to start with your expectations about the product and its desired usage from the learner’s perspective.
\r\nLet’s take a popular product such as a Zoom. It is not hard to guess what its key metric is. Zoom’s primary use is to host or attend a meeting. So its key metric is tied to the number of meetings hosted (per week). In a product with multiple use cases such as SoloLearn, the choice of metric becomes harder. There are several ways that you can use data to confirm/reject your hypotheses about your key metric, almost all of which use qualitative research and quantitative analysis (mainly correlational analysis). We will cover these concepts and how SoloLearn applies them in future blog posts.
\r\nWhile some people may say data is a mirror of reality, and others claim data serves as a window as a way to look at the world, one thing is clear: it is difficult to make good decisions if they are not grounded in carefully selected and analyzed data.
\r\nMake sure you track all necessary data. The answer to when we need to implement tracking of data is “always”. Even if there is no urgent need to have data tracking in place, it is best to create a framework for collecting it and for mining it from the start. Doing so will ultimately pay off, and the more data you ultimately have, the more tools you have in your arsenal to inform future decisions.
\r\nThere is always a question about what data to track, so keep a balance between not overloading your applications with data logging requests but also having enough data to support the ideas and hypotheses that you want to test.
\r\nThere are two primary inputs when it comes to growth. Many companies emphasize the role of an experimental growth engine where you test as many ideas as quickly as possible. In a future post we will cover how we run multiple A/B tests simultaneously while keeping the data and results clean. Execution is critically important but success hinges on the properly defining the problem and crafting a clear hypothesis about the causes of the problem and how we propose to solve it.
\r\nAfter we define a specific problem or propose an idea about how to make the users’ experience better, the team moves on to the stage of initial data validation to see what we can ascertain based on observational data analysis. Observational studies are, in general, less costly, since they do not require inputs from other teams, but the tradeoff is often very noisy data. So, more often than not, the team determines that there is a need for a properly designed experiment which will help us validate or reject the hypotheses. At SoloLearn, a Product Manager leads the process of designing an experiment that, if properly implemented, will determine if the problem is solved, or the idea does make the users’ experience on the app better. This design includes the metrics by which we will determine success and the specific data we will track to make that determination objectively.
\r\nIn future posts, we will discuss how to achieve growth in SAAS apps as well as describe Sololearn’s experimental culture.
\r\nGagik holds a Ph.D. in Mathematical Economics and serves as SoloLearn’s Data Science Team Lead and currently teaches Data Science at Yerevan State University. His areas of interest are Statistics, Probabilities, Data Visualization and Deep Learning. He began his career in financial risk management before shifting to software at SoloLearn.
\r\n',createdDate:"2020-10-07T00:00:00.000Z",metaKeywords:null,metaDescription:"SoloLearn practices data-driven decision-making to evolve and iterate on its app to improve the experience of its users.",topicId:4,topicName:"Company News",minutesToRead:6,mainImageUrl:"https://api.sololearn.com/Uploads/How_SoloLearn_Uses_Data_To_Improve_Its_App_Part_1_blogpost_1.jpg",userAvatarUrl:"https://blob.sololearn.com/avatars/49672056-8d51-4398-b153-25125d627390.jpg",relatedPosts:[{id:235,userId:18654115,userName:"Seth",title:"Ready to rise up our Leaderboard? Get rewarded for learning!",shortContent:"
We’re happy to announce the successful launch of the Leaderboard spaceship.
Wonder what that means for you? As you go through lessons, you’ll compete, level up, and get rewarded for learning! Want to know how to win on the Leaderboard? Read our blogpost to learn all about it. Or just try it out today and fly to the stars!
",fullContent:null,createdDate:"2022-08-11T00:00:00.000Z",metaKeywords:null,metaDescription:"Want to know how to win on the Leaderboard? Read our blogpost to learn all about it. Or just try it out today and fly to the stars !",topicId:4,topicName:"Company News",minutesToRead:3,mainImageUrl:"https://api.sololearn.com/Uploads/blog_cover.png",userAvatarUrl:"https://blob.sololearn.com/avatars/eadec77c-c7a4-4b21-aa0b-029bd599b2d9.jpg",relatedPosts:null},{id:226,userId:18654115,userName:"Seth",title:"Sololearn’s New Course: Web Development Fundamentals",shortContent:"
Sololearn offers industry-leading coding classes and resources in order to best prepare you for careers in modern web, application, and technology development. We are proud to announce our newest course offering, Web Development Fundamentals. Integrating components of CSS, HTML, and JavaScript, the course gives learners a comprehensive understanding of the knowledge and skills needed to be a successful web developer today. Learn more about it here!
\r\n",fullContent:null,createdDate:"2022-01-31T00:00:00.000Z",metaKeywords:null,metaDescription:"Learn more about the newest course offering available to members of the Sololearn community, the Web Development Fundamentals course.",topicId:4,topicName:"Company News",minutesToRead:4,mainImageUrl:"https://api.sololearn.com/Uploads/Web_development_basics_announcement_blogpost.png",userAvatarUrl:"https://blob.sololearn.com/avatars/eadec77c-c7a4-4b21-aa0b-029bd599b2d9.jpg",relatedPosts:null},{id:223,userId:18654115,userName:"Seth",title:"Meet the Newest Members of the Sololearn Community!",shortContent:'
You may have already noticed them popping up in a few places in the app. If so, great job – that means you’ve been studying! Over the coming months, you’ll be seeing more of them as they come to life in all the new features we’re busy developing.
\r\n',fullContent:null,createdDate:"2021-12-25T00:00:00.000Z",metaKeywords:null,metaDescription:" Over the coming months, you’ll be seeing more of them as they come to life in all the new features we’re busy developing.",topicId:4,topicName:"Company News",minutesToRead:3,mainImageUrl:"https://api.sololearn.com/Uploads/Christmas_blog_post.png",userAvatarUrl:"https://blob.sololearn.com/avatars/eadec77c-c7a4-4b21-aa0b-029bd599b2d9.jpg",relatedPosts:null}]}}}