Breaking Through the Data Ceiling

Ankur Uttam

VP of Marketing Analytics at Breaking Through the Data Ceiling

Learning Objectives

How many times have you struggled to get alignment (and adoption) on insights and models coming out of great data projects? How many times have you murmured "but the data shows that we can...?" If you are someone who can relate to the questions above then this presentation will walk you through the journey of successful data projects from inception to business impact.


Key Takeaways:



  • Always keep business metrics and stakeholders in mind - seek alignment throughout the process, not only at the beginning or end

  • Good to have a sophisticated/accurate versions but great to have simple/almost there versions of the results/insights

  • Understand the available tools that can be used to make a change and work within those constraints


"Understanding each person's motivation and rewarding accordingly is very important."

Ankur Uttam

VP of Marketing Analytics at Breaking Through the Data Ceiling

Transcript

Hi, everyone. Welcome and thank you for taking time to attend the talk today. My name is Ankur, and I’m currently a VP of Analytics and Data science at Poshmark, one of the leading social commerce platform in the world. Prior to Poshmark, I’ve spent 15 years building and scaling data science teams in different technology, e-commerce, and gaming companies.


Today, I am going to talk about how insights arising from data projects can be converted into high business impact worksheets. I will discuss how to transform your teams from being a service organization to an idea generation powerhouse. This will include building an environment to foster innovation and motivation, navigating the organization to have your ideas heard and implemented, and how to go about finding solution and making this a repeatable process.


Let me start with what I believe are the foundational blocks for success. There are two aspects to the whole process, getting the right idea and converting that idea into a successful project driving business outcome. The first part on people and ecosystem covers how to systematize idea generation and foster an innovation culture. Next, I will discuss how these ideas can be taken forward to produce positive business outcomes.


Let’s start by talking about people. People, unarguably, are the most important asset for any organization. Hiring, training, mentoring, and then motivating and retaining the best talent is half the battle what. I will spend more time on things that need to happen after any hiring and pleasure training is completed since a lot of you will have fell form teams already. In many cases, you entered these teams and not build them from ground up.


One of the first things that should be part of the initial experience and mentoring is the perspective. The key point here is to keep the larger picture in mind. I have chosen these pictures to highlight the stark difference that exist when picking up a problem to prioritize with the end objective of harvesting more fruits. The topside might be more intellectually stimulating and gratifying as a problem as it involves with complex solution to determine how to grow more fruits on a tree that sits in the middle of a desert. It will have multiple constraints, so that solution would be more challenging, but there is only so much fruit that you can grow on a single tree. Contrast this with the image at the bottom. Here you have a forest full of trees laden with fruits. The problem here is to determine the order in which the tree should be harvested to maximize production. This may be a less challenging problem analytically, but could use significantly better results for the organization.


Having this perspective that an organization needs to get more fruit and not how hard is it to produce the fruit is very important. A related outcome of the mindset is where to focus your effort and energy on. Do you want to spend more time on the solution or on the outcome? It’s not an either or situation, but more of a balancing act where you’re not tilting too far in either direction. Depending on the audience, you may want to shift your focus or discussion one way or the other. On one end of the spectrum is focusing and optimizing the solution to come up with the most accurate answer towards this and most of the effort is spent trying to create an optimum solution and optimizing model metrics like R square, F value, etc.


On the other side is focusing on the outcome of the solution. Here you are translating the results into business terms and talking about what are the key factors that influence the business KPI. You are discussing scenario planning, what ifs of different possibilities. Training people on keeping the business outcome in mind ensures that the results are presented in a simple and actionable way.


Let us move to some elements that encourage and motivate the organization to go into the direction of innovation and new ideas. What we are trying to do here is to create an environment, an ecosystem that helps us move in the direction of our vision. One of the most important aspect of any team is proactive generation of business ideas. These may be disruptive, but a lot of them may be incremental. As long as the idea adds to the top or the bottom line, they will be supported with organizational resources. As leaders, we try to create that ecosystem. As analysts, we try to thrive in that ecosystem and create opportunities for self and the organization.


Before we get into the elements of the ecosystem, I want to throw one of my beliefs in front of you. This is something that has helped me come up with a view of the ecosystem where ideation flourishes. We all know that ideas formed through neural connections in our brains. You can think of these points as hunches or thoughts or intuition you have due to your experiences and knowledge, which get connected in that moment, and the idea about life itself.


Archimedes, for example, did not suddenly come up with the idea of buoyancy. When he immersed himself in that path he had a few scattered thoughts and hunches, which in that moment, collected to form the idea. Ideas evolve and become more concrete as more thought is put into it. You discuss your thoughts with different people in your organization, gaining feedback, and new nuggets of information. This helps formalize the idea which you can go after. Hence, I believe, ideas formed through neural connections in the collective brain of the group.


There are three key elements to creating an ecosystem where this collective group can develop and flourish. First and foremost is the organizational structure. There are two standard structures: centralized and distributed. Both of these have pros and cons, and can be implemented efficiently and productively in different environments. However, they do not necessarily encourage the development of the collective brain, or the ears of building and managing teams, I have come to a structure, which I call the federated model. In this model, the larger team is still centralized, sitting under a C-level exec, but each of the sub teams have a lot of autonomy in their day to day operations. This structure brings together benefits of both the centralized and the distributed teams. Additionally, it allows movement of people across teams to learn and partner on cross functional projects. Both these enable broadening the perspective and mindset of analysts by giving them exposure to different functional areas within the organization.


The second aspect is process. From a process perspective, the leaders need to create an ecosystem that systematizes interactions and cross learnings. They have to ensure that these interactions are deep, where everyone is leveling up and learning and contributing.


Few things that have worked for me in the past are learning series, where a person or team can present an idea or a project and lay it out in front of the larger group for discussion.


Then, group learning sessions—in this, we pick up a topic, which everyone comes prepared to discuss. There is an assigned moderator or teacher who enables the discussion, but everyone learns as a group.


Lastly, it’s temporary project based rotations. Besides helping and learning different functions. It also enables technical learning as different teams use different tools. For example, a BI team is likely to use a lot of tools to create reports, while a product team is likely to use a lot of stats around AV testing. Why I say temporary is because it is very important for every analyst to have a home team to maintain their identity. Otherwise, it is very easy to get disoriented and lose focus and motivation.


Lastly, incentives. I believe nothing motivates people better than reports. However, one thing to keep in mind is that different people are motivated by different things. Understanding each person’s motivation and rewarding accordingly is very important. Rewards could be monetary or professional growth related. It could be as simple as giving an additional day off. Anything that makes the person or a team member happy—we should do that. A related thing is to create an environment of celebration of people’s achievements, victories, and even losses. These celebrations could be where the rewards are announced. Having the presence of leaders from other groups within the company to increase the visibility and prestige of the rewards will be important as well.


Let me now transition to discuss how to proceed once you have a couple of these ideas identified through the collective brain of your group. What you as a leader needs to do to get the green light for the project.


First and foremost, you need to develop a business case, almost like a pitch to sell the idea to a larger group and eventually to the decision maker who prioritize projects and resource allocation. A good business case should cover at least the following: What is the problem area we are trying to address? Why is that important? Is it an existing pain point or is it something that will become important for the organization in the future?


Next, we need to identify the KPI we are trying to move and why our proposal would move the KPI. I would like to note here that it may initially not always be a revenue or a cost number, it could be something like increased retention. However, nothing more to eights leadership from approving something that has dollar number associated with it. As an example of improving your attention by 2%, we’ll lead to 1% more interactions on the platform and this will lead to .5% increase in transactions. Essentially, you’re moving the business by about .5%. I would suggest that you always complete this follow to the dollar number.


Next, you should cover a brief plan on how you envision the execution of the project. Will it be done in phases? Is on a test and learn approach? What resources would be needed? What is the expected ROI based on the incremental revenue and the cost at work? A lot of times you don’t even need to do the ROI calculation if the top line movement is significant.


The next thing I’m going to talk about is Nemawashi. The literal meaning is preparing the roots for transplanting. In the professional and organizational setup, it is commonly dubbed as the meetings before the meeting. The idea is that before you go to the final pitch in front of the decision committee, you have run the idea with all the stakeholders, gotten their feedback, and gotten their software through. During this process, you may have to adjust your ideas, which you should be open to, so that you get everyone aligned and there are no objections or surprises in the final beat.


Listed below are some of the key aspects of the Nemawashi. I will not get into the details, but it can be summarized into two broad areas. One is around how you should build and maintain relationships in the entire organization and understand the informal power structure. Second is around how you should proactively communicate to and build consensus with the entire organization relevant for a particular project.


Let us move on to the design of the solution. I want to cover two key aspects of solution design today. First is the state of data availability in the organization. The ideal state is that we have data for all the factors the entire range of values those factors can take and for as long as history as possible. That would be represented by all green cube. However, what you see is what the reality is—the data is always limited. For some factors, you have everything, but for other factors, you do not have any data. As a group, it is important to conduct this exercise to understand our limitations and our strengths. This can help us attach confidence levels to our outcomes and also provide a timeline of next few phases based on how the data collection goes. This also sets the foundation for the need of experimentation to learn and improve our models.


The second is experiment design. I believe no solution is ever complete, it is always in a state of local optima. With time, data sources increase, environment changes, products change, so there is a need to continuously monitor and evolve any solution. The frequency would depend on the business. It is also important to explain the business partners and get them on board early on.


Next, I want to cover an important concept which I have found extremely critical and efficient in execution of projects, especially ones pertaining to data. Business stakeholders and data teams speak two very different languages. A lot of solutions are never implemented because they fail to correctly address the business questions on hand. The leading cause for this is the lack of understanding of the business problem by the team working on the problem. Hence, the need for the translate a person adept in both business and solution worlds. It is up to us, as leaders, to help in this translation and overtime, train the team to be independently thinking. What does translation entail?


Most analysis starts with the business problem. Example, growth seems to be slowing. For an analyst, it is just another line or bar chart. The translator needs to convert this problem into an analytical problem. The analyst is then able to come up with an analytical solution, and the translator comes into the picture here to interpret this model back to the business and give an actionable recommendation. Let’s continue the example of slowing growth. The translator translates that question into: what are the growth drivers and how does each driver impact growth? This is something that analysts understands. The analyst then builds the model, listing the significant variables and their coefficient. The translator comes into the picture and translates this back into the business by giving a recommendation that says, “You can increase the growth by percent and by allocating some spend from Channel A channel B.” That’s how this whole process works.


Once the project is approved, and the initial solution design is complete, it’s time to hit the crop. However, a few times, projects get installed or even canceled at this late stage. The primary reason is that the champion for the project is missing, so what should you do? You should get a seat at the table. You should always seek to be part of the core group that brainstorms and discusses the project going forward. You act as the guardian of the project ensuring that the essence and true objective of the project is always front and center in all the discussions. Do not let the discussions derail or veer off in unintended directions. Besides the solution design, you should also participate in creation of the test plan and strategies to measurement plan, and keep reminding the team of the objective of the project and what the Northstar metric is. If the project needs iterations, you should be there to help plan and execute them. Learn incrementally and work towards the success of the project.


Now that you have had a successful project, congratulations, you just completed your first indigenous project. You should celebrate it with your team and the broader team who helped make that success, and reward the appropriate team. You have built a lot of credibility and equity with the partners, stakeholders, and the company. How do you utilize that? First is nurture your newly formed relationships. You will have just formed multiple new professional relationships. As with all relationships, you should segregate them into professional and professional plus personal, and allocate time to develop some of them into long-lasting ones. Continue to propose new ideas and make it a systemic process. Go ahead and commit to executing a couple of these ideas every year and include it as part of your team OKRs.


Another thing to keep in mind is to use your equity responsibly. Use it when you are certain of success to increase its value. Even if you fail, you should always have enough equity that you are able to continue on this journey of strategic innovation to drive business impact.


In a nutshell, what I want to say is build the right environment around rightly train people to encourage and reward innovation. Build a strong business case, incorporating feedback from all partners and getting soft approvals. Act as a translator to ensure project is correctly executed, communicate limitations, and build it in an iterative test and learn phases. Lastly, be continually involved to create success and then build on that equity to make it the way you work. That brings me to the end of the presentation. Feel free to send me your questions and thoughts. Thank you.


Get full Q/N Access

Sign up to Q/N with a few details to watch this presentation.

  • Hidden
  • Hidden