Building an amazing team is an incredible feeling, but finding the right people can be a tedious slog. There are the role cards, the phone screens, the test problems, and something people call “culture fit” that really just seems to mean someone with strong communication skills that you won’t mind working with.
As your company matures, you inevitably get more and more focused on creating a clear strategy for growth. Oftentimes this means an incredibly complex process involving spreadsheets, alignment meetings, and a dozen drafts circulating among team members. Eventually all this effort produces team goals, which are then turned into individual responsibilities for the quarter.
Some people say entrepreneurship is more art than science, but that’s likely because applying rigorous scientific methods to developing a business is hard, not ineffective. But in a world where more and more data is easily available to founders, it’s becoming easier to learn and disprove bad hypotheses more quickly and effectively — and to get lost in a haystack of product analytics.
“It’s a great problem to have,” says everyone who hears about the rapid growth of a successful company. However, it doesn’t always seem so great when the phone is ringing off the hook, your email backlog is a week old, and your team is buried in work.
This article is part of a series called “The Data-Driven Product Manager,” interviews with product experts to help you use data to improve your product.
Though it might be an undeserved reputation, the word “data” suggests “objective” or “impartial” information to drive decisions. And sometimes data even can help make team culture less biased. At the Industry Product Conference last month, I spoke to Rohini Pandhi, a Product Lead at Square, the credit card processing solution for small business. She shared how data has made an impact on both the business goals and culture at Square.
Before joining Square, Rohini held senior product roles at a number of companies including Rackspace, PubNub, and Nodeable. Rohini is both technical and business-oriented, holding a Computer Engineering degree from the University of Michigan and an MBA from the University of Chicago’s Booth School of Business.
The first thing I asked Rohini was how she thought the role of the Product Manager has evolved since she started, and what product managers are responsible for today.
I’ve worked at early stage, growth stage, and public big companies and product is a different theme at each scale. There’s little nuances in what “product” really means at different places because the problems you’re trying to solve will be different.
When you’re smaller, you’re trying to find that product-market fit and you’re doing all kinds of hand-on work yourself; as the product owner, you’re prototyping; you’re coding; you’re also making the marketing collateral; you’re taking sales calls; you’re doing support. You’re basically saying, ‘let’s throw everything against the wall and see what sticks’.
As you scale, you have these bigger departments that can help you with that depth, so your role as a product manager becomes more about how you align a set of troops that you’re not commanding. You’re nudging your team into a direction and creating the focus for all of the cross-functional efforts. The product person is like the central command — you hear about new ideas from across the company, you know the technical considerations for building features. At that point, you understand the competitive landscape. So then you can drive consensus, but you need to recommend the direction the team travels in.
The PM is the hub, but how do you convey everything you’re learning and synthesizing? You need good ways to get everyone on the same page. I asked Rohini how she communicates how she’s making decisions to other people on her team and up the chain.
Regardless of scale, it’s all about transparency to me. I don’t like thinking that product manager is this person behind the curtain, Wizard of Oz-style. When I’ve talked to my team at any of the companies that I’ve worked at, I’ve always used the analogy of a hurricane map: ‘I can tell you where we need to need to be right now or when that hurricane’s gonna hit the coast, but then after that it becomes this broad spectrum of possibilities’. And you need to have that flexibility in the future as you learn new insights about your product and users.”
Being very candid about where we might be going based on inputs and data that we get has really helped create trust amongst the team. They have a sense of security because they know someone is looking at all of the data. And some folks may even become more interested knowing more about the inputs and actively participate in future product discussions, which is great.”
All-Hands meetings are awesome because they give the sense that everyone’s a part of the team. You’re supposed to ask as many questions as you want, be inquisitive, be involved, have that sense of ownership, so we’re all here working together on this one purpose.
I like asynchronous modes of communication, too. Meetings can sometimes suck all of the life out of the joy of working and in product, the bigger you get, sometimes you’re just stuck in meetings back-to-back. You look at your day and it can feel like you haven’t accomplished anything.
You need to know what the best means of communication is for whatever decision you’re making. It may not be a meeting, it may be just a Slack message, it might just be a JIRA conversation, it could be an email. Some people don’t feel comfortable asking questions in group settings so I also like embedding myself into 1-on-1 conversations with people across functional teams, and my own team, just making sure everyone’s on the same page and they all feel confident about the direction we’re going in.
Getting everyone aligned seems essential, but I wondered how Rohini gets a global sense of whether people are onboard.
In the past, I’ve stepped into roles where there’s been distrust or negative feelings about a previous PM, like, “I don’t know where this came from, I don’t know what happened,” almost like a bad break-up or something! I’d have to tell the team, “I have no idea what happened in the past, but here’s what we’re going to do to ease into a new rhythm.”
No one comes into a role or team with bad intentions. Team members strive for the same goals, but sometimes politics or emotions create an aggressive culture. But those kinds of distrustful conversations lead nowhere.
I like the idea of everyone having a piece, or a sense of ownership, because that means that no one person is going to be responsible for everything. There’s no crystal ball that’s going to tell you where to go but if we all have that sense of, “We’re being listened to and we’re considering all the different inputs, and I’m willing to share what the different inputs are of why we prioritize one thing versus another,” it becomes a much healthier conversation. I’m okay with people saying, “No, that’s not right,” or “I don’t believe this;” that’s a great conversation to have. We need to get to a place where we can have a healthy debate.
As Rohini was talking about how data plays a role in terms of making or collecting information, I was curious what kind of data her team was using and how they were using it.
There’s qualitative and quantitative. From the quantitative side, there’s so much that you could get through web analytics. The funny thing is, there are certain data points that you could translate in any number of ways. For example, ‘time on site is 25% more than it was before’. Does that mean that the content is more confusing and that’s why people are staying on the site, or does that mean it’s more engaging and that’s why they’re staying on the site? There’s so many different ways to translate data.
We balance that with qualitative data because it provides a fuller story. User feedback is a huge piece of the puzzle. Of course, you often can’t talk to every user you have on your system, so the quant with the qual helps you figure out, directionally, where you want to go with your product.
You often can’t talk to every user you have on your system, so the quant with the qual helps you figure out, directionally, where you want to go with your product.
I also quantify our prioritization matrix. I feel like numbers, sometimes, are easier to understand. Sometimes quantification is a nice way to look at a bunch of pieces of information and how they relate to one another. The internal roadmap that I share with everyone across the company has numbers and weights associated with it so they can see, “Okay, we’re picking this as a higher priority because it scored higher for these reasons.”
Data can be an amazing way to convey what your goals are and if you’re meeting them, but it can also be used to tell a story that isn’t complete. I asked Rohini if she had ever run into a situation where she had to question the data or discovered misleading data being presented?
Of course. I think the data’s only as good as the programming. The code’s not gonna lie, the data’s not gonna lie, but what lies is your definition of it. The hardest part about collecting more data is a common definition. At Square, if we look at our revenue and we look at our revenue per customer, how are we defining the customer? Is it an active 30-day customer, or active 90-day? Is it that they’ve ever used the product once? The revenue changes completely when you look at the different profiles. You also have to ask, ‘who was pulling that number before? Do we have the same definition of what we’re tracking now?’
Data may not “lie” but the challenge is in details of what your definitions are and how you define you hypotheses and your success metrics. It can really gets you into trouble. At least it’s gotten me into trouble several times!
Rohini works with a number of companies as an advisor in addition to her work at Square. I wanted to find out what she thinks are the metrics that teams should focus on, and how they can get started with data.
One that I really like, and I share it with all the startups that I help as well, is Dave McClure’s Pirate Metrics. It’s a really clean way of seeing the funnel, and being able to say, ‘Here’s what’s important.’ At its base, it’s a very solid starting point for most teams.
As a fan of Pirate Metrics, and also of the One Metric described in the book Lean Analytics, I wondered if Rohini had focused on different data points at the variety of companies she’d worked with and how much the lifecycle stage of the company played into what was important.
At a bigger company, your metrics fall into he larger set of company goals and your funnel and focus areas may change accordingly. For example, at a startup with a new product, if you’re trying to A/B test some sort of engagement lower in the funnel but it’ll take you the entire year to run this A/B test because you don’t have enough traffic, maybe you should be focusing on top of funnel to get more traffic in order to do that A/B test in a shorter amount of time. You’re optimizing for something that doesn’t necessarily need optimization just yet.
For a lot of middle-stage companies, the goals and data that supports them is changing rapidly. I asked Rohini how product managers at that size use data, and if they actually use customer interviews, customer surveys and customer feedback in their process. How does that all come together into information that you can use to make a decision?
There’s more art than science on that one. I think at the middle-stage companies, you as the PM are thinking ‘We’ve already figured out product market fit, now we need to figure out scale.” You’re thinking about that top of funnel growth to get to the tipping point and so a lot of the data becomes proving out, ‘if you put ‘X’ dollars into sales and marketing, because we know that the product is fundamental and solid, we get ‘Y’ dollars in return out of that’.
From the product side, when you do qualitative stuff, if you see a pattern that happens with 5 people, 5 customer interviews, that’s a real pattern. Especially at the small-to-middle stage.
I’ve also done customer advisory boards where I bring in 8–10 people from different companies whose profiles are pretty representative for our entire user base. We share our roadmap with them or considerations to our roadmap and we’ll kind of give them a scarcity problem where we say, “If you could only pick 3 options, which ones would you pick?”
Otherwise, people will say they want everything tomorrow! But if you give them only three options from the entire set to choose, there will be bucketing in certain categories that starts to happen and you can drill into why your customers chose those certain pain points. It’s not necessarily quantitative but it gives you some data points to use.
Square is a data-driven company; you might even say it is a data-focused company. I was curious how that translated in terms of company culture.
It’s actually fantastic. A lot of our data is financial data, obviously, and there’s a lot of money moving back and forth. At Square, what I really like about their data-driven culture is that there isn’t any ego attached to a decision. People are willing to say, “Okay, that hypothesis was wrong because the data came in and proved me otherwise.”
Thanks so much to Rohini for sitting down with me for this very informative discussion. If you’re interested in finding more Data-Driven PM articles, check out these interviews with Laura Klein and Hunter Walk. You can also sign up for The School of Little Data, our free email courses to learn how to use your data to collaborate with your team. Notion is a new tool for teams who want to communicate with data, and we’d love your feedback on how we can best serve your needs.
A Key Performance Indicator (KPI) is a simple, measurable way of monitoring the performance of your business. Regardless of what kind of business you run, they are designed to quickly answer the question, “How are we doing?” This is true for company KPIs, team KPIs, and individual KPIs.
I found myself constantly nodding while reading Kaegan Donnelly’s recent article, The startup guide to growing your business with Intercom.
Like his team at Baremetrics, the tool we use at Notion as often as Notion to help us run our business is Intercom. And it’s always reassuring to learn new tricks and see similarities in how other teams are running their company and customer support through Intercom.
If you haven’t read it yet, I highly recommend checking it out.
And if you also found yourself nodding along, I’d like to share some additional tips and show how we’re adding more value to Intercom.
Collecting CSAT (Customer Satisfaction) data is Customer Support 101. It can be an invaluable tool for understanding customer sentiment and improving operations, and almost every team includes it in their suite of top metrics (Click here for more info on the top customer support/success metrics).
Data driven software development team leaders know that there is magic and insight in their team data. With the right information at the right time, great managers can identify which teams need help, spot and fix troublesome bottlenecks, and prevent team burnout and attrition. In the world of software engineering metrics, however, there are a lot of KPIs to consider and a lot of competing views on which are the most important.
Getting a team motivated is challenging in itself, but we've all read and heard that setting goals for your team is a critical component. When goals and objectives seem to come down from on high without involving the people tasked with achieving them in the process, it can be downright excruciating. Incorporating your team into the process of team goal setting can be the critical difference between frustration, resentment, or apathy and motivation, excitement, and a sense of teamwork.