B2c approach in which the conversation starts with a comment about the product.

Conversation with Mohammad Musa, former lead PM Google

In each entry in this “conversation” series I talk to a designer/product manager/engineer on a topic. I want to make basic practical skills education transparent and free.

Today I’m talking to Mohammad Musa, a product manager who has worked for Google in the past and is currently working on his own startup. Mohammad has managed multiple internal Google products and has also worked as part of the launch teams on many Google products.

Incorporating customer feedback is one of the most important jobs of the PM. Today we are talking about how to incorporate feedback, specifically in the context with B2C companies.

What are the channels used to capture user feedback?

Here is one way of thinking about user feedback — 1:1 and at scale. User researchers help us capture 1:1 feedback and we use tools to get feedback at scale. 1:1 feedback is qualitative and the at scale feedback is quantitative and a lot of times you have to combine both to get the answers you are seeking.

The channels for user feedback we have in Google are:

1. User research teams: User research teams through interviews and focus groups. Also internal teams and Googlers give feedback during the dogfooding phase of the product.
2. Feedback tools: Google Feedback is embedded in all Google products.

3. Happiness Tracking surveys: Surveys we sent out to understand user happiness. We used Google surveys for this.

4. Bug tracking systems: Some products have open channels for collecting user bugs — for example, Tensor Flow.

5. Help center articles: Have a place to indicate if something was useful or not in help articles.

5. Google product forums: Google product forums were mailing groups for users to discuss feedback amongst each other. We had community managers that would escalate and monitor these threads constantly.

6. Feature request forms: For internal products, we also have a feature request form which employees can leave feedback as well.

At Google there is no dearth of user feedback, which is atypical for most companies. Once feedback comes in, the PM categorizes it into different categories — UX, bugs, etc. The ownership of capturing feedback is between the user research team and the PM.

How do you make a decision as to what feedback to listen to and what not to take into account?

We make sure every piece of feedback passes these questions:

1. Does it align with our company mandate?

Google has company mandates every quarter which all products are required to follow. These mandates can come from anyone in leadership. For example: there was a Google mandate of “Mobile first” so all products had to align with that mandate when shipping new features.

2. Does it align with product vision?

Everything you do has to fall in the overall product vision. The vision is set by engineering and product leadership at Google.

Finally, is it a bug? If so, it’s triaged and filed, if it’s urgent it’s fixed right away.

On what to take and what not to — do not listen to feedback directly, instead seek to understand what problem it solves and not necessarily follow the solutions.

One example of this is when I led an internal tool at Google which was a crowdsourcing feedback tool for Googlers for AMA’s (like Reddit). They got feedback to de-dupe questions — i.e. it should weed out duplicates when people submit questions.

In this case we didn’t follow the exact solution right away. Instead of having a de-dupe flow, they first implemented machine learning to understand if the questions were similar. After they launched that, they measured results and realized there were still some errors. Post that, we implemented deduping questions but used the ML technology. We didn’t do exactly what people wanted but gave them the exact outcome they expected.

How do you prioritize within the feedback you have received to understand what to implement?

After we have a subset of feedback which passes through the above criteria, we need to optimize for maximum impact.

Questions to ask for maximum impact:

How many people are being affected by this?

How many paying customers are being affected by it? This applies more to enterprise business — Google did hands on for large customers but for the majority of the long tail they were hands off.

The way we also planned the roadmap was in themes, for example the theme for one quarter could be user happiness. In that quarter we would prioritize feedback which fell into that theme.

Finally, you have to take into account engineering estimates as well before finalizing anything.

In general, PMs at Google make sure they adhere to Google’s philosophy around building products. Most of the focus is on users and not competitors. And finally, they are interested in being products which are core to people’s lives everyday — something which they refer to as “toothbrush” products.

How does the style of feedback change once you launch something vs something that’s been around a long time?

A product at Google goes through the following stages:

Fish food phase: This is when your own team and some internal teams are the only users — at this point it’s very high touch engagement. Tools used are emails and chat.

Dog food phase: There are hundreds to thousands of people using your product. At this point, you start tracking using feedback tools.

Trusted tester phase: Closed alpha — for consumer products.

Final Launch: When you open it up to all customers — usually a staged roll out. 1%, 5% and 10%. If everything works at 10% then you just roll it out to all the users.

Feedback and tools change accordingly, depending on the phase they are at.

How do you make users feel heard because you can’t implement most of the feedback?

For internal teams this is what we followed: Hand draft a response and tell them which category the feedback falls in, because we care about their opinion. These are the categories we had:

p0 = Do it right now

p1 = Do it in the next day (prioritize it, we really need to get it done)

p2 = We are going to do it but it’s not urgent

p3 = We like it and it’s desirable and we will do it at some point if possible

For non-internal products we send an automated response. One of the things we had in the pipeline (still not implemented) was sending more automated responses with content like — “Thank you for your feedback. Here is a list of all the top priorities we are working on this quarter — if your feedback falls among those, great! We are working on it. If your request isn’t a priority for us and you feel strongly that it should, please escalate using this form.”

Finally, look at the data and be honest — are consumers using your product? Google PMs in consumer teams had a set of HEART metrics they would track — Happiness, Engagement, Acquisition, Retention, Task success. Enterprise teams would always look at NPS scores.

Hopefully this article has been useful for you. Please click on the heart button if you like it and leave a comment with any further questions!

Is the method of showing the major selling features of a product and how it benefits the customer?

What is the way in which a business provides services before during and after a purchase?

What is selling is direct contact with a prospective customer with the objective of selling a good or service?

What is making contact with people who are not expecting a sales contact?