[UI] New Getting ratings and reviews flow

[tpregueiro]

Background

Reviews and ratings are known to have great influence in user acquisition. Users are more likely to install apps with more reviews and, obviously, with better ratings. Part of that user acquisition also has to do with discoverability and whether your app shows up at the top of the list for a given search query. Ratings affect the ranking, and so, the lower the rating we get, the harder it will be for our users to find our product.

Historically, our rating in the PlayStore has been decreasing month after month and is now (August 2017) at 4.085★. At Eyeo we are working hard to create a better product for our users and making sure that the product in itself solves users' problems. However, the cummulative historical ranking makes it harder to move the needle on the ratings. Every new improvement we make, even if our users love it and leave us a lot of great 5-star reviews will only slightly move the rating by a few centesimals.

So we have to fight in two fronts separately at the same time: 1) make a better product that users love and cherish, for which we will organically get good reviews and over time average up our overall ratings; 2) funnel positive reviews that might not ever exist unless we prompt users for those. With this feature we aim at solving problem 2, i.e., prompting users whether they are enjoying their app experience, and if so, ask them if they want to leave a review in the PlayStore to help other users find our app.

In addition to all of the abovementioned, we actually have an existing functionality to do just that inherented from Fennec for Android (Firefox for Android). You can check out the existing functionality in version 1.2.0 of ABB for Android by going to about:feedback. In newer versions of Fennec (45+), Mozilla decided to redesign that functionality and instead of using a local page like about:feedback they went with a server-side page that does pretty much the same. So, as we merge to newer versions of Fennec, we will loose the functionality we inherented from Firefox and so we had to decide whether to keep the 1) about:feedback page, 2) do something similar to Firefox, with a server-side page with a similar UX as that of the about:feedback page or 3) rethink the feature from scratch and do something more natively integrated that doesn't require a server-side page, and that is less disruptive than sending users to a new tab. We decided option 3) was the right way to go, and this document aims at describing the functionality required to achieve that.

Goals
Funnel more reviews to the PlayStore.
Funnel the most negative feedback to our support team to be address asap.
Drive more user acquisition as consequence of increase the number of reviews in the PlayStore.
Go back up

Success criteria
Increase the overall number of reviews.
Increase the number of positive reviews, by funneling 5-stars feedback to the PlayStore.

What to change

Please refer to the Functional Specification:
​https://bitbucket.org/adblockplus/spec/src/8b2e577c0bdd9763ec19854df249d4d078e2ee58/spec/abb/abb-android/abb-android-ask-for-ratings.md?at=63-new-onboarding-abb-android&fileviewer=file-view-default

A: Trigger
​https://bitbucket.org/adblockplus/spec/src/8b2e577c0bdd9763ec19854df249d4d078e2ee58/spec/abb/abb-android/abb-android-ask-for-ratings.md?at=63-new-onboarding-abb-android&fileviewer=file-view-default#markdown-header-a-trigger

B: Initial prompt
​https://bitbucket.org/adblockplus/spec/src/8b2e577c0bdd9763ec19854df249d4d078e2ee58/spec/abb/abb-android/abb-android-ask-for-ratings.md?at=63-new-onboarding-abb-android&fileviewer=file-view-default#markdown-header-b-initial-prompt

C: Positive feedback dialog
​https://bitbucket.org/adblockplus/spec/src/8b2e577c0bdd9763ec19854df249d4d078e2ee58/spec/abb/abb-android/abb-android-ask-for-ratings.md?at=63-new-onboarding-abb-android&fileviewer=file-view-default#markdown-header-c-positive-feedback-dialog

D: Negative feedback dialog
​https://bitbucket.org/adblockplus/spec/src/8b2e577c0bdd9763ec19854df249d4d078e2ee58/spec/abb/abb-android/abb-android-ask-for-ratings.md?at=63-new-onboarding-abb-android&fileviewer=file-view-default#markdown-header-d-negative-feedback-dialog

E: Data requirements
​https://bitbucket.org/adblockplus/spec/src/8b2e577c0bdd9763ec19854df249d4d078e2ee58/spec/abb/abb-android/abb-android-ask-for-ratings.md?at=63-new-onboarding-abb-android&fileviewer=file-view-default#markdown-header-e-data-requirements

UI

Prototype in Invision: ​https://eyeogmbh.invisionapp.com/share/BHD1GY4PQ#/248682812_02_-_Feedback_Prompt_-_Positive_Rating

Inspect project in InVision: ​https://eyeogmbh.invisionapp.com/d/#/console/11786429/248682812/inspect

[jwangenheim]

I really appreciate that we integrate that kind of feedback trigger into ABB. I am sure it will enable us to get a lot more reviews and a better overall rating. I just have two comments on the specs:

• Don't we want to add an option, that allows the user to permanently dismiss the rating dialog?
• When the user hits REVIEW or YES (to send an email), we should make sure the rating dialog is never triggered again.

[tpregueiro]

@jwangenheim those are both great points.

Don't we want to add an option, that allows the user to permanently dismiss the rating dialog?

I agree. If user keeps hitting dismiss, we should just probably just take the hint and stop annoying that user. Here's a couple of alternatives to implement a permanent or somewhat-permanent dismiss:

1) After the user has dismissed the prompt more than X times (say x=3), never show the rating dialog/prompt again to that user. This is simple and straighforward. Has the disadvantage that this user could potentially be more inclined to give us a rating at the X+1th attempt. If we wanted to be more conservative we could define a higher X value, say x=5.

2) After the user has dismissed the prompt more than X times (say x=3), the X+1th time the prompt is shown, change the copy of the button to "Permanently dismiss" (final copy tbd). To be honest, I think this puts a huge burden on the user because they have to make a very irrevokable choice, whereas in 1) we can just intrinsically assume the user doesn't want to be bothered with this anymore.

3)After the user has dismissed the prompt more than X times (say x=3), reset the counters (like already defined in the spec), but add a kinf of a "snooze" for N number of days (say N = 90 days), during which we don't show the prompt. After N days, the counters are reset, and everything goes back to normal. Personally I think this a more self-correcting approach, and we could even define a high threshold like N=180 days.

4) Same as 3) but instead of using a fixed timeframe, using major releases as the "snooze" condition. After the user has dismissed the prompt more than X times (say x=3), stop prompting the user until the user upgrades to a major release (e.g. major releases could be defined as X.5.0, or X.0.0, or some other definition).

I am more inclined to do something like 3) or 4) since it prevents from completely loosing the possibility of asking that user ever again. What do you think?

--

When the user hits REVIEW or YES (to send an email), we should make sure the rating dialog is never triggered again.

As this feature is more sophisticated, it probably makes sense that we have the ability to show the dialog in the future, especially for users who go through the negative rating path, e.g. if a user hits 2 stars, and then sends us an email, perhaps we should ask again once a new major version is released. Whereas if the user has gone through the 5-star path, we probably never want to ask again.

That said, to avoid adding even more complexity to this initial version, we could just do as you suggest, when the user hits REVIEW or YES (goes to email), we store that info, and don't show the prompt to that user again. What do you think?

Alternative we could do as you suggest, simply never show again, but still store the path which the user chosen (negative or positive). And maybe in a future release we could add the logic that looks at that value, and maybe triggers it again. ?

[jwangenheim]

@tpregueiro Thanks for sharing your ideas on those two points. Good ones!

@jwangenheim those are both great points.

Don't we want to add an option, that allows the user to permanently dismiss the rating dialog?

I agree. If user keeps hitting dismiss, we should just probably just take the hint and stop annoying that user. Here's a couple of alternatives to implement a permanent or somewhat-permanent dismiss:

1) After the user has dismissed the prompt more than X times (say x=3), never show the rating dialog/prompt again to that user. This is simple and straighforward. Has the disadvantage that this user could potentially be more inclined to give us a rating at the X+1th attempt. If we wanted to be more conservative we could define a higher X value, say x=5.

2) After the user has dismissed the prompt more than X times (say x=3), the X+1th time the prompt is shown, change the copy of the button to "Permanently dismiss" (final copy tbd). To be honest, I think this puts a huge burden on the user because they have to make a very irrevokable choice, whereas in 1) we can just intrinsically assume the user doesn't want to be bothered with this anymore.

3)After the user has dismissed the prompt more than X times (say x=3), reset the counters (like already defined in the spec), but add a kinf of a "snooze" for N number of days (say N = 90 days), during which we don't show the prompt. After N days, the counters are reset, and everything goes back to normal. Personally I think this a more self-correcting approach, and we could even define a high threshold like N=180 days.

4) Same as 3) but instead of using a fixed timeframe, using major releases as the "snooze" condition. After the user has dismissed the prompt more than X times (say x=3), stop prompting the user until the user upgrades to a major release (e.g. major releases could be defined as X.5.0, or X.0.0, or some other definition).

I am more inclined to do something like 3) or 4) since it prevents from completely loosing the possibility of asking that user ever again. What do you think?

--

I really like 3), as it does not introduce to much complexity and we still keep the chance to reach out to the user again after some time. I think I would not feel annoyed as a user, if that dialog pops up again after 3-6 month.

When the user hits REVIEW or YES (to send an email), we should make sure the rating dialog is never triggered again.

As this feature is more sophisticated, it probably makes sense that we have the ability to show the dialog in the future, especially for users who go through the negative rating path, e.g. if a user hits 2 stars, and then sends us an email, perhaps we should ask again once a new major version is released. Whereas if the user has gone through the 5-star path, we probably never want to ask again.

That said, to avoid adding even more complexity to this initial version, we could just do as you suggest, when the user hits REVIEW or YES (goes to email), we store that info, and don't show the prompt to that user again. What do you think?

Alternative we could do as you suggest, simply never show again, but still store the path which the user chosen (negative or positive). And maybe in a future release we could add the logic that looks at that value, and maybe triggers it again. ?

You are right, we should probably have the chance to reach out to users who gave negative feedback. I prefer the idea to do that after a major version is released. As we already check for updates this shouldn't be too much effort.

I think it makes sense to share those two points on the ABB channel to hear what the others think about it. Would you mind doing this?

[karthikpriya]

spam

[madhusreeinfo]

spam

[madhusreeinfo]

spam

[johnlewisjohn]

spam

[nps1337]

spam