Opened 4 years ago

Last modified 3 years ago

#4672 closed change

Introduce HockeyApp event-based error reporting — at Version 20

Reported by: mario Assignee:
Priority: P3 Milestone: Adblock-Browser-for-iOS-1.5.2
Module: Adblock-Browser-for-iOS Keywords:
Cc: pavelz, lisabielik, scheer, mario Blocked By:
Blocking: Platform: Adblock Browser for iOS
Ready: yes Confidential: no
Tester: Rraceanu Verified working: yes
Review URL(s):

Description (last modified by mario)

Background

As of 2016 HockeyApp provides a service called "HockeyApp Events" which can be used to intercept non-crashing errors/exceptions and collect them as error reports. This function is based on HockeyApp's crash report, which has been introduced with #3272.
Introducing this enables us to better understand potential problems and react faster to non-crashing errors.

What to change

HA Events
Implement HA events as described in this documentation.

Opt-In UI

  • Change Crash Reports to Crash and Error Reports in the settings menu
  • Change SEND CRASH REPORT to SEND CRASH AND ERROR REPORTS in the crash reports settings menu
  • Change Ask me after a crash to Ask me after a Crash or Error in the crash reports settings menu
  • Change Only anonymized data containing device information and crash details are sent. to Only anonymized data containing device information and crash or error details are sent. in the crash reports settings menu
  • Add a modal dialog which is shown as soon as an error is intercepted the same way as the modal which is shown when a crash occurs:
    • Title: Adblock Plus Encountered an Error
    • Description: Would you like to send a report to fix the problem?
    • Button 1: Send Report
    • Button 2: Always Send
    • Button 3: Don't Send

Opt-In Logic

  • As soon as an error is intercepted for the first time, show the modal dialog as described above.
    • Tapping "Send Report" will send the report once
    • Tapping "Always Send" will send the report once and change the error reporting settings to "Always"
    • Tapping "Don't Send" will not send any report
  • As soon as an error is intercepted the next time, base action on the error reporting settings:
    • If "Ask me after a crash" is active, show the modal dialog above
    • If "Always" is active, send the error report without showing the modal dialog again
    • If "Don't Send" is active, send no report without showing the modal dialog again

Restrictions
Due to the fact, that non-crashing errors could happen multiple times in a row, apply the following limitations:

  • Limit the error report dialog to be shown only once every 10 minutes
  • Limit events to be sent once every 10 minutes in order not to slow down ABB or clog the user's internet connection if a repeating error is intercepted in a loop
  • Have a backlog of event types with their count
  • Upon any event, add to backlog, increment count as needed
  • If timeout expired, take 5 most common event types
  • If there is less than 5 types, fill up to 5 repeating counts of available types
  • If user denied sending (permanent or once), keep growing the backlog until app restart (negligible memory consumption)
  • Upon approval (permanent or once), send, clear the backlog and reset timeout

Change History (23)

Changed 4 years ago by mario

Changed 4 years ago by mario

Changed 4 years ago by mario

comment:1 Changed 4 years ago by mario

  • Description modified (diff)

comment:2 Changed 4 years ago by mario

  • Cc lisabielik added

@Lisa, could you please review the text changes introduced with this issue (Opt-In UI section)? Thanks!

comment:3 Changed 4 years ago by lisabielik

Change:

Only anonymized data containing containing device information and crash details are send.

to:

Only anonymized data containing device information and crash or error details are sent.

comment:4 Changed 4 years ago by mario

  • Description modified (diff)

Oh, that one slipped through. Thanks! Changed the description.

comment:5 Changed 4 years ago by mario

  • Ready set

comment:8 Changed 4 years ago by pavelz

@mario @lisabielik forgotten texting update needed:
former options per crash reports settings menu: Always/Never/Ask me after a Crash

Third option should still say only ...Crash or rather Crash or Error (evt. Crash/Error) per all other requested changes to that dialog?

comment:9 Changed 4 years ago by mario

  • Description modified (diff)
  • Ready unset

Updated the description. Waiting for lisabielik's feedback.

comment:10 Changed 4 years ago by lisabielik

@mario @pavelz

For consistency, I think we should use:

Ask me after a crash or error

comment:11 Changed 4 years ago by pavelz

@lisabielik so drop the existing capitalization?

comment:12 Changed 4 years ago by lisabielik

hi @pavelz,

Sorry...no. It should be:

Ask me after a Crash or Error

Thanks for catching that.

comment:13 Changed 4 years ago by mario

  • Description modified (diff)

Updated description to reflect text changes

comment:14 Changed 4 years ago by pavelz

@mario the requested opt-in logic breaks the logic already established by HockeyApp SDK. We should follow HA SDK logic, unless the error reporting configuration is meant to be separated from crash reporting. Which i understood is not the intent, to simplify things for the user as much as possible.

Tapping "Send Report" will send the report once and change the error reporting settings to "Ask me after a crash or error"

The crash/error modal dialog appears ONLY if the app config is "Ask me", so changing the settings to it is redundant

Tapping "Always Send" will send the report once and change the error reporting settings to "Always"

This is correct and HA SDK does it.

Tapping "Don't Send" will send the report once and change the error reporting settings to "Never"

This not correct. Choosing not to send one specific crash report must not turn off whole error/crash reporting. HA SDK does not do it for a good reason. There would be too many users who turn off the error/crash reporting completely by rejecting sending one specific crash (being on untrusted network at the moment or whatever). The only way to turn the sending off completely is through the Settings.

comment:15 Changed 4 years ago by pavelz

@mario technical:

Limit the events to be sent to a technical reasonable amount in order not to slow down ABB or clog the user|s internet connection if a repeating error is intercepted in a loop

That could be the same rate as proposed for limiting the error modal popping, i.e. a minute. Having an error every minute is a disaster big enough already to not need receiving the same even more often.

On a second thought, even 1 minute may be unnecessarily short. Make it 5 minutes?

comment:16 Changed 4 years ago by mario

  • Description modified (diff)

The crash/error modal dialog appears ONLY if the app config is "Ask me", so changing the settings to it is redundant

Addressed.

This not correct. Choosing not to send one specific crash report must not turn off whole error/crash reporting

Interesting, I thought HA does handle this otherwise.
Addressed.

On a second thought, even 1 minute may be unnecessarily short. Make it 5 minutes?

Agreed and addressed.

Feel free to mark the ticket ready.

comment:17 Changed 4 years ago by mario

  • Description modified (diff)

comment:18 Changed 4 years ago by mario

I've rephrased this limitation slightly:

Limit duplicate events to be sent once every 5 minutes in order not to slow down ABB or clog the user's internet connection if a repeating error is intercepted in a loop

The reason for that is, that we might "lose" error events in case one single event is firing regularly (e.g. every 10 seconds) and thus prevents other events from firing due to the "one event per 5 minutes"-rule. Thus limiting only "duplicate" events might make sense. Is there still any dangers of running into firing too many events? If so I'd suggest to not only limit duplicate events to one event per 5 minutes, but also limit individual events to e.g. 10 every minute or the like.

Last edited 4 years ago by mario (previous) (diff)

comment:19 Changed 4 years ago by pavelz

This is starting to smell of overengineering. I will rephrase what i said above: the app is supposed to produce errors RARELY, which means at most once per hour of sustained usage. Firing more often than 5 minutes is a type of misbehavior which i genuinely hope for not getting through unit tests and QA.

Let's have only one timeout, please. Actually 10 minutes sounds even better than 5 minutes. Folding of duplicate events would somewhat naturally occur anyway, because the only identifier we can influence is a fixed string - fixed for all events of the same type. On the other hand, one such event represent very small packet on the wire - the event string, timestamp, app build identification - a hundred bytes at most. So sending multiple events in one shot shouldn't be a problem.

So i suggest the following logic:

  • have a backlog of event types with their count
  • upon any event, add to backlog, increment count as needed
  • if timeout expired, take 5 most common event types
  • if there is less than 5 types, fill up to 5 repeating counts of available types
  • if user denied sending (permanent or once), keep growing the backlog until app restart (negligible memory consumption)
  • upon approval (permanent or once), send, clear the backlog and reset timeout

comment:20 Changed 4 years ago by mario

  • Description modified (diff)

Sounds good to me. Addressed in the issue's description and added your proposed logic.

Note: See TracTickets for help on using tickets.