Opened 3 months ago

Last modified 2 months ago

#7168 new change

Introduce query string parameter "firstVersion" in resource downloads

Reported by: sporz Assignee:
Priority: P3 Milestone:
Module: Core Keywords:
Cc: fhd, sebastian, kzar, mjethani, wspee, greiner Blocked By:
Blocking: Platform: Unknown / Cross platform
Ready: yes Confidential: no
Tester: Unknown Verified working: no
Review URL(s):

Description (last modified by sporz)

Background

We want to gain insight into our users retention behavior and create the ability to do cohort-based analyses.

Initial brainstorming for this was done internally.

What to change

We recommend to add an additional query string parameter when downloading resources (filter lists, notification.json, uninstallation):

  • &firstVersion=YYYYMM(DD).
  • firstVersion would be the version-number of the first successful download of ANY resource (any filterlist or notification) within the client.
  • If there is no firstVersion recorded yet, record lastVersion as firstVersion. If it's not a new user (but one that upgraded from a previous version of Adblock Plus), add "-E" to the firstVersion string.
  • For downloads before the first known successful one, please send a single 0 (new users) or 0-E (for existing users that just upgraded).
  • We recommend sensible truncating when sending firstVersion due to privacy concerns: during the first full 30 days, 8 characters would be sent (the full date), between 31 and 365 days only 6 characters would be sent (year & month only), after that 4 characters (year only).
    • Date calculations should be independent of the clients clock.

Change History (21)

comment:1 Changed 3 months ago by sebastian

  • Cc sebastian kzar added; snoak removed

comment:2 Changed 3 months ago by kzar

  • Cc mjethani added

To submit firstVersion we'd have to keep track of it in the extension. What about for users who we didn't yet track it for, what would we submit?

comment:3 Changed 2 months ago by sporz

  • Description modified (diff)

comment:4 Changed 2 months ago by sporz

  • Updated the description to now cover this case.
  • Updated the description to now do even more truncating after 1 year.

comment:5 Changed 2 months ago by sporz

  • Description modified (diff)

comment:6 Changed 2 months ago by sporz

  • Description modified (diff)

comment:7 Changed 2 months ago by sporz

  • Description modified (diff)

comment:8 Changed 2 months ago by sebastian

Either we keep sending 0 forever for users who upgraded from a previous version, or we we pretend that the first filter list download after updating to a version of Adblock Plus that is aware of firstVersion is the first version for those users. Please clarify?

comment:9 Changed 2 months ago by sporz

You're completely right - and either has serious consequences with regards to ambiguity between users that just installed, existing users that upgraded (or upgraded late) and clients that come with a bundled list. I'll need to follow up on this.

Last edited 2 months ago by sporz (previous) (diff)

comment:10 Changed 2 months ago by wspee

  • Cc wspee added

comment:11 follow-up: Changed 2 months ago by sporz

So, I synced with Kirill and we got a couple of ideas and questions:

Goals:

  • A) For making analyses without ambiguities we should clearly differentiate long-term users that just upgraded from new users. [must have]
  • B) We would like to have workable data on all users right away without a delay (like waiting for new users). [nice to have]

Ideas:

  • a) We were thinking about adding a flag like "-E" to the string sent (like 20190101-E) in case of a long-term user that just upgraded.
  • b) Felix suggested that we already might have an "installation date" for all existing users in their clients database. So for existing users, we could try and use this "installation date" (which wouldn't need to be independent of the clients clock) for creating the string to send.
  • c) In order to check the validity of data from existing users that would send "installation date" (if it's based on the clients clock), for new users we could send it separately (truncated in the same way as firstVersion) as it's own query parameter.
  • d) This would be a temporary thing and removed one release later (when (non-)validity is established).

Questions:

  • For A) and a) - If I interpret your statement above correctly, we can already properly identify in the client if a user just installed OR if he just upgraded. Just to clarify: is this correct? We should make sure this also works when there is a filter list being bundled together with the client.
  • With regards to B) and b) - is there an installation date present on all existing clients already that we could use for this purpose? Is it based on the clients clock?
  • With regards to B) and c)+d) - would a temporary query parameter be something you'd be ok with?
Last edited 2 months ago by sporz (previous) (diff)

comment:12 in reply to: ↑ 11 Changed 2 months ago by sebastian

Replying to sporz:

  • For A) and a) - If I interpret your statement above correctly, we can already properly identify in the client if a user just installed OR if he just upgraded. Just to clarify: is this correct? We should make sure this also works when there is a filter list being bundled together with the client.

Yes, as long as we account for that in the implementation.

  • With regards to B) and b) - is there an installation date present on all existing clients already that we could use for this purpose? Is it based on the clients clock?

There is no API that provides you with the installation date of a Web Extension.

  • With regards to B) and c)+d) - would a temporary query parameter be something you'd be ok with?

I'm not sure how temporary this condition is. We will never have 100% acurrate data until every user that had Adblock Plus installed before we release this change either uninstalled or reinstalled Adblock Plus.

Last edited 2 months ago by sebastian (previous) (diff)

comment:13 Changed 2 months ago by sporz

For A) a) great =]

Clarification for B) - I'd say that using "firstVersion" for all new installations is as reliable as we can get. I don't think we need extra validation on that. I wanted to use it to validate OTHER means of getting information, for upgrading users.

For B) b) - for upgrading users, if there is any date in the client we have access to that's close to the installation date I'd be happy to use it - please suggest any. Otherwise we should create a 'delayed' "first version" for those (starting right after the upgrade), but clearly marking those users (using "20190109-E", and for their first download after upgrade when there is no firstVersion recorded yet "0-E").

I'll wait for your reply before upgrading the tickets description accordingly.

comment:14 Changed 2 months ago by sebastian

So it seems the best option we have is: If there is no firstVersion recorded yet, record lastVersion as firstVersion. If it's not a new user (but one that upgraded from a previous version of Adblock Plus), add "-E" to the firstVersion string.

comment:15 Changed 2 months ago by sporz

  • Description modified (diff)

comment:16 Changed 2 months ago by sporz

Exactly, I added your statement to the description.

comment:17 Changed 2 months ago by sporz

  • Description modified (diff)

comment:18 Changed 2 months ago by sebastian

  • Priority changed from Unknown to P3
  • Ready set

comment:19 Changed 2 months ago by greiner

  • Cc greiner added

comment:20 Changed 2 months ago by mjethani

Maybe a silly question, but how do we ensure that the date calculation is independent of the client's clock?

comment:21 Changed 2 months ago by sebastian

firstVersion and lastVersion aren't client-side timestamps, they refer to the value that was given in the Version: comment in the filter list.

Note: See TracTickets for help on using tickets.