[Downloader] Mark second downloads

[fhd]

Background

See #536.

What to change

Downloads made for a resource should add the parameter downloadCount which needs to be incremented for each download and max out at 5+.

Hints for testers

Filter subscription updates should work the same as before, without any user visible changes. The download URLs will have an additional parameter.

[greiner]

What about making this more generic and instead of a secondDownload=true add a downloadCount=12 parameter or overloading the lastVersion one (e.g. lastVersion=12/201410060850)? This increases the request's entropy and could allow us to make out usage patterns which at a later point we could use to detect abnormalities.

[trev]

I would rather not collect more data than absolutely necessary here. However, I am also afraid that anomalies might not stop at the initial download. So something like downloadCount=5+ might make sense (counting to 5 downloads and sending only 5+ after that). Larger download counts might allow recognizing individual users here and we definitely don't want that.

Note that the additional entropy is also a problem for our server-side processing, it makes caching more problematic.

[greiner]

Actually, one important detail that's missing for implementation of the downloadCount=5+ approach is where to store the counter. In the initially suggested approach the second download would be determined implicitly by checking for the last version to be zero.

Therefore I see the following options:

• Implement only for second downloads (problem: won't allow us to distinguish downloads 3-5+)
• Attach counter to downloadables (problem: we currently don't store downloadables)
• Attach counter to notification data and subscriptions (problem: problematic to reuse code)

None of these options is optimal so let me know if you can think of a better way to store the counter. Otherwise we might want to go with the original approach but for the sake of forwards-compatibility I'd recommend sticking with downloadCount instead of secondDownload.

[trev]

In the initially suggested approach the second download would be determined implicitly by checking for the last version to be zero.

No, you cannot distinguish first and second download via lastVersion.

Implement only for second downloads (problem: won't allow us to distinguish downloads 3-5+)

Won't help here, see above.

Attach counter to downloadables (problem: we currently don't store downloadables)

No, downloadables aren't stored directly. They will need the download count as an additional field however.

Attach counter to notification data and subscriptions (problem: problematic to reuse code)

Yes, that's the approach to be chosen here - same as for lastVersion and a bunch of similar fields.

[greiner]

Fixed:
​https://hg.adblockplus.org/adblockplus/rev/344b10d19064
​https://hg.adblockplus.org/adblockplustests/rev/b1fa8a6663f8