Opened 4 years ago

Last modified 16 months ago

#1501 new defect

Filter lists are not always downloaded and are not re-downloaded.

Reported by: sergz Assignee:
Priority: Unknown Milestone:
Module: Libadblockplus Keywords:
Cc: fhd, mapx Blocked By:
Blocking: Platform: Unknown
Ready: no Confidential: no
Tester: Verified working: no
Review URL(s):

Description

Environment

The revision from which libadblockplus is built is 4872480b63c3 (the latest currently). I observe it in abpshell as well as in IE plugin.

How to reproduce

It's not easy to reproduce but when it happens it's very annoying. So, for now there are no concrete steps.
To reproduce the re-downloading trouble:

  • disable internet connection
  • launch abpshell
  • exit from abpshell
  • enable internet connection
  • launch abpshell
  • try to match some known url

Observed behaviour

It does not match any rule and filters are not downloaded.
The content of patterns.ini is

  • from abpshell
    # Adblock Plus preferences
    version=4
    
    [Subscription]
    url=https://easylist-downloads.adblockplus.org/easylist.txt
    title=EasyList
    homepage=https://easylist.adblockplus.org/
    lastDownload=1414506464
    downloadStatus=synchronize_connection_error
    errors=1
    
  • from IE (exception rules are not downloaded)
    # Adblock Plus preferences
    version=4
    
    [Subscription]
    url=https://easylist-downloads.adblockplus.org/easylist.txt
    title=EasyList
    fixedTitle=true
    homepage=https://easylist.adblockplus.org/
    lastDownload=1414493512
    downloadStatus=synchronize_ok
    lastSuccess=1414493512
    lastCheck=1414505459
    expires=1415184712
    softExpiration=1414834921
    version=201410281040
    requiredVersion=2.0
    
    [Subscription filters]
    ! Last modified: 28 Oct 2014 10:40 UTC
    
    ....
    
    @@||b10f.jp/flv2/player_1280x1024.swf?*&movieurl=http://ads.b10f.jp/flv/$object,~third-party
    ! *** easylist:easylist_adult/adult_whitelist_popup.txt ***
    @@||imagebam.com/image/$popup
    
    [Subscription]
    url=https://easylist-downloads.adblockplus.org/exceptionrules.txt
    title=https://easylist-downloads.adblockplus.org/exceptionrules.txt
    

Expected behaviour

If filter list is not completely downloaded it should download filter list again and match should report that information is not available instead of "No match".

Change History (5)

comment:1 Changed 4 years ago by mapx

  • Cc mapx added

comment:2 Changed 4 years ago by trev

The retry interval after a download failure is one day - your abpshell won't redownload immediately, that's by design. The reason is: if the download failure is due to our servers really being unreachable, then as soon as they are online again they will be bombarded with requests. Everybody who didn't manage to reach the servers during the downtime will immediately try to redownload, causing a massive traffic spike and likely overloading the servers. We've had this situation before, and we definitely don't want to deal with it again.

The specific scenario where there is no internet connection should be addressed by #271 (would need to propagate up to MSIE of course).

So from what I can tell, this issue should be resolved as "rejected."

comment:3 Changed 4 years ago by sergz

One day delay is clear. But it should not behave like the everything is OK. When we test whether some request matches the filter entry the result should also contain the information about broken subscriptions or at least there should be a way to query subscription status.

comment:4 Changed 4 years ago by fhd

Yeah, I can see what you mean. How I see it, we have two sorts of error here:

  1. A subscription download failed.
  2. There are no filters at all. (Probably mostly caused by subscription downloads failing, but could also be the user removing all subscriptions.)

IMO, neither of this is always a problem and something we should warn the user about. But in combination, it's most definitely a problem worth mentioning.

So, when a subscription download fails, we could have code that checks if the subscription was ever downloaded successfully, and if not (and maybe if it's more than, say, 14 days old), invokes some callback clients can react to.

Seems like something we should have in Core actually - I don't think we tackle this particular problem on any platform so far.

comment:5 Changed 16 months ago by fhd

  • Cc trev removed
Note: See TracTickets for help on using tickets.