Filter lists are not always downloaded and are not re-downloaded.

[sergz]

Environment

The revision from which libadblockplus is built is 4872480b63c3 (the latest currently). I observe it in abpshell as well as in IE plugin.

How to reproduce

It's not easy to reproduce but when it happens it's very annoying. So, for now there are no concrete steps.
To reproduce the re-downloading trouble:

• disable internet connection
• launch abpshell
• exit from abpshell
• enable internet connection
• launch abpshell
• try to match some known url

Observed behaviour

It does not match any rule and filters are not downloaded.
The content of patterns.ini is

• from abpshell

  # Adblock Plus preferences
  version=4
  
  [Subscription]
  url=https://easylist-downloads.adblockplus.org/easylist.txt
  title=EasyList
  homepage=https://easylist.adblockplus.org/
  lastDownload=1414506464
  downloadStatus=synchronize_connection_error
  errors=1
  

• from IE (exception rules are not downloaded)

  # Adblock Plus preferences
  version=4
  
  [Subscription]
  url=https://easylist-downloads.adblockplus.org/easylist.txt
  title=EasyList
  fixedTitle=true
  homepage=https://easylist.adblockplus.org/
  lastDownload=1414493512
  downloadStatus=synchronize_ok
  lastSuccess=1414493512
  lastCheck=1414505459
  expires=1415184712
  softExpiration=1414834921
  version=201410281040
  requiredVersion=2.0
  
  [Subscription filters]
  ! Last modified: 28 Oct 2014 10:40 UTC
  
  ....
  
  @@||b10f.jp/flv2/player_1280x1024.swf?*&movieurl=http://ads.b10f.jp/flv/$object,~third-party
  ! *** easylist:easylist_adult/adult_whitelist_popup.txt ***
  @@||imagebam.com/image/$popup
  
  [Subscription]
  url=https://easylist-downloads.adblockplus.org/exceptionrules.txt
  title=https://easylist-downloads.adblockplus.org/exceptionrules.txt
  

Expected behaviour

If filter list is not completely downloaded it should download filter list again and match should report that information is not available instead of "No match".

[trev]

The retry interval after a download failure is one day - your abpshell won't redownload immediately, that's by design. The reason is: if the download failure is due to our servers really being unreachable, then as soon as they are online again they will be bombarded with requests. Everybody who didn't manage to reach the servers during the downtime will immediately try to redownload, causing a massive traffic spike and likely overloading the servers. We've had this situation before, and we definitely don't want to deal with it again.

The specific scenario where there is no internet connection should be addressed by #271 (would need to propagate up to MSIE of course).

So from what I can tell, this issue should be resolved as "rejected."

[sergz]

One day delay is clear. But it should not behave like the everything is OK. When we test whether some request matches the filter entry the result should also contain the information about broken subscriptions or at least there should be a way to query subscription status.

[fhd]

Yeah, I can see what you mean. How I see it, we have two sorts of error here:

1. A subscription download failed.
2. There are no filters at all. (Probably mostly caused by subscription downloads failing, but could also be the user removing all subscriptions.)

IMO, neither of this is always a problem and something we should warn the user about. But in combination, it's most definitely a problem worth mentioning.

So, when a subscription download fails, we could have code that checks if the subscription was ever downloaded successfully, and if not (and maybe if it's more than, say, 14 days old), invokes some callback clients can react to.

Seems like something we should have in Core actually - I don't think we tackle this particular problem on any platform so far.