Missing error handling in updateMalwareDomainsList

[ferris]

Environment

Cron <rsync@filtermaster2> python -m sitescripts.subscriptions.bin.updateMalwareDomainsList

How to reproduce

This happens regularly on the filtermasters. I'm guessing it can be reproduced by mocking one of the malwaredomain servers to be down.

Observed behaviour

We're getting these alerts occasionally:

Traceback (most recent call last):
  File "/usr/lib/python2.7/runpy.py", line 162, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/usr/lib/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "/opt/sitescripts/sitescripts/subscriptions/bin/updateMalwareDomainsList.py", line 63, in <module>
    data = try_mirror(mirror)
  File "/opt/sitescripts/sitescripts/subscriptions/bin/updateMalwareDomainsList.py", line 40, in try_mirror
    response = urllib2.urlopen(mirror + MALWAREDOMAINS_PATH)
  File "/usr/lib/python2.7/urllib2.py", line 126, in urlopen
    return _opener.open(url, data, timeout)
  File "/usr/lib/python2.7/urllib2.py", line 400, in open
    response = self._open(req, data)
  File "/usr/lib/python2.7/urllib2.py", line 418, in _open
    '_open', req)
  File "/usr/lib/python2.7/urllib2.py", line 378, in _call_chain
    result = func(*args)
  File "/usr/lib/python2.7/urllib2.py", line 1207, in http_open
    return self.do_open(httplib.HTTPConnection, req)
  File "/usr/lib/python2.7/urllib2.py", line 1177, in do_open
    raise URLError(err)
urllib2.URLError: <urlopen error [Errno 110] Connection timed out>

Expected behaviour

I'd expect this error to be caught, and that the next malwaredomain server would be attempted.

In case all the mirrors are failing, it would also be good to get the reasons why, instead of just a message saying 'Unable to fetch malware domains list.'.

[abpbot]

A commit referencing this issue has landed:
​Fixes 4784 - Improve error reporting in updateMalwareDomainsList and add tests