StopIteration Exception raised in notification FCGI handler

[matze]

While investigating #3209, we stumbled upon the following error raised by the notification.json FCGI backend:

2015/10/21 00:27:35 [error] 30801#0: *502601808 FastCGI sent in stderr: "Traceback (most recent call last):
  File "/usr/lib/pymodules/python2.7/flup/server/fcgi_base.py", line 558, in run
    protocolStatus, appStatus = self.server.handler(self)
  File "/usr/lib/pymodules/python2.7/flup/server/fcgi_base.py", line 1118, in handler
    result = self.application(environ, start_response)
  File "/opt/sitescripts/sitescripts/web.py", line 111, in multiplex
    try:
  File "/opt/sitescripts/sitescripts/notifications/web/notification.py", line 108, in notification
    response = _create_response(notifications, groups)
  File "/opt/sitescripts/sitescripts/notifications/web/notification.py", line 96, in _create_response
    "notifications": _get_notifications_to_send(notifications, groups)
  File "/opt/sitescripts/sitescripts/notifications/web/notification.py", line 79, in _get_notifications_to_send
    active_variant = _get_active_variant(notifications, groups)
  File "/opt/sitescripts/sitescripts/notifications/web/notification.py", line 62, in _get_active_variant
    notification = next(x for x in notifications if x["id"] == group_id)
StopIteration", client: ***, server: notification.adblockplus.org, request: "GET /notification.json?addonName=adblockplus&addonVersion=2.6.11&application=firefox&applicationVersion=41.0.2&platform=gecko&platformVersion=41.0.2&lastVersion=201509280005-2%2F1&downloadCount=4%2B HTTP/1.1", host: "notification.adblockplus.org"

There are a bunch of those in a day, per server:

$ zgrep StopIteration /var/log/nginx/error.log.1 | wc -l
145979

Interestingly they do not cause the server to report an error, it still reports them with status 200.

[sporz]

This error seems to first occur around the time some of the fcgi servers went down, about September 29. Before that date, it seems to not have existed.

[sporz]

The error persists up to today.
It seems to only affect clients that have "lastVersion=YYYYMMDDHHMM-2%2F[123]", i.e. are in one of the three test-groups.
It occurs in about half of the requests of clients with those lastVersions.

[fhd]

Looks like a sitescripts issue to me, I'll look into it.

[fhd]

OK, finally got around to actually looking into this. It's definitely a bug in the handler, here's a test that reproduces it:

  def test_inactive_notification_variant_not_returned(self):
    self.load_notifications_mock.return_value = [
      {"id": "a", "inactive": True}
    ]
    result = json.loads(notification.notification({
      "QUERY_STRING": "lastVersion=197001010000-a/1"
    }, lambda *args: None))
    self.assertEqual(len(result["notifications"]), 0)

Shouldn't be too hard to fix, I'll look into that in a bit.

[fhd]

Finally got around to fixing this, patch is under review.

[fhd]

Pushed: ​https://hg.adblockplus.org/sitescripts/rev/2c8e1fc65e8a