Opened 4 years ago

Closed 4 years ago

#2900 closed defect (fixed)

Revive filter27.adblockplus.org

Reported by: matze Assignee: matze
Priority: P1 Milestone:
Module: Infrastructure Keywords:
Cc: fhd, fred, Kirill, palant Blocked By:
Blocking: Platform: Unknown / Cross platform
Ready: yes Confidential: no
Tester: Unknown Verified working: no
Review URL(s):

Description

***** Nagios *****

Notification Type: PROBLEM
Host: filter27.adblockplus.org
State: DOWN
Address: filter27.adblockplus.org
Info: PING CRITICAL - Packet loss = 100%

Date/Time: Thu Aug 13 00:09:49 UTC 2015

See also #2889.

Change History (5)

comment:1 Changed 4 years ago by matze

  • Cc Kirill palant added

Since that host was behaving off the norm ever since we've set it up a few days ago, I've backed up keys and logs and just re-installed the entire box. Provisioning has finished already, but the RAID sync will take around five to six hours. The server will re-join balancing afterwards.

@palant, @Kirill
Retaining the current logs without rotating them for today should allow for processing them tomorrow without manual intervention. Any objections?

comment:2 Changed 4 years ago by matze

  • Resolution set to fixed
  • Status changed from new to closed

The host is back in balancing.

comment:3 Changed 4 years ago by matze

  • Resolution fixed deleted
  • Status changed from closed to reopened

The host went down again, without further information visible in any logs. Thus I've asked Hetzner to analyze it's hardware, awaiting results in the next hours.

@Kirill Logs from that host haven't been processed by the stats server yet, because the log rotation (which creates the .1 file downloaded) has never taken place so far. I assume the same goes for your analysis?

comment:4 Changed 4 years ago by matze

After the RAM has been replaced and the CPU thermal grease has been renewed, I've put the host back into balancing.

@palant, @Kirill
The logs have now been rotated (implicitly because Hetzner rebooted the host before the rotation last night), but it seems like they haven't been processed yet. I'd suppose they now need manual attention, as the current chunk will become rotated again before processing kicks in.

comment:5 Changed 4 years ago by matze

  • Resolution set to fixed
  • Status changed from reopened to closed
Note: See TracTickets for help on using tickets.