Opened on 08/28/2014 at 08:06:14 AM

Last modified on 09/05/2019 at 11:35:13 AM

#1281 reviewing change

Set up a build server

Reported by: fhd Assignee: matze
Priority: P2 Milestone:
Module: Infrastructure Keywords:
Cc: fhd, sergz, kvas Blocked By:
Blocking: #3151, #3152 Platform: Unknown
Ready: yes Confidential: no
Tester: Unknown Verified working: no
Review URL(s):

Description (last modified by matze)

We're currently running builds on the main server, in regular intervals. We should move that to a dedicated server.

And I'm pretty sure we want to use a CI server while we're at it, some reasons:

  1. It will handle build scheduling for us - we can build on push and/or in regular intervals, and we can chain dependent tasks (i.e. libadblockplus build and tests, Android build)
  2. We can run builds on several slaves (e.g. one for Linux, one for Windows)
  3. There's a UI we can use to see which builds failed when and for what reason, and which builds are currently in progress and in which stage

What to change

Buildbot seems like the best option:

  1. It is very popular
  2. The whole configuration is in Python - highly flexible

We should set up two nodes: One for the buildbot master and a single slave - that should suffice for now. Running master and slave on the same node will make things unnecessarily messy and harder to scale.

Attachments (0)

Change History (30)

comment:1 Changed on 08/28/2014 at 08:07:15 AM by fhd

  • Description modified (diff)

comment:2 Changed on 08/28/2014 at 08:08:49 AM by fhd

  • Cc trev added

Wladimir: I've deliberately not set this to ready. Please have a look and set it if you agree.

comment:3 Changed on 08/28/2014 at 08:09:19 AM by fhd

  • Component changed from Extensions-for-Adblock-Plus to Libadblockplus

Was in the wrong component for some reason.

comment:4 Changed on 08/28/2014 at 08:13:58 AM by fhd

  • Component changed from Libadblockplus to Infrastructure

Wrong component _again_. I blame this horrible touchpad.

comment:5 Changed on 08/28/2014 at 08:18:59 AM by philll

Use the g-spot instead.

comment:6 Changed on 08/28/2014 at 08:21:01 AM by matze

  • Cc added

comment:7 Changed on 08/28/2014 at 03:22:34 PM by trev

  • Ready set

comment:8 Changed on 10/13/2014 at 10:10:55 PM by fhd

  • Owner set to fhd

comment:9 Changed on 10/17/2014 at 09:18:29 AM by fhd

  • Owner fhd deleted

comment:10 Changed on 11/18/2014 at 02:47:12 PM by AAlvz

  • Owner set to AAlvz, poz2k4444

comment:11 Changed on 01/06/2015 at 11:30:39 AM by poz2k4444

FYI: We're currently developing the puppet module for buildbot, we have different approaches for the installation of buildbot:

  1. Installing via easy_install/pip: This will fetch the 0.8.9 version, we can choose whatever version we want, there is documentation for it.
  1. Installing via apt: with this approach it'll fetch the 0.8.5 version, there is documentation for this version.
  1. Fork the code source and compile it by ourselves: With this approach we can get whatever version we want, but for the latest version of buildbot (0.8.10) there is no documentation available.

We already have working the first approach, we can migrate to one of the others at any time with a little bit of changes on the configuration files if we decide to change the versions.

comment:12 Changed on 01/06/2015 at 01:35:12 PM by fhd

IMO we should rather go with the second option: Installing from APT. That's what we do with pretty much all third party software we deploy, unless modified by us, plus stuff from APT tends to be more stable and less of a hassle in my experience.

comment:13 Changed on 01/06/2015 at 01:37:31 PM by trev

Are there any differences between 0.8.5 and 0.8.9 that are relevant to us? If not, then I would certainly prefer apt, already because of the better support in Puppet.

comment:14 Changed on 01/16/2015 at 10:46:38 AM by AAlvz

Puppet supports pip very good as well (provider => 'pip'). That's the way the buildbot module is working so far.

The main difference is that the config files for the buildbot master changes between versions, and another important difference is that we have everything working properly (& under revision) using version 0.8.9.

Getting the other version to work has been more of a hassle so far.

comment:15 Changed on 01/16/2015 at 10:47:18 AM by AAlvz

  • Review URL(s) modified (diff)

comment:16 Changed on 02/23/2015 at 11:11:10 AM by poz2k4444

  • Cc removed
  • Owner changed from AAlvz, poz2k4444 to matze

comment:17 Changed on 02/23/2015 at 09:52:32 PM by fhd

Can we please get this on Rietveld? It's way too big for GitHub, there's no reviewers listed etc.

comment:18 Changed on 02/23/2015 at 09:52:57 PM by fhd

  • Cc fhd added

comment:19 Changed on 02/24/2015 at 07:53:42 AM by matze

Sure, no problem. Just note that none of these tickets are meant to be reviewed at GitHub. We've used that internally to track and sync only. Wladimir asked a while ago to also include the links here though, for documentation reasons and "why not".

comment:20 Changed on 02/24/2015 at 10:38:19 AM by fhd

Ah, so it's not actually under review (in the normal sense) yet. Then nevermind :)

comment:21 Changed on 09/18/2015 at 09:45:13 AM by matze

  • Review URL(s) modified (diff)
  • Status changed from new to reviewing
  • Tester set to Unknown

comment:22 Changed on 09/18/2015 at 09:45:38 AM by matze

  • Blocking 1092 added

comment:23 Changed on 10/02/2015 at 11:22:19 AM by fhd

  • Blocking 3151 added

comment:24 Changed on 10/02/2015 at 11:32:05 AM by fhd

  • Priority changed from P3 to P2

Not sure why that's a P3 - at least P2 in my book. We really need this. (Since I'm the reviewer, it would suffice for me to know that, but still :D)

comment:25 Changed on 10/02/2015 at 12:27:07 PM by fhd

  • Blocking 3152 added

comment:26 Changed on 10/02/2015 at 12:28:46 PM by fhd

  • Description modified (diff)

Since we're first setting up buildbot separately, I've created #3152 for building libadblockplus with it. I still think this is the first job we should set up, because it's so simple.

comment:27 Changed on 02/20/2016 at 01:31:17 PM by matze

  • Cc sergz kvas added
  • Description modified (diff)
  • Review URL(s) modified (diff)

The first version of the Buildbot integration has been pushed to the infrastructure repository. It will act as the basis for further integration.

@fhd: There's a minor improvement we should introduce now, before starting to work on the slave setups.

Last edited on 02/20/2016 at 01:31:39 PM by matze

comment:28 Changed on 02/20/2016 at 05:05:00 PM by matze

comment:29 Changed on 04/28/2017 at 09:47:37 AM by fhd

  • Blocking 1092 removed

comment:30 Changed on 12/21/2017 at 11:25:42 AM by fhd

  • Cc trev removed

Add Comment

Modify Ticket

Change Properties
as reviewing .
as The resolution will be set. Next status will be 'closed'.
The resolution will be deleted. Next status will be 'reopened'.
to The owner will be changed from matze.
Note: See TracTickets for help on using tickets.