Set up a build server

[fhd]

We're currently running builds on the main server, in regular intervals. We should move that to a dedicated server.

And I'm pretty sure we want to use a CI server while we're at it, some reasons:

1. It will handle build scheduling for us - we can build on push and/or in regular intervals, and we can chain dependent tasks (i.e. libadblockplus build and tests, Android build)
2. We can run builds on several slaves (e.g. one for Linux, one for Windows)
3. There's a UI we can use to see which builds failed when and for what reason, and which builds are currently in progress and in which stage

What to change

Buildbot seems like the best option:

1. It is ​very popular
2. The whole configuration is in Python - highly flexible

We should set up two nodes: One for the buildbot master and a single slave - that should suffice for now. Running master and slave on the same node will make things unnecessarily messy and harder to scale.

[fhd]

Wladimir: I've deliberately not set this to ready. Please have a look and set it if you agree.

[fhd]

Was in the wrong component for some reason.

[fhd]

Wrong component _again_. I blame this horrible touchpad.

[philll]

Use the g-spot instead.

[poz2k4444]

FYI: We're currently developing the puppet module for buildbot, we have different approaches for the installation of buildbot:

1. Installing via easy_install/pip: This will fetch the 0.8.9 version, we can choose whatever version we want, there is documentation for it.

2. Installing via apt: with this approach it'll fetch the 0.8.5 version, there is documentation for this version.

3. Fork the code source and compile it by ourselves: With this approach we can get whatever version we want, but for the latest version of buildbot (0.8.10) there is no documentation available.

We already have working the first approach, we can migrate to one of the others at any time with a little bit of changes on the configuration files if we decide to change the versions.

[fhd]

IMO we should rather go with the second option: Installing from APT. That's what we do with pretty much all third party software we deploy, unless modified by us, plus stuff from APT tends to be more stable and less of a hassle in my experience.

[trev]

Are there any differences between 0.8.5 and 0.8.9 that are relevant to us? If not, then I would certainly prefer apt, already because of the better support in Puppet.

[AAlvz]

Puppet supports pip very good as well (provider => 'pip'). That's the way the buildbot module is working so far.

The main difference is that the config files for the buildbot master changes between versions, and another important difference is that we have everything working properly (& under revision) using version 0.8.9.

Getting the other version to work has been more of a hassle so far.

[fhd]

Can we please get this on Rietveld? It's way too big for GitHub, there's no reviewers listed etc.

[matze]

Sure, no problem. Just note that none of these tickets are meant to be reviewed at GitHub. We've used that internally to track and sync only. Wladimir asked a while ago to also include the links here though, for documentation reasons and "why not".

[fhd]

Ah, so it's not actually under review (in the normal sense) yet. Then nevermind :)

[fhd]

Not sure why that's a P3 - at least P2 in my book. We really need this. (Since I'm the reviewer, it would suffice for me to know that, but still :D)

[fhd]

Since we're first setting up buildbot separately, I've created #3152 for building libadblockplus with it. I still think this is the first job we should set up, because it's so simple.

[matze]

The ​first version of the Buildbot integration has been pushed to the infrastructure repository. It will act as the basis for further integration.

@fhd: There's a ​minor improvement we should introduce now, before starting to work on the slave setups.

[matze]

​https://hg.adblockplus.org/infrastructure/rev/830134f7b357