Opened 3 years ago

Last modified 8 weeks ago

#1281 reviewing change

Set up a build server

Reported by: fhd Assignee: matze
Priority: P2 Milestone:
Module: Infrastructure Keywords:
Cc: trev, fhd, sergz, kvas Blocked By:
Blocking: #3151, #3152 Platform: Unknown
Ready: yes Confidential: no
Tester: Unknown Verified working: no
Review URL(s):

https://codereview.adblockplus.org/29325436/
https://codereview.adblockplus.org/29336742/

Description (last modified by matze)

We're currently running builds on the main server, in regular intervals. We should move that to a dedicated server.

And I'm pretty sure we want to use a CI server while we're at it, some reasons:

  1. It will handle build scheduling for us - we can build on push and/or in regular intervals, and we can chain dependent tasks (i.e. libadblockplus build and tests, Android build)
  2. We can run builds on several slaves (e.g. one for Linux, one for Windows)
  3. There's a UI we can use to see which builds failed when and for what reason, and which builds are currently in progress and in which stage

What to change

Buildbot seems like the best option:

  1. It is very popular
  2. The whole configuration is in Python - highly flexible

We should set up two nodes: One for the buildbot master and a single slave - that should suffice for now. Running master and slave on the same node will make things unnecessarily messy and harder to scale.

Change History (29)

comment:1 Changed 3 years ago by fhd

  • Description modified (diff)

comment:2 Changed 3 years ago by fhd

  • Cc trev added

Wladimir: I've deliberately not set this to ready. Please have a look and set it if you agree.

comment:3 Changed 3 years ago by fhd

  • Component changed from Extensions-for-Adblock-Plus to Libadblockplus

Was in the wrong component for some reason.

comment:4 Changed 3 years ago by fhd

  • Component changed from Libadblockplus to Infrastructure

Wrong component _again_. I blame this horrible touchpad.

comment:5 Changed 3 years ago by philll

Use the g-spot instead.

comment:6 Changed 3 years ago by matze

  • Cc mathias@… added

comment:7 Changed 3 years ago by trev

  • Ready set

comment:8 Changed 3 years ago by fhd

  • Owner set to fhd

comment:9 Changed 3 years ago by fhd

  • Owner fhd deleted

comment:10 Changed 3 years ago by AAlvz

  • Owner set to AAlvz, poz2k4444

comment:11 Changed 2 years ago by poz2k4444

FYI: We're currently developing the puppet module for buildbot, we have different approaches for the installation of buildbot:

  1. Installing via easy_install/pip: This will fetch the 0.8.9 version, we can choose whatever version we want, there is documentation for it.
  1. Installing via apt: with this approach it'll fetch the 0.8.5 version, there is documentation for this version.
  1. Fork the code source and compile it by ourselves: With this approach we can get whatever version we want, but for the latest version of buildbot (0.8.10) there is no documentation available.

We already have working the first approach, we can migrate to one of the others at any time with a little bit of changes on the configuration files if we decide to change the versions.

comment:12 Changed 2 years ago by fhd

IMO we should rather go with the second option: Installing from APT. That's what we do with pretty much all third party software we deploy, unless modified by us, plus stuff from APT tends to be more stable and less of a hassle in my experience.

comment:13 Changed 2 years ago by trev

Are there any differences between 0.8.5 and 0.8.9 that are relevant to us? If not, then I would certainly prefer apt, already because of the better support in Puppet.

comment:14 Changed 2 years ago by AAlvz

Puppet supports pip very good as well (provider => 'pip'). That's the way the buildbot module is working so far.

The main difference is that the config files for the buildbot master changes between versions, and another important difference is that we have everything working properly (& under revision) using version 0.8.9.

Getting the other version to work has been more of a hassle so far.

comment:15 Changed 2 years ago by AAlvz

  • Review URL(s) modified (diff)

comment:16 Changed 2 years ago by poz2k4444

  • Cc mathias@… removed
  • Owner changed from AAlvz, poz2k4444 to matze

comment:17 Changed 2 years ago by fhd

Can we please get this on Rietveld? It's way too big for GitHub, there's no reviewers listed etc.

comment:18 Changed 2 years ago by fhd

  • Cc fhd added

comment:19 Changed 2 years ago by matze

Sure, no problem. Just note that none of these tickets are meant to be reviewed at GitHub. We've used that internally to track and sync only. Wladimir asked a while ago to also include the links here though, for documentation reasons and "why not".

comment:20 Changed 2 years ago by fhd

Ah, so it's not actually under review (in the normal sense) yet. Then nevermind :)

comment:21 Changed 21 months ago by matze

  • Review URL(s) modified (diff)
  • Status changed from new to reviewing
  • Tester set to Unknown

comment:22 Changed 21 months ago by matze

  • Blocking 1092 added

comment:23 Changed 21 months ago by fhd

  • Blocking 3151 added

comment:24 Changed 21 months ago by fhd

  • Priority changed from P3 to P2

Not sure why that's a P3 - at least P2 in my book. We really need this. (Since I'm the reviewer, it would suffice for me to know that, but still :D)

comment:25 Changed 21 months ago by fhd

  • Blocking 3152 added

comment:26 Changed 21 months ago by fhd

  • Description modified (diff)

Since we're first setting up buildbot separately, I've created #3152 for building libadblockplus with it. I still think this is the first job we should set up, because it's so simple.

comment:27 Changed 16 months ago by matze

  • Cc sergz kvas added
  • Description modified (diff)
  • Review URL(s) modified (diff)

The first version of the Buildbot integration has been pushed to the infrastructure repository. It will act as the basis for further integration.

@fhd: There's a minor improvement we should introduce now, before starting to work on the slave setups.

Last edited 16 months ago by matze (previous) (diff)

comment:29 Changed 8 weeks ago by fhd

  • Blocking 1092 removed
Note: See TracTickets for help on using tickets.