Opened 5 years ago

Closed 4 years ago

Last modified 4 years ago

#1299 closed change (fixed)

Generate docs outside the devbuild build process

Reported by: fhd Assignee: fhd
Priority: P2 Milestone:
Module: Sitescripts Keywords:
Cc: trev, sebastian Blocked By: #116
Blocking: #3621, #3637 Platform: Unknown
Ready: yes Confidential: no
Tester: Unknown Verified working: no
Review URL(s):

http://codereview.adblockplus.org/5396204603047936/ (Abandoned)
https://codereview.adblockplus.org/29335805/

Description (last modified by fhd)

Background

We are currently generating the docs for adblockplus and adblockpluschrome in createNightlies.py, i.e. as part of the devbuild build process. That's not the best place for it, because we also need to generate docs for libadblockplus and adblockpluscore (see #3618).

What to change

Trigger doc generation through a separate cron job or post-receive hook, not through createNightlies.py. This should generate docs for adblockplus, adblockpluschrome, adblockpluscore, and libadblockplus (https://adblockplus.org/docs/libadblockplus/).

Things to consider:

  1. We should use hg archive to clone the repositories, it's faster.
  2. We shouldn't remove the clones after the script, but keep it around and update it instead.
  3. We should keep the clone on the same volume as the target dir (i.e. /srv) to ensure the mv is fast enough.
  4. We should move the target docs dir and replace it before removing it - will reduce docs downtime.

Change History (18)

comment:1 Changed 5 years ago by fhd

  • Blocked By 116 added

comment:2 Changed 5 years ago by fhd

  • Review URL(s) modified (diff)
  • Status changed from new to reviewing

comment:3 Changed 5 years ago by fhd

  • Status changed from reviewing to reopened

Reopening, the approach in the review was too hacky. We should do this in sitescripts.

Notes from the review:

  1. We should use hg archive for the clone, it's faster.
  2. We shouldn't remove the clone after the script, but keep it around and update it instead.
  3. We should keep the clone on the same volume (i.e. /srv) to ensure the mv is fast enough.
  4. We should move the target docs dir and replace it before removing it - will reduce docs downtime.

comment:4 Changed 5 years ago by fhd

  • Description modified (diff)

comment:5 Changed 5 years ago by trev

  • Priority changed from P2 to P4

comment:6 Changed 4 years ago by sebastian

  • Component changed from Infrastructure to Sitescripts

comment:7 Changed 4 years ago by fhd

  • Cc sebastian added
  • Description modified (diff)
  • Priority changed from P4 to P2
  • Tester set to Unknown

This is getting important now, because we also need it for the adblockpluscore documentation.

comment:8 Changed 4 years ago by fhd

  • Blocking 3621 added

comment:9 Changed 4 years ago by fhd

  • Summary changed from Generate the libadblockplus API documentation on adblockplus.org to Generate docs outside the devbuild build process

comment:10 Changed 4 years ago by fhd

  • Owner set to fhd

comment:11 Changed 4 years ago by fhd

  • Review URL(s) modified (diff)
  • Status changed from reopened to reviewing

comment:12 Changed 4 years ago by sebastian

  • Blocked By 3635 added

comment:13 Changed 4 years ago by fhd

  • Blocking 3637 added

comment:14 follow-up: Changed 4 years ago by fhd

Sebastian: I don't think #3635 should block this one. Even if we change ensure_dependencies.py, we cannot update it in ABP for Firefox since that one is still in code freeze until #3621 lands, which is blocked by this issue here. I think we should land this here first, then fix #3635, then remove the hack introduced here.

BUT: I just realised we have to run this on server16 if we stick to the hg archive (see #3637), which is bad. So I think it would be best to just use hg clone and hg update in this script, that would make #3635 irrelevant here. What do you think?

comment:15 Changed 4 years ago by sebastian

  • Blocked By 3635 removed

comment:16 in reply to: ↑ 14 Changed 4 years ago by sebastian

Replying to fhd:

Sebastian: I don't think #3635 should block this one. Even if we change ensure_dependencies.py, we cannot update it in ABP for Firefox since that one is still in code freeze until #3621 lands, which is blocked by this issue here. I think we should land this here first, then fix #3635, then remove the hack introduced here.

I agree. When I first triaged #3635 I assumed that we are still going to use hg archive but rather want to update the repositories than going with the workaround. I updated the issue.

comment:17 Changed 4 years ago by fhd

  • Resolution set to fixed
  • Status changed from reviewing to closed
Note: See TracTickets for help on using tickets.