Opened 16 months ago

Closed 8 months ago

#5335 closed change (rejected)

Adding dependencies and build steps to websites

Reported by: juliandoucette Assignee:
Priority: Unknown Milestone: Websites code sharing
Module: Sitescripts Keywords:
Cc: wspee, kvas, jsonesen, ire, saroyanm, matze, ferris, juliandoucette Blocked By:
Blocking: Platform: Unknown / Cross platform
Ready: no Confidential: no
Tester: Unknown Verified working: no
Review URL(s):

Description (last modified by kvas)

Background

  • We would like to share SCSS across websites
  • We are currently building SCSS into CSS locally and pushing it to websites
    • Including SCSS from website-defaults, which we are installing locally via npm config

We would like to move the building step onto a staging and/or production server. The easiest (especially in terms of deployment) way to allow this is to make CMS able to perform pre-build and post-build steps. Then website authors could indicate additional actions (i.e. shell commands) to perform before and after generation and generate_static_pages.py would perform those actions during the generation process.

What to change

Introduce a configuration section into settings.ini of the website called [build-steps]. The section would have two configuration variables, each of which is interpreted as a list of strings (separated by newlines):

  • before-build -- commands to be run before the generation takes place (in the directory where the sources of the website are located).
  • after-build -- commands to be run after the generation takes place (in the directory where the output resides).

Interaction with preview

  • Preview should run the before-build steps before launching the web server.

Documentation

New section and options should be added to documentation. It should be emphasized that it's the responsibility of the website developers to ensure that the commands listed in the build steps are present on target production server and that they can be run by the CMS site generation server.

Example

[build-steps]
before-build=
    python ensure_dependencies.py
    npm install
    gulp scss
after-build= html-prettify -r *

Change History (14)

comment:1 Changed 16 months ago by juliandoucette

  • Description modified (diff)

comment:2 Changed 16 months ago by juliandoucette

  • Description modified (diff)

comment:3 Changed 15 months ago by ire

  • Cc ire added; iaderinokun removed

comment:4 Changed 15 months ago by kvas

  • Component changed from Infrastructure to Sitescripts
  • Description modified (diff)

comment:5 Changed 15 months ago by kvas

@juliandoucette, @saroyanm, @ire: what do you think about the implementation plan for this ticket? In particular, are you comfortable with (1) the responsibility for ensuring that the build steps are available in production (I would imagine a request to Ops should be enough for that) and (2) with the fact that anyone who would want to generate or preview a website that contains build steps would need to make sure the build steps can be run on their computer?

comment:6 Changed 15 months ago by juliandoucette

@kvas I think this is backwards. I'd rather rely on infrastructure to run npm install and gulp and call generate_static_pages or runserver (among other things) using gulp.

Last edited 15 months ago by juliandoucette (previous) (diff)

comment:7 Changed 15 months ago by juliandoucette

  • Cc juliandoucette added

comment:8 follow-up: Changed 15 months ago by kvas

That would be cool with me (since the CMS developers wouldn't need to do anything ;) but I though it might be useful, as Matze suggested, to support this in CMS, since then these steps would be controlled by website developers rather than infrastructure.

But I think what you're proposing also has advantages: it's more robust, when it's in the infrastructure, and it interacts with preview in a more manual way, which might in some cases also be an advantage.

Anyway, if you would prefer to do this in the infrastructure, we should change this ticket back to "Infrastructure" (or maybe convert to a Hub ticket) and then you should give them exact steps for the websites that need them.

comment:9 in reply to: ↑ 8 ; follow-up: Changed 15 months ago by juliandoucette

Replying to kvas:

That would be cool with me (since the CMS developers wouldn't need to do anything ;) but I though it might be useful, as Matze suggested, to support this in CMS, since then these steps would be controlled by website developers rather than infrastructure.

I think that just running gulp gives me the exact same control. Am I missing something?

But I think what you're proposing also has advantages: it's more robust, when it's in the infrastructure, and it interacts with preview in a more manual way, which might in some cases also be an advantage.

I don't know what you mean.

Anyway, if you would prefer to do this in the infrastructure, we should change this ticket back to "Infrastructure" (or maybe convert to a Hub ticket) and then you should give them exact steps for the websites that need them.

I agree, and prefer, if I'm not missing anything.

comment:10 in reply to: ↑ 9 ; follow-up: Changed 15 months ago by kvas

Replying to juliandoucette:

Replying to kvas:

But I think what you're proposing also has advantages: it's more robust, when it's in the infrastructure, and it interacts with preview in a more manual way, which might in some cases also be an advantage.

I don't know what you mean.

For the first part I mean that if you ask the Ops to run gulp for website X, they will probably make sure gulp is available there. If the CMS runs gulp, based on settings.ini, that might not be the case.

And for preview, imagine there's some long build process that takes a while, but the user knows that they don't need to run it now, because what they changed doesn't affect the build artifacts. If CMS would be taking care of build steps, they would run anyway before the preview (unless we'd be very smart about it) but if the build steps are run manually, the user will just do it when they need to.

Anyway, would you prefer to write this ticket as an infrastructure ticket yourself, or would you need some assistance from the CMS team (i.e. from me ;)?

comment:11 in reply to: ↑ 10 Changed 15 months ago by matze

For the first part I mean that if you ask the Ops to run gulp for website X, they will probably make sure gulp is available there. If the CMS runs gulp, based on settings.ini, that might not be the case.

Actually the one who operates the CMS in the current environment needs to make sure the dependencies being available anyway. Irregardless of whether those are designated in code, in the documentation, configuration, any combination of the aforementioned or even no proper information.

And for preview, imagine there's some long build process that takes a while, but the user knows that they don't need to run it now, because what they changed doesn't affect the build artifacts. If CMS would be taking care of build steps, they would run anyway before the preview (unless we'd be very smart about it) but if the build steps are run manually, the user will just do it when they need to.

We spoke about that scenario in our call and agreed that a simple flag to activate or suppress configured hooks would render this concern obsolete.


However, I strongly recommend to boil the production version down to a single command (be it gulp or something built into the CMS or whatever).

It should not be a problem if that one is for triggering a "clean" or "complete" build, even if that is not what's usually done in development: Just having the option to perform the exact same operation fulfills the associated requirements for being able to reproduce and test.

comment:12 Changed 15 months ago by juliandoucette

And for preview, imagine there's some long build process that takes a while, but the user knows that they don't need to run it now, because what they changed doesn't affect the build artifacts. If CMS would be taking care of build steps, they would run anyway before the preview (unless we'd be very smart about it) but if the build steps are run manually, the user will just do it when they need to.

Exactly. Using gulp, we can break down the build steps into pieces and run them individually and in different combinations (the default being a complete build).

Anyway, would you prefer to write this ticket as an infrastructure ticket yourself, or would you need some assistance from the CMS team (i.e. from me ;)?

I don't think I need your help.

comment:13 Changed 14 months ago by juliandoucette

  • Milestone set to Websites code sharing

comment:14 Changed 8 months ago by juliandoucette

  • Resolution set to rejected
  • Status changed from new to closed

This is irrelevant now that we are addressing this and related issues separately in eyeo hub.

Note: See TracTickets for help on using tickets.