Opened 2 years ago

Last modified 2 years ago

#5336 closed change

Allow additional include, page, and template paths using CMS — at Version 7

Reported by: juliandoucette Assignee:
Priority: P2 Milestone: Websites code sharing
Module: Sitescripts Keywords:
Cc: wspee, kvas, jsonesen, ire, saroyanm, juliandoucette Blocked By:
Blocking: Platform: Unknown / Cross platform
Ready: yes Confidential: no
Tester: Unknown Verified working: no
Review URL(s):

https://codereview.adblockplus.org/29555839/

Description (last modified by kvas)

Background

  • We would like to share code across websites
  • If I am able to add dependencies to websites (see #5335)
    • Then I could also add generic includes, pages, and templates to website-defaults
      • If CMS would allow me to load from additional paths

So we need to add a possibility to add additional source paths to the standard paths that the CMS uses (${siteroot}/pages, ${siteroot}/globals, etc).

What to change

Add a section to settings.ini called [extra-paths] with the following variables:

  • includes -- list of strings, separated by newlines, each of which is an additional path where includes can be loaded from,
  • pages -- same for content pages,
  • templates -- same for templates,
  • filters -- same for Jinja filters,
  • globals -- same for Jinja globals.

Notes

  • The additional paths should apply to generation and preview equally.
  • Currently generation uses Mercurial a committed revision (last revision of default branch or something else if specified) instead of the directory content as a source. The directories added by extra-paths option would need to refer to the local file system, not to the repository (usually the wouldn't exist in the repository). This further complicates current confusing behavior but should become ok once we uncomplicate that (TODO).

Example

If I specify the following in settings.ini:

[extra-paths]
includes=
    node_modules/website-defaults/includes
    other-website/includes
filters=node_modules/website-defaults/includes

Then I:
`<? include example ?>`

Then:
1. CMS should check `includes/examlpe`
2. If it doesn't find `example` there then it should check the paths specified in `extra-paths/includes`, if any.

Change History (7)

comment:1 Changed 2 years ago by juliandoucette

  • Description modified (diff)

I forgot about globals :D


Real world example: If implemented, I could define a standard set of meta data fields inside includes in website-defaults and share these includes across websites (if I am able to add dependencies - #5335) instead of duplicating them in each website repository.

comment:2 follow-up: Changed 2 years ago by kvas

While I agree that code sharing across websites would be a nice feature (and the practice makes sense to me), I feel uneasy about the feature as it is specified now. The problem here is that we would start relying on some mechanisms outside of control of CMS (and the version control system) to get those dependencies into place and this makes the builds not reproducible. To have reliable code sharing we'd need to add something like dependencies between websites (or website depending on repository with website fragments). How to do this without reinventing too many wheels is an interesting question.

One approach would be to rely on #5335 for getting the dependencies into place. If we have build steps, we could add a pre-build step that will clone the repositories that we depend on into a subfolder and then we would know that they will be there and the build is repeatable. This is still a bit fragile for my taste, and it's not completely clear how this would interact with the preview feature of CMS (although preview interaction needs to be solved for #5335 to work anyway). What I like about this approach, though, is that it's flexible for the website developers and doesn't add new concepts to the CMS.

Another option would be to add the notion of dependencies between websites directly to CMS. Then CMS would know that it needs to clone some other repos during the build, put them somewhere and make the globals/includes/pages/templates from them available. This seems a bit more robust and easier to use, but then CMS would need to know even more about Mercurial (or potentially also Git and God knows what in the future) and that seems like too much stuff to bring into the CMS domain.

Does anybody have any other ideas or thoughts?

comment:3 Changed 2 years ago by wspee

Another consideration might be a potential staging system. It makes sense to be able to setup a production like staging system of a future version for testing or similar. Currently this is not possible, which means we cannot test dynamic functionality, which means dynamic stuff is likely to break, see for example #5004.

It seems like it might be good idea to incorporate all this aspects into a joint effort?!

comment:4 in reply to: ↑ 2 ; follow-up: Changed 2 years ago by juliandoucette

Replying to kvas:

Does it make you more comfortable if I remove the fallback requirement (not searching one directory and then another)?

comment:5 in reply to: ↑ 4 Changed 2 years ago by kvas

Replying to juliandoucette:

Replying to kvas:

Does it make you more comfortable if I remove the fallback requirement (not searching one directory and then another)?

Not really. I think the searching actually makes sense this way but we need to ensure that the content of those directories is the same in development, testing, production, etc. without requiring that someone maintains them so. Ideally we would like adding and removing dependencies to a website to be transparent.

comment:6 Changed 2 years ago by ire

  • Cc ire added; iaderinokun removed

comment:7 Changed 2 years ago by kvas

  • Description modified (diff)
Note: See TracTickets for help on using tickets.