Failure on missing .git/info folder

[kzlomek]

Environment

Issue was lately seen with Gitlab CI dedicated runner in libadblockplus-android project.

How to reproduce

1. Checkout code from ​https://gitlab.com/eyeo/adblockplus/libadblockplus-android
2. Run ensure_dependencies.py script

Observed behaviour

Script fails with:

INFO: Cloning repository https://github.com/adblockplus/adblockpluscore into ./adblockpluscore
Traceback (most recent call last):
  File "./ensure_dependencies.py", line 380, in <module>
    resolve_deps(repo)
  File "./ensure_dependencies.py", line 323, in resolve_deps
    ensure_repo(repodir, parenttype, target, vcs, _root.get(vcs, ''), source)
  File "./ensure_dependencies.py", line 268, in ensure_repo
    repo_types[parenttype].ignore(target, parentrepo)
  File "./ensure_dependencies.py", line 130, in ignore
    _ensure_line_exists(exclude_file, module)
  File "./ensure_dependencies.py", line 355, in _ensure_line_exists
    with open(path, 'a+') as f:
IOError: [Errno 2] No such file or directory: './.git/info/exclude'

Expected behaviour

Script ensure_dependencies.py works without an error.

[sebastian]

We ran into the same issue when setting up the CI for adblockpluscore and adblockpluschrome. IIRC, this is a bug with GitLab, and the canonical workaround is ​to create the missing folder before calling ensure_dependencies.py, or at least this is what we did there.

[kzlomek]

The same is for us, we have several Gitlab CI projects and cofnigurations and in every one of them we need to repeat this step (mkdir -p .git/info).
Is there some case when lack of .git/info folder should make this script to fail instead of proceeding without parsing exclude file? My point is if this workaround is not dangerous I'd vote for putting that into ensure_dependencies.py instead of repeating the same workaround in projects which uses this script.

[sebastian]

Tristan, what do you think?

[tlucas]

Since we eventually want to get rid of ensure_dependencies.py (at least that's what i interpret from our current direction), i'd rather not add a new feature + update the dependencies everywhere.

In order to not having to adjust every single .yml file with some boilerplate code, i guess we could use ​gitlab's include feature, with a subproject or something alike.

[kzar]

Since we eventually want to get rid of ensure_dependencies.py...

I'm not sure that's true, ensure_dependencies.py works pretty well for me.

[tlucas]

Replying to kzar:

Since we eventually want to get rid of ensure_dependencies.py...

I'm not sure that's true, ensure_dependencies.py works pretty well for me.

If we indeed decide to keep ensure_dependencies.py, despite our efforts to get rid of almost everything python-related - then yes, having ensure_dependencies.py check for .git/info/exclude might be the way to go. But i feel the overall course is to have either one repository or - on the long run - use sub repositories instead (which would solve this issue as a side-effect).

What do you guys think?

[sebastian]

We are currently in the progress of rewriting our build scripts in JavaScript. Eventually we want to get rid of the Python dependency altogether, and therefore also of ensure_dependencies.py. Moreover, once we completely migrated to Git and abandoned the Mercurial repository, there is no reason anymore to not just use submodules. We'd still have to call npm install in the adblockplusui submodule (which ensure_dependencies.py currently takes care of), but that can be done as well by our build scripts.

Still I have no strong opinion whether it's worth to address this issue in ensure_dependencies.py for the time being, or whether it rather belongs into .gitlab-ci.yml in the first place.

[tlucas]

Replying to sebastian:

We are currently in the progress of rewriting our build scripts in JavaScript. Eventually we want to get rid of the Python dependency altogether, and therefore also of ensure_dependencies.py. Moreover, once we completely migrated to Git and abandoned the Mercurial repository, there is no reason anymore to not just use submodules. We'd still have to call npm install in the adblockplusui submodule (which ensure_dependencies.py currently takes care of), but that can be done as well by our build scripts.

Still I have no strong opinion whether it's worth to address this issue in ensure_dependencies.py for the time being, or whether it rather belongs into .gitlab-ci.yml in the first place.

That given, i strongly favor simply adding that one line into the .gitlab-ci.yml file, which is (most likely) subject to more changes anyway.

[sebastian]

Fine with me.