Opened 4 months ago

Closed 3 months ago

#7186 closed defect (rejected)

Failure on missing .git/info folder

Reported by: kzlomek Assignee:
Priority: Unknown Milestone:
Module: Automation Keywords:
Cc: sebastian, tlucas Blocked By:
Blocking: Platform: Unknown / Cross platform
Ready: no Confidential: no
Tester: Unknown Verified working: no
Review URL(s):

https://github.com/adblockplus/buildtools/pull/8

Description

Environment

Issue was lately seen with Gitlab CI dedicated runner in libadblockplus-android project.

How to reproduce

  1. Checkout code from https://gitlab.com/eyeo/adblockplus/libadblockplus-android
  2. Run ensure_dependencies.py script

Observed behaviour

Script fails with:

INFO: Cloning repository https://github.com/adblockplus/adblockpluscore into ./adblockpluscore
Traceback (most recent call last):
  File "./ensure_dependencies.py", line 380, in <module>
    resolve_deps(repo)
  File "./ensure_dependencies.py", line 323, in resolve_deps
    ensure_repo(repodir, parenttype, target, vcs, _root.get(vcs, ''), source)
  File "./ensure_dependencies.py", line 268, in ensure_repo
    repo_types[parenttype].ignore(target, parentrepo)
  File "./ensure_dependencies.py", line 130, in ignore
    _ensure_line_exists(exclude_file, module)
  File "./ensure_dependencies.py", line 355, in _ensure_line_exists
    with open(path, 'a+') as f:
IOError: [Errno 2] No such file or directory: './.git/info/exclude'

Expected behaviour

Script ensure_dependencies.py works without an error.

Change History (9)

comment:1 Changed 4 months ago by sebastian

  • Cc tlucas added

We ran into the same issue when setting up the CI for adblockpluscore and adblockpluschrome. IIRC, this is a bug with GitLab, and the canonical workaround is to create the missing folder before calling ensure_dependencies.py, or at least this is what we did there.

comment:2 Changed 4 months ago by kzlomek

The same is for us, we have several Gitlab CI projects and cofnigurations and in every one of them we need to repeat this step (mkdir -p .git/info).
Is there some case when lack of .git/info folder should make this script to fail instead of proceeding without parsing exclude file? My point is if this workaround is not dangerous I'd vote for putting that into ensure_dependencies.py instead of repeating the same workaround in projects which uses this script.

comment:3 Changed 4 months ago by sebastian

Tristan, what do you think?

comment:4 Changed 3 months ago by tlucas

Since we eventually want to get rid of ensure_dependencies.py (at least that's what i interpret from our current direction), i'd rather not add a new feature + update the dependencies everywhere.

In order to not having to adjust every single .yml file with some boilerplate code, i guess we could use gitlab's include feature, with a subproject or something alike.

comment:5 follow-up: Changed 3 months ago by kzar

Since we eventually want to get rid of ensure_dependencies.py...

I'm not sure that's true, ensure_dependencies.py works pretty well for me.

comment:6 in reply to: ↑ 5 Changed 3 months ago by tlucas

Replying to kzar:

Since we eventually want to get rid of ensure_dependencies.py...

I'm not sure that's true, ensure_dependencies.py works pretty well for me.

If we indeed decide to keep ensure_dependencies.py, despite our efforts to get rid of almost everything python-related - then yes, having ensure_dependencies.py check for .git/info/exclude might be the way to go. But i feel the overall course is to have either one repository or - on the long run - use sub repositories instead (which would solve this issue as a side-effect).

What do you guys think?

comment:7 follow-up: Changed 3 months ago by sebastian

We are currently in the progress of rewriting our build scripts in JavaScript. Eventually we want to get rid of the Python dependency altogether, and therefore also of ensure_dependencies.py. Moreover, once we completely migrated to Git and abandoned the Mercurial repository, there is no reason anymore to not just use submodules. We'd still have to call npm install in the adblockplusui submodule (which ensure_dependencies.py currently takes care of), but that can be done as well by our build scripts.

Still I have no strong opinion whether it's worth to address this issue in ensure_dependencies.py for the time being, or whether it rather belongs into .gitlab-ci.yml in the first place.

comment:8 in reply to: ↑ 7 Changed 3 months ago by tlucas

Replying to sebastian:

We are currently in the progress of rewriting our build scripts in JavaScript. Eventually we want to get rid of the Python dependency altogether, and therefore also of ensure_dependencies.py. Moreover, once we completely migrated to Git and abandoned the Mercurial repository, there is no reason anymore to not just use submodules. We'd still have to call npm install in the adblockplusui submodule (which ensure_dependencies.py currently takes care of), but that can be done as well by our build scripts.

Still I have no strong opinion whether it's worth to address this issue in ensure_dependencies.py for the time being, or whether it rather belongs into .gitlab-ci.yml in the first place.

That given, i strongly favor simply adding that one line into the .gitlab-ci.yml file, which is (most likely) subject to more changes anyway.

comment:9 Changed 3 months ago by sebastian

  • Resolution set to rejected
  • Status changed from new to closed

Fine with me.

Note: See TracTickets for help on using tickets.