Opened 3 years ago

Closed 3 years ago

#4933 closed change (fixed)

Mirror repositories

Reported by: sergz Assignee: ferris
Priority: P3 Milestone:
Module: Infrastructure Keywords:
Cc: fhd, fred, ferris Blocked By:
Blocking: #4907 Platform: Unknown / Cross platform
Ready: yes Confidential: no
Tester: Unknown Verified working: no
Review URL(s):

https://codereview.adblockplus.org/29398680
https://codereview.adblockplus.org/29398663/

Description

Background

In order to update v8 (#4907) we need to mirror several repositories:

What to change

Create our mirrors of these repos (mercurial and on github).

Change History (19)

comment:1 Changed 3 years ago by eric@…

I don't see any need to create our own mirrors.

What's actually needed is a way to get code into the right place on a developer's machine. One way to do that is to operate mirrors and use the the simpler syntax supported by ensure_dependencies.py that assumes an ABP server. Another is to use the more complex syntax, also already supported by ensure_dependencies.py, that allows specifying both a server and a revision identifier.

comment:2 Changed 3 years ago by sergz

I don't see any need to create our own mirrors.

Many of repositories which are dependencies of currently used version of v8 are already not available. To be sure that we still can compile the code we need to store a copy of these dependencies.

One way to do that is to operate mirrors and use the the simpler syntax supported by ensure_dependencies.py that assumes an ABP server. Another is to use the more complex syntax, also already supported by ensure_dependencies.py, that allows specifying both a server and a revision identifier.

Indeed I'm going to use ensure_dependencies.py to manage these dependencies but I'm sure that all repositories should be controlled by us.

comment:3 Changed 3 years ago by fhd

Indeed I'm going to use ensure_dependencies.py to manage these dependencies but I'm sure that all repositories should be controlled by us.

Yeah, that's generally how we do it with source dependencies. That way our stuff will keep on working even if the original repo is down/moved/broken/whatever.

comment:4 Changed 3 years ago by sergz

FYI: ensure_dependencies.py does not support to fetch git repo, e.g.
third_party/v8/build = git:https://github.com/xxxz/chromium-build.git@3983535ba9ae6e2dc501e59fde158018d811f95c
when the root repo is mercurial. From git repo it does work.

comment:5 Changed 3 years ago by sergz

  • Cc ferris added

comment:6 Changed 3 years ago by ferris

  • Owner set to ferris

Me and Vasily want to do this Wednesday 30th of March.

comment:7 Changed 3 years ago by matze

  • Ready set

comment:8 Changed 3 years ago by matze

  • Priority changed from Unknown to P3

comment:9 Changed 3 years ago by ferris

Me and Vasily discussed how this is done yesterday. I'll try setting it up today. The repositories in question aren't as large as we feared, so the plan is to just git clone and then convert locally on hg1. We'd like to name the repos like this:

  • chromium-deps-icu
  • chromium-src-base-trace_event-common
  • chromium-src-build
  • chromium-src-tools-clang

The names are based on their paths, obviously. We also wondered if we should give them some sort of mirror prefix...

comment:10 Changed 3 years ago by ferris

I've manually cloned and placed these repos under ~hg/web now.

comment:11 Changed 3 years ago by ferris

  • Review URL(s) modified (diff)

Sergei will see if he can clone the mirrors to github (I'm not member of adblockplus).

Sergei now has push rights to the mirrors, ref https://codereview.adblockplus.org/29398663/

The mirrors also have the description and contact info now.

Awaiting https://codereview.adblockplus.org/29398680/ to be merged into web.adblockplus.org for the modules page.

comment:12 Changed 3 years ago by ferris

  • Status changed from new to reviewing

comment:13 Changed 3 years ago by abpbot

A commit referencing this issue has landed:
Issue 4933 - add more mirrors to libadblockplus

comment:14 Changed 3 years ago by sergz

Sergei will see if he can clone the mirrors to github (I'm not member of adblockplus).

I don't have enough rights to create repositories on github under adblockplus.
Could someone please do it? @fred, @fhd.

There is actually an option to import a repository which can be done only via mouse and browser :) I have just tried it on one repo on my private account.

comment:15 Changed 3 years ago by ferris

Hi Sergei, can you try importing the repositories on github now from their original coordinates? It's not clear whether we want to mirror the original git repositories, or we want to set up a hg->git sync from hg.abp.org, but I think the former approach seems quite natural for now.

comment:16 Changed 3 years ago by sergz

Hi Sergei, can you try importing the repositories on github now from their original coordinates?

I don't have enough permissions.

It's not clear whether we want to mirror the original git repositories, or we want to set up a hg->git sync from hg.abp.org, but I think the former approach seems quite natural for now.

Converting git->hg->git sounds indeed questionable. These mirrors are not something which is getting updated every day, so I think it's fine to do it manually, though it would be better to clearly document that somewhere. BTW, is there by any chance of an automatic conversion git->hg? Disregard our current plans regarding mercurial I think that functionality would be very useful.

comment:17 Changed 3 years ago by fred

Since one needs admin access to the Github organization I did the import of the git repositories from the original sources together with Sergei.

As they also belong to the libadblockplus module I gave the Libadblockplus Owners Github team Admin-level access to these repos.

comment:18 Changed 3 years ago by ferris

Great, thanks Fred! Sergei, can you close this ticket if everything is OK now?

comment:19 Changed 3 years ago by sergz

  • Resolution set to fixed
  • Status changed from reviewing to closed
Note: See TracTickets for help on using tickets.