Opened on 08/24/2015 at 05:11:58 PM
Closed on 10/14/2015 at 05:43:45 PM
#2949 closed defect (fixed)
Fix adblockpluschrome Git sync
Reported by: | matze | Assignee: | |
---|---|---|---|
Priority: | P2 | Milestone: | |
Module: | Infrastructure | Keywords: | |
Cc: | fhd, fred, sebastian, kzar | Blocked By: | |
Blocking: | #2906 | Platform: | Unknown / Cross platform |
Ready: | yes | Confidential: | no |
Tester: | Unknown | Verified working: | no |
Review URL(s): |
Description (last modified by matze)
The hg-git snyc for adblockpluschrome has not been setup during #276. It turns out this being one of the repositories where I could not reproduce the exact Hashes from the former manual synchronization, which was done by Felix.
Thus we have basically two options: Either trying to reproduce it using the information Felix' original/local clone or just break references on GitHub..
Attachments (0)
Change History (14)
comment:1 Changed on 08/24/2015 at 05:12:12 PM by matze
comment:2 Changed on 08/24/2015 at 05:12:29 PM by matze
- Blocking 2906 added
comment:3 Changed on 08/24/2015 at 06:00:29 PM by fhd
Just tried it again with a fresh clone from hg, pushing to a (new) local bare git repository, hashes seem identical. My hg-git version is 0.8.1, installed via pip.
comment:4 Changed on 08/24/2015 at 06:17:40 PM by matze
I've been using the same version (hgrc aligned with #273):
hg@hg1:~/web/adblockpluschrome$ python -c 'import hggit; print hggit.__version__' 0.8.2
The error, however, is reproducible with a fresh clone from the legacy repository:
hg@hg1:~/web/adblockpluschrome$ hg bookmark -r default master hg@hg1:~/web/adblockpluschrome$ hg push -b default github pushing to git+ssh://git@github.com/adblockplus/adblockpluschrome.git searching for changes abort: revision 4af64b38d99f cannot be pushed since it doesn't have a ref
hg@hg1:~/web/adblockpluschrome$ hg log -r 4af64b38d99f changeset: 546:4af64b38d99f parent: 362:68fa41ef96b5 user: Wladimir Palant <trev@adblockplus.org> date: Thu Dec 02 18:21:37 2010 +0100 summary: Closing branch, all the relevant changes seem to have been integrated already
Since all unmatched references (see .hg/git-mapfile) cause the synchronization to fail, I have limited it to branch default / master by default. It seems like after the branch above has been closed, hg-git still requires a reference to be present. Whatever the exact cause, in theory we should be able to reconstruct the missing bits from the .hg directory which has been used for pushing.
We could, however, also just go the shortcut and break the references for adblockpluschrome once again. If that's fine with you, @snoack.
comment:5 Changed on 09/10/2015 at 07:44:54 AM by sebastian
FWIW, I 'd prefer to not break the hashes. However, if this isn't possible or would delay the fix, I guess we have to accept that we'll have to break them. I don't remember that we linked to git hashes of that repo anywhere, anyway. However, we need the mirror back quite soon.
comment:6 Changed on 09/10/2015 at 08:27:03 AM by fhd
In my sync script back then, I created a new clone for each commit, like this:
$ git init --bare adblockpluschrome-git $ hg push -R adblockpluschrome adblockpluschrome-git
I've sent Matze the results of that.
The relevant versions of stuff I'm using:
Python: 2.7.6 (as shipped with OS X 10.10.4)
Git: 2.3.4 (via MacPorts)
Mercurial: 3.2.4 (via pip)
hg-git: 0.8.1 (via pip)
comment:7 Changed on 09/10/2015 at 08:54:40 AM by kzar
- Cc kzar added
comment:8 Changed on 09/10/2015 at 12:56:22 PM by matze
Unfortunately, the meta-data gathered that way was not enough to recover the synchronization with the current hashes. But I could confirm the actual issue here:
In a few circumstances, the hg-git module does not implement a reproducible way to create the hash mappings. Not even with the original .hg/git-mapfile. Thus a few of the entries there are different for each sync.
comment:9 Changed on 09/10/2015 at 01:41:01 PM by sebastian
- Cc sebastian added; snoack removed
comment:10 Changed on 09/16/2015 at 08:27:58 AM by matze
- Description modified (diff)
See also #3080.
comment:11 Changed on 09/21/2015 at 03:17:20 PM by kzar
I thought maybe I could help with this so I attempted to reproduce the issue. I installed hg_git using easy_install, cloned adblockpluschrome from the Mercurial repository and then performed a push to a temporary Git repository using hg_git. It worked for me first try, the commit hashes matched.
Commands I ran:
hg clone ssh://hg@hg.adblockplus.org/adblockpluschrome sudo easy_install hg_git # Then added hg_git to the extensions section of my .hgrc file cd adblockpluschrome hg push git+ssh://git@github.com/kzar/tmp.git
I'm running on Ubuntu 15.04 with Python 2.7.9, hg_git 0.8.2, Mercurial 3.1.2 and Git 2.1.4
comment:12 Changed on 09/22/2015 at 10:07:35 AM by matze
Fair enough, one can reproduce the initial hashes. The problem occurs when trying to synchronize a change made subsequent to the initial sync - that's when it cannot reproduce all references properly:
$ hg outgoing -S github comparing with git+ssh://git@github.com/adblockplus/adblockpluschrome.git abort: branch 'refs/tags/1.5.2' changed on the server, please pull and merge before pushing
comment:13 Changed on 09/22/2015 at 02:37:52 PM by kzar
Matze demonstrated this issue for me:
- He cloned the Mercurial repository twice on one of our servers
- He then pushed both to separate temporary Git repositories hosted on GitHub using hg-git.
- He then performed a commit using Mercurial in the first repository.
- Then when attempting to push that repository to GitHub again, it failed in the way he described. (An error was shown because refs/tags/1.5.2 no longer matched.)
- Comparing the ./hg/git-mapfile of the two repositories showed that multiple references no longer matched.
- He then performed a commit in the second repository, pushing that to GitHub also failed in the same way.
- Interesting comparing the two .hg/git-mapfiles again showed less differences. E.g. the references were somehow being changed in a reliable way.
Unfortunately I was not able to reproduce this issue myself locally. I was able to push new commits without trouble and I didn't find the references had changed.
comment:14 Changed on 10/14/2015 at 05:43:45 PM by matze
- Resolution set to fixed
- Status changed from new to closed
@fhd Please either try to fix this on your own (you should have all access and info necessary) or provide me with a copy of your local adblockpluschrome/.hg directory.