Migrating repos from Mercurial to Git 6


Because Ogre uses Mercurial hosted on Bitbucket, and Mercurial is being removed, this guide is aimed to helping our users to migrate as painless as possible.

Git differences

We need to explain a few differences from hg to git. If you’re a git expert, you can skip this section.

The first main difference is that hg treats the repo history as immutable. Only advanced operations (which must be explicitly enabled) can manipulate history. And doing so can cause issues as replicating those history edits can be painful.
Git on the other hand is completely mutable and history can change.

The second main difference, and linked to this mutability, is how both treat branches.

Mercurial has a root-to-leaf approach to branches: a commit creates a branch and becomes the root of that branch. All of its children belong to that branch unless they explicitly open a new branch or are merged into an existing one. Technically speaking hg can have more than one root per branch, but this is discouraged.

Git has a leaf-to-root approach to branches. Branches are more like pointer references that reference-count the commits.
If a commit has a reference count of 0, it gets garbage collected and disappears from the official timeline. You can still access them via ‘git log -g’ as long as the garbage collector hasn’t been run.

A git ‘branch’ is a reference to a commit that increases its reference count, and this forces all of its parents to stay alive. When you create a new commit, the branch tag is moved from pointing to its current commit to now pointing at the newly created commit so that your new one has a non-zero ref count, and all of its parents are kept alive.

In git it is the leaf commit which decides what a branch is until the root(s) are reached, while in hg it is the root commit(s) which decide what a branch is.

This means that in git, it is possible for a commit to belong to multiple branches (because it has multiple children, each belonging to different branches)

Because git branches are just like ‘tags’ that can be easily switched around, this means that an entire history of commits can disappear if the branch tag is moved somewhere else.

A picture is a thousand words, and this GIF is 4000 words, what happens when you move the branch tag to a different commit:

With the animation it should become clear that:

  1. Unlike hg, git commits can belong to multiple branches
  2. Branch tags can be moved around easily
  3. The branch tag is defined by the leaf commit, not by the root one.

Because there’s at least two versions of the repo: your local changes and the repository’s, git uses the convention of tagging the repository’s commits as ‘origin/branch_a’ and your local version tags the commit as ‘branch_a’.
The remote’s commit must be tagged because otherwise it would have no reference count and thus could not show up in the timeline.

OK so now that we have explained how git works, it’s become clear that the concept of hg branches does not translate well to git’s branches at all. This is why conversion tools tend to ignore the branch metadata in hg, and use hg bookmarks instead.

Hg bookmarks are just ‘tags’ that can be easily moved to point between commits and don’t alter the history of hg, thus they behave very much like git ‘branches’ do.

Btw this is why I don’t like git. It exposes internal mechanisms (like reference counting and garbage collection) to the user. This results in a horrible user experience. It’s the equivalent of a chat program asking its users to write an SQL query to update their nickname.

hg-git plugin

Hg-git plugin is wonderful. It lets you use TortoiseHg with git. Beware however, the plugin does not support LFS. Another source of problems is that git supports octopus merges (merging multiple commits into one) whereas hg only supports merging two commits into one at a time.

Because you’re going hg -> git route, octopus merges shouldn’t be a problem. But if you want to keep using TortoiseHg with your git repo, then you shouldn’t allow octopus merges (i.e. by accepting a pull request that contains them)

Another thing: hg-git cannot use the https protocol. You can only pull/push to remote respositories via ssh by writing ‘git+ssh://git@github.com/username/reponame.git’, and to local repositories.

See Github and Bitbucket on how to setup an ssh key. The steps are really similar.

hg-git can preserve Mercurial’s branching information by writing extra metadata at the end of the commit. It looks ugly, but there’s no other way if you want to fully preserve lossless information between git <-> hg.

Once you’ve installed hg-git, set it up alongside the bookmarks, and you’re ready to go!

If you’ve done it correctly, your TortoiseHg should look like this:

This repo is hosted at https://github.com/darksylinc/colibrigui

  • ‘master’ is my local branch.
  • ‘github/master’ is my remote branch.

git-remote-hg

git-remote-hg is another alternative CLI tool very similar to hg-git, but that allows you to pull/push to hg repositories from git.

Once installed you can just convert an hg repo to git via:

    git clone hg::/path/to/hg-repository my-converted-git-repo

Hg bookmarks can’t use the same name as a branch

hg-git has an option to add a prefix to bookmarks:

[git]
branch_bookmark_suffix=_bookmark

Thus you can now create ‘branch_a_bookmark’ and it will be exported as ‘branch_a’
Alternatively you can force creating a bookmark with the same name as a branch via CLI typing:

hg bookmarks -f branch_a

How do I list all bookmarks (aka git branches)?

Aside from right click -> Bookmarks which lets you list, add, remove and move bookmarks, you can also type boomarks() in the filter:

How do I filter by bookmark in TortoiseHg?

Type ancestors(bookmark(“branch name”)) in the filter:

Unfortunately there’s no one-click-button for this one, like there is for true hg branches.

After typing it a few times, TortoiseHg will autocomplete as soon as you type the first letter

Git specific: How do I ‘pull’ when there are modified files?

A common workflow with TortoiseHg is to pull and auto-merge. However if you make ‘git pull’ with pending changes, you will see that git refuses to pull. A common recommendation on the web is to stash your changes pull, the pop the stash. But this will alter the timestamps, which can trigger huge recompiles if a troublesome header that’s included everywhere must be stashed.
The closest solution is:

git fetch
git reset origin/branch_name

Now you will pull without touching the modified files. It is up to you to revert these files or to revert specific lines to their newer versions.

How do I clone Ogre from Github in TortoiseHg?

After you’ve setup hg-git, bookmarks and SSH credentials with Github, clone the following URL:

git+ssh://git@github.com:OGRECave/ogre-next.git

The clone will take some time though, as hg-git plugin must first convert all the git commits to hg so TortoiseHg can understand it.

I want to use git natively. Do you have an UI to recommend?

I’ve tried git-kraken, Sourcetree, TortoiseGit, Github Desktop, gitg and git-cola.

To be honest the only one that worked reliably is git-cola. However git-cola is not for those who like pretty UIs. The skin is ugly, Windows 3.11 ugly. But it gets the job done, it is fast, and reliable. I got used to its ugliness.

The other GUIs either hide important stuff, leave the repo in an inconsistent or broken state, would refuse to do certain commit/merge/push/pull operations or become unusable with large repos.

It’s either that, or they make it impossible hard to search commit history.

After git-cola. Git-kraken is the most reasonable out of the bunch, and at least has a pretty UI.


6 thoughts on “Migrating repos from Mercurial to Git

  • Hotshot5000

    I discovered this https://hg.sr.ht/ (I am not affiliated in any way with them) and migrated my hg repos to them from bitbucket. I found this much easier than migrating to git. It’s also free (for now at least). Anyway 50 bucks per year sounds much better than git :). And I get to continue using TortoiseHg.

    • Matias Post author

      Hi!

      Thanks!

      Nice to see you’re alive! 🙂
      I still want to merge your GLES support (in fact we already use part of your code as it was used to run in macOS GL).

      I had heard about hg.sr.ht, it’s just something about it didn’t make me feel like it’s trustworthy for private projects (public have no danger). Call it a baseless feeling.

      We were already planning to move to Github, it’s just *I* like Mercurial a lot more. I explain the reasons for moving to Github (rather than sticking with Mercurial on another host) in a forum post: https://forums.ogre3d.org/viewtopic.php?f=25&t=95300#p545838

      • Hotshot5000

        Hi!

        Sorry for not contributing more often but I am caught up with work on the actual game and don’t have time right now. I haven’t kept up GLES3 with the v2.2 branch and don’t want to make any promises for when I will be back because last time my estimations were horribly inaccurate.

        I found sourcehut as I hate git with passion (I use it everyday at work and sometimes the headaches it gives and the unintuitive commands make me want to scream. It’s the C++ of DVCS’es) and I am still hoping that bitbucket will reverse that decision (or at least give us 5 years of support, even paid, 10 months is absurd). Another alternative might be Perforce Helix TeamHub but sourcehut is free for now. Also git is a PITA on Windows and my game works on both Windows and iOS. It’s really an inferior tool that won because it had Torvalds’ name on it. I call it fashion driven development.

        I tried converting a repo to git and use hg-git but I have some large files and there is no LFS support so… I could remove those files but I am lazy 🙂

Comments are closed.