Using Git submodules to work on two repos at the same time

Git submodules are a useful tool when you need to work on two closely coupled source repositories. My current use case involves wanting to maintain a Hugo theme together with the actual site that is using the theme. These are my notes on the basics of working with submodules.

Overview

When you’re working with a git submodule, you are starting with a one repository (let’s call it parent-repo) and adding the source of a second repository (child-repo) as a subdirectory. In this way, your local checkout of parent-repo will contain code from both repos, and you will be able to make edits and track changes in both repositories.

For every commit, you’ll have to decide if you’re modifying child-repo or parent-repo. Because these are still two separate repositories, there is no way to make one atomic commit to both simultaneously. But since the code is all in one tree, you can test and develop on both repos “together”.

When you make changes to child-repo, if you want those new commits to be available in parent-repo, you will need to additionally update parent-repo to point at the latest child-repo commit.

Setting up the submodule

To begin, we need to add the child repo as a sub-module of the parent repo, and tell git in which subdirectory of parent-repo do we want the child-repo code to live:

~/src/github.com/anaulin/parent-repo (master) $ git submodule add https://github.com/anaulin/child-repo child-repo

If we check the git status of the parent repo, we can see that this command has added a new .gitmodules file, and a new child-repo subdirectory:

~/src/github.com/anaulin/parent-repo (master) $ git status
On branch master
Your branch is up to date with 'origin/master'.

Changes to be committed:
    (use "git reset HEAD <file>..." to unstage)

   	new file:   .gitmodules
   	new file:   child-repo

The .gitmodules file lists the submodules of the parent-repo, and which directory each module is mapped to:

~/src/github.com/anaulin/parent-repo (master) $ cat .gitmodules
[submodule "child-repo"]
	path = child-repo
	url = https://github.com/anaulin/child-repo

If we do a git commit and push this to GitHub, we can see that GitHub now recognizes that parent-repo has child-repo as a submodule:

Clicking on the child-repo directory on this GitHub page takes you out of parent-repo and into child-repo at the specified commit.

We now have the files from child-repo available locally:

~/src/github.com/anaulin/parent-repo (master) $ ls child-repo/
README.md

Working with files in parent and child repos

Operating on files in parent-repo, works in the usual way: edit, commit, push, rinse and repeat.

To edit a file from child-repo when working within parent-repo, you can work in the child-repo directory in the same way as if you were in a regular repo:

~/src/github.com/anaulin/parent-repo (master) $ cd child-repo/
~/src/github.com/anaulin/parent-repo/child-repo (master) $ vim README.md
# ...make some edits...
~/src/github.com/anaulin/parent-repo/child-repo (master) $ git commit -m 'Commit to child-repo'
# [...]
~/src/github.com/anaulin/parent-repo/child-repo (master) $ git push

This git push will push the changes to your child-repo remote (because you are performing these operations from within the child-repo part of parent-repo).

Now in parent-repo we see a diff:

~/src/github.com/anaulin/parent-repo/child-repo (master) $ cd ..
# ...move out of child-repo, into a different part of parent-repo...

~/src/github.com/anaulin/parent-repo (master) $ git status
On branch master
Your branch is up to date with 'origin/master'.
Changes not staged for commit:
(use "git add <file>..." to update what will be committed)
(use "git checkout -- <file>..." to discard changes in working directory)
modified: child-repo (new commits)
no changes added to commit (use "git add" and/or "git commit -a")

~/src/github.com/anaulin/parent-repo (master) $ git diff
diff --git a/child-repo b/child-repo
index a454f1f..7274e3b 160000
--- a/child-repo
+++ b/child-repo
@@ -1 +1 @@
-Subproject commit a454f1f517674e2f71e4dc40c70bf4a93f696d16
+Subproject commit 7274e3b1de0edf6624ec784c66f5291c62be2bfd

This is git telling us that parent-repo expects its submodule to be at a certain commit, but the local version of the submodule is now at a different commit.

We can get rid of this discrepancy by updating the commit of child-repo that parent-repo expects, using git submodule --update remote. (There are more options you can pass to git submodule --update if you ever want to update to a specific commit, instead of the latest one.)

You can also work on child-repo from outside of parent-repo in the usual way. In order to have parent-repo be aware of those changes, use git submodule update --remote as well.

Checking out a project with submodules

When a project is set up with submodules, to clone and pull it for the first time you’ll need to additionally run git submodule update --init --recursive. This command will get you a local copy of the submodule:

~/src/github.com/anaulin $ git clone git@github.com:anaulin/parent-repo.git
Cloning into 'parent-repo'...
remote: Counting objects: 17, done.
remote: Compressing objects: 100% (13/13), done.
remote: Total 17 (delta 4), reused 12 (delta 2), pack-reused 0
Receiving objects: 100% (17/17), 2.02 KiB | 689.00 KiB/s, done.
Resolving deltas: 100% (4/4), done.

~/src/github.com/anaulin $ cd parent-repo/
~/src/github.com/anaulin/parent-repo (master) $ ls
README.md child-repo

~/src/github.com/anaulin/parent-repo (master) $ ls child-repo/

# Local copy of child-repo directory is empty
# Initialize the contents of child-repo by updating the submodule:

~/src/github.com/anaulin/parent-repo (master) $ git submodule update --init --recursive
Submodule 'child-repo' (https://github.com/anaulin/child-repo) registered for path 'child-repo'
Cloning into '/Users/ana/src/github.com/anaulin/parent-repo/child-repo'...
Submodule path 'child-repo': checked out '7239d8d33f5d1d58f428b124cb91ee727a8fa2f0'

~/src/github.com/anaulin/parent-repo (master) $ git status
On branch master
Your branch is up to date with 'origin/master'.
nothing to commit, working tree clean

~/src/github.com/anaulin/parent-repo (master) $ ls child-repo/
README.md

# Success!

If you are going to develop with a submodule, you’ll probably want to throw some parts of this workflow into a script to save yourself some typing, and to standardize the specific workflow you want to use across developers on your team.

More resources

Tags: tech

« Older entries · Newer entries »