un-submodule a git submodule

How do I un-submodule a git submodule (bring all the code back into the core) ?

As in how “should” I, as in “Best procedure” …

  • Is there a way to reconstruct a .git directory for a submodule?
  • Reuse a Part of a git Repository
  • Git submodule workflow pull from upstream but push to own repository
  • Git discard changes in working directory submodule
  • 'git submodule update --init --recursive' VS 'git submodule foreach --recursive git submodule update --init'
  • is git submodules what I need?
  • What is a good workflow for submodule forks
  • Completed with errors, not showing in source tree
  • 11 Solutions collect form web for “un-submodule a git submodule”

    If all you want is to put your submodule code into the main repository, you just need to remove the submodule and re-add the files into the main repo:

    git rm --cached submodule_path # delete reference to submodule HEAD (no trailing slash)
    git rm .gitmodules             # if you have more than one submodules,
                                   # you need to edit this file instead of deleting!
    rm -rf submodule_path/.git     # make sure you have backup!!
    git add submodule_path         # will add files instead of commit reference
    git commit -m "remove submodule"
    

    If you also want to preserve the history of the submodule, you can do a small trick: “merge” the submodule into the main repository so that the result will be the same as it was before, except that the submodule files are now in the main repository.

    In the main module you will need to do the following:

    # Fetch the submodule commits into the main repository
    git remote add submodule_origin git://url/to/submodule/origin
    git fetch submodule_origin
    
    # Start a fake merge (won't change any files, won't commit anything)
    git merge -s ours --no-commit submodule_origin/master
    
    # Do the same as in the first solution
    git rm --cached submodule_path # delete reference to submodule HEAD
    git rm .gitmodules             # if you have more than one submodules,
                                   # you need to edit this file instead of deleting!
    rm -rf submodule_path/.git     # make sure you have backup!!
    git add submodule_path         # will add files instead of commit reference
    
    # Commit and cleanup
    git commit -m "removed submodule"
    git remote rm submodule_origin
    

    The resulting repository will look a bit weird: there will be more than one initial commit. But it won’t cause any problems for git.

    In this second solution you will have the big advantage that you can still run git blame or git log on the files which were originally in submodules. In fact what you did here is to rename many files inside one repository, and git should autodetect this. If you still have problems with git log, try some options (–follow, -M, -C) which do better rename/copy detection.

    Since git 1.8.5 (Nov 2013) (without keeping the history of the submodule):

    mv yoursubmodule yoursubmodule_tmp
    git submodule deinit yourSubmodule
    git rm yourSubmodule
    mv yoursubmodule_tmp yoursubmodule
    git add yoursubmodule
    

    That will:

    • unregister and unload (ie delete the content of) the submodule (deinit, hence the mv first),
    • clean up the .gitmodules for you (rm),
    • and remove the special entry representing that submodule SHA1 in the index of the parent repo (rm).

    Once the removal of the submodule is complete (deinit and git rm), you can rename the folder back to its original name and add it to the git repo as a regular folder.

    Note: if the submodule was created by an old Git (< 1.8), you might need to remove the nested .git folder within the submodule itself, as commented by Simon East


    If you need to keep the history of the submodule, see jsears’s answer, which uses git filter-branch.

    It happened to us that we created 2 repositories for 2 projects that were so coupled that didn’t make any sense to have them separated, so we merged them.

    I’ll show how to merge the master branches in each first and then I will explain how you can extend this to every branches you got, hope it helps you.

    If you got the submodule working, and you want to convert it to a directory in place you can do:

    git clone project_uri project_name
    

    Here we do a clean clone to work. For this process you don’t need to initialize or update the submodules, so just skip it.

    cd project_name
    vim .gitmodules
    

    Edit .gitmodules with your favorite editor (or Vim) to remove the submodule you plan to replace. The lines you need to remove should look something like this:

    [submodule "lib/asi-http-request"]
        path = lib/asi-http-request
        url = https://github.com/pokeb/asi-http-request.git
    

    After saving the file,

    git rm --cached directory_of_submodule
    git commit -am "Removed submodule_name as submodule"
    rm -rf directory_of_submodule
    

    Here we remove the submodule relation completely so we can create bring the other repo to the project in-place.

    git remote add -f submodule_origin submodule_uri
    git fetch submodel_origin/master
    

    Here we fetch the submodule repository to merge.

    git merge -s ours --no-commit submodule_origin/master
    

    Here we start a merge operation of the 2 repositories, but stop before commit.

    git read-tree --prefix=directory_of_submodule/ -u submodule_origin/master
    

    Here we send the content of master in the submodule to the directory where it was before prefixing a directory name

    git commit -am "submodule_name is now part of main project"
    

    Here we complete the procedure doing a commit of the changes in the merge.

    After finishing this you can push, and start again with any other branch to merge, just checkout the branch in you repository that will receive the changes and change the branch you bringing in the merge and read-tree operations.

    1. git rm --cached the_submodule_path
    2. remove the submodule section from the .submodules file
    3. do a commit “removed submodule xyz”
    4. git add the_submodule_path
    5. another commit “added codebase of xyz”

    I didn’t find any easier way yet. You can compress 3-5 into one step via git commit -a – matter of taste.

    Lots of answers here but all of them seem to be overly complex and likely do not do what you want. I am sure most people want to keep their history.

    For this example the main repo will be git@site.com:main/main.git and the submodule repo will be git@site.com:main/child.git. This assumes that the submodule is located in the root directory of the parent repo. Adjust the instructions as needed.

    Start by cloning the parent repo and removing the old submodule.

    git clone git@site.com:main/main.git
    git submodule deinit child
    git rm child
    git add --all
    git commit -m "remove child submodule"
    

    Now we will add the child repos upstream to the main repo.

    git remote add upstream git@site.com:main/child.git
    git fetch upstream
    git checkout -b merge-prep upstream/master
    

    The next step assumes that you want to move the files on the merge-prep branch into the same location as the submodule was above although you can easily change the location by changing the file path.

    mkdir child
    

    move all folders and files except the .git folder into the child folder.

    git add --all
    git commit -m "merge prep"
    

    Now you can simply merge your files back into the master branch.

    git checkout master
    git merge merge-prep # --allow-unrelated-histories merge-prep flag may be required 
    

    Look around and make sure everything looks good before running git push

    The one thing you have to remember now is that git log does not by default follow moved files however by running git log --follow filename you can see the full history of your files.

    I’ve created a script that will translate a submodule to a simple directory, while retaining all file history. It doesn’t suffer from the git log --follow <file> issues that the other solutions suffer from. It’s also a very easy one-line invocation that does all of the work for you. G’luck.

    It builds on the excellent work by Lucas Jenß, described in his blog post “Integrating a submodule into the parent repository”, but automates the entire process and cleans up a few other corner cases.

    The latest code will be maintained with bugfixes on github at https://github.com/jeremysears/scripts/blob/master/bin/git-submodule-rewrite, but for the sake of proper stackoverflow answer protocol, I’ve included the solution in its entirety below.

    Usage:

    $ git-submodule-rewrite <submodule-name>
    

    git-submodule-rewrite:

    #!/usr/bin/env bash
    
    # This script builds on the excellent work by Lucas Jenß, described in his blog
    # post "Integrating a submodule into the parent repository", but automates the
    # entire process and cleans up a few other corner cases.
    # https://x3ro.de/2013/09/01/Integrating-a-submodule-into-the-parent-repository.html
    
    function usage(){
      echo "Merge a submodule into a repo, retaining file history."
      echo "Usage: $0 <submodule-name>"
      echo ""
      echo "options:"
      echo "  -h, --help                Print this message"
      echo "  -v, --verbose             Display verbose output"
    }
    
    function abort {
        echo "$(tput setaf 1)$1$(tput sgr0)"
        exit 1
    }
    
    function request_confirmation {
        read -p "$(tput setaf 4)$1 (y/n) $(tput sgr0)"
        [ "$REPLY" == "y" ] || abort "Aborted!"
    }
    
    function warn() {
      cat << EOF
        This script will convert your "${sub}" git submodule into
        a simple subdirectory in the parent repository while retaining all
        contents and file history.
    
        The script will:
          * delete the ${sub} submodule configuration from .gitmodules and
            .git/config and commit it.
          * rewrite the entire history of the ${sub} submodule so that all
            paths are prefixed by ${path}.
            This ensures that git log will correctly follow the original file
            history.
          * merge the submodule into its parent repository and commit it.
    
        NOTE: This script might completely garble your repository, so PLEASE apply
        this only to a fresh clone of the repository where it does not matter if
        the repo is destroyed.  It would be wise to keep a backup clone of your
        repository, so that you can reconstitute it if need be.  You have been
        warned.  Use at your own risk.
    
    EOF
    
      request_confirmation "Do you want to proceed?"
    }
    
    function git_version_lte() {
      OP_VERSION=$(printf "%03d%03d%03d%03d" $(echo "$1" | tr '.' '\n' | head -n 4))
      GIT_VERSION=$(git version)
      GIT_VERSION=$(printf "%03d%03d%03d%03d" $(echo "${GIT_VERSION#git version}" | tr '.' '\n' | head -n 4))
      echo -e "${GIT_VERSION}\n${OP_VERSION}" | sort | head -n1
      [ ${OP_VERSION} -le ${GIT_VERSION} ]
    }
    
    function main() {
    
      warn
    
      if [ "${verbose}" == "true" ]; then
        set -x
      fi
    
      # Remove submodule and commit
      git config -f .gitmodules --remove-section "submodule.${sub}"
      if git config -f .git/config --get "submodule.${sub}.url"; then
        git config -f .git/config --remove-section "submodule.${sub}"
      fi
      rm -rf "${path}"
      git add -A .
      git commit -m "Remove submodule ${sub}"
      rm -rf ".git/modules/${sub}"
    
      # Rewrite submodule history
      local tmpdir="$(mktemp -d -t submodule-rewrite-XXXXXX)"
      git clone "${url}" "${tmpdir}"
      pushd "${tmpdir}"
      local tab="$(printf '\t')"
      local filter="git ls-files -s | sed \"s/${tab}/${tab}${path}\//\" | GIT_INDEX_FILE=\${GIT_INDEX_FILE}.new git update-index --index-info && mv \${GIT_INDEX_FILE}.new \${GIT_INDEX_FILE}"
      git filter-branch --index-filter "${filter}" HEAD
      popd
    
      # Merge in rewritten submodule history
      git remote add "${sub}" "${tmpdir}"
      git fetch "${sub}"
    
      if git_version_lte 2.8.4
      then
        # Previous to git 2.9.0 the parameter would yield an error
        ALLOW_UNRELATED_HISTORIES=""
      else
        # From git 2.9.0 this parameter is required
        ALLOW_UNRELATED_HISTORIES="--allow-unrelated-histories"
      fi
    
      git merge -s ours --no-commit ${ALLOW_UNRELATED_HISTORIES} "${sub}/master"
      rm -rf tmpdir
    
      # Add submodule content
      git clone "${url}" "${path}"
      rm -rf "${path}/.git"
      git add "${path}"
      git commit -m "Merge submodule contents for ${sub}"
      git config -f .git/config --remove-section "remote.${sub}"
    
      set +x
      echo "$(tput setaf 2)Submodule merge complete. Push changes after review.$(tput sgr0)"
    }
    
    set -euo pipefail
    
    declare verbose=false
    while [ $# -gt 0 ]; do
        case "$1" in
            (-h|--help)
                usage
                exit 0
                ;;
            (-v|--verbose)
                verbose=true
                ;;
            (*)
                break
                ;;
        esac
        shift
    done
    
    declare sub="${1:-}"
    
    if [ -z "${sub}" ]; then
      >&2 echo "Error: No submodule specified"
      usage
      exit 1
    fi
    
    shift
    
    if [ -n "${1:-}" ]; then
      >&2 echo "Error: Unknown option: ${1:-}"
      usage
      exit 1
    fi
    
    if ! [ -d ".git" ]; then
      >&2 echo "Error: No git repository found.  Must be run from the root of a git repository"
      usage
      exit 1
    fi
    
    declare path="$(git config -f .gitmodules --get "submodule.${sub}.path")"
    declare url="$(git config -f .gitmodules --get "submodule.${sub}.url")"
    
    if [ -z "${path}" ]; then
      >&2 echo "Error: Submodule not found: ${sub}"
      usage
      exit 1
    fi
    
    if ! [ -d "${path}" ]; then
      >&2 echo "Error: Submodule path not found: ${path}"
      usage
      exit 1
    fi
    
    main
    

    The best answer to this I have found is here:

    http://x3ro.de/2013/09/01/Integrating-a-submodule-into-the-parent-repository.html

    This article explains the procedure very well.

    For when

    git rm [-r] --cached submodule_path
    

    returns

    fatal: pathspec 'emr/normalizers/' did not match any files
    

    Context: I did rm -r .git* in my submodule folders before realizing that they needed to be de-submoduled in the main project to which I had just added them. I got the above error when de-submoduling some, but not all of them. Anyway, I fixed them by running, (after, of course, the rm -r .git*)

    mv submodule_path submodule_path.temp
    git add -A .
    git commit -m "De-submodulization phase 1/2"
    mv submodule_path.temp submodule_path
    git add -A .
    git commit -m "De-submodulization phase 2/2"
    

    Note that this doesn’t preserve history.

    Here’s a slightly improved version (IMHO) of the current top answer:

    In a separate dir (to make mistakes easier to clean up and try again) check out both the top repo and the subrepo.

    git clone ../main_repo main.tmp
    git clone ../main_repo/sub_repo sub.tmp
    

    First edit the subrepo to move all files into the desired subdirectory

    cd sub.tmp
    mkdir sub_repo_path
    git mv `ls | grep -v sub_repo_path` sub_repo_path/
    git commit -m "Moved entire subrepo into sub_repo_path"
    

    Make a note of the HEAD

    SUBREPO_HEAD=`git reflog | awk '{ print $1; exit; }'`
    

    Now remove the subrepo from the main repo

    cd ../main.tmp
    rmdir sub_repo_path
    vi .gitmodules  # remove config for submodule
    git add -A
    git commit -m "Removed submodule sub_repo_path in preparation for merge"
    

    And finally, just merge them

    git fetch ../sub.tmp
    git merge $SUBREPO_HEAD
    

    And done! Safely and without any magic.

    Based on VonC’s answer, I have created a simple bash script that does this. The add at the end has to use wildcards otherwise it will undo the previous rm for the submodule itself. It’s important to add the contents of the submodule directory, and not to name the directory itself in the add command.

    In a file called git-integrate-submodule:

    #!/usr/bin/env bash
    mv "$1" "${1}_"
    git submodule deinit "$1"
    git rm "$1"
    mv "${1}_" "$1"
    git add "$1/**"
    

    I found it more convenient to (also?) fetch local commit data from the submodule, because otherwise I would loose them. (Could not push them as I have not access to that remote). So I added submodule/.git as remote_origin2, fetched it commits and merged from that branch.
    Not sure if I still need the submodule remote as origin, since I am not familiar enough with git yet.

    Git Baby is a git and github fan, let's start git clone.