Why does git-rebase give me merge conflicts when all I'm doing is squashing commits?

We have a Git repository with over 400 commits, the first couple dozen of which were a lot of trial-and-error. We want to clean up these commits by squashing many down into a single commit. Naturally, git-rebase seems the way to go. My problem is that it ends up with merge conflicts, and these conflicts are not easy to resolve. I don’t understand why there should be any conflicts at all, since I’m just squashing commits (not deleting or rearranging). Very likely, this demonstrates that I’m not completely understanding how git-rebase does its squashes.

Here’s a modified version of the scripts I’m using:

  • Git branch not returning any results
  • Eclipse Internal error makes it impossible to use git
  • Do git conflict resolution files not always represent every line in the source files?
  • Generate documentation on commit
  • Feature branches and Pull Requests - Git
  • Rebase onto branch without common ancestor

  • repo_squash.sh (this is the script that is actually run):


    rm -rf repo_squash
    git clone repo repo_squash
    cd repo_squash/
    GIT_EDITOR=../repo_squash_helper.sh git rebase --strategy theirs -i bd6a09a484b8230d0810e6689cf08a24f26f287a
    

    repo_squash_helper.sh (this script is used only by repo_squash.sh):


    if grep -q "pick " $1
    then
    #  cp $1 ../repo_squash_history.txt
    #  emacs -nw $1
      sed -f ../repo_squash_list.txt < $1 > $1.tmp
      mv $1.tmp $1
    else
      if grep -q "initial import" $1
      then
        cp ../repo_squash_new_message1.txt $1
      elif grep -q "fixing bad import" $1
      then
        cp ../repo_squash_new_message2.txt $1
      else
        emacs -nw $1
      fi
    fi
    

    repo_squash_list.txt: (this file is used only by repo_squash_helper.sh)


    # Initial import
    s/pick \(251a190\)/squash \1/g
    # Leaving "Needed subdir" for now
    # Fixing bad import
    s/pick \(46c41d1\)/squash \1/g
    s/pick \(5d7agf2\)/squash \1/g
    s/pick \(3da63ed\)/squash \1/g
    

    I’ll leave the “new message” contents to your imagination. Initially, I did this without the “–strategy theirs” option (i.e., using the default strategy, which if I understand the documentation correctly is recursive, but I’m not sure which recursive strategy is used), and it also didn’t work. Also, I should point out that, using the commented out code in repo_squash_helper.sh, I saved off the original file that the sed script works on and ran the sed script against it to make sure it was doing what I wanted it to do (it was). Again, I don’t even know why there would be a conflict, so it wouldn’t seem to matter so much which strategy is used. Any advice or insight would be helpful, but mostly I just want to get this squashing working.

    Updated with extra information from discussion with Jefromi:

    Before working on our massive “real” repository, I used similar scripts on a test repository. It was a very simple repository and the test worked cleanly.

    The message I get when it fails is:

    Finished one cherry-pick.
    # Not currently on any branch.
    nothing to commit (working directory clean)
    Could not apply 66c45e2... Needed subdir
    

    This is the first pick after the first squash commit. Running git status yields a clean working directory. If I then do a git rebase --continue, I get a very similar message after a few more commits. If I then do it again, I get another very similar message after a couple dozen commits. If I do it yet again, this time it goes through about a hundred commits, and yields this message:

    Automatic cherry-pick failed.  After resolving the conflicts,
    mark the corrected paths with 'git add <paths>', and
    run 'git rebase --continue'
    Could not apply f1de3bc... Incremental
    

    If I then run git status, I get:

    # Not currently on any branch.
    # Changes to be committed:
    #   (use "git reset HEAD <file>..." to unstage)
    #
    # modified:   repo/file_A.cpp
    # modified:   repo/file_B.cpp
    #
    # Unmerged paths:
    #   (use "git reset HEAD <file>..." to unstage)
    #   (use "git add/rm <file>..." as appropriate to mark resolution)
    #
    # both modified:      repo/file_X.cpp
    #
    # Changed but not updated:
    #   (use "git add/rm <file>..." to update what will be committed)
    #   (use "git checkout -- <file>..." to discard changes in working directory)
    #
    # deleted:    repo/file_Z.imp
    

    The “both modified” bit sounds weird to me, since this was just the result of a pick. It’s also worth noting that if I look at the “conflict”, it boils down to a single line with one version beginning it with a [tab] character, and the other one with four spaces. This sounded like it might be an issue with how I’ve set up my config file, but there’s nothing of the sort in it. (I did note that core.ignorecase is set to true, but evidently git-clone did that automatically. I’m not completely surprised by that considering that the original source was on a Windows machine.)

    If I manually fix file_X.cpp, it then fails shortly afterward with another conflict, this time between a file (CMakeLists.txt) that one version thinks should exist and one version thinks shouldn’t. If I fix this conflict by saying I do want this file (which I do), a few commits later I get another conflict (in this same file) where now there’s some rather non-trivial changes. It’s still only about 25% of the way through the conflicts.

    I should also point out, since this might be very important, that this project started out in an svn repository. That initial history very likely was imported from that svn repository.

    Update #2:

    On a lark (influenced by Jefromi’s comments), I decided to do the change my repo_squash.sh to be:

    rm -rf repo_squash
    git clone repo repo_squash
    cd repo_squash/
    git rebase --strategy theirs -i bd6a09a484b8230d0810e6689cf08a24f26f287a
    

    And then, I just accepted the original entries, as is. I.e., the “rebase” shouldn’t have changed a thing. It ended up with the same results describe previously.

    Update #3:

    Alternatively, if I omit the strategy and replace the last command with:

    git rebase -i bd6a09a484b8230d0810e6689cf08a24f26f287a
    

    I no longer get the “nothing to commit” rebase problems, but I’m still left with the other conflicts.

    Update with toy repository that recreates problem:

    test_squash.sh (this is the file you actually run):

    #========================================================
    # Initialize directories
    #========================================================
    rm -rf test_squash/ test_squash_clone/
    mkdir -p test_squash
    mkdir -p test_squash_clone
    #========================================================
    
    #========================================================
    # Create repository with history
    #========================================================
    cd test_squash/
    git init
    echo "README">README
    git add README
    git commit -m"Initial commit: can't easily access for rebasing"
    echo "Line 1">test_file.txt
    git add test_file.txt
    git commit -m"Created single line file"
    echo "Line 2">>test_file.txt 
    git add test_file.txt 
    git commit -m"Meant for it to be two lines"
    git checkout -b dev
    echo Meaningful code>new_file.txt
    git add new_file.txt 
    git commit -m"Meaningful commit"
    git checkout master
    echo Conflicting meaningful code>new_file.txt
    git add new_file.txt 
    git commit -m"Conflicting meaningful commit"
    # This will conflict
    git merge dev
    # Fixes conflict
    echo Merged meaningful code>new_file.txt
    git add new_file.txt
    git commit -m"Merged dev with master"
    cd ..
    
    #========================================================
    # Save off a clone of the repository prior to squashing
    #========================================================
    git clone test_squash test_squash_clone
    #========================================================
    
    #========================================================
    # Do the squash
    #========================================================
    cd test_squash
    GIT_EDITOR=../test_squash_helper.sh git rebase -i HEAD@{7}
    #========================================================
    
    #========================================================
    # Show the results
    #========================================================
    git log
    git gc
    git reflog
    #========================================================
    

    test_squash_helper.sh (used by test_sqash.sh):

    # If the file has the phrase "pick " in it, assume it's the log file
    if grep -q "pick " $1
    then
      sed -e "s/pick \(.*\) \(Meant for it to be two lines\)/squash \1 \2/g" < $1 > $1.tmp
      mv $1.tmp $1
    # Else, assume it's the commit message file
    else
    # Use our pre-canned message
      echo "Created two line file" > $1
    fi
    

    P.S.: Yes, I know some of you cringe when you see me using emacs as a fall-back editor.

    P.P.S.: We do know we’ll have to blow away all of our clones of the existing repository after the rebase. (Along the lines of “thou shalt not rebase a repository after it’s been published”.)

    P.P.P.S: Can anyone tell me how to add a bounty to this? I’m not seeing the option anywhere on this screen whether I’m in edit mode or view mode.

  • Ignore files with names starting with 'output'
  • Git push fails because everything is alledgely up-to-date
  • How to squash a lot of commits automatically?
  • Failed, DIRTY_WORKTREE in eclipse, how to solve it?
  • Is there a reliable way to import cvsnt to git?
  • How to translate git command line to tortoiseproc?
  • 5 Solutions collect form web for “Why does git-rebase give me merge conflicts when all I'm doing is squashing commits?”

    All right, I’m confident enough to throw out an answer. Maybe will have to edit it, but I believe I know what your problem is.

    Your toy repo test case has a merge in it – worse, it has a merge with conflicts. And you’re rebasing across the merge. Without -p (which doesn’t totally work with -i), the merges are ignored. This means that whatever you did in your conflict resolution isn’t there when the rebase tries to cherry-pick the next commit, so its patch may not apply. (I believe this is shown as a merge conflict because git cherry-pick can apply the patch by doing a three-way merge between the original commit, the current commit, and the common ancestor.)

    Unfortunately, as we noted in the comments, -i and -p (preserve merges) don’t get along very well. I know that editing/rewording work, and that reordering doesn’t. However, I believe that it works fine with squashes. This is not documented, but it worked for the test cases I describe below. If your case is way, way more complex, you may have a lot of trouble doing what you want, though it’ll still be possible. (Moral of the story: clean up with rebase -i before merging.)

    So, let’s suppose we have a very simple case, where we want to squash together A, B, and C:

    - o - A - B - C - X - D - E - F (master)
       \             /
        Z -----------
    

    Now, like I said, if there were no conflicts in X, git rebase -i -p works as you’d expect.

    If there are conflicts, things get a little trickier. It’ll do fine squashing, but then when it tries to recreate the merge, the conflicts will happen again. You’ll have to resolve them again, add them to the index, then use git rebase --continue to move on. (Of course, you can resolve them again by checking out the version from the original merge commit.)

    If you happen to have rerere enabled in your repo (rerere.enabled set to true), this will be way easier – git will be able to reuse the recorded resolution from when you originally had the conflicts, and all you have to do is inspect it to make sure it worked right, add the files to the index, and continue. (You can even go one step farther, turning on rerere.autoupdate, and it’ll add them for you, so the merge won’t even fail). I’m guessing, however, that you didn’t ever enable rerere, so you’re going to have to do the conflict resolution yourself.*


    * Or, you could try the rerere-train.sh script from git-contrib, which attempts to “Prime [the] rerere database from existing merge commits” – basically, it checks out all the merge commits, tries to merge them, and if the merge fails, it grabs the results and shows them to git-rerere. This could be time-consuming, and I’ve never actually used it, but it might be very helpful.

    I was looking for a similar requirement , i.e. discarding intermeiate commits of my development branch , I’ve found this procedure worked for me.
    on my working branch

    git reset –hard mybranch-start-commit
    git checkout mybranch-end-commit . // files only of the latest commit
    git add -a
    git commit -m”New Message intermediate commits discarded”
    

    viola we have connected the latest commit to the start commit of the branch!
    No merge conflict issues!
    In my learning practice I have come to this conclusion at this stage , Is there a better approach for the purpose .

    If you don’t mind creating a new branch, this is how I dealt with the problem:

    Being on mainline:

    # create a new branch
    git checkout -b new_clean_branch
    
    # apply all changes
    git merge original_messy_branch
    
    # forget the commits but have the changes staged for commit
    git reset --soft mainline        
    
    git commit -m "Squashed changes from original_messy_branch"
    

    Note that -X and strategy options were ignored when used in an interactive rebase.

    See commit db2b3b820e2b28da268cc88adff076b396392dfe (July 2013, git 1.8.4+),

    Do not ignore merge options in interactive rebase

    Merge strategy and its options can be specified in git rebase, but with -- interactive, they were completely ignored.

    Signed-off-by: Arnaud Fontaine

    That means -X and strategy now work with interactive rebase, as well as plain rebase, and your initial script could now work better.

    I was running into a simpler but similar issue, where I had
    1) resolved a merge conflict on a local branch,
    2) kept working adding lots more little commits,
    3) wanted to rebase and hit merge conflicts.

    For me, git rebase -p -i master worked. It kept the original conflict resolution commit and allowed me to squash the others on top.

    Hope that helps someone!

    Git Baby is a git and github fan, let's start git clone.