Remove all except certain folders from git history
I have a complex git repo from which I would like to delete ALL files and history except for two folders, let’s say:
git filter-branch --subdirectory-filter would let me select one folder, and make that the new root, it doesn’t seem to give me any option for selecting two directories, and preserving their placement.
git filter-branch --tree-filter or
--index-filter seem like it will let me iterate through every commit in history, where I can use
git rm on an unwanted folder.
I cannot seem to find any working way to get these commands to just preserve the two folders I desire while clearing everything else.
2 Solutions collect form web for “Remove all except certain folders from git history”
You are correct: a tree filter or an index filter would be the way to do this with
The tree filter is easier, but much slower (easily 10 to 100 times slower). The way a tree filter works is that your supplied command is run in a temporary directory that contains all, and only, the files that were present in the original (now being copied) commit. Any files your command leaves behind, remain in the copied commit. Any files your command creates in the temporary directory, are also in the copied commit. (You may create or remove directories within the temporary directory with no effect either way, since Git stores only the files.) Hence, to remove everything except A and B, write a command that removes every file that is in something other than either A or B:
find . -name A -prune -o -name B -prune -o -print0 | xargs -0 rm
The index filter is harder, but faster because Git does not have to copy all the files out to a file tree and then re-scan the file tree to build a new index, in order to copy the original commit. Instead, it provides only an index, which you can then manipulate with commands like
git rm -rf --cached --ignore-unmatch for instance, or
git update-index for the most general case. But, now the only tools you have are those in Git that manipulate the index. There is no fancy Unix
You do, of course, have
git ls-files, which reads out the current contents of the index. Hence you can write a program in whatever language you like (I would use Python first here, probably, others might start with Perl) that in essence does:
for (all files in the index) if (file name starts with 'A/' or 'B/') do nothing else add to removal list invoke "git rm --cached" on paths in removal list
If you are willing to trust that no file name has an embedded newline, the above can be done in regular shell as:
git ls-files | IFS=$'\n' while read path; do case "$path" in A/*|B/*) continue;; esac git rm --cached "$path" done
which is not terribly efficient (one
git rm --cached per path!) but should work “out of the box” as an
(Untested, but probably works and should be significantly more efficient: pipe
git ls-files output through
grep -v to remove desired files, and pipe
grep output into
git update-index --force-remove --stdin. This still assumes no newlines in path names.)
For files, I’ve done this with
git fast-export. But I’m not sure that would work recurseively on directories. So I’d suggest using a combination of
git fast-export and
git fast-export HEAD -- `find foo/a bar/x/y -type f` >../myfiles.fi
Then create a new repo, and import the streams.
git init git fast-import <../myfiles.fi