My project uses over 100 git submodules, which submodule alternative can handle a lot of repositories gracefully

I’ve been researching git subtree and other alternatives to git submodules. My project has well over 100 submodules and it’s very unwieldy to manage them all.

Can anyone recommend a workflow that works really well with a large number of repositories that need to be kept in sync.

  • Add github library as dependency to Android-Studio project
  • Yeoman install and collaboration (over git for instance)
  • How Can I Seamlessly Use A Local Bower Dependency?
  • Sharing Git Submodules
  • Best git structure for importing android dependent project with egit in eclipse
  • Should I commit Godeps/_workspace or is Godeps.json enough?
  • How to add a folder of a repository as a submodule to other repository using tortoiseGIT
  • Get error message ''fatal: sha1 information is lacking or useless“ when apply a patch using ”git am -3"
  • How can I un-do a git commit AFTER a git push?
  • Is TortoiseGit ready for prime time yet?
  • local git config during remote ssh session
  • gitosis-admin git push failed, exec hooks/post-update
  • One Solution collect form web for “My project uses over 100 git submodules, which submodule alternative can handle a lot of repositories gracefully”

    If you project has over 100 git submodules of components and dependencies, their management will be unwieldy no matter which approach you use 🙂 I suggest look for ways to script and automate as many parts as possible. Trust me, the novelty of playing with and chaining git commands wear out very quickly for most people, especially when deadlines are approaching. There is already a very good answer here on the comparison of the different approaches to manage git sub-projects.

    Regarding workflow, I will first separate repositories that are under your control from those that aren’t i.e. 3rd party repositories.

    For 3rd party repositories which don’t change often (either via merges or upstream PRs), you can still use submodules. Typically, you will point these submodules to the HEAD of some stable tags. Sync-ing them it’s just a matter of running (or scripting) git submodule update --recursive --remote. If these 3rd party dependencies can be specified in package management tools like bundler (for ruby projects), it will help to simplify your subprojects management.

    For repositories that your own and change often, either gitslave or git-subtree are two alternatives, depending on your team’s preferences.

    gitslave multiplexes git operations into multiple branches. IOW, when you branch, merge, commit, push, pull etc., each command will be run on the parent project and all slaves in turn. This mandates the team to work in a top-down manner, starting from the super-project down to the slaves.

    gitsubtree uses Git’s subtree merge functionality to achieve a similar effect as submodules, by actually storing the files in the main repository and merging in changes directly to that repository. The end result is a canonical repository with the option of including all the subprojects’ history. In a way, this allows team members to focus more on the subtrees they are responsible for, but will require extra work to merge back to the parent tree.

    As a developer, my preference is to work at the lower sub-projects level (to do my “red, green, refactor” cycle), and touch the parent projects only when necessary. But regardless of whether you choose a top-down or bottom-up workflow, try to identify repetitive error-prone steps in your branching & merging strategy, and script them as much as possible.

    Git Baby is a git and github fan, let's start git clone.