My project uses over 100 git submodules, which submodule alternative can handle a lot of repositories gracefully

I’ve been researching git subtree and other alternatives to git submodules. My project has well over 100 submodules and it’s very unwieldy to manage them all.

Can anyone recommend a workflow that works really well with a large number of repositories that need to be kept in sync.

  • Jar Dependencies in GitHub
  • Resolve recursive git cookbook dependencies with Berkshelf
  • Pulling my own private repository from Bitbucket via Composer
  • How to handle dependencies when using git topic branch workflow?
  • Convert Doc to PDf with documents4j
  • Subversion and dependencies
  • splitting a git repository
  • Recover a commit sent as a pull-request from a deleted fork on GitHub
  • Git branch merged on local repo but still visible on Github
  • Best practice with .gitignore for connection strings inside App.config or Web.config
  • Edit a commit with gitpython
  • GIT creating extraneous ~HEAD files during rebase
  • One Solution collect form web for “My project uses over 100 git submodules, which submodule alternative can handle a lot of repositories gracefully”

    If you project has over 100 git submodules of components and dependencies, their management will be unwieldy no matter which approach you use 🙂 I suggest look for ways to script and automate as many parts as possible. Trust me, the novelty of playing with and chaining git commands wear out very quickly for most people, especially when deadlines are approaching. There is already a very good answer here on the comparison of the different approaches to manage git sub-projects.

    Regarding workflow, I will first separate repositories that are under your control from those that aren’t i.e. 3rd party repositories.

    For 3rd party repositories which don’t change often (either via merges or upstream PRs), you can still use submodules. Typically, you will point these submodules to the HEAD of some stable tags. Sync-ing them it’s just a matter of running (or scripting) git submodule update --recursive --remote. If these 3rd party dependencies can be specified in package management tools like bundler (for ruby projects), it will help to simplify your subprojects management.

    For repositories that your own and change often, either gitslave or git-subtree are two alternatives, depending on your team’s preferences.

    gitslave multiplexes git operations into multiple branches. IOW, when you branch, merge, commit, push, pull etc., each command will be run on the parent project and all slaves in turn. This mandates the team to work in a top-down manner, starting from the super-project down to the slaves.

    gitsubtree uses Git’s subtree merge functionality to achieve a similar effect as submodules, by actually storing the files in the main repository and merging in changes directly to that repository. The end result is a canonical repository with the option of including all the subprojects’ history. In a way, this allows team members to focus more on the subtrees they are responsible for, but will require extra work to merge back to the parent tree.

    As a developer, my preference is to work at the lower sub-projects level (to do my “red, green, refactor” cycle), and touch the parent projects only when necessary. But regardless of whether you choose a top-down or bottom-up workflow, try to identify repetitive error-prone steps in your branching & merging strategy, and script them as much as possible.

    Git Baby is a git and github fan, let's start git clone.