How to migrate all URLs in svn:externals properties across a repository?

We are in the process of moving our SVN repositories from one machine to another one, and with it will come a new domain name for the new repo. The problem is, that within the repository, there are lots of svn:externals references to other projects within the repository. So for example, we have projectA, which has in the svn:externals properties:

external/libraryA svn://oldserver.net/repo/libraryA
external/libraryB svn://oldserver.net/repo/libraryB

…and so on. All of the URL’s reference this particular domain name, so it can be easily parsed. Having already learned my lesson, I will migrate these URLs to be “svn://localhost/”, but I need to find a way to go through the repository history and rewrite all of the old URLs, so that we can still check out older revisions of these projects without having broken links.

  • Don't show svn:externals in svn status
  • Trying to create a pre-commit hook that verify externals
  • Subversion - where should the svn:externals come from?
  • What's the benefits of “svn:externals”?
  • svn externals … yes or no?
  • Help in understanding SVN Externals
  • How would I go about doing this?

  • git project vs repository, what's the fundamental difference?
  • How to create a repository with ssh:// access on a server
  • Correspondece between Git client and server versions
  • How do I clone a subdirectory only of a Git repository?
  • What are the purpose of the bare git repositories in .repo/projects/ created by the Android repo script?
  • Export git repository with history into another repository
  • 6 Solutions collect form web for “How to migrate all URLs in svn:externals properties across a repository?”

    As you indicated that you still want to be able to check out older revisions, the only solution is really to “rewrite” the entire history (solution D mentioned earlier).

    To do this, you should:

    1) Dump the contents of the entire repository using svnadmin dump:

    $ svnadmin dump /path/to/repos > original-dumpfile
    * Dumped revision 0.
    * Dumped revision 1.
    * Dumped revision 2.
    * Dumped revision 3.
    

    2) Edit the dump file, to change the svn:externals URLs. This is the most difficult part: Assuming the repository contains binary data as well, opening the dump file in a plain text editor will most likely corrupt the dump file. I’ve had good experiences using a so-called “hex-editor”, for instance the Freeware Hex Editor XVI32

    3) Create a new repository and load the modified dumpfile into it:

    $ svnadmin create newrepos
    $ svnadmin load newrepos < modified-dumpfile
    

    For more information, you might also be interested in this link:
    http://svnbook.red-bean.com/en/1.1/ch05s03.html

    NOTE: Subversion 1.5 actually added support for relative URLs in the svn:externals property, which can precisely prevent these sort of problems in the future:
    http://subversion.tigris.org/svn_1.5_releasenotes.html#externals

    I’d use SvnDumpTool for this. It has exactly what you’re looking for:

    svndumptool transform-prop svn:externals "(\S*) (|-r ?\d* ?)http://oldserver.net(/\S*)" "\2\3 \1" source.dumpfile source-fixed-externals.dumpfile
    

    This fixes up each external to the subversion 1.5 format, and uses relative URLs.

    So svn:externals like:

    external/libraryA svn://oldserver.net/repo/libraryA
    

    become:

     /repo/libraryA external/libraryA
    

    using server root relative URLs.

    I edited my dump file with vi but I had to use the “-b” switch to edit in binary mode such that any characters that could be interpreted to be line endings did not get converted.

    e.g. vi -b filename.dump

    Also, I found that, if your URL length changes, there are string lengths that also had to be modified.
    For example, consider an entry that looks like this:

    Node-path: trunk/src/include

    Node-kind: dir

    Node-action: change

    Prop-content-length: 192

    Content-length: 192

    K13

    svn:externals

    V 156

    MGL_ABC svn://server_name/dir1/dir2

    MGL_DEF svn://server_name/dir1/dir3

    When you modify those URLs, if the length of the string changes, you need to also change the “192”, “192” and “156” to match the new length.
    I found it difficult to compute the absolute length but easy to find the differential.
    For example, let’s say URL 1 becomes shorter by 3 characters and URL 2 becomes shorter by 4 characters. Then, you would have to subract ‘7’ from each of those three string length numbers.

    You could:

    a) check out the old revision, and change your hosts-file to point the old name to the new address, then svn update. In case the URL-path also changed… well then you might as well:

    b) take the time to write a script that find the properties in the current (old revision-) working copy and changes the URLs there, without committing them. OR:

    c) make a note of the revision(-s) where you checked in the new property values, check out the old version, and simply do a merge those revisions (-that only affect the properties) into your working copy.

    d) or, possibly, use svndump to dump the repository data, string-replace the URL in the dump, then restore it.. I would not give you any guarantee that that even works 😉

    I had to relocate 12 working copies across 9 users and 4 deployments. It was a simple change, replacing a domain with an IP, i.e. thing.domain.net -> 192.168.0.1

    Expecting svn relocate to behave as described (traverse nested externals) I wrote a simple DOS instruction to run at each location:

    for /D %G in (*) do (
    cd ./%G
    & svn relocate http://thing.domain.net http://192.168.0.1
    & cd ..)

    This didn’t work as expected, only relocating the parent WC.

    My solution was to edit the repositories themselves (I used Tortoise Repo Browser) to change the location of the externals. Following this change an update to the relocated parent was all that was required to bring everything into line.

    It would probably be a good idea to get all the Tortoise users to clear their URL history so they don’t inadvertently perform operations using the old URL (it still exists in the DNS lookup):

    Settings->Saved Data->URL history->Clear

    All my externals were in directories named flow. I fixed the URLs in my externals with this one-liner (bash shell):

    for p in $(find -maxdepth 4 -name flow); do svn ps svn:externals "$(svn pg svn:externals $p/.. | perl -pe 's/^(\w+) svn\+ssh.*thing\.domain\.net(.*)/$2 $1/')" $p/..; done
    
    Git Baby is a git and github fan, let's start git clone.