Process only changed files

With jenkins I want to process periodically only changed files from SVN and commit the output of the processing back to SVN.

We are committing binary files into SVN (we are working with Oracle Forms and are committing fmb-Files). I created a script which exports the fmb’s to xml (with the original Fmb2XML tool from Oracle) and than I convert the XML to plain source which we also want to commit. This allows us greping, viewing the changes, ….

At the moment I am only able to checkout everything, convert the whole directory and committing the whole directory back to SVN. But since all plain text files are newly generated they appear changed in SVN. I want to commit only the changed ones.

Can anyone help me with this?

  • Merge two git repository histories
  • SVN: How to make a file read-only with subversion?
  • Strategy to migrate from StarTeam 5 to Subversion?
  • How to import svn branch to git with trunk history
  • svn deployment strategies for multiple groups of developers (not co-located) working on different components of the same project
  • Gitolite mirroring on SVN server
  • xcode change from svn to git
  • SVNserve giving Error 1053 The service did not respond to the start or control request
  • 2 Solutions collect form web for “Process only changed files”

    I installed the Groovy plugin, configured the Groovy language and created a script which I execute as “system Groovy script”. The scripts looks like:

    import java.lang.ProcessBuilder.Redirect
    import hudson.model.*
    import hudson.util.*
    import hudson.scm.*
    import hudson.scm.SubversionChangeLogSet.LogEntry
    // uncomment one of the following def build = ... lines
    // work with current build
    def build = Thread.currentThread()?.executable
    // for testing, use last build or specific build number
    //def item = hudson.model.Hudson.instance.getItem("Update_SRC_Branch") 
    //def build = item.getLastBuild()   
    //def build = item.getBuildByNumber(35)   
    // get ChangesSets with all changed items
    def changeSet= build.getChangeSet()
    List<LogEntry> items = changeSet.getItems()
    def affectedFiles = items.collect { it.paths }
    // get filtered file names (only fmb) without path
    def fileNames = affectedFiles.flatten().findResults {
        if (it.path.substring(it.path.lastIndexOf(".") + 1) != "fmb") return null
        it.path.substring(it.path.lastIndexOf("/") + 1)
    // setup log files
    def stdOutFile = "${build.rootDir}\\stdout.txt"
    def stdErrFile = "${build.rootDir}\\stderr.txt"
    // now execute the external transforming
    fileNames.each {
        def params = [...]
        def processBuilder = new ProcessBuilder(params)
        // redirect stdout and stderr to log files
        processBuilder.redirectOutput(new File(stdOutFile))
        processBuilder.redirectError(new File(stdErrFile))
        def process = processBuilder.start()
        // print log files
        println new File(stdOutFile).readLines()
        System.err.println new File(stdErrFile).readLines()

    Afterwards I use command line with “svn commit” to commit the updated files.

    Preliminary note: getting files from repo in SVN-jargon is “checkout”, saving to repo – “commit”. Don’t mix CVS and SVN terms, it can lead to misinterpretation

    In order to get list of changed files in revision (or revset) you can use

    • Easy way – svn log with options -q -v. For single revision you also add -c REVNO, for revision range: -r REVSTART:REVEND. Probably additional –xml will produce more suitable output, than plain-text

    You must to post-process output of log in order to get pure list, because: log contain some useless for you data, in case of log for range you can have the same file included in more than one revision

    z:\>svn log -q -v -r 1190
    r1190 | lazybadger | 2012-09-20 13:19:45 +0600 (Чт, 20 сен 2012)
    Changed paths:
       M /trunk/Abrikos.ini
       M /trunk/ER-Telecom.ini
       M /trunk/GorNet.ini
       M /trunk/KrosLine.ini
       M /trunk/Rostelecom.ini
       M /trunk/Vladlink.ini

    example of single revision: you have to log | grep trunk | sort -u, add repo-base to filenames

    • Harder way: with additional SCM (namely – Mercurial) and hgsubversion you’ll get slightly more (maybe) log with hg log --template "{files}\n" – only slightly because you’ll get only filelist, but filesets in different revisions are newline-separated, filenames inside revision are space-separated
    Git Baby is a git and github fan, let's start git clone.