docker add cache when git checkout same file

I need checkout project in my CI server build image reusing docker cache.

Docker ADD not use cache when checkout same file.

  • Dockerhub automated builds with multiple private repos
  • Dockerfile strategies for Git
  • How to manage secrets in a Microservice / Container / Cloud environment?
  • Run docker after building own image
  • npm install fails in jenkins pipeline in docker
  • Dockerfile versioning best practice
  • I am in git branch A execute docker build -t somename . it use docker cache normally, but if I go to branch B via git checkout B, do nothing, go to branch A via git checkout A and run again docker build -t somename . the docker cache is using only before the first ADD.

    here example:

    Dockerfile

    # DOCKER-VERSION 0.10.0
    FROM myregistry:5000/ruby:2.1.2-1
    MAINTAINER ME
    
    # Gem sources
    RUN gem source -r https://rubygems.org/
    RUN gem source -a http://gems.mydomain
    
    # Never install a ruby gem docs
    RUN echo "gem: --no-rdoc --no-ri" >> ~/.gemrc
    
    # gems install
    RUN mkdir /foo
    WORKDIR /foo
    
    RUN gem install bundler
    ADD Gemfile /foo/Gemfile
    RUN bundle install
    
    # expose ports
    EXPOSE 8080
    

    log from build

    Sending build context to Docker daemon 19.54 MB
    Sending build context to Docker daemon 
    Step 0 : FROM myregistry:5000/ruby:2.1.2-1
     ---> 9ce683a713b4
    Step 1 : MAINTAINER ME
     ---> Using cache
     ---> 8f54114fd2e7
    Step 2 : RUN gem source -r https://rubygems.org/
     ---> Using cache
     ---> f58a08708863
    Step 3 : RUN gem source -a http://gems.mydomain
     ---> Using cache
     ---> 3e69e17c5954
    Step 4 : RUN echo "gem: --no-rdoc --no-ri" >> ~/.gemrc
     ---> Using cache
     ---> 1edb37962dd4
    Step 5 : RUN mkdir /foo
     ---> Running in 3d3441c34ee3
     ---> aeb90f00bc9b
    Removing intermediate container 3d3441c34ee3
    Step 6 : WORKDIR /foo
     ---> Running in 84b881d8f621
     ---> 10e1d8984458
    Removing intermediate container 84b881d8f621
    Step 7 : RUN gem install bundler
     ---> Running in 31e98523ce46
    Successfully installed bundler-1.6.2
    1 gem installed
     ---> 84d5195ab831
    Removing intermediate container 31e98523ce46
    Step 8 : ADD Gemfile /foo/Gemfile
     ---> 3e5f2675ee22
    Removing intermediate container c90e8be5ea17
    Step 9 : RUN bundle install
     ---> Running in ac0e83e5eebb    
    Fetching gem metadata from http://gems.mydomain/......
    Fetching additional metadata from http://gems.mydomain/..
    Resolving dependencies...
    Installing rake 10.3.2
    Installing i18n 0.6.9
    Installing multi_json 1.10.1
    .
    .
    .
    Installing railties 3.2.19
    Installing responders 0.9.3
    Using bundler 1.6.2
    Your bundle is complete!
    Use `bundle show [gemname]` to see where a bundled gem is installed.    
     ---> 19beae703adb
    Removing intermediate container ac0e83e5eebb
    Step 10 : EXPOSE 8080
     ---> Running in 1b1e55d349e5
     ---> 32405bdac6d1
    Removing intermediate container 1b1e55d349e5
    Successfully built 32405bdac6d1
    

    git checkout B

    git checkout A

    docker build -t somename .

    second build log

    Sending build context to Docker daemon 19.52 MB
    Sending build context to Docker daemon 
    Step 0 : FROM myregistry:5000/ruby:2.1.2-1
     ---> 9ce683a713b4
    Step 1 : MAINTAINER ME
     ---> Using cache
     ---> 8f54114fd2e7
    Step 2 : RUN gem source -r https://rubygems.org/
     ---> Using cache
     ---> f58a08708863
    Step 3 : RUN gem source -a http://gems.mydomain
     ---> Using cache
     ---> 3e69e17c5954
    Step 4 : RUN echo "gem: --no-rdoc --no-ri" >> ~/.gemrc
     ---> Using cache
     ---> 1edb37962dd4
    Step 5 : RUN mkdir /foo
     ---> Using cache
     ---> aeb90f00bc9b
    Step 6 : WORKDIR /foo
     ---> Using cache
     ---> 10e1d8984458
    Step 7 : RUN gem install bundler
     ---> Using cache
     ---> 84d5195ab831
    Step 8 : ADD Gemfile /foo/Gemfile
     ---> 4977e35c80f7
    Removing intermediate container bd59cc0d5e51
    Step 9 : RUN bundle install
     ---> Running in 6ff16f32e94a
    Fetching gem metadata from http://gems.mydomain/......
    Fetching additional metadata from http://gems.mydomain/..
    Resolving dependencies...
    Installing rake 10.3.2
    Installing i18n 0.6.9
    Installing multi_json 1.10.1
    .
    .
    .    
    Installing railties 3.2.19
    Installing responders 0.9.3
    Using bundler 1.6.2
    Your bundle is complete!
    Use `bundle show [gemname]` to see where a bundled gem is installed.
     ---> d9332f9035c3
    Removing intermediate container 6ff16f32e94a
    Step 10 : EXPOSE 8080
     ---> Running in b20252a00160
     ---> 4d9932882e06
    Removing intermediate container b20252a00160
    Successfully built 4d9932882e06
    

  • EGit - pull= failled and no conflicts
  • Git Extensions: No Windows Explorer shell on Windows 7 and all commands disabled in vs 2010 plugin?
  • How to re-merge a branch, after a partial merge was performed
  • How to receive local Git branch name with Jenkins Git plugin?
  • Git: ignore specific file(s) both on Commit and Pull
  • gitolite repository getting created but projects.list is empty
  • 2 Solutions collect form web for “docker add cache when git checkout same file”

    Docker invalidates the docker build cache when a file’s mtime value has changed, and git does not track file mtime values. This cache invalidation also shows up in other situations too, like in continuous integration or build environments involving docker, git and branches.

    I have been using a “touch” target in a Makefile that I run prior to asking docker to build a container:

    touch:
        @echo "Reset timestamps on git working directory files..."
        find ./ | grep -v .git | xargs touch -t 200001010000.00
    

    Next, always run make touch prior to docker build or any docker-based “build” target in the same Makefile…

    Another option is to setup a git hook that modifies mtime values automatically:
    https://git.wiki.kernel.org/index.php/ExampleScripts#Setting_the_timestamps_of_the_files_to_the_commit_timestamp_of_the_commit_which_last_touched_them

    Another possible solution is to fork docker and remove mtime from its definition of caches:
    https://github.com/docker/docker/blob/master/pkg/tarsum/tarsum.go

    Note: as of docker 1.8, mtime is no longer taken into account
    when invalidating the cache. Pull request #12031 updated this behavior

    git checkout, git clone, git fetch, an so on modify the creation date of file. Then docker see another file even is the same.

    docker ADD command fails caching when creation date of file change.

    Git Baby is a git and github fan, let's start git clone.