git clone S3 error: 403 Forbidden

I want to be able to sync git repositories to AWS S3 for backups. Furthermore I want the public to be able to git clone my backups. My steps were:

s3cmd mb s3://lktesting
git update-server-info
s3cmd -P sync .git/ s3://lktesting
s3cmd ws-create s3://lktesting
s3cmd ws-info s3://lktesting

I thought this used to work, but now I get:

  • Best strategy to deploy static site to s3 on github push?
  • Amazon secret keys and Heroku
  • Git deleting things mysteriously (edit: actually django-storages)
  • Amazon S3, jgit Push a really large file
  • Add encrypted variables to travis.yml and push to open source project
  • Heroku: Using external mount in local filesystem
  • git clone http://lktesting.s3-website-ap-southeast-1.amazonaws.com/
    Cloning into 'lktesting.s3-website-ap-southeast-1.amazonaws.com'...
    error: The requested URL returned error: 403 Forbidden (curl_result = 22, http_code = 403, sha1 = bf866b95d9517ea38e213740cead5cf1c313f5aa)
    Checking connectivity... done.
    

    Does anyone know what I am missing?

  • Should I gitignore my .gitignore?
  • git global config issue
  • Recover a commit sent as a pull-request from a deleted fork on GitHub
  • Deploying project to live server with Gitlab
  • Your configuration specifies to merge with the <branch name> from the remote, but no such ref was fetched.?
  • git submodules , gitslave, git subtree or a simpler solution
  • 3 Solutions collect form web for “git clone S3 error: 403 Forbidden”

    Git objects under .git may exists as single files or inside git packs. The Git dumb HTTP protocol will try to fetch an object as a single file, and only if this fails with “404 Not found”, it will look for a pack.

    Apparently, an Amazon S3 bucket will only return the 404 code if you give the “List” permission to everyone: How do I have an S3 bucket return 404 (instead of 403) for a key that does not exist in the bucket/

    Update: You can assign the necessary permission using AWS CLI with put-bucket-acl from s3api.

    Complete sequence of commands to host a clonable git repository in an S3 bucket:

    BUCKET=my-bucket-name
    
    # Setup
    aws s3 mb s3://$BUCKET
    aws s3api put-bucket-acl --bucket $BUCKET --acl public-read
    
    # Sync
    git update-server-info
    aws s3 sync --acl public-read .git s3://$BUCKET
    
    # Clone
    git clone https://$BUCKET.s3.amazonaws.com
    

    If you want to avoid any sync issue (like a .git/objects/... missing), do not sync the content of .git

    Use a git bundle in order to copy only one file representing the compressed version of your git repository (see “How can I email someone a git repository?”).
    That one file acts as a full-fledged git repo: you can git clone from it.

    cd /path/to/your/repo
    git bundle create /tmp/myrepo.bundle --all
    s3cmd -P sync /tmp/myrepo.bundle s3://lktesting
    git clone http://lktesting.s3-website-ap-southeast-1.amazonaws.com/myrepo.bundle
    

    You can not push to it though, so you might want to clone it direcly in your s3 instance, and clone from that uncompressed s3 repo.

    It looks like running the exact same approach with an empty repository works okay.

    Running the same command (git clone) with debug flags[0], while I get some contents copied locally, it looks like certain objects[1] referenced in the git repo aren’t present in the S3 bucket (403 is the default response code thrown when a key isn’t present). Did your sync complete fully?

    [0]

    GIT_CURL_VERBOSE=1 GIT_TRACE=1 git clone http://lktesting.s3-website-ap-southeast-1.amazonaws.com/
    [...]
    GET /objects/03/4261c96d614614344a1b618c8ec3d8d2ff7d3c HTTP/1.1
    Host: lktesting.s3-website-ap-southeast-1.amazonaws.com
    User-Agent: git/2.5.4 (Apple Git-61)
    Accept: */*
    
    * The requested URL returned error: 403 Forbidden
    

    [1] /objects/03/4261c96d614614344a1b618c8ec3d8d2ff7d3c

    Git Baby is a git and github fan, let's start git clone.