git clone S3 error: 403 Forbidden

I want to be able to sync git repositories to AWS S3 for backups. Furthermore I want the public to be able to git clone my backups. My steps were:

s3cmd mb s3://lktesting
git update-server-info
s3cmd -P sync .git/ s3://lktesting
s3cmd ws-create s3://lktesting
s3cmd ws-info s3://lktesting

I thought this used to work, but now I get:

git clone http://lktesting.s3-website-ap-southeast-1.amazonaws.com/
Cloning into 'lktesting.s3-website-ap-southeast-1.amazonaws.com'...
error: The requested URL returned error: 403 Forbidden (curl_result = 22, http_code = 403, sha1 = bf866b95d9517ea38e213740cead5cf1c313f5aa)
Checking connectivity... done.

Does anyone know what I am missing?

  • How can I remove the history of one file in git, leaving only the most recent version?
  • Tried clone files from Git repo. Only (.git) folder is downloaded and it has nothing
  • What's the best way to look up why/when changes were made in a git?
  • Can't push to new gitlab install
  • how to revert in Git?
  • .gitignore to ignore all files, then recursively allows files of a certain type
  • Using multiple Git accounts
  • How to use git behind a proxy
  • 3 Solutions collect form web for “git clone S3 error: 403 Forbidden”

    Git objects under .git may exists as single files or inside git packs. The Git dumb HTTP protocol will try to fetch an object as a single file, and only if this fails with “404 Not found”, it will look for a pack.

    Apparently, an Amazon S3 bucket will only return the 404 code if you give the “List” permission to everyone: How do I have an S3 bucket return 404 (instead of 403) for a key that does not exist in the bucket/

    Update: You can assign the necessary permission using AWS CLI with put-bucket-acl from s3api.

    Complete sequence of commands to host a clonable git repository in an S3 bucket:

    BUCKET=my-bucket-name
    
    # Setup
    aws s3 mb s3://$BUCKET
    aws s3api put-bucket-acl --bucket $BUCKET --acl public-read
    
    # Sync
    git update-server-info
    aws s3 sync --acl public-read .git s3://$BUCKET
    
    # Clone
    git clone https://$BUCKET.s3.amazonaws.com
    

    If you want to avoid any sync issue (like a .git/objects/... missing), do not sync the content of .git

    Use a git bundle in order to copy only one file representing the compressed version of your git repository (see “How can I email someone a git repository?”).
    That one file acts as a full-fledged git repo: you can git clone from it.

    cd /path/to/your/repo
    git bundle create /tmp/myrepo.bundle --all
    s3cmd -P sync /tmp/myrepo.bundle s3://lktesting
    git clone http://lktesting.s3-website-ap-southeast-1.amazonaws.com/myrepo.bundle
    

    You can not push to it though, so you might want to clone it direcly in your s3 instance, and clone from that uncompressed s3 repo.

    It looks like running the exact same approach with an empty repository works okay.

    Running the same command (git clone) with debug flags[0], while I get some contents copied locally, it looks like certain objects[1] referenced in the git repo aren’t present in the S3 bucket (403 is the default response code thrown when a key isn’t present). Did your sync complete fully?

    [0]

    GIT_CURL_VERBOSE=1 GIT_TRACE=1 git clone http://lktesting.s3-website-ap-southeast-1.amazonaws.com/
    [...]
    GET /objects/03/4261c96d614614344a1b618c8ec3d8d2ff7d3c HTTP/1.1
    Host: lktesting.s3-website-ap-southeast-1.amazonaws.com
    User-Agent: git/2.5.4 (Apple Git-61)
    Accept: */*
    
    * The requested URL returned error: 403 Forbidden
    

    [1] /objects/03/4261c96d614614344a1b618c8ec3d8d2ff7d3c

    Git Baby is a git and github fan, let's start git clone.