git clone S3 error: 403 Forbidden
I want to be able to sync git repositories to AWS S3 for backups. Furthermore I want the public to be able to
git clone my backups. My steps were:
s3cmd mb s3://lktesting git update-server-info s3cmd -P sync .git/ s3://lktesting s3cmd ws-create s3://lktesting s3cmd ws-info s3://lktesting
I thought this used to work, but now I get:
git clone http://lktesting.s3-website-ap-southeast-1.amazonaws.com/ Cloning into 'lktesting.s3-website-ap-southeast-1.amazonaws.com'... error: The requested URL returned error: 403 Forbidden (curl_result = 22, http_code = 403, sha1 = bf866b95d9517ea38e213740cead5cf1c313f5aa) Checking connectivity... done.
Does anyone know what I am missing?
3 Solutions collect form web for “git clone S3 error: 403 Forbidden”
Git objects under
.git may exists as single files or inside git packs. The Git dumb HTTP protocol will try to fetch an object as a single file, and only if this fails with “404 Not found”, it will look for a pack.
Apparently, an Amazon S3 bucket will only return the 404 code if you give the “List” permission to everyone: How do I have an S3 bucket return 404 (instead of 403) for a key that does not exist in the bucket/
Update: You can assign the necessary permission using AWS CLI with put-bucket-acl from s3api.
Complete sequence of commands to host a clonable git repository in an S3 bucket:
BUCKET=my-bucket-name # Setup aws s3 mb s3://$BUCKET aws s3api put-bucket-acl --bucket $BUCKET --acl public-read # Sync git update-server-info aws s3 sync --acl public-read .git s3://$BUCKET # Clone git clone https://$BUCKET.s3.amazonaws.com
If you want to avoid any sync issue (like a
.git/objects/... missing), do not sync the content of
git bundle in order to copy only one file representing the compressed version of your git repository (see “How can I email someone a git repository?”).
That one file acts as a full-fledged git repo: you can
git clone from it.
cd /path/to/your/repo git bundle create /tmp/myrepo.bundle --all s3cmd -P sync /tmp/myrepo.bundle s3://lktesting git clone http://lktesting.s3-website-ap-southeast-1.amazonaws.com/myrepo.bundle
You can not push to it though, so you might want to clone it direcly in your s3 instance, and clone from that uncompressed s3 repo.
It looks like running the exact same approach with an empty repository works okay.
Running the same command (git clone) with debug flags, while I get some contents copied locally, it looks like certain objects referenced in the git repo aren’t present in the S3 bucket (403 is the default response code thrown when a key isn’t present). Did your sync complete fully?
GIT_CURL_VERBOSE=1 GIT_TRACE=1 git clone http://lktesting.s3-website-ap-southeast-1.amazonaws.com/ [...] GET /objects/03/4261c96d614614344a1b618c8ec3d8d2ff7d3c HTTP/1.1 Host: lktesting.s3-website-ap-southeast-1.amazonaws.com User-Agent: git/2.5.4 (Apple Git-61) Accept: */* * The requested URL returned error: 403 Forbidden