Authenticating REST Requests
Authentication is the process of proving your identity to the system. Identity is an important factor in Amazon S3 access control decisions. Requests are allowed or denied in part based on the identity of the requester. For example, the right to create buckets is reserved for registered developers and (by default) the right to create objects in a bucket is reserved for the owner of the bucket in question. As a developer, you'll be making requests that invoke these privileges so you'll need to prove your identity to the system by authenticating your requests. This section shows you how.
The content in this section does not apply to HTTP POST. For more information, see Browser-Based Uploads Using POST.
The Amazon S3 REST API uses a custom HTTP scheme based on a keyed-HMAC (Hash Message Authentication Code) for authentication. To authenticate a request, you first concatenate selected elements of the request to form a string. You then use your AWS Secret Access Key to calculate the HMAC of that string. Informally, we call this process "signing the request," and we call the output of the HMAC algorithm the "signature" because it simulates the security properties of a real signature. Finally, you add this signature as a parameter of the request, using the syntax described in this section.
When the system receives an authenticated request, it fetches the AWS Secret Access Key that you claim to have, and uses it in the same way to compute a "signature" for the message it received. It then compares the signature it calculated against the signature presented by the requester. If the two signatures match, then the system concludes that the requester must have access to the AWS Secret Access Key, and therefore acts with the authority of the principal to whom the key was issued. If the two signatures do not match, the request is dropped and the system responds with an error message.
Example Authenticated Amazon S3 REST Request
GET /photos/puppy.jpg HTTP/1.1 Host: johnsmith.s3.amazonaws.com Date: Mon, 26 Mar 2007 19:37:58 +0000 Authorization: AWS 0PN5J17HBGZHT7JJ3X82:frJIUN8DYpKDtOLCwo//yllqDzg=
The Authentication Header
The Amazon S3 REST API uses the standard HTTP
Authorization header to pass
authentication information. (The name of the standard header is unfortunate, sinceit
carries authentication information, not authorization).Under the Amazon S3 authentication
scheme, the Authorization header has the following form.
Developers are issued an AWS Access Key ID and AWS SecretAccess Key when they
register. For request authentication, the
identifies the secret key that was used to compute the signature, and(indirectly) the
developer making the request.
Signature element is the RFC 2104HMAC-SHA1 of selected
elements from the request, and so the
Signature part of the
Authorization header will vary from request to request. If the request signature
calculated by the system matches the
Signature included with the
request,then the requester will have demonstrated possession to the AWSSecret Access
Key. The request will then be processed under the identity, and with the authority, of
the developer to whom the key was issued.
Following is pseudo-grammar that illustrates the construction of the
Authorization request header (
\nmeans the Unicode code
U+000A commonly called newline).
Authorization = "AWS" + " " + AWSAccessKeyId + ":" + Signature; Signature = Base64( HMAC-SHA1( UTF-8-Encoding-Of( YourSecretAccessKeyID, StringToSign ) ) ); StringToSign = HTTP-Verb + "\n" + Content-MD5 + "\n" + Content-Type + "\n" + Date + "\n" + CanonicalizedAmzHeaders + CanonicalizedResource; CanonicalizedResource = [ "/" + Bucket ] + <HTTP-Request-URI, from the protocol name up to the query string> + [ sub-resource, if present. For example "?acl", "?location", "?logging", or "?torrent"]; CanonicalizedAmzHeaders = <described below>
HMAC-SHA1 is an algorithm defined by RFC 2104 (go to RFC
2104 - Keyed-Hashing for Message Authentication ). The algorithm takes as
input two byte-strings: a key and a message. For Amazon S3 Request authentication, use your
AWS Secret Access Key (
YourSecretAccessKeyID) as the key, and the
UTF-8 encoding of the
StringToSign as the message. The output of
HMAC-SHA1 is also a byte string, called the digest. The
request parameter is constructed by Base64 encoding this digest.
Request Canonicalization for Signing
Recall that when the system receives an authenticated request, it compares the
computed request signature with the signature provided in the request. In order for the
system-computed signature to match the developer-computed signature, the
StringToSign for a request must be constructed by both parties in
exactly the same way. We call the process of putting a request in an agreed-upon form
for signing "canonicalization".
Constructing the CanonicalizedResource Element
CanonicalizedResource represents the Amazon S3 resource targeted by
the request. Construct it for a REST request as follows:
Start with the empty string (
If the request specifies a bucket using the HTTP Host header
(virtual hosted-style), append the bucket name preceded by a
Append the path part of the un-decoded HTTP Request-URI, up-to but not including the query string.
If the request addresses a sub-resource, like
Elements of the CanonicalizedResource that come from the HTTP Request-URI should be signed literally as they appear in the HTTP request, including URL-Encoding metacharacters.
CanonicalizedResource might be different than the HTTP
Request-URI. In particular, if your request uses the HTTP
Host header to
specify a bucket, the bucket does appear in the HTTP Request-URI. However, the
CanonicalizedResource continues to include the bucket. Query
string parameters other than sub-resource flags (e.g., "?acl", "?location", "?logging",
or "?torrent") will also appear in the Request-URI but are not included in
CanonicalizedResource. For more information, see Virtual Hosting of Buckets.
Constructing the CanonicalizedAmzHeaders Element
To construct the CanonicalizedAmzHeaders part of
select all HTTP request headers that start with 'x-amz-' (using a case-insensitive
comparison) and use the following process.
|1||Convert each HTTP header name to lower-case. For example,
|2||Sort the collection of headers lexicographically by header name.|
|3||Combine header fields with the same name into one
"header-name:comma-separated-value-list" pair as prescribed by RFC 2616,
section 4.2, without any white-space between values. For example, the
two metadata headers '
|4||"Unfold" long headers that span multiple lines (as allowed by RFC 2616, section 4.2) by replacing the folding white-space (including new-line) by a single space.|
|5||Trim any white-space around the colon in the header. For example, the
|6|| Finally, append a new-line (
Positional versus Named HTTP Header StringToSign Elements
The first few header elements of
Date, and Content-MD5) are positional in nature.
does not include the names of these headers, only their values from the request. In
contrast, the '
x-amz-' elements are named; Both the header names and the
header values appear in
If a positional header called for in the definition of
StringToSign is not present in your request, (
Content-MD5, for example, are optional for PUT requests, and meaningless
for GET requests), substitute the empty string ("") in for that position.
Time Stamp Requirement
A valid time stamp (using either the HTTP
Date header or an
x-amz-date alternative) is mandatory for authenticated requests.
Furthermore, the client time-stamp included with an authenticated request must be within
15 minutes of the Amazon S3 system time when the request is received. If not, the request
will fail with the
RequestTimeTooSkewed error status code. The
intention of these restrictions is to limit the possibility that intercepted requests
could be replayed by an adversary. For stronger protection against eavesdropping, use
the HTTPS transport for authenticated requests.
Some HTTP client libraries do not expose the ability to set the
header for a request. If you have trouble including the value of the 'Date' header in
the canonicalized headers, you can set the time-stamp for the request using an
x-amz-date' header instead. The value of the
header must be in one of the RFC 2616 formats (http://www.ietf.org/rfc/rfc2616.txt). When an
x-amz-date header is present in a request, the system will ignore any
Date header when computing the request signature. Therefore, if you
x-amz-date header, use the empty string for the
Date when constructing the
StringToSign. See the
next section for an example.
The examples in this section use the (non-working) credentials in the following table.
In the example
StringToSigns, formatting is not significant and
\n means the Unicode code point
U+000A commonly called
Example Object GET
This example gets an object from the johnsmith bucket.
GET /photos/puppy.jpg HTTP/1.1 Host: johnsmith.s3.amazonaws.com Date: Tue, 27 Mar 2007 19:36:42 +0000
GET\n \n \n Tue, 27 Mar 2007 19:36:42 +0000\n /johnsmith/photos/puppy.jpg
Note that the CanonicalizedResource includes the bucket name, but the HTTP Request-URI does not (it is specified by the Host header)
Example Object PUT
This example puts an object into the johnsmith bucket.
PUT /photos/puppy.jpg HTTP/1.1 Content-Type: image/jpeg Content-Length: 94328 Host: johnsmith.s3.amazonaws.com Date: Tue, 27 Mar 2007 21:15:45 +0000
PUT\n \n \n image/jpeg\n Tue, 27 Mar 2007 21:15:45 +0000\n /johnsmith/photos/puppy.jpg
Note the Content-Type header in the request and in the StringToSign. Also note that the Content-MD5 is left blank in the StringToSign since it is not present in the request.
This example lists the content of the johnsmith bucket.
GET /?prefix=photos&max-keys=50&marker=puppy HTTP/1.1 User-Agent: Mozilla/5.0 Host: johnsmith.s3.amazonaws.com Date: Tue, 27 Mar 2007 19:42:41 +0000
GET\n \n \n Tue, 27 Mar 2007 19:42:41 +0000\n /johnsmith/
Note the trailing slash on the CanonicalizedResource, and the absence of query string parameters.
This example fetches the access control policy sub-resource for the 'johnsmith' bucket.
GET /?acl HTTP/1.1 Host: johnsmith.s3.amazonaws.com Date: Tue, 27 Mar 2007 19:44:46 +0000
GET\n \n \n Tue, 27 Mar 2007 19:44:46 +0000\n /johnsmith/?acl
Notice how the sub-resource query string parameter is included in the CanonicalizedResource.
This example deletes an object from the 'johnsmith' bucket using the path-style and Date alternative.
DELETE /johnsmith/photos/puppy.jpg HTTP/1.1 User-Agent: dotnet Host: s3.amazonaws.com Date: Tue, 27 Mar 2007 21:20:27 +0000 x-amz-date: Tue, 27 Mar 2007 21:20:26 +0000
DELETE\n \n \n \n x-amz-date:Tue, 27 Mar 2007 21:20:26 +0000\n /johnsmith/photos/puppy.jpg
Note how we used the alternate 'x-amz-date' method of specifying the date (because our client library prevented us from setting the date, say). In this case the field for the actual 'Date' header is left blank in the StringToSign.
This example uploads an object to a CNAME style virtual hosted bucket with metadata.
PUT /db-backup.dat.gz HTTP/1.1 User-Agent: curl/7.15.5 Host: static.johnsmith.net:8080 Date: Tue, 27 Mar 2007 21:06:08 +0000 x-amz-acl: public-read content-type: application/x-download Content-MD5: 4gJE4saaMU4BqNR0kLY+lw== X-Amz-Meta-ReviewedBy: [email protected] X-Amz-Meta-ReviewedBy: [email protected] X-Amz-Meta-FileChecksum: 0x02661779 X-Amz-Meta-ChecksumAlgorithm: crc32 Content-Disposition: attachment; filename=database.dat Content-Encoding: gzip Content-Length: 5913339
PUT\n 4gJE4saaMU4BqNR0kLY+lw==\n application/x-download\n Tue, 27 Mar 2007 21:06:08 +0000\n x-amz-acl:public-read\n x-amz-meta-checksumalgorithm:crc32\n x-amz-meta-filechecksum:0x02661779\n x-amz-meta-reviewedby: [email protected],[email protected]\n /static.johnsmith.net/db-backup.dat.gz
Notice how the 'x-amz-' headers are sorted, white-space trimmed, converted to lowercase, and multiple headers with the same name have been joined using a comma to separate values.
Note how only the
entity headers appear in the
StringToSign. The other
Content-* entity headers do not.
Again, note that the
CanonicalizedResource includes the
bucket name, but the HTTP Request-URI does not (the bucket is specified by the Host
Example List All My Buckets
GET / HTTP/1.1 Host: s3.amazonaws.com Date: Wed, 28 Mar 2007 01:29:59 +0000
GET\n \n \n Wed, 28 Mar 2007 01:29:59 +0000\n /
Example Unicode Keys
GET /dictionary/fran%C3%A7ais/pr%c3%a9f%c3%a8re HTTP/1.1 Host: s3.amazonaws.com Date: Wed, 28 Mar 2007 01:49:49 +0000
GET\n \n \n Wed, 28 Mar 2007 01:49:49 +0000\n /dictionary/fran%C3%A7ais/pr%c3%a9f%c3%a8re
The elements in
REST Request Signing Problems
When REST request authentication fails, the system responds to the request with an
XML error document. The information contained in this error document is meant to help
developers diagnose the problem. In particular, the
element of the
SignatureDoesNotMatch error document tells you
exactly what request canonicalization the system is using.
Some toolkits silently insert headers that you do not know about beforehand, such as
adding the header
Content-Type during a PUT. In most of these cases, the
value of the inserted header remains constant, allowing you to discover the missing
headers using tools such as Ethereal or tcpmon.
Query String Request Authentication Alternative
You can authenticate certain types of requests by passing the required information as
query-string parameters instead of using the
Authorization HTTP header.
This is useful for enabling direct third-party browser access to your private Amazon S3 data,
without proxying the request. The idea is to construct a "pre-signed" request and encode
it as a URL that an end-user's browser can retrieve. Additionally, you can limit a
pre-signed request by specifying an expiration time.
Following is an example query string authenticated Amazon S3 REST request.
GET /photos/puppy.jpg ?AWSAccessKeyId=0PN5J17HBGZHT7JJ3X82&Expires=1141889120&Signature=vjbyPxybdZaNmGa%2ByT272YEAiv4%3D HTTP/1.1 Host: johnsmith.s3.amazonaws.com Date: Mon, 26 Mar 2007 19:37:58 +0000
The query string request authentication method doesn't require any special HTTP headers. Instead, the required authentication elements are specified as query string parameters:
|Query String Parameter Name||Example Value||Description|
||Your AWS Access Key Id. Specifies the AWS Secret Access Key used to sign the request, and (indirectly) the identity of the developer making the request.|
||The time when the signature expires, specified as the number of seconds since the epoch (00:00:00 UTC on January 1, 1970). A request received after this time (according to the server), will be rejected.|
||The URL encoding of the Base64 encoding of the HMAC-SHA1 of StringToSign.|
The query string request authentication method differs slightly from the ordinary
method but only in the format of the
Signature request parameter
StringToSign element. Following is pseudo-grammar that
illustrates the query string request authentication method.
Signature = URL-Encode( Base64( HMAC-SHA1( YourSecretAccessKeyID, UTF-8-Encoding-Of( StringToSign ) ) ) ); StringToSign = HTTP-VERB + "\n" + Content-MD5 + "\n" + Content-Type + "\n" + Expires + "\n" + CanonicalizedAmzHeaders + CanonicalizedResource;
YourSecretAccessKeyID is the AWS Secret Access Key ID Amazon
assigns to you when you sign up to be an Amazon Web Service developer. Notice how the
Signature is URL-Encoded to make it suitable for placement in
the query-string. Also note that in
StringToSign, the HTTP
Date positional element has been replaced with
CanonicalizedResource are the same.
Example Query String Request Authentication
GET /photos/puppy.jpg?AWSAccessKeyId=0PN5J17HBGZHT7JJ3X82& Signature=rucSbH0yNEcP9oM2XNlouVI3BH4%3D& Expires=1175139620 HTTP/1.1 Host: johnsmith.s3.amazonaws.com
GET\n \n \n 1175139620\n /johnsmith/photos/puppy.jpg
We assume that when a browser makes the GET request, it won't provide a
Content-MD5 or a Content-Type header, nor will it set any x-amz- headers, so those
parts of the
StringToSign are left blank.