Setting Up Server Access Logging

Amazon Simple Storage Service: 2006-03-01

Setting Up Server Access Logging

	Important
	This section describes Beta functionality that is subject to change in future releases. Please provide feedback on this functionality in the

Amazon S3 Developer Forum.

The Amazon S3 server access logging feature lets you generate access log files for buckets that you own. These log files are delivered to you by writing them into a (possibly different) bucket that you own. Once delivered, the access logs are ordinary objects that you can read, list or delete at your convenience.

These instructions assume that you want to enable server access logging on one of your pre-existing buckets, and that you want to have those logs delivered into a new bucket you will create just for logging. We suppose that the bucket you want to log access to is called 'mybucket' and the new bucket you will create to hold your access logs is called 'mylogs'. This makes 'mybucket' the source bucket for logging and 'mylogs' the target bucket for logging. Whenever you see 'mybucket' or 'mylogs' in the example, replace them with the name of your bucket that you want to log, and the bucket you want to store your access logs, respectively.

This tutorial makes use of s3curl (go to s3curl.pl sample program) to work with the Amazon S3 REST API. Make sure you use the most recent version of s3curl, as it has been updated to support this tutorial. After invoking s3curl, always check for a 200 OK HTTP response. If you get some other response code, refer to the XML error response which likely contains information about what went wrong.

Preparing the Target Bucket

To prepare the target bucket

First, decide if you want your logs delivered to an existing bucket, or if you want to create a new bucket just for access log files. Following is a command that creates a new target bucket for logging. Notice the canned ACL argument that grants the system permission to write log files to this bucket.

Note

The source and the target buckets must be in the same location. For more information about bucket location constraints, see How to Select a Region for Your Buckets
```
$ ./s3curl.pl --id YOUR_AWS_ACCESS_KEY_ID --key YOUR_AWS_SECRET_ACCESS_KEY --acl log-delivery-write --put /dev/null -- -s -v http://s3.amazonaws.com/mylogs
```
If you just created a new bucket for logging, skip to the next section. Otherwise, to have your access logs files delivered to an existing bucket, you must modify the access control policy of that bucket by hand. Fetch the ?acl sub-resource of the target bucket and save it to a local file:
```
$ ./s3curl.pl --id YOUR_AWS_ACCESS_KEY_ID --key YOUR_AWS_SECRET_ACCESS_KEY -- -s -v 'http://s3.amazonaws.com/mylogs?acl' > mylogs.acl
```

Now open the local copy of the logging resource in your favorite text editor and insert a new <Grant> element to the <AccessControlList> section that gives the log delivery group WRITE and READ_ACP permission to your bucket.


<Grant>

    <Grantee xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:type="Group">

        <URI>http://acs.amazonaws.com/groups/s3/LogDelivery</URI>

    </Grantee>

    <Permission>WRITE</Permission>

</Grant>

<Grant>

    <Grantee xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:type="Group">

        <URI>http://acs.amazonaws.com/groups/s3/LogDelivery</URI>

    </Grantee>

    <Permission>READ_ACP</Permission>

</Grant>

Finally, apply the modified access control policy by writing it back to Amazon S3.


$ ./s3curl.pl --id YOUR_AWS_ACCESS_KEY_ID --key YOUR_AWS_SECRET_ACCESS_KEY --put mylogs.acl -- -s -v 'http://s3.amazonaws.com/mylogs?acl'

Enabling Server Access Logging on the Source Bucket

Now that the target bucket can accept log files, we'll update the ?logging sub-resource of the source bucket to turn on server access logging. Remember that you must be the bucket owner to read or write this resource.

Fetch the ?logging sub-resource for modification using the command shown in the following example.

Example


$ ./s3curl.pl --id YOUR_AWS_ACCESS_KEY_ID --key YOUR_AWS_SECRET_ACCESS_KEY -- -s -v 'http://s3.amazonaws.com/mybucket?logging' > mybucket.logging

Open mybucket.logging in your favorite text editor and uncomment the <LoggingSettings> section. Replace the contents of the <TargetBucket> and <TargetPrefix> with 'mylogs' and 'mybucket-access_log-' respectively.

Additionally, to grant users access to log files within the bucket, you can specify one or more users in the <TargetGrants> section, You can specify users through their e-mail address (EmailAddress) or canonical user ID (CanonicalUser). Permissions include READ, WRITE, and FULL_CONTROL. The result should be similar to the following.

Example


<?xml version="1.0" encoding="UTF-8"?>

<BucketLoggingStatus xmlns="http://doc.s3.amazonaws.com/2006-03-01">

    <LoggingEnabled>

        <TargetBucket>mylogs</TargetBucket>

        <TargetPrefix>mybucket-access_log-/</TargetPrefix>

            <TargetGrants>

	        <Grant>

	        <Grantee xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:type="AmazonCustomerByEmail">

	            <EmailAddress>[email protected]</EmailAddress>

	        </Grantee>

	        <Permission>READ</Permission>

	    </Grant>

        </TargetGrants>

    </LoggingEnabled>

</BucketLoggingStatus>

	Note
	For general information about authentication, see Authentication and Access Control.

Now apply your modifications by writing the document back to the ?logging sub-resource in Amazon S3.

Example


$ ./s3curl.pl --id YOUR_AWS_ACCESS_KEY_ID --key YOUR_AWS_SECRET_ACCESS_KEY --put mybucket.logging -- -s -v 'http://s3.amazonaws.com/mybucket?logging'

You can confirm your changes by fetching the ?logging sub-resource and comparing it to what you just wrote.

Server access logging should now be enabled. Make a few requests against the source bucket now, and your access logs should begin to be delivered to the target bucket within the next few hours.

Disabling Server Logging for a Bucket

Fetch, modify, and apply the ?logging sub resource in the same way as described in the preceding procedure, except use your text editor to remove the <LoggingEnabled> element.

	Note
	Log changes do not take effect immediately; logs will be delivered for a while after disabling logging.

previous page start next page

	Note
	The source and the target buckets must be in the same location. For more information about bucket location constraints, see How to Select a Region for Your Buckets