Iterating Through Multi-Page Results
As buckets can contain a virtually unlimited number of keys, the complete results of a list query can be extremely large. To manage large result sets, Amazon S3 uses pagination to split them into multiple responses. Following is a pseudo-code procedure that demonstrates how to iteratively fetch an exhaustive list of results, given a prefix, marker and delimiter.
function exhaustiveList(bucket, prefix, marker, delimiter) : do { result = AmazonS3.list(bucket, prefix, marker, delimiter); // ... work with incremental list results ... marker = max(result.Contents.Keys, result.CommonPrefixes.Prefixes) // or more conveniently, when delimiter != null // marker = result.NextMarker; } while (result.IsTruncated);