Page tree
Skip to end of metadata
Go to start of metadata

AWS S3 Use Cases

Provide Temporary AWS Credentials limited to single S3 Bucket Upload and Download

20210320:1600

The driver for this use case it to prototype AWS Organizations and Landing Zone automation around providing 2 S3 limited accounts on AWS - one account to read/write to specific S3 bucket - the second account with S3 bucket read only permissions.


Options for S3 access


Access
IAM (quick but insecure)

https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_temp.html
https://aws.amazon.com/premiumsupport/knowledge-center/s3-console-access-certain-bucket/

Create IAM user
Create s3-bucket-only Group
(reference bucket ARN) - list bucket and get/put and  s3:getBucketVersioning (not working) or s3:* to delete files

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "s3:ListBucket",
                "s3:getBucketVersioning",
                "s3:*"
            ],
            "Resource": "arn:aws:s3:::vesna2021"
        },
        {
            "Effect": "Allow",
            "Action": [
                "s3:PutObject",
                "s3:GetObject"            ],
            "Resource": "arn:aws:s3:::vesna2021/*"
        }
    ]}



Assign IAM user to group

Full console access
Full CLI access
AWS Security Token Servicehttps://docs.aws.amazon.com/STS/latest/APIReference/welcome.html
Temporary Credentials and
Signed URLs
https://medium.com/@ashan.fernando/upload-files-to-aws-s3-using-signed-urls-fa0a0cf489db


Options for S3 Upload

OptionDetailsExample

AWS CLI

s3 put

Include muiltipart upload (perf)
https://docs.aws.amazon.com/AmazonS3/latest/userguide/mpuoverview.html

AWS Console

(not recommended)

need temporary access creds
Transfer SFTP


Options for S3 Download


OptionsDetailsExample
AWS CLI
s3 cp









Create S3 bucket


// get ARNarn:aws:s3:::bucket2021


Create IAM Users

Provide S3 Upload URLs

Provide S3 Download URLs


Upload Large Datasets to S3

S3 Upload Options

Use the same region for S3 and consuming services like EC2

https://docs.aws.amazon.com/AmazonS3/latest/userguide/optimizing-performance.html

https://docs.aws.amazon.com/AmazonS3/latest/userguide/optimizing-performance-guidelines.html

https://docs.aws.amazon.com/AmazonS3/latest/userguide/optimizing-performance-design-patterns.html

AWS CLI

https://aws.amazon.com/premiumsupport/knowledge-center/s3-upload-large-files/

https://aws.amazon.com/s3/pricing/

AWS S3 Transfer Accelleration

AWS SFTP

https://aws.amazon.com/aws-transfer-family/pricing/

Costly

AWS Snowball




  • No labels