Permalink
Switch branches/tags
Find file Copy path
Fetching contributors…
Cannot retrieve contributors at this time
208 lines (154 sloc) 8.26 KB

General Assembly Logo

Simple Storage Service on Amazon Web Services setup

Instructions

Fork and clone this repository.

Read over all the instructions before proceeding.

Follow the steps outlined to create and gain programmatic access to an AWS S3 bucket.

Prerequisites

  • An AWS (Amazon Web Services) account
  • A Credit card is required to verify your AWS account.

If you do not have an account, open AWS and click Create a Free Account. Amazon provides a free tier, with some limitations, for twelve months after you sign-up for an AWS account.

Motivation

Storing large static files is a common need for a web application. Accepting image uploads from authorized users but allowing public read access is a frequent example.

AWS provides a variety of APIs, one of which is easily used for this purpose. This guide helps ensure access to these APIs is restricted.

Why is the important?

Using any metered API has financial risks. Using many APIs may have data risks (information loss or exposure).

Using restrictive access control with AWS ensures that even if an identity is compromised, the actual risks, financial and otherwise, are limited.

AWS S3 access control

  1. Open the AWS Console in your browser
  2. From the AWS console, select Services, and open tabs for IAM (Identity and Access Management) and S3 (Simple Storage Service).

Identity and Access Management (IAM)

Identities are how we grant access to AWS APIs.

In the IAM tab:

Getting to the IAM tab:

image

image

Identity and Access Management (IAM) image

  1. Select Users in the left sidebar. image

  2. Click Add User near the top of the page.

  3. Enter wdi-upload into the text box. image

  4. Under access type, check Programmatic Access image

  5. Click Next: Permissions

  6. Highlight Add User to Group image

  7. Click Next: Review

  8. Click Create user image

  9. Click Close

Then

  1. Click on your newly created user.
  • Make sure wdi-upload is checked.
  • Click directly on wdi-upload image
  1. Click on the security credentials tab.
  2. Click the small x to the right of your existing access key to delete it.
  3. Click Create access key
  4. When you recieve a Success response, click Download .csv file and save the CSV to your wdi folder. (this is the only time you'll be able to see your access key, but you can generate a new one anytime and are encouraged to rotate them frequently)

image

  1. Click Close
  2. Copy the User ARN (Amazon Resource Name) at the top of the page and save it in arn.txt.

We'll need the User ARN to grant access to an S3 bucket we'll use for uploads. We'll also need an Access Key (Access Key Id and Secret Access Key) for this IAM User to upload files via the S3 API. The Access Key is contained in the csv file that we just downloaded: accessKeys.csv.

Note well: accessKeys.csv contains secrets! Do not share them or store them in git. The .gitignore in this repository explicitly ignores this file. Altering the .gitignore file in this repository could result in your AWS credentials (credentials linked to your credit card information) being visible on Github. NEVER COMMIT SECRETS TO GIT

Simple Storage Service (S3)

S3 stores files you upload in buckets. A bucket is a top-level namespace for your files.

In the S3 tab: image

  1. Click Create Bucket. This opens the Create a Bucket - Select a Bucket Name and Region modal.

  2. Enter a name in the Bucket name box. It must be unique among all S3 buckets and in all lowercase characters.

  3. Select US East (N. Virginia) for the Region.

  4. Click Create in the lower lefthand corner.

  5. Highlight your bucket and select the Permissions tab.

  6. Click Bucket Policy near the bottom of the Permissions tab. image

  7. At the bottom of the Bucket policy editor page, click Policy generator. This opens the AWS Policy Generator page.

  8. On the AWS Policy Generator page

  9. Step 1: Select Policy Type

  10. For Select Type of Policy use S3 Bucket Policy.

  11. Step 2: Add Statement(s)

  12. Select Allow for Effect.

  13. Paste the User ARN that you saved in the arn.txt file into the Principal box.

  14. Select PutObject and PutObjectAcl for Actions. image

  15. Enter arn:aws:s3:::<bucket_name>/* into the Amazon Resource Name (ARN) box.

    • Make sure to remove '<' & '>' and keep /* at the end of your user ARN for this step. image
  16. Click Add Statement. image

  17. Step 3: Generate Policy

  18. Click Generate Policy

  19. Copy the JSON from the Policy JSON Document modal.

  20. Click Close

  21. Return to the S3 tab.

  22. Paste the bucket policy into the Bucket policy editor field. image

  23. Click Save.

  24. Click on Access Control List

  25. Click on your account

  26. A modal will pop up.

  27. Click Save in the modal.

You have now created and granted access to an S3 bucket.

These steps limit upload access to one bucket for the identity wdi-upload.

This is one specific and restrictive way of implementing access control. AWS provides many different mechanisms to grant and restrict access.

Example bucket policy JSON

{
  "Version": "2012-10-17",
  "Id": "Policy1439826519004",
  "Statement": [
    {
      "Sid": "Stmt1439826516658",
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:iam::<AWS Account Id>:user/<IAM User Name>"
      },
      "Action": [
        "s3:PutObjectAcl",
        "s3:PutObject"
      ],
      "Resource": "arn:aws:s3:::<bucket_name>/*"
    }
  ]
}

Checklist

  • Create (or select) an AWS Identity.
  • Create and download an access key for this identity.
  • Save said access key csv inside this repo.
  • Save your ARN to arn.txt in this repo.
  • Create an S3 bucket.
  • Set AWS Region to US East (N. Virginia)
  • Create a bucket policy.
  • DO NOT ALTER THE .gitignore FILE. Note well: credentials.csv contains secrets! Do not share them or store them in git. The .gitignore in this repository explicitly ignores this file. Altering the .gitignore file in this repository could result in your AWS credentials (credentials linked to your credit card information) being visible on Github. NEVER COMMIT SECRETS TO GIT.

License

  1. All content is licensed under a CC­BY­NC­SA 4.0 license.
  2. All software code is licensed under GNU GPLv3. For commercial use or alternative licensing, please contact legal@ga.co.