Configuring AWS#

Important

We recommend using the command-line setup option in the AWS Quickstart. If this option does not work for you, then continue with the steps below to use the AWS console.

In this guide you will use the AWS console to configure Coiled to run Dask computations on your AWS account. You will first create an IAM user from your AWS acccount then configure your Coiled cloud account to use your AWS account.

Before you start#

This guide assumes you have already created your Coiled account and have an AWS account. If you don’t have an AWS account, you can sign up for AWS Free Tier.

Create IAM user#

First, you will create an IAM user in your AWS account. Watch the video below to follow along, then you can configure your Coiled cloud account.

1. Sign in to the Console#

Sign in to the AWS console (see these instructions in the AWS documentation if you’re having trouble).

2. Create IAM policies#

Coiled requires a limited set of IAM permissions to provision infrastructure and compute resources in your AWS account. Follow the steps in the AWS user guide on how to create new IAM policies.

We’ll create two policies, one for initial setup and another for ongoing use. When you arrive at the step to insert a JSON policy document, copy and paste the following JSON policy documents. You can name them coiled-setup and coiled-ongoing, respectively, to make them easy to locate in the next step.

3. Create a new IAM user#

Follow the steps in the AWS guide on creating a new IAM user. This IAM user must have programmatic access. Creating an IAM role with temporary access keys will not be sufficient.

4. Attach IAM policies#

Now that you have created the two IAM policies with all necessary permissions, you can attach these policies to the IAM user you created in step 2. Follow the steps in the AWS user guide on attaching IAM identity permissions.

5. Obtain AWS credentials#

Coiled provisions resources on your AWS account through the use of AWS security credentials. Select the user you created previously. Follow the steps in the AWS user guide on programmatic access to obtain (or create) your access key ID and secret access key. They will resemble the following:

Example AWS Secret Access ID: AKIAIOSFODNN7EXAMPLE
Example AWS Secret Access Key: wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY

Keep these credentials handy since you will configure them in Coiled Cloud in the next step.

Note

The AWS credentials you supply must be long-lived (not temporary) tokens.

Configure your Coiled cloud account#

Now you’re ready to configure the cloud backend in your Coiled account to use your AWS account.

Watch the video below to follow along:

1. Log in to your Coiled account#

First, log in to your Coiled account. In the navigation bar on the left, click on Setup. Select Cloud Provider Configuration, then click the Edit button:

../_images/cloud-backend-start.png

Note

You can configure a different cloud backend for each Coiled account (i.e., your personal/default account or your Team account). Be sure that you’re configuring the correct account by switching accounts at the top of the left navigation bar in your Coiled dashboard if needed.

2. Select your cloud provider#

On the Select Your Cloud Provider step, select the AWS option, then click the Next button:

../_images/cloud-backend-provider-aws.png

3. Configure AWS#

On the Configure AWS step, select your default AWS region (i.e., when a region is not specified in the Coiled Python client). Enter your AWS Access Key ID and AWS Secret Access Key from the previous step, then click the Next:

../_images/cloud-backend-keys-aws.png

4. Network configuration#

On the Network Configuration step, select whether you would like Coiled to automatically create new or manually use existing VPC and network resources (see Bring your own network):

../_images/cloud-backend-network.png

5. Container registry#

On the Container Registry step, select whether you want to store Coiled software environments in Amazon Elastic Container Registry (ECR), the default option, or Docker Hub, then click Next:

../_images/cloud-backend-registry-aws.png

6. Review#

Review your cloud backend provider options, then click the Submit button:

../_images/cloud-backend-review-aws.png

On the next page, you will see the resources provisioned by Coiled in real time.

Next Steps#

Congratulations, Coiled is now configured to use your AWS account!

Note

Now that you have completed these configuration steps, you can detach the coiled-setup policy to restrict Coiled to only use the IAM permissions defined in the coiled-ongoing policy.

Follow the Getting Started tutorial to create a Coiled cluster and run a computation. When you create your first cluster, Coiled will create a new VPC, subnets, AMI, EC2 instances, and other resources on your AWS account that are used to power your Dask clusters. See Amazon Web Services (AWS) for a more detailed description of these resources and additional configuration options.