Amazon Web Services (AWS)#

IAM Policies#

Coiled requires a limited set of IAM permissions to provision infrastructure and compute resources in your AWS account. See the guide for configuring AWS for more information.

Resources#

When you create a Dask cluster with Coiled on your own AWS account, Coiled will provision the following resources on your AWS account:

../_images/backend-coiled-aws-architecture.png

AWS resources for a Dask cluster with 5 workers#

When you create additional Dask clusters with Coiled, then another scheduler VM and additional worker VMs will be provisioned within the same public and private subnets, respectively. As you create additional Dask clusters, Coiled will reuse and share the existing VPC and other existing network resources that were initially created.

Warning

If you get a permissions error when reading from an S3 bucket, you may need to attach S3 policies to the role that Coiled creates to attach to EC2 instances. The role name is the same as your account slug.

See also

If you encounter any issues when setting up resources, you can use the method coiled.get_notifications() to have more visibility into this process. You might also be interested in reading our Troubleshooting guide.

See also

You might be interested in reading the tutorial on How to limit Coiled’s access to your AWS resources.

You might be interested in reading the tutorial on Managing resources created by Coiled.

Quotas#

Each AWS service has pre-defined quotas, or limits, which are the maximum values for the resources in your AWS account (see the AWS Service Quotas guide). You may want to request an increase in these quotas, especially if you have received LimitExceeded error messages while using Coiled. You can view the default quotas or request an increase from the Service Quotas console. You can follow these instructions from AWS Support to request an increase.

Backend Options#

There are several AWS-specific options that you can specify (listed below) to customize Coiled’s behavior. Additionally, the next section contains an example of how to configure these options in practice.

Name

Description

Default

region_name

AWS region to create resources in

us-east-1

zone_name

AWS Availability Zone to create cluster

depends on region

spot

Whether or not to use spot instances for cluster workers

False

firewall

Ports and CIDR block for the security groups that Coiled creates

{"ports": [22, 8787, 8786], "cidr": "0.0.0.0/0"}

The currently supported AWS regions are:

  • us-east-1

  • us-east-2

  • us-west-1

  • us-west-2

  • ap-southeast-1

  • ca-central-1

  • ap-northeast-1

  • ap-northeast-2

  • ap-south-1

  • ap-southeast-1

  • ap-southeast-2

  • eu-central-1

  • eu-north-1

  • eu-west-1

  • eu-west-2

  • eu-west-3

  • sa-east-1

Note

Coiled will choose the us-east-1 region by default. If you don’t wish to use this region, you should provide a different region.

You can specify backend options directly in Python:

import coiled

cluster = coiled.Cluster(backend_options={"region_name": "us-west-1"})

Or save them to your Coiled configuration file:

# ~/.config/dask/coiled.yaml

coiled:
  backend-options:
    region: us-west-1

GPU support#

This backend allows you to run computations with GPU-enabled machines if your account has access to GPUs. See the GPU best practices documentation for more information on using GPUs with this backend.

Workers currently have access to a single GPU, if you try to create a cluster with more than one GPU, the cluster will not start, and an error will be returned.

Coiled logs#

If you are running Coiled on your own AWS account, cluster logs will be saved within your AWS account. Coiled will use CloudWatch to store logs.

Coiled will create a log group with your account name and add a log stream for each instances that Coiled creates. These logs will be stored for 30 days.

Log Storage

Storage time

Cloudwatch

30 days

Availability Zone#

The availability of different VM instance types varies across AZs, so choosing a different AZ may make it easier to create a cluster with the desired number and type of instances.

This option allows you to pick the Availability Zone (AZ) to use for a cluster. Each AZ is one or more distinct data centers located within a region. For example, the us-east-1 region contains the us-east-1a zone, (as well as b, c, d, and f zones).

You can specify the zone to use when creating an individual cluster like so:

cluster = coiled.Cluster(backend_options={"zone_name": "us-east-1b"})

In order to create a Dask cluster in a given AZ, we need a subnet for that specific zone.

When you configure Coiled to use your AWS account (as described above), Coiled attempts to create a subnet for every zone in the selected region instead of just the default zone (note that there are no additional AWS or Coiled costs associated with each subnet).

When creating a Dask cluster, you can specify the zone to use for that cluster. Ideally the specified zone already has the required subnet (created when you configured Coiled to use your AWS account) but if not, we’ll attempt to create a subnet at cluster-creation time. This may fail if Coiled no longer has “setup” IAM permissions; you’ll get an error message if we are unable to find or create a subnet in the specified zone.

Assuming we are able to find or create the required subnet, then we’ll then create your Coiled cluster in the specified availability zone.

If no zone is specified when creating an individual cluster, we’ll use the zone set at the account level (currently this can only be set if you configure your account backend using the the Python API), and if that isn’t set, we’ll use the default zone for the region your account is configured to use.

Refer to the AWS documentation on Regions and Availability Zones for additional information.

Networking#

When Coiled is configured to run in your own AWS account, you can customize the security group ingress rules for resources that Coiled creates in your AWS account.

By default, Dask schedulers created by Coiled will be reachable via ports 22, 8787 and 8786 from any source network. This is consistent with the default ingress rules that Coiled configures for its AWS security groups:

Protocol

Port

Source

tcp

8787

0.0.0.0/0

tcp

8786

0.0.0.0/0

tcp

22

0.0.0.0/0

Note

Ports 8787 and 8786 are used by the Dask dashboard and Dask protocol respectively. Port 22 optionally supports incoming SSH connections to the virtual machine.

Configuring firewall rules#

While allowing incoming connections on the default Dask ports from any source network is convenient, you might want to configure additional security measures by restricting incoming connections. This can be done by using coiled.set_backend_options() or by using the backend_options.