Configure Firewalls#

By default, when you create a Dask cluster with Coiled, it is configured to allow incoming network connections on the default Dask ports from any source network for the sake of convenience. For additional security, you can restrict incoming connections to Dask clusters using the ingress option. This option can be used to specify the account-level default firewall settings for all newly created clusters via set_backend_options, or this option can be used when creating a cluster via the backend_options in coiled.Cluster.

Opening ports for a specific CIDR block#

If you need more control over the security groups or firewalls for Dask clusters created by Coiled, use the ingress argument to specify ingress rules for a source cidr block for a specified list of ports. If you configure the ingress setting, then Coiled will use these firewall rules as each new Dask cluster and its associated security group are created.

Note that ingress is a list, so you can open different ports to different CIDR blocks. This can be useful if (for instance) you need to open access to a VPN with one CIDR and a paired VPC with a different CIDR block.

For example, you can use backend_options to specify Coiled account-level default firewall settings:

import coiled

coiled.set_backend_options(
    backend="aws",
    aws_access_key_id="<your-access-key-id-here>",
    aws_secret_access_key="<your-access-key-secret-here>",
    ingress=[{"ports": [22, 443], "cidr": "10.1.0.0/16"}],
    workspace="my-team-workspace-slug",  # if you are using a Coiled team workspace
)

which will result in the following ingress rules configured for all newly created Dask clusters in your Coiled account:

Protocol

Port

Source

tcp

22

10.1.0.0/16

tcp

443

10.1.0.0/16

You can also use the backend_options option to modify the firewall settings for a specific cluster. Here’s an example opening multiple CIDR blocks:

import coiled

coiled.Cluster(
    backend_options={
        "ingress": [
            # 443 for client -> scheduler, and dashboard
            {"ports": [443], "cidr": "10.1.0.0/16"},
            # 22 optional, used for `coiled run` and notebook file sync
            {"ports": [8787], "cidr": "10.32.0.0/16"},
        ]
    }
)

Or, you can specify ingress settings in your Coiled configuration file:

# ~/.config/dask/coiled.yaml

coiled:
  backend-options:
    ingress: [{
            "ports": [22, 443],
            "cidr": "10.1.0.0/16"
        }]

Connecting on a private IP address#

By default the Coiled client will attempt to connect to the Dask scheduler using its public IP address, which causes traffic to be routed over the public internet. If you wish traffic between the Coiled client and the Dask scheduler to be routed over a private network you can pass the use_scheduler_public_ip argument to coiled.Cluster calls:

import coiled

coiled.Cluster(use_scheduler_public_ip=False)

If you wish to set this behavior as default, you can set this in your Coiled configuration file:

# ~/.config/dask/coiled.yaml

coiled:
  use_scheduler_public_ip: false

Custom networking setups#

If you have more complex security or networking requirements and prefer to use an existing VPC, subnets, and security groups you can bring your own network (see this guide for AWS or this guide for GCP).

On this page