Automatic Setup

In this guide you will configure Coiled to run Dask computations entirely within your own Google Cloud account using the coiled setup gcp command-line tool.

Before you start

This guide relies on using the gcloud CLI. If you do not already have the gcloud CLI installed, you can follow these instructions from the Google Cloud documentation.

Setting up Coiled

You can use the coiled setup gcp CLI tool to connect Coiled to your Google Cloud account:

coiled setup gcp

With your permission, this command will create the IAM roles and service accounts Coiled needs so you can create clusters in your Google Cloud project. You will be prompted with an explanation at each step, so you can choose to say “yes” (or “no”) at any point (see Manual Setup for more details).

Here’s a demonstration of the process:

If you would like to use the default options for network configuration and container registry, where Coiled creates a VPC in your account and uses Google Artifact Registry for storing your Python environments you can run:

coiled setup gcp

Otherwise, you can configure Coiled to use a custom network configuration (e.g. your pre-existing VPC) or a different container registry for your Python environment (e.g. Docker Hub):

coiled setup gcp --manual-final-setup

After which you will be provided with the required access key and prompted to complete configuration in the Coiled web app at https://cloud.coiled.io/settings/setup/update-backend-options. Select GCP, then select Custom network configuration to upload your credentials.

Note

Especially if you are using a new Google Cloud account, you may encounter a Quota <resource> exceeded error (e.g. Quota 'CPUS' exceeded). You can view and adjust quota limits using the coiled setup gcp --quotas (see Quotas).

coiled setup gcp

coiled setup gcp [OPTIONS]

Options

-y, --yes

Don’t prompt for confirmation, just do it!

--iam-user <iam_user>

Name for role and user to create

--export <export>

Allows you to export role definitions as files

Options:

role | data-role

--account, --workspace <account>

Coiled workspace that will be configured. By default, uses your default Coiled workspace. Note: –account is deprecated, please use –workspace instead.

--quota-link

Just show link to GCP Console quota page

--quotas

Check and potentially request GCP quota increases

--manual-final-setup

Don’t automatically send credentials to Coiled, finish setup manually in the web UI

--enable-log-sink-permissions, --ignore-log-sink-permissions

Temporarily adjust organization policy so that service account for log sink can be granted permission

--enable-gar

Configure Google Artifact Registry for use with Coiled

--region <region>

GCP region to use when setting up your VPC/subnets