In this guide you will configure Coiled to run Dask computations entirely within
your own Google Cloud account using the
coiled setup gcp command-line tool.
Before you start#
This guide relies on using the
gcloud CLI. If you do not already have the
gcloud CLI installed, you can follow these instructions from the Google Cloud documentation.
Setting up Coiled#
You can use the
coiled setup gcp CLI tool to connect Coiled to your Google Cloud account:
coiled setup gcp
With your permission, this command will create the IAM roles and service accounts Coiled needs so you can create clusters in your Google Cloud project. You will be prompted with an explanation at each step, so you can choose to say “yes” (or “no”) at any point (see Manual Setup for more details).
Here’s a demonstration of the process:
If you would like to use the default options for network configuration and container registry, where Coiled creates a VPC in your account and uses Google Artifact Registry for storing your Python environments you can run:
coiled setup gcp
Otherwise, you can configure Coiled to use a custom network configuration (e.g. your pre-existing VPC) or a different container registry for your Python environment (e.g. Docker Hub):
coiled setup gcp --manual-final-setup
After which you will be provided with the required access key and prompted to complete configuration in the Coiled web app at https://cloud.coiled.io/settings/setup/update-backend-options. Select
GCP, then select
Custom network configuration to upload your credentials.
Especially if you are using a new Google Cloud account, you may encounter a
Quota <resource> exceeded error (e.g.
Quota 'CPUS' exceeded). You can view and adjust quota limits using the
coiled setup gcp --quotas (see Quotas).
coiled setup gcp#
coiled setup gcp [OPTIONS]
- -y, --yes#
Don’t prompt for confirmation, just do it!
- --iam-user <iam_user>#
Name for role and user to create
- --export <export>#
Allows you to export role definitions as files
- role | data-role
- --account <account>#
Coiled account that will be configured. By default, uses your default Coiled account.
Just show link to GCP Console quota page
Check and potentially request GCP quota increases
Don’t automatically send credentials to Coiled, finish setup manually in the web UI
Configure Google Artifact Registry for use with Coiled
- --region <region>#
GCP region to use when setting up your VPC/subnets