Google Cloud Platform (GCP)#
IAM custom role#
Coiled requires a limited set of IAM permissions to provision infrastructure and compute resources in your GCP account. See the guide on configuring Google Cloud for more information.
IAM custom role
Save the below YAML to a file on your local machine called coiled.yaml
:
title: coiled
description: coiled-externally-hosted
stage: GA
includedPermissions:
- bigquery.datasets.create
- bigquery.jobs.create
- bigquery.datasets.get
- bigquery.datasets.update
- compute.acceleratorTypes.list
- compute.addresses.list
- compute.disks.create
- compute.disks.delete
- compute.disks.list
- compute.disks.useReadOnly
- compute.firewalls.create
- compute.firewalls.delete
- compute.firewalls.get
- compute.firewalls.list
- compute.globalOperations.get
- compute.globalOperations.getIamPolicy
- compute.globalOperations.list
- compute.images.create
- compute.images.delete
- compute.images.get
- compute.images.list
- compute.images.setLabels
- compute.images.useReadOnly
- compute.instances.create
- compute.instances.delete
- compute.instances.get
- compute.instances.getSerialPortOutput
- compute.instances.list
- compute.instances.setLabels
- compute.instances.setMetadata
- compute.instances.setServiceAccount
- compute.instances.setTags
- compute.instanceTemplates.create
- compute.instanceTemplates.delete
- compute.instanceTemplates.get
- compute.instanceTemplates.useReadOnly
- compute.machineTypes.get
- compute.machineTypes.list
- compute.networks.create
- compute.networks.delete
- compute.networks.get
- compute.networks.list
- compute.networks.updatePolicy
- compute.projects.get
- compute.projects.setCommonInstanceMetadata
- compute.regionOperations.get
- compute.regionOperations.list
- compute.regions.get
- compute.regions.list
- compute.routers.create
- compute.routers.delete
- compute.routers.get
- compute.routers.list
- compute.routers.update
- compute.routes.delete
- compute.routes.list
- compute.subnetworks.create
- compute.subnetworks.delete
- compute.subnetworks.get
- compute.subnetworks.getIamPolicy
- compute.subnetworks.list
- compute.subnetworks.use
- compute.subnetworks.useExternalIp
- compute.zoneOperations.get
- compute.zoneOperations.list
- compute.zones.list
- iam.serviceAccounts.actAs
- logging.buckets.create
- logging.buckets.get
- logging.buckets.list
- logging.logEntries.create
- logging.logEntries.list
- logging.sinks.create
- logging.sinks.get
- logging.sinks.list
- storage.buckets.create
- storage.buckets.get
- storage.objects.create
- storage.objects.get
- storage.objects.list
- storage.objects.update
Quotas#
Each Google Cloud resource type has pre-defined quotas, which are the maximum number of resources you can create for a given resource type. You can view existing quotas and request increases from your Google Cloud console. If you have received error messages such as Quota <resource-type> exceeded
, you may want to request an increase. In particular, the following resource types often have insufficient quotas:
Persistent disk SSD (GB) (see the Google Cloud documentation on Disk quotas). You may need to request an increase if you see the
Quota 'SSD_TOTAL_GB' exceeded
error message.In-use external IP addresses (see the Google Cloud documentation on External IP addresses). You may need to request an increase if you see the
Quota 'IN_USE_ADDRESSES' exceeded
error message.CPU (see the Google Cloud documentation on CPU quota). You may need to request an increase if you see the
Quota 'CPUS' exceeded
error message.
Backend options#
There are several GCP-specific options that you can specify (listed below) to customize Coiled’s behavior.
Name |
Description |
Default |
---|---|---|
|
GCP region to create resources in |
|
|
GCP zone to create resources in |
|
|
Whether or not to use preemptible instances for cluster workers |
|
|
Ports and CIDR block for the security groups that Coiled creates - Under active development and should be considered to be in an early experimental/testing phase |
|
You can specify backend options directly in Python:
import coiled
cluster = coiled.Cluster(
backend_options={"region_name": "us-central1", "preemptible": False}
)
Or save them to your Coiled configuration file:
# ~/.config/dask/coiled.yaml
coiled:
backend-options:
region_name: us-central1
GPU support#
Coiled supports running computations with GPU-enabled machines if your account has access to GPUs. See the GPU best practices documentation for more information on using GPUs with GCP.
Coiled logs#
If you are running Coiled on your GCP account, cluster logs will be saved within your GCP account. Coiled will send logs to GCP Logging and GCP BigQuery (if BigQuery is enabled in the project).
We send logs to GCP Logging so that you can easily view logs with GCP Logs Explorer, and we use GCP Cloud Storage/GCP BigQuery to back the logs views we display on the Cluster Dashboard.
Note
Coiled will only use BigQuery if you have BigQuery enabled in your project and if
you have the following permissions in your service account: bigquery.datasets.create
,
bigquery.datasets.get
, bigquery.datasets.update
and bigquery.jobs.create
Log Storage |
Storage time |
---|---|
|
30 days |
|
90 days |
|
10 days |
When you configure your backend to use GCP, Coiled creates a bucket
named coiled-logs
GCP Logging.
Networking#
Warning
The features below are currently under active development and should be considered to be in an early experimental/testing phase.
When Coiled is configured to run in your own GCP account, you can customize the firewall ingress rules for resources that Coiled creates in your GCP account.
By default, Dask schedulers created by Coiled will be reachable via ports 8787 and 8786 from any source network. This is consistent with the default ingress rules that Coiled configures for its GCP firewalls:
Protocol |
Port |
Source |
---|---|---|
tcp |
8787 |
|
tcp |
8786 |
|
tcp |
22 |
|
Note
Ports 8787 and 8786 are used by the Dask dashboard and Dask protocol respectively. Port 22 optionally supports incoming SSH connections to the virtual machine.
Configuring firewall rules#
While allowing incoming connections on the default Dask ports from any source
network is convenient, you might want to configure additional security measures
by restricting incoming connections. This can be done by using
coiled.set_backend_options()
or by using the backend_options
.