Manage Hardware#

Coiled makes it easy to create a Dask cluster:

import coiled

cluster = coiled.Cluster(
    n_workers=50,
)
client = cluster.get_client()

# … your Dask code here

You should get feedback about your cluster starting:

Terminal dashboard displaying the Coiled cluster status overview, configuration, and Dask worker states.

Motivation#

Your configuration impacts the performance and cost of your computation.

Options like worker size and type, chip architecture (Intel vs AMD vs ARM vs NVIDIA), whether to use spot/preemptible instances, in which region to run your computations and more can determine if your computation completes, how quickly, and with what cost.

Fortunately, Coiled gives you full flexibility over these choices. You can choose any VM type in any region with easy selection of cost-saving options like Spot or ARM. These documents help explain how to make those decisions.

Advanced Example#

The defaults are sensible, but there are additional configuration options and operations that you can choose. Here is a more complex example:

import coiled

cluster = coiled.Cluster(
    n_workers=50,                # More workers
    worker_memory="256 GiB",     # Bigger workers
    region="us-east-2",          # Specific region
    arm=True,                    # Change architecture
    spot_policy="spot_with_fallback",  # Use spot
    tags={"workload": "prod"},   # Tag clusters
)
# your dask code here

cluster.scale(200)               # Scale cluster size

# more dask code

cluster.shutdown()               # Clean up

Dashboard#

Coiled also provides a robust management dashboard, available at cloud.coiled.io

Additional Configuration#

There are many options to configure your clusters. Coiled has the full flexibility of your cloud provider.

The rest of this section, including the reference, documents commonly used features. Please explore.