Manage Hardware¶
Coiled makes it easy to create a Dask cluster:
import coiled
cluster = coiled.Cluster(
n_workers=50,
)
client = cluster.get_client()
# … your Dask code here
You should get feedback about your cluster starting:
Motivation¶
Your configuration impacts the performance and cost of your computation.
Options like worker size and type, chip architecture (Intel vs AMD vs ARM vs NVIDIA), whether to use spot/preemptible instances, in which region to run your computations and more can determine if your computation completes, how quickly, and with what cost.
Fortunately, Coiled gives you full flexibility over these choices. You can choose any VM type in any region with easy selection of cost-saving options like Spot or ARM. These documents help explain how to make those decisions.
Advanced Example¶
The defaults are sensible, but there are additional configuration options and operations that you can choose. Here is a more complex example:
import coiled
cluster = coiled.Cluster(
n_workers=50, # More workers
worker_memory="256 GiB", # Bigger workers
region="us-east-2", # Specific region
arm=True, # Change architecture
spot_policy="spot_with_fallback", # Use spot
tags={"workload": "prod"}, # Tag clusters
)
# your dask code here
cluster.scale(200) # Scale cluster size
# more dask code
cluster.shutdown() # Clean up
Dashboard¶
Coiled also provides a robust management dashboard, available at cloud.coiled.io
Additional Configuration¶
There are many options to configure your clusters. Coiled has the full flexibility of your cloud provider.
The rest of this section, including the reference, documents commonly used features. Please explore.