How Coiled works in your cloud account#
Coiled creates and manages Dask clusters in your own cloud provider account. This allows you to make use of security/data access controls, compliance standards, and promotional credits that you already have in place.
Use Coiled with AWS
Use Coiled with GCP
Coiled is only responsible for provisioning resources for clusters you create. Once a Dask cluster is created, all computations, data transfer, and client-to-scheduler communication occurs entirely within your cloud provider account.
Need a cloud provider account?#
Coiled creates and manages Dask clusters in your own AWS or Google Cloud account. There’s no cost for having an account and both have free-tier offerings for many services, though you’ll pay for the resources you use when you create Dask clusters.
If you’re not ready to create your own cloud provider account, but would still like to try Coiled, we can set you up with a trial on our account. Reach out to us at hello@coiled.io with a brief note about how you’re using Dask.
If you would like to learn more about how to use Dask, Coiled offers live tutorials.
To learn about other ways to deploy Dask, see the Dask documentation on deploying Dask clusters.
How do I choose a cloud provider?#
Coiled offers the same, or equivalent, features on both AWS and Google Cloud, and both cloud providers offer fairly similar features at fairly similar costs.
One consideration is where you currently store your data. If you are using Amazon S3, then we recommend AWS. If you are using Google BigQuery, then we recommend using Google Cloud.
If you’re starting from scratch, either should work well! AWS is more mature and has data centers in more locations across the world. More people are familiar with AWS, but many people think that Google Cloud has a more developer-friendly user experience.
Will this be expensive?#
The cost from your cloud provider to run a cluster with 200 cores for a hour is about $5 (that’s using 100 t3.medium instances on AWS or e2-standard-2 instances on Google Cloud).
You can use Coiled for up to 10,000 CPU-hours per month with no additional cost (see https://www.coiled.io/pricing).
Cloud providers charge you to store and transmit data; you can reduce this cost by keeping your data with a single cloud provider and in the same region as your cluster.
Very large instance types and GPUs can quickly increase the cost per hour, but they can also bring down your total cost by letting you process large workloads in less time.
As always, we’re happy to chat about your specific needs. Feel free to reach out to us at support@coiled.io.