How do I access my data?
You can access many different file formats using Dask and related libraries.
Large datasets should live on the cloud in services like S3 to avoid data transmission costs.
Additionally, if you have small data locally you can use other Dask APIs, like Dask-ML or Dask Futures to do heavy computation on small datasets comfortably.
How do I install libraries into my Coiled clusters?
Coiled helps you manage software environments both on your local machine and
on cloud resources. You can specify custom environments with conda or pip
environments files with the
and Coiled will manage building Docker images for you that can then be used
in Dask clusters or other jobs on the cloud.
See Software Environments for more information.
Can I run computation in my own cloud account?
Yes. Coiled accounts run by default in our hosted AWS account, but for pricing or security reasons you may prefer to run things on your own account.
See Backends for more information.
Do I have to migrate my data to Coiled?
No. Your data stays in its native location(s). Coiled manages computation. Coiled helps you get data from your existing data stores, process it, and write out to those same data stores.
Can I use Coiled to read private data on AWS?
Yes. If you create a Coiled cluster from a location that has AWS credentials then Coiled will generate a secure token from those credentials and forward it to your Dask workers. Your Dask workers will have the same rights and permissions that you have by default.
Alternatively, for additional control Coiled can be deployed within your own AWS account and allow you to specify and manage IAM roles directly.
See Security & Privacy for more information.
Is my computation and data secure?
Coiled provides end-to-end network security with both cloud networking policies and with TLS encryption. Coiled does not persist or store any of your data, except in memory as you are doing computations.
You can also use Coiled to manage computation in your own account or on your own hardware (see the next question below)
See Security & Privacy for more information.
How much does Coiled cost?
Coiled is currently in beta. During this time Coiled is free for all beta users.
You will not be charged for any of the compute resources you use, however there is a limit of 100 concurrently running cores per user. This policy will change in the future when Coiled is opened up to a broader audience, but until then we are happy to provide beta users cloud computing resources at no cost. Thank you for trying out Coiled!
For more information on pricing, see coiled.io/pricing
Does Coiled support other clouds?
Coiled currently supports running on our managed AWS environment or within your own AWS account. Coiled also supports running on our managed Azure and GCP environments. See Backends for more information on setup, supported regions, and GPUs.
Please contact email@example.com if you’re interested in using Coiled with other backends such as your own Azure account, your own GCP account, or Kubernetes.
How can I report a feature request, bug, etc?
Please open an issue on the Coiled issue tracker. Feel free to report bugs, submit feature requests, ask questions, or provide other input. Your feedback is valued and will help influence the future of Coiled.
For generic questions, please join our Coiled Community Slack where you can ask questions and interact with our engineers and with the Coiled community.
How do I invite my colleagues / students / …?
We’re glad that you’re having a good time and want to invite colleagues or students. Coiled is currently open access, so your colleagues can join without any special setup.
Additionally, if you want to organize a team account send a quick e-mail to firstname.lastname@example.org with a team name and we’ll set you up as an administrator over your new team.
See Teams for more information.
Why do I get Version Mismatch warnings?
When running cloud computations from your local machine we need to ensure some level of consistency between your local and remote environments. For example your Python versions should match, and if you want to use a library like PyTorch or Pandas remotely we should probably also install it locally. When Dask notices a mismatch, it will tell you with a warning.
Matching versions can be challenging if handled manually. Fortunately Coiled provides services to help build and maintain software environments that match across local and remote environments.
See the Software Environments documentation for more information.
Does Coiled support GPUs?
Yes, but GPU support is only available today to paying users.
Can I use Coiled from Sagemaker/VS Code/PyCharm/…?
Yes, you can use Coiled from anywhere that you can use Python. Coiled is agnostic to user environment.
Does Coiled support Jupyter notebooks?
Yes, see the Notebooks tab in the application for example notebooks for notes on how to get started. You may also want to try some of our notebooks at cloud.coiled.io/examples/notebooks.
Coiled can run and manage arbitrary Python processes in the cloud, including Jupyter notebooks. Note that today Coiled does not yet persist user state across notebook sessions.
Can I run Coiled on-prem?
There are two types of on-prem that we support. For those looking to run Coiled in your own cloud account, we support that today. See the question “Can I run computation in my own account?” above.
For those looking to run Coiled on your own machines in your own data center, we would love to hear from you. Please contact email@example.com to start a conversation with us.
Why do I need a local software environment?
When performing distributed computation with Dask, you’ll create a
distributed.Client object which connects your local Python process
(e.g. your laptop) to your remote Dask cluster (e.g. running on AWS). Dask
Client s are the user-facing entry point for submitting tasks to a Dask
cluster. When using a
Client to submit tasks to your cluster, Dask will
package up and send data, functions, and other Python objects needed for
your computations from your local Python process where your
running to your remote Dask cluster in order for them to be executed.
This means that if you want to run a function on your Dask cluster, for
numpy.mean() function, then you must have NumPy
installed in your local Python process so Dask can send the
function from your local Dask
Client to the workers in your Dask
cluster. For this reason, it’s recommended to have the same libraries
installed on both your local machine and on the remote workers in your
See the Software Environments section for more details on how to easily synchronize your local and remote software environments using Coiled.