Environment Variables

To set environment variables—including credentials and other secrets—that can be used by code running on your cluster, you can use send_private_envs after your cluster has started. send_private_envs accepts a dictionary, for example:

cluster = coiled.Cluster(...)
cluster.send_private_envs({"AUTH_TOKEN_FOR_CUSTOM_DATABASE": "some-token"})

This method sends the (potentially) private environment variables directly from your local machine to the cluster over an encrypted connection and are never stored by Coiled except in memory on your cluster.

Note that because this environment variable is set after Dask is running on the cluster, it cannot be used for environment variables that need to be set before Python starts or before Dask dependencies are imported.

In the rare case when you need to set a non-secret environment variable before Python starts, use the environ keyword argument of coiled.Cluster. This accepts a dictionary, for example:

import coiled

cluster = coiled.Cluster(
    ...
    environ={"PYTHONOPTIMIZE": "1", "PYTHONTRACEMALLOC": "1"},
)
client = cluster.get_client()