CLI Jobs#
Run simple commands and scripts in the cloud
Sometimes you just want to run a simple script on a VM in the cloud. This is common for different reasons:
You want to run close to your data
You want a GPU
You want a bigger machine
You want to run the script many times while scaling out
This is easy with the coiled run
command. Coiled will run your command on
the infrastructure of your choosing.
coiled run python my_script.py
coiled run bash my_bash_script.sh
coiled run --gpu python pytorch_model_train.py
coiled run --vm-type m6i.32xlarge python parallel_script.py
for i in {1..10};
do coiled run python my_script.py --param $i &
done
Use Cases#
Features#
Coiled Run benefits from the standard Coiled features:
Easy to use API
Creates and manages VMs for you in the cloud
Automatically synchronizes your local software and credentials
Gives access to any cloud hardware (like GPUs) in any region
Hardware#
It’s easy to customize the hardware and software resources a run needs. You can select any VM type available on your cloud (see VM Size and Type). For example:
# 128 Cores and 512 GB of RAM
coiled run --vm-type m6i.32xlarge python myscript.py
# A single T4 NVIDIA GPU
coiled run --vm-type g4dn.xlarge python train.py
Software for Coiled Run will work the same as with Coiled clusters. By default, Coiled will automatically synchronize your local software environment. This works well in most cases, but you can also specify an manual software environment or Docker image:
# Manual software environment
coiled run --software my-env python my_script.py
# Docker image
coiled run --container nvcr.io/nvidia/rapidsai/base:23.08-cuda11.8-py3.10 python rapids_script.py
Files#
Any files listed in your command will automatically be uploaded to the cloud
for your run to complete. If additional files are needed (for example, your Python
script imports code from another local Python module) you can use the
--file
option to ensure additional files are also included with your run.
coiled run --file utils.py python my_script.py
Feedback#
Coiled Run is under active development. We highly
value feedback from users and encourage you to play with coiled run
and then let us know about your experience on
this issue tracker
or by reaching out to support@coiled.io.
API#
coiled run#
Run a command on the cloud.
coiled run [OPTIONS] [COMMAND]...
Options
- --name <name>#
Run name. This identifier controls whether coiled run invocations are dispatched to the same cloud VM or not. Use the same name to run multiple commands on the same VM. Defaults to a unique name with no VM reuse.
- --account, --workspace <account>#
Coiled workspace (uses default workspace if not specified). Note:
--account
is deprecated, please use--workspace
instead.
- --software <software>#
Software environment name to use. If neither software nor container is specified, all the currently-installed Python packages are replicated on the VM using package sync.
- --container <container>#
Container image to use. If neither software nor container is specified, all the currently-installed Python packages are replicated on the VM using package sync.
- --vm-type <vm_type>#
VM type to use. Specify multiple times to provide multiple options.
- --gpu#
Have a GPU available.
- --region <region>#
The cloud provider region in which to run the notebook.
- --disk-size <disk_size>#
Use larger-than-default disk on VM, specified in GiB.
- --keepalive <keepalive>#
Keep your VM running for the specified time, even after your command completes. In seconds (
--keepalive 60
) unless you specify units (--keepalive 3m
for 3 minutes).Default to shutdown immediately after the command finishes.
- -f, --file <file>#
Local files required to run command. Can be either individual file or entire directory. Multiple values can be specified, such as
--file foo.txt --file my-subdir/
.
- -e, --env <env>#
Environment variables securely transmitted to run command environment. Format is
KEY=val
, multiple vars can be set with separate--env
for each.
- --subdomain <subdomain>#
Custom subdomain for the VM hostname.
- --allow-ssh-from <allow_ssh_from>#
IP address or CIDR from which connections to port 22 (SSH) are open; can also be specified as ‘everyone’ (0.0.0.0/0) or ‘me’ (automatically determines public IP detected for your local client). Note that
coiled run
relies on SSH connection for executing commands on VM.
- --port <port>#
Open extra ports in network firewall for inbound connections (multiple ports can be set with separate
--port
for each).
- -it, --interactive#
Open an interactive session, e.g.,
coiled run --interactive bash
orcoiled run --interactive python
.
- --detach#
Start the run in the background, don’t wait for the results.
- --sync#
Sync files between local working directory and
/scratch/synced
.
- --root#
Act as root in Docker container.
- --forward-gcp-adc#
Forward long-lived Google Cloud Application Default Credentials to VM for data access.
- -t, --tag <tag>#
Tags. Format is
KEY=val
, multiple vars can be set with separate--tag
for each.
- --sync-ignore <sync_ignore>#
Paths to not sync when using
--sync
.
- --mount-bucket <mount_bucket>#
S3 or GCS bucket(s) to mount as volumes.
Arguments
- COMMAND#
Optional argument(s)