Posts tagged coiled

Coiled observability wins: Chunksize

Distributed computing is hard, distributed debugging is even harder. Dask tries to simplify this process as much as possible. Coiled adds additional observability features for your Dask clusters and processes them to help users understand their workflows better.

../../_images/chunksize_task_stream.png

Read more ...


Parallel Serverless Functions at Scale

The cloud offers amazing scale, but it can be difficult for Python data developers to use. This post walks through how to use Coiled Functions to run your existing code in parallel on the cloud with minimal code changes.

Comparing code runtime between a laptop, single cloud VM, and multiple cloud VMs in parallel

Read more ...


Reduce training time for CPU intensive models with scikit-learn and Coiled Functions

Patrick Hoefler

Read more ...


Fine Performance Metrics and Spans

While it’s trivial to measure the end-to-end runtime of a Dask workload, the next logical step - breaking down this time to understand if it could be faster - has historically been a much more arduous task that required a lot of intuition and legwork, for novice and expert users alike. We wanted to change that.

Populated Fine Performance Metrics dashboard

Read more ...


Data-proximate Computing with Coiled Functions

Coiled Functions make it easy to improve performance and reduce costs by moving your computations next to your cloud data.

../../_images/data-proximate.png

Read more ...


Process Hundreds of GB of Data with DuckDB in the Cloud

Patrick Hoefler

Read more ...


How to Train a Neural Network on a GPU in the Cloud with coiled functions

Patrick Hoefler

Read more ...


Dask performance benchmarking put to the test: Fixing a pandas bottleneck

Patrick Hoefler, Hendrik Makait

Read more ...


Coiled notebooks

We recently pushed out a new, experimental notebooks feature for easily launching Jupyter servers in the cloud from your local machine. We’re excited about Coiled notebooks because they:

Read more ...


Observability for Distributed Computing with Dask

Hendrik Makait

2023-05-16

Read more ...


Performance testing at Coiled

At Coiled we develop Dask and automatically deploy it to large clusters of cloud workers (sometimes 1000+ EC2 instances at once!). In order to avoid surprises when we publish a new release, Dask needs to be covered by a comprehensive battery of tests — both for functionality and performance.

Nightly tests report

Read more ...