Posts by Sarah Johnson

Scaling AI-Based Data Processing with Hugging Face + Dask

Oct 9, 2024

../../_images/dask-hf.png

Read more ...

DataFrames at Scale Comparison: TPC-H

May 14, 2024

Read more ...

Dask vs. Spark

Apr 19, 2024

Bar chart comparing the relative difference in TPC-H query runtime for Dask vs. PySpark when executed on a M1 MacBook Pro with 8 cores. Orange represents queries where Dask is faster and blue where PySpark is faster.

Read more ...

One Trillion Row Challenge

Feb 5, 2024

Read more ...

One Billion Row Challenge (1BRC) in Python with Dask

Jan 16, 2024

Read more ...

Run Jupyter Notebooks on a GPU on the Cloud

Oct 10, 2023

Read more ...

Processing a 250 TB dataset with Coiled, Dask, and Xarray

We processed 250TB of geospatial cloud data in twenty minutes on the cloud with Xarray, Dask, and Coiled. We do this to demonstrate scale and to think about costs.

County-level heat map of the continental US showing mean depth to soil saturation (in meters) in 2020.

Read more ...

How well does Dask run on Graviton?

May 5, 2023

bar chart of AWS cost vs. processor type

Read more ...

Just in time Python environments

Feb 23, 2023

../../_images/senvs2_build_push_pull.svg

Read more ...