Posts tagged duckdb

Process Hundreds of GB of Data with DuckDB in the Cloud

DuckDB is great tool for running efficient queries on large datasets. When you want cloud data proximity or need more RAM, Coiled makes it easy to run your Python function in the cloud. In this post we’ll use Coiled Functions API to process the 150 GB Uber-Lyft dataset on a single machine with DuckDB.

Code snippet of using the coiled.function decorator to run a query with DuckDB on a large VM in the cloud.

Read more ...