Draft. This page is reachable and it is not indexed until the guide QA checklist passes.
Data engineering project: load a CSV into DuckDB
A small, runnable ingestion project. Read a CSV into DuckDB, check a row count, and query it with SQL. No cloud account.
By Vincent Charles. Last updated .
Vincent Charles is a data engineer and founder of dataengineers.club and Unchain Data (unchaindata.xyz). He has worked on data at Dailymotion's big data team, Carrefour and Binance, starting in SQL and GIS location intelligence.
On this page
Who this is for
- Not written yet. This draft does not state a result.
Methodology
- Installed
- Nothing was installed for this draft.
- Versions
- No versions were recorded.
- Environment
- No environment was used.
- What was run
- No shared test was run.
- What was not run
- The test described in the brief was not run.
- Repo
- No public repo for this draft.
Feature matrix
Feature and fit matrix. Cells stay empty until a shared test or a dated doc check exists.
| Tool | Model | Deployment | Open source | Managed offer | Shared-test result | Best when |
|---|---|---|---|---|---|---|
| Data engineering project: load a CSV into DuckDB | Not measured | Not measured | Not measured | Not measured | not run | Not measured |
Pricing as of 2026-11-19
Pricing as of 2026-11-19. No official price was copied into this draft.
| Tool | Public starting price or see source | Billing unit | What we verified | Source and date |
|---|---|---|---|---|
| Data engineering project: load a CSV into DuckDB | Not on the public page as of 2026-11-19 | Not measured | Not measured | Not checked for this draft |
Pick this if
Pick this if. No row chooses a tool yet.
| If you need | Start with | Avoid or postpone |
|---|---|---|
| A reason to leave the incumbent | Not measured | Not measured |
| A reason to stay on the incumbent | Not measured | Not measured |
| An open-source path | Not measured | Not measured |
What we actually ran
What you will build
Earlier draft note, not a completed test.
You will load a small orders file into a local DuckDB database, reject a bad row with a check you can see, and answer one question in SQL: revenue by status. That is the whole project. It is one pipeline, not a career syllabus.
Install the DuckDB CLI from duckdb.org, or use Python if you already have the duckdb package. The SQL file in this repo runs without Python.
The files
Earlier draft note, not a completed test.
orders.csv has five rows, including one with a missing amount so you can see a filter drop it. The SQL creates the table from the CSV and a clean view.
CREATE OR REPLACE TABLE raw_orders AS SELECT * FROM read_csv_auto('orders.csv'); CREATE OR REPLACE TABLE orders AS SELECT order_id, status, amount_usd FROM raw_orders WHERE amount_usd IS NOT NULL; SELECT status, count(*) AS orders, sum(amount_usd) AS revenue_usd FROM orders GROUP BY status ORDER BY status;
Run it
Earlier draft note, not a completed test.
From the folder that contains orders.csv:
duckdb warehouse.duckdb < load_orders.sql
What good looks like
Earlier draft note, not a completed test.
raw_orders has 5 rows. orders has 4, because the row with no amount is excluded. The grouped query returns paid and pending only. If you see the blank amount in the clean table, the filter did not run.
Next steps that stay in the same pattern: add a second CSV, a primary key check, and a scheduled command. Tools such as dlt can replace read_csv_auto when the source is an API. This page does not claim a row-count or speed result beyond this file.
Day-2 operations
Not written yet. This draft does not state a result.
Limitations
Not written yet. This draft does not state a result.
FAQ
What do I need installed?
The DuckDB CLI from duckdb.org, or the duckdb Python package if you already have it. The SQL file runs without Python.
What counts should I see?
raw_orders has 5 rows. orders has 4, because the row with no amount is excluded. The grouped query returns paid and pending only.
Sources
- DuckDB documentation. Not checked for this draft.
- Sample files in this repo. Not checked for this draft.
Get the newsletter
One email when a comparison on this page is actually finished.