All work

Agent skills for data engineering

Data Artisan

Ask a coding agent for a database schema and you get working code. You often do not get the indexes, the partition strategy, or the data quality checks, because nothing in the request said the thing has to survive contact with a terabyte. Data Artisan is a set of skill files that supply that missing context.

Published, MIT licensed, CI green

skills covering schema, ETL, quality and analytics
5
licensed and portable across agents
MIT

The problem

Generated data code tends to be correct and naive at the same time. It runs, it passes a smoke test, and it is missing indexes on high-cardinality columns, has no partitioning story, no data quality checks, and no lineage or observability. Designs that are fine at a gigabyte fall over at a terabyte.

The gap is not the model's ability to write SQL. It is that nobody told it which patterns separate a working schema from a production one.

How it works

Each skill is a single portable file describing the patterns and the reasoning behind them, in a format agents can load directly.

Schema design covers indexing, partitioning and data type strategy across Postgres, Snowflake and BigQuery. The ETL library covers slowly changing dimensions, incremental loads and full refreshes with idempotency.

The quality skill generates dbt tests, Great Expectations suites or plain SQL validation for completeness, uniqueness and referential integrity. Two further skills cover DuckDB analytics patterns and running against local models.

Because the files are plain and portable, they work in several agent tools rather than being tied to one vendor, and they can be pasted directly if the tooling is not supported.

Where it landed

The repository is published under MIT with continuous integration running against it, and the skills install through the agent skills CLI or by copying the file.

What it taught me

Writing the patterns down exposed the ones I was improvising

Turning working knowledge into instructions another system has to follow is a good way to discover which parts were habit rather than reasoning. Several skills got substantially clearer once I had to justify each rule instead of just applying it.

Built with

  • SQL
  • PostgreSQL
  • dbt
  • DuckDB
  • Agent Skills
  • Markdown