Skip to content

Blog

Dashboards as Code for your dbt Project

dbt is great for transforming data. You write modular SQL, track everything in Git, and run automated tests to make sure your data is correct.

But dbt doesn’t help you visualize your data.

When it’s time to build dashboards on top of a dbt project, you usually end up clicking around in a separate BI tool - outside of Git and disconnected from your code workflow.

That’s where Shaper comes in: In Shaper, dashboards are also just SQL files stored in Git.

Clinic Overview Dashboard Preview

Recently Jarich, creator of model2data, and I built a demo project to show how simple it is to go from a data model to a tested dbt project and an interactive Shaper dashboard.

We auto-generated the dbt project with model2data then, generated the Shaper dashboard with a single AI prompt.

You can check out the live dashboard and the code right here:

When building a demo, fake data is usually a pain. If you just generate random values, all your charts look flat and unrealistic. Or the data doesn’t even make sense - like invoices dated before an appointment was booked, or payments on cancelled visits.

That’s the problem Jarich solved with model2data.

For our demo, we modeled a network of 8 healthcare clinics with practitioners, treatments, insurers, patients, and appointments in a single schema file.

With model2data, you define your schema once and specify how the generated data should behave:

# Excerpt from healthcare_clinics.model2data.yml
appointments:
role: fact
columns:
status:
type: appointment_status
not_null: true
generate:
weights: {completed: 16, cancelled: 3, no_show: 1}
booked_at:
type: timestamp
not_null: true
generate: {growth: 0.5}
scheduled_for:
type: timestamp
not_null: true
generate: {after: booked_at, business_hours: true}
actual_duration_minutes:
type: int
generate:
min: 10
max: 120
when: {status: [completed]}
distribution: {kind: normal, mean: 35, stddev: 12}
billed_amount:
type: numeric
generate:
min: 25
max: 600
when: {status: [completed]}
distribution: {kind: lognormal, median: 95, spread: 0.5}
paid_at:
type: timestamp
generate:
after: checked_in_at
when: {status: [completed]}
null_rate: 0.3

From this single file, model2data generates:

  1. A DuckDB database with 1,500 patients and 6,000 appointments scheduled during realistic weekday business hours.
  2. A complete dbt project with staging models and 83 automated dbt data tests - checking primary keys, foreign key relationships, non-null constraints, and valid status values.

Once you have a ready dbt project, building dashboards in Shaper follows the exact same workflow. Shaper directly connects to the database generated by model2data and queries the exact tables dbt defined.

Shaper is directly built on top of DuckDB and lets you build interactive dashboards completely in SQL:

  • Dashboards live as .dashboard.sql files right next to your dbt models in Git.
  • You use standard DuckDB SQL and turn results into filters, metrics, and charts using custom SQL types like ::DROPDOWN, ::BARCHART_STACKED, or ::LINECHART.
  • Because dbt already tests your foreign keys and constraints, you can join tables and calculate metrics without worrying about broken references or messy data.

Here is how Clinic Overview.dashboard.sql sets up filters at the top and caches the filtered dbt tables in a temporary table for fast queries across the dashboard:

-- 1. Main Header & Controls
SELECT 'Healthcare Clinic Network'::SECTION, 'Executive Overview & Clinical Operations'::SUBTITLE;
SELECT
min(scheduled_for)::DATE::DATEPICKER_FROM AS start_date,
max(scheduled_for)::DATE::DATEPICKER_TO AS end_date
FROM staging.stg_appointments;
SELECT 'Clinic'::LABEL;
SELECT 'All Clinics'::DROPDOWN AS selected_clinic
UNION ALL
(SELECT name FROM staging.stg_clinics ORDER BY name);
-- 2. Cache filtered dataset in memory
CREATE TEMP TABLE filtered_appointments AS (
SELECT
a.id AS appointment_id,
a.status,
a.channel,
a.booking_source,
a.scheduled_for,
a.actual_duration_minutes,
a.billed_amount,
a.paid_at,
c.name AS clinic_name,
p.specialty AS practitioner_specialty,
t.name AS treatment_name,
COALESCE(i.name, 'Uninsured / Self-Pay') AS insurer_name
FROM staging.stg_appointments a
JOIN staging.stg_practitioners p ON a.practitioner_id = p.id
JOIN staging.stg_clinics c ON p.clinic_id = c.id
JOIN staging.stg_treatments t ON a.treatment_id = t.id
JOIN staging.stg_patients pat ON a.patient_id = pat.id
LEFT JOIN staging.stg_insurers i ON pat.insurer_id = i.id
WHERE a.scheduled_for::DATE BETWEEN getvariable('start_date') AND getvariable('end_date')
AND (getvariable('selected_clinic') = 'All Clinics' OR c.name = getvariable('selected_clinic'))
);

From there, every chart on the dashboard is just a simple SQL query:

SELECT 'Monthly Appointments by Status'::LABEL;
SELECT
date_trunc('month', scheduled_for)::DATE::XAXIS,
status::CATEGORY,
count(*)::BARCHART_STACKED
FROM filtered_appointments
GROUP BY ALL
ORDER BY ALL;

Generating the Dashboard in a Single Prompt

Section titled “Generating the Dashboard in a Single Prompt”

What’s not in the repo is how I created the dashboard.

I didn’t write those 300 lines of SQL by hand.

Instead, I used the Shaper AI Skills. The skills teach your AI coding agent (like Claude Code, Cursor, or Codex) how to inspect the database schema, write Shaper SQL, validate queries, and preview dashboards via the Shaper CLI.

With Shaper connected to the DuckDB database from the dbt project, I gave the agent a single prompt:

I got this DBT project. Shaper is connected to the database from this project. Please generate me an overview dashboard for the data in this project. ask me for suggestions of what to show as different sections.

  1. The agent then read the dbt project and ran shaper schema to inspect the tables and columns in DuckDB.
  2. It suggested 6 sections for the dashboard and I didn’t have to adjust anything. I just replied “ok”.
  3. The agent generated Clinic Overview.dashboard.sql, ran shaper validate in the terminal to verify that all SQL queries run without errors, and opened the live dashboard in my browser using shaper preview.
Clinic Overview Dashboard Preview

Because dbt projects are well-structured and Shaper dashboards are just SQL validated via the CLI, the AI agent created a working dashboard without me having to fix anything.

And since shaper schema and shaper validate only check the schema and SQL syntax locally, the LLM never even sees your actual data.

If you want to try the demo locally, all you need is the Shaper CLI:

  1. Clone the demo repository:
    Terminal window
    git clone https://github.com/taleshape-com/model2data-shaper-demo.git
    cd model2data-shaper-demo
  2. Start Shaper with the included DuckDB file and deploy the dashboard:
    Terminal window
    shaper -d .shaperdata --duckdb healthcare_clinics.duckdb --deploy .
  3. Open http://localhost:5454/dashboards/clinic-overview in your browser.

To use the AI workflow on your own dbt project, check out the AI Agent Workflow docs. And if you need realistic test data and dbt models from a simple schema definition, definitely check out model2data on Github.

Claude Dashboards vs Shaper

Shaper is an open source dashboarding product. Now Anthropic released Claude Dashboards. What does that mean for Shaper?

The last days everyone on Data Analytics LinkedIn has been talking about how Claude Dashboards just killed the need for BI software.

This whole discussion only partially affects Shaper though: The Business Intelligence market has been crowded even two years ago when I started building Shaper.

Shaper never tried to compete with traditional BI tooling:

The problem Shaper solves:

  1. You want to share dashboards and reports with customers,
  2. but you are working with sensitive personal data that must stay in-house,
  3. and you need to be able to verify and audit results for correctness.

That’s why Shaper:

  1. focuses on embedded analytics,
  2. is fully open source and runs in your infrastructure,
  3. and defines dashboards as SQL code you store in Git.

None of these problems are solved by Claude Dashboards or the tools that came before it. I don’t expect that any healthcare company or finance org I work with to be willing to send PII to Claude or share AI generated dashboards with customers without a review process to approve the underlying analytics. Claude Dashboards looks like a great tool to query less critical data and to prototype dashboards before letting Claude translate them into Shaper SQL.


If you like to Shaper and want the ease of Claude to generate dashboards, checkout how to use our AI Agent Workflow to generate dashboards with Claude Code.

Why we open sourced Single Sign-On

We just added Single Sign-On functionality to Shaper. And we made it open source.

Single Sign-On (SSO) is the most obvious feature to separate from the main product and sell at a premium to enterprise clients.

You don’t need it to start using the product, but any bigger organization cannot do without it.

Why is SSO crucial to bigger organizations?

Section titled “Why is SSO crucial to bigger organizations?”

Obviously it’s not because large organizations care so much about UX and cannot live without the convenience of not having to type a password. Instead, it’s all about security: No weak passwords. Centralized access rights. Audit logging.

Should they care less about security? Is security something to build later on after you are already successful? I and many others don’t think so. That’s why there are websites like https://sso.tax/.

We decided to make SSO part of open source Shaper. But with a nuance:

JWT-based SSO is completely open source. You implement a single endpoint in your application and get complete control over how the user is authenticated. This is the simplest way to integrate Shaper into your existing application.

In addition to this, we provide OpenID Connect integration (OICD) and Proxy Header Auth support as part of our commercial offer. If you have an OICD provider or an internal API gateway integrated in your infrastructure, then integrating with them requires zero custom code. But investing in this infrastructure just for Shaper is definitely more complex than creating a JWT authentication endpoint.

Please reach out if you have any feedback or if you are looking for support with implementing SSO.

Build Your Own Data Analytics Slack Agent With Shaper

At Taleshape, we are building Shaper - an open source SQL-based tool to visualize data and build dashboards.

And we think Shaper is a great fit for building custom data analytics AI agents.

Let’s look at an agent in action:

  • Talk to the agent directly in Slack, and it uses Shaper to answer your data questions.
  • The agent can create anything from charts and Excel files to complete PDF reports, and then shares these files in Slack.
  • Since any chart or report in Shaper is just a SQL query, we can easily verify and reproduce any answer the agent gives.
Play
  1. The agent itself is a simple Python application that listens to Slack messages.
  2. When it receives a message, it uses an LLM to answer the user request.
  3. We give the LLM tools to use Shaper’s API to generate png/pdf/xlsx files directly from SQL queries.
  4. We then upload the generated files to Slack. While the LLM generates the SQL, it doesn’t touch the results, avoiding the risk of hallucinated data or charts.

Architecture Diagram

Since we control the agent and Shaper runs in our own infrastructure, we are also in full control of what data the LLM sees and which LLM we use. This setup can be easily adapted to fit the privacy and security needs of any organization.

Shaper’s interactive dashboards are great for tracking known metrics or giving customers an overview of their data.

But what do we do if we have ad-hoc questions or want to explore our data beyond what any existing dashboard was built for?

That’s where AI agents can help:

  • Build an agent that answers your business team’s data questions without the help of a data analyst.
  • Build an agent to explore new use cases for your data.

Shaper’s SQL-based approach means that instead of answering one-off questions your data team can focus on:

  1. Reviewing and verifying answers the agent gives.
  2. Turn any ad-hoc request into customer-facing dashboards and reusable reports.

Building your own agent not only means that you stay in full control of your data, but also that you can build an agent that is truly useful in practice. You can narrow the problem scope and build an agent for exactly the workflows most useful to you, integrating it into your processes with the context it needs.

Shaper provides the foundation to access data, create charts and reports, while ensuring correctness and keeping everything reproducible.

At Taleshape, we help our customers build systems exactly like this agent.

If you’d like to learn more, don’t hesitate to reach out.

Self-hosted Data Lake Platform with DuckDB, DuckLake and Shaper

Do you need to store and analyze large amounts of data without breaking the bank? Do you need to run in your own infrastructure instead of relying on the big cloud providers?

You can build a simple data lake platform that scales from a $5 VM to petabytes of data with these three open source tools: DuckDB, DuckLake, and Shaper.

Architecture Diagram

A minimal data platform requires at least three components:

  1. Data Storage
  2. Data Ingestion and Transformation
  3. Data Analysis and Visualization

Let’s go over them one by one:

When you have only a small amount of data you can use DuckDB to directly query your production database such as Postgres, but as your data grows that gets slow and expensive.

One alternative is to replicate data to a data warehouse such as Snowflake, BigQuery, Redshift or even Clickhouse. But warehouses are complex to operate and expensive.

That’s where data lakes come in:

Building a data lake architecture means storing data as files in a cheap object storage such as S3.

Data is then stored in a format optimized for analytics such as Parquet files. And DuckDB can query those files really efficiently - fast enough for most use cases even with large amounts of data.

But Parquet files are read-only. To update data you need to create a new file. So you quickly end up with a large number of files that you need to manage and the need to rewrite files for better performance.

That’s why a data lake needs a metadata layer - a catalog of all the data.

Once you have a catalog, using a data lake feels just like using a database:

You create, update and insert into tables without thinking about Parquet files. The catalog automatically rewrites and optimizes the files for you as needed.

The most popular implementation of such a catalog is Apache Iceberg. Cloud providers offer Iceberg as a service (for example AWS S3 Tables and Cloudflare R2 Data Catalog).

And DuckDB has great support for Iceberg.

If your infrastructure provider already supports Iceberg on top of its object storage and you don’t mind paying the premium for that, that’s a great option for a data lake.

However, to run your own data lake on top of any object storage, DuckLake is much simpler than running your own Iceberg implementation.

DuckLake doesn’t require any additional services to run. It works with any object storage and supports many databases for its catalog.

If you have an S3-compatible service and a Postgres database, you can start using DuckLake without any new infrastructure.

We can run a complete data lake platform in a simple docker-compose setup:

  • Minio for object storage
  • Postgres for the DuckLake catalog
  • Shaper to run DuckDB and provide a UI for querying and scheduling jobs
docker-compose.yml
services:
minio:
image: minio/minio
restart: unless-stopped
command: server /data --console-address ":9001"
environment:
MINIO_ROOT_USER: minioadmin
MINIO_ROOT_PASSWORD: minioadmin
volumes:
- minio_data:/data
healthcheck:
test: ["CMD", "mc", "ready", "local"]
interval: 5s
timeout: 5s
retries: 10
minio-init:
image: minio/mc
depends_on:
minio:
condition: service_healthy
entrypoint: >
/bin/sh -c "
mc alias set local http://minio:9000 minioadmin minioadmin &&
mc mb --ignore-existing local/ducklake &&
echo 'Bucket ducklake ready.'
"
restart: "no"
postgres:
image: postgres:16
restart: unless-stopped
environment:
POSTGRES_USER: ducklake
POSTGRES_PASSWORD: ducklake
POSTGRES_DB: ducklake
volumes:
- postgres_data:/var/lib/postgresql/data
healthcheck:
test: ["CMD-SHELL", "pg_isready -U ducklake -d ducklake"]
interval: 5s
timeout: 5s
retries: 10
shaper:
image: taleshape/shaper
restart: unless-stopped
depends_on:
postgres:
condition: service_healthy
minio-init:
condition: service_completed_successfully
ports:
- "5454:5454" # UI → http://localhost:5454
volumes:
- ./init.sql:/init.sql:ro
environment:
SHAPER_INIT_SQL_FILE: /init.sql
MINIO_KEY_ID: minioadmin
MINIO_SECRET: minioadmin
MINIO_ENDPOINT: minio:9000
MINIO_BUCKET: ducklake
PG_HOST: postgres
PG_PORT: "5432"
PG_DB: ducklake
PG_USER: ducklake
PG_PASSWORD: ducklake
volumes:
minio_data:
postgres_data:
shaper_data:

We also need to configure Shaper to set up the secrets for DuckLake. We can do that in a separate init.sql file that Shaper executes on startup:

init.sql
CREATE OR REPLACE SECRET minio_secret (
TYPE s3,
KEY_ID '${MINIO_KEY_ID}',
SECRET '${MINIO_SECRET}',
REGION 'us-east-1',
ENDPOINT '${MINIO_ENDPOINT}',
URL_STYLE 'path',
USE_SSL false
);
CREATE OR REPLACE SECRET pg_secret (
TYPE postgres,
HOST '${PG_HOST}',
PORT ${PG_PORT},
DATABASE '${PG_DB}',
USER '${PG_USER}',
PASSWORD '${PG_PASSWORD}'
);
CREATE SECRET ducklake_secret (
TYPE ducklake,
METADATA_PATH '',
DATA_PATH 's3://${MINIO_BUCKET}/',
METADATA_PARAMETERS MAP {'TYPE': 'postgres', 'SECRET': 'pg_secret'}
);
ATTACH 'ducklake:ducklake_secret' AS ducklake;

You can start the setup by running docker compose up -d in the directory where you created the docker-compose.yml and init.sql files.

We can use Shaper’s Task feature as a simple scheduling system to run SQL queries that ingest and transform data in our data lake:

  1. Open http://localhost:5454 in your browser.
  2. Click on New and select Task at the top.
  3. Copy and paste the following SQL:
SELECT NULL::SCHEDULE;
CREATE TABLE IF NOT EXISTS ducklake.sessions AS
SELECT * FROM 'https://taleshape.com/sample-data/sessions.parquet';
  1. Click on “Create” at the top right and give the task a name such as “Ingest Sessions Data”.
  2. Now click on “Run” in the top right corner.

Congrats, you’ve loaded your first data into DuckLake!

For real use cases you would want to schedule tasks to run automatically and then you would use for example MERGE INTO to update your data lake continuously.

And of course not only Shaper can access DuckLake. For more complex setups you can introduce other tools for your data pipelines such as DBT and Dagster.

Once we have data in our data lake we can now query it.

Let’s build a first Shaper dashboard on top of DuckLake:

  1. In the Shaper UI click on New and select Dashboard at the top.
  2. Copy and paste the following SQL:
SELECT count(*) AS "Total Sessions"
FROM ducklake.sessions;
SELECT 'Sessions per Week'::LABEL;
SELECT
date_trunc('week', created_at)::XAXIS,
category::CATEGORY,
count(*)::BARCHART_STACKED,
FROM ducklake.sessions GROUP BY ALL ORDER BY ALL;
  1. Click on “Create” at the top right and give the dashboard a name such as “Sessions Dashboard”.

Congrats, you just built your first dashboard on top of your data lake!

You can learn more about building dashboards in Shaper in the Getting Started Guide.

You now have a minimal, but complete data lake platform that you can build interesting data products on top of.

But for now this is only running on your computer locally. Now it’s time to deploy this setup into a production infrastructure.

The cheapest way to get started is to simply run all services on a single server, almost identical to how we did above.

But you wouldn’t need a data lake if you have so little data that it fits on a single machine.

Instead, you want to make use of an existing object storage service which almost every hosting provider offers.

If you are looking for a more affordable hosting provider or prefer a European provider over the big American cloud providers, I recommend you look into Hetzner (referral to get free 20€).

You get a VM for 3.49€ and 1TB of object storage for 4.99€, hosted in Germany.

Just be careful that there are some gotchas with Hetzner’s object storage. To make sure your setup is properly secured, check out this detailed guide by my friend Floyd:


And if you are looking for fully-managed Shaper connected to your data lake and integrated into your systems, check out Taleshape’s Managed Hosting and Support Plans.