Deploy to Production

Managed Hosting

If you are looking for help with installing and operating Shaper, have a look at our services:

Managed Hosting and Support Plans

Deploying with Docker Compose

The simplest way to self-host Shaper is to run Docker Compose on a Virtual Machine.

1. Install Docker and Docker Compose on the server.
2. Configure your DNS to point to the server’s IP address.
3. Create a basic docker-compose.yml file:

services:
  caddy:
    image: lucaslorentz/caddy-docker-proxy:ci-alpine
    ports:
      - 80:80
      - 443:443
      - 443:443/udp
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
      - caddy_data:/data
    restart: unless-stopped

  shaper:
    image: taleshape/shaper:0
    environment:
      - SHAPER_DUCKDB_SECRET_DIR=/data/duckdb_secrets
      # Security flags (uncomment and adjust as needed, see Hardening below):
      # Restrict DuckDB file access and extension loading
      # - SHAPER_NO_DUCKDB_COMMUNITY_EXTENSIONS=true
      # - SHAPER_ALLOWED_DUCKDB_EXTENSIONS=httpfs,parquet,json
      # - SHAPER_INIT_SQL=SET allowed_directories = ['/tmp']; SET allowed_paths = ['/tmp']; SET enable_external_access = false; SET autoload_known_extensions = false; SET autoinstall_known_extensions = false;
      # Disable Shaper features you are not using
      # - SHAPER_NO_TASKS=true
      # - SHAPER_NO_PUBLIC_SHARING=true
      # - SHAPER_NO_PASSWORD_PROTECTED_SHARING=true
      # Deploy from code and disable live editing completely
      # - SHAPER_NO_EDIT=true
      # Restrict embedding to known domains
      # - SHAPER_CORS_DOMAINS=domain1.com,domain2.com
      # Set your own JWT Secret instead of auto-generating and storing in NATS
      # - SHAPER_JWT_SECRET=...
      # Encrypt NATS Jetstream
      # - SHAPER_NATS_JS_KEY=...
    labels:
      caddy: yourdomain.com
      caddy.reverse_proxy: "{{upstreams 5454}}"
    volumes:
      - shaperdata:/data
    restart: unless-stopped
    logging:
      driver: "json-file"
      options:
        max-size: "10m"
        max-file: "3"

volumes:
  shaperdata:
    driver: local
  caddy_data:
    driver: local

4. Run docker compose up -d

With this setup, caddy-docker-proxy automatically generates Caddy configuration from Docker labels and obtains and renews TLS certificates for your domain via Let’s Encrypt / ZeroSSL. This allows you to add TLS without maintaining a separate reverse proxy config file.

Data is stored in the shaperdata volume, which is persisted across container restarts and updates. Certificates and Caddy state are stored in the caddy_data volume.

You likely also want to mount a file at /var/lib/shaper/init.sql to setup credentials and attach remote databases and such. Shaper automatically executes this SQL on startup. See the Connecting to Data Sources docs for more.

The commented-out environment variables demonstrate common security hardening options—such as restricting DuckDB file access and extensions, disabling unused features, locking down live editing, restricting CORS domains, and encrypting NATS Jetstream. Review the Hardening: Secure your Shaper Instance section below to choose the settings that fit your environment.

Hardening: Secure your Shaper Instance

Danger

By default, no authentication is required to access the Shaper web interface.

When exposing Shaper on the internet you must enable authentication to prevent unauthorized access to your data.

To enable authentication, click on “Admin” in the bottom left corner and then click on “Create User”.

Disclaimer

Shaper lets you use the full power of DuckDB. Only let trusted individuals create dashboards and tasks.

Take any measures of the following that fit your use case to protect your system:

General measures

• Run Shaper in a restricted Docker container
• Run multiple Shaper containers to separate use cases and have clear security boundaries
• Give Shaper only access to the data it strictly needs
• Give Shaper only read access when possible
• Configure your firewall to restrict incoming and outgoing traffic

Lock down the Shaper configuration

# Restrict DuckDB file access and extension loading
SHAPER_NO_DUCKDB_COMMUNITY_EXTENSIONS=true
SHAPER_ALLOWED_DUCKDB_EXTENSIONS=httpfs,parquet,json
SHAPER_INIT_SQL=SET allowed_directories = ['/tmp']; SET allowed_paths = ['/tmp']; SET enable_external_access = false; SET autoload_known_extensions = false; SET autoinstall_known_extensions = false;
# Disable Shaper features you are not using
SHAPER_NO_TASKS=true
SHAPER_NO_PUBLIC_SHARING=true
SHAPER_NO_PASSWORD_PROTECTED_SHARING=true
# Deploy from code and disable live editing completely
SHAPER_NO_EDIT=true
# Restrict embedding to known domains
SHAPER_CORS_DOMAINS=domain1.com,domain2.com
# Set your own JWT Secret instead of auto-generating and storing in NATS
SHAPER_JWT_SECRET=...
# Encrypt NATS Jetstream
SHAPER_NATS_JS_KEY=...

See the Configuration Options and the Security Whitepaper for more details.

Backing up your data

Shaper can automatically create daily snapshots of your data and upload it to an S3-compatible object storage.

Shaper uses two file-based databases: SQLite and DuckDB. Internal data such as users and dashboards are stored in SQLite and you can persist data in DuckDB via the tasks or ingest API. When snapshots are enabled, both databases are snapshotted and uploaded to the configured S3 bucket.

By default Shaper automatically restores the latest snapshot on startup if no local data is found.

Setting up snapshots

1. Create an S3 bucket in your preferred object storage. Make sure to set a lifecycle policy to automatically delete old snapshots after a certain period of time.
2. Set the following environment variables in your Shaper instance:

• SHAPER_SNAPSHOT_S3_BUCKET: The name of the S3 bucket (required)
• SHAPER_SNAPSHOT_TIME: The time of day when the snapshot should be created in HH:MM format (optional, defaults to 01:00)

3. Configure S3 credentials using one of the following methods:

Option A: AWS Credential Chain (Recommended for AWS)

If you’re using AWS S3, you can use the standard AWS credential chain:

• Set AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, and AWS_REGION environment variables
• Or use IAM roles (if running on EC2)
• Or use AWS credentials file (~/.aws/credentials)

Option B: Shaper-specific configuration

Alternatively, you can use Shaper-specific environment variables:

• SHAPER_SNAPSHOT_S3_ENDPOINT: The endpoint URL of the S3 service (optional, defaults to AWS S3 if not provided).
• SHAPER_SNAPSHOT_S3_ACCESS_KEY: The access key.
• SHAPER_SNAPSHOT_S3_SECRET_KEY: The secret key.
• SHAPER_SNAPSHOT_S3_REGION: The region of the S3 bucket.

The Shaper-specific options will override the AWS credential chain if both are provided.

How snapshots work

SQLite and DuckDB snapshots are created one after the other, SQLite first.

When running in a multi-node setup, only one node creates snapshots to avoid conflicts.

SQLite snapshots

Shaper uses VACUUM INTO to write a new SQLite database to a temporary file. The temporary file is then uploaded to the S3 bucket with a name like shaper-snapshots/shaper-sqlite-2006-01-02_15-04-05.db The SQLite snapshot contains the NATS consumer offsets in the consumer_state table and will automatically resume from the last processed message on restore if there is newer data in the NATS stream.

DuckDB snapshots

Shaper uses EXPORT DATABASE and IMPORT DATABASE to create and restore DuckDB snapshots directly to/from S3 from within DuckDB. The data is stored in S3 as Parquet files (zstd compressed) and two SQL scripts. The DuckDB snapshot is stored in a folder like shaper-snapshots/shaper-duckdb-2006-01-02_15-04-05/. To do this from within DuckDB, Shaper creates a DuckDB S3 secret for the configured S3 bucket. This secret is always available which means you can interactively query the snapshots directly from DuckDB if needed. When restoring a DuckDB snapshot, Shaper first runs all INSTALL, LOAD and ATTACH statements found in the init.sql file since some views and macros might depend on them.

Proxying Shaper

If you are using a custom reverse proxy to proxy requests to Shaper, make sure to set x-real-ip/x-forwarded-for for Shaper’s rate limiting to work correctly.

You can also serve Shaper at a non-root base-path by setting basepath

It’s also common to serve embedded dashboards, but keep the rest of Shaper private. In that case, proxy only the following routes:

GET /embed/shaper.js
GET /embed/custom.css
GET /api/dashboards/:id
GET /api/dashboards/:id/download/:file
GET /api/download/:token/:file

Clustering

Work in Progress

You can already run a multi-node Shaper cluster by using an external NATS server. Once implemented, you will also be able to cluster the built-in NATS server directly. More detailed docs will follow after. See Github for progress.

Monitoring

Work in Progress

Basic metrics are exposed at /metrics in Prometheus format. In production you need an API Key (that has “Read Metrics” permissions) as bearer token to access the metrics endpoint. More metrics and detailed documentation are coming soon. See Github for progress.

NATS Stream Retention

By default the NATS streams keep all messages indefinitely. If you are seeing issues because you are ingesting a lot of data, run a lot of tasks or have a huge amount of dashboard edits, you can limit the retention period of the streams via these options:

• ingest-max-age
• state-max-age
• task-results-max-age