Skip to main content

Local deployment

When to use

runtimeType=local is the default backend. It is the right choice for:

  • Developer workstations and single-host setups where all DuckDB Quack nodes run as child processes of the manager JVM.
  • Smoke-testing and integration work where Kubernetes is not available or not needed.
  • Small deployments where the total node count fits comfortably on one machine.

For multi-host or container-orchestrated deployments, see the Kubernetes backend instead.

How nodes are spawned

LocalQuackBackend spawns each DuckDB Quack node by forking a child process via scripts/spawn-quack-node.sh. The manager controls the full lifecycle: it allocates a port from the configured range, generates a random authentication token, sets the metastore environment variables, and hands those to the script. When the manager stops (SIGTERM or ./scripts/stop-jar.sh), it sends SIGTERM to every tracked child and waits up to 5 seconds per process before issuing SIGKILL.

Port allocation. The manager maintains a PortAllocator bounded by minPort and maxPort (defaults: 21900 to 22500). Ports are leased on spawn and released on stop. At most maxNodesTotal (default: 50) child nodes may be alive at the same time across all pools.

Do not run scripts/spawn-quack-node.sh directly. The supervisor owns port allocation, token generation, and the metastore environment contract. Manual invocations bypass all of that, leak ports into the allocator's range, and produce orphan processes that cause silent "Authentication failed" errors at query time.

Orphan detection. scripts/run-jar.sh runs a port preflight check at startup. If any listener is found inside the node port range it aborts with an actionable error message. To clear orphans:

pkill -f 'quack_serve|spawn-quack-node'

The spawn script path can be overridden via QOD_SPAWN_SCRIPT when the script is installed outside the repository (for example under /opt/... in a container image).

Node environment contract

LocalQuackBackend.start() populates each child process's environment from two sources in order: the defaultMetastore block in application.conf (or its QOD_PG_* env-var overrides), then any per-pool metastore entries supplied in the CreatePoolRequest. Pool-level keys override the defaults.

The spawn script reads the following variables:

VariableMeaning
pgHostHostname of the Postgres server used by DuckLake as its catalog store.
pgPortPort of the Postgres server (default 5432).
pgUserPostgres login user (default postgres).
pgPasswordPostgres password for pgUser.
dbNamePostgres database that holds the DuckLake catalog metadata. Also used as the DuckDB catalog name.
schemaNameDuckLake schema inside dbName where tables are registered. Must differ from dbName to avoid ambiguous two-part identifiers in DuckDB.
dataPathLocal or object-store path where DuckLake writes Parquet data files. Accepts s3://, az://, and abfss:// URIs for remote storage.

In addition the supervisor injects two internal variables that the spawn script receives as positional arguments: the TCP port the node must listen on, and a 256-bit URL-safe base64 bearer token used to authenticate the manager's HTTP requests to that node. Neither appears in application.conf; they are generated fresh for each spawn.

The kind variable is also required. It selects the catalog mode: ducklake (DuckLake catalog backed by Postgres), duckdb-file (standalone .duckdb file), or memory (no persistent catalog; useful for federation-only pools).

Operate locally

Configuration knobs. All relevant settings accept QOD_* environment variable overrides. See /reference/configuration for the full list. The most commonly adjusted ones for a local deployment are:

Env varDefaultPurpose
QOD_RUNTIME_TYPElocalBackend type. Leave as local for single-host deployments.
QOD_MIN_PORT21900Lower bound of the child node port range.
QOD_MAX_PORT22500Upper bound of the child node port range.
QOD_MAX_NODES_TOTAL50Hard cap on simultaneous child node processes.
QOD_SPAWN_SCRIPT./scripts/spawn-quack-node.shPath to the node spawn script.
QOD_PG_HOSTlocalhostPostgres host for the control plane and default metastore.
QOD_PG_PASSWORDazizamPostgres password. Rotate before any non-local exposure.
QOD_DUCKLAKE_DATA_PATH./ducklake/tpchDefault data path propagated to spawned nodes.

Wiping local state. To discard all state and start fresh, pass NUKE=1 to run-jar.sh:

NUKE=1 ./scripts/run-jar.sh

This stops any running manager, drops the Postgres control-plane database and the bootstrap tenant database, and removes the ducklake/, state/, and certs/ directories. This action is irreversible.

Respawn and recovery behavior. When the manager restarts after a crash or SIGTERM, it reads the persisted node registry and calls adopt() on each node that was alive at shutdown. Adopted nodes have their port re-claimed in the allocator and are tracked via ProcessHandle so a subsequent stop() still delivers SIGTERM. Nodes that died while the manager was down are detected by the health-check loop and replaced automatically. For full details on how the manager handles node failures, drains, and replacements, refer to the Resilience guide.