QueueTable

zio.blocks.data.migration.QueueTable

Helpers for migration queue tables used by migration workers, dialect-driven to support both PostgreSQL and SQLite.

Queue tables store IDs of aggregates pending migration. Workers dequeue batches via SELECT ... FOR UPDATE SKIP LOCKED on PostgreSQL; SQLite uses a plain LIMIT with the write lock held via IMMEDIATE (SQLite connections via JdbcTransactor.fromUrl/sqlite or any pooled/wrapped DataSource are configured per-transaction with busy_timeout=5000 and IMMEDIATE via isWrapperFor/unwrap, so transact reserves the write lock and waits on contention; connect is configured similarly but does not begin a transaction or reserve the lock — only transact issues BEGIN IMMEDIATE; see PR #1534 discussion_r3863454094).

Attributes

Source: QueueTable.scala
Graph
Supertypes: class Object

trait Matchable

class Any
Self type: QueueTable.type

Members list

Value members

Concrete methods

Creates a queue table with columns (id, op, payload) for tracking dirty keys. Uses the given dialect to generate database-appropriate DDL.

Note: trigger installation is a separate step handled by the dialect's createTriggerDDL; this method only creates the queue table itself.

Value parameters

tableName: Name of the queue table (validated as a bare SQL identifier)
transactor: Transactor for executing DDL

Attributes

Source: QueueTable.scala

Pure queue-table CREATE TABLE fragment.

Attributes

Source: QueueTable.scala

Dequeues up to batchSize IDs. Uses the given dialect to generate database-appropriate locking — FOR UPDATE SKIP LOCKED on PostgreSQL, plain LIMIT on SQLite.

On SQLite the transaction must already hold the write lock before the SELECT; JdbcTransactor configures any SQLite Connection (including pooled/wrapped via isWrapperFor/unwrap) per-transaction with busy_timeout=5000 and IMMEDIATE (also fromUrl properties and sqlite(DataSource) for concrete SQLiteDataSource), so transact reserves the write lock and waits on contention instead of failing the later DELETE with SQLITE_BUSY. Single-consumer only on SQLite. See PR #1534 discussion_r3863454094.

Attributes

Source: QueueTable.scala

Pure dequeue SELECT template for the given dialect.

Attributes

Source: QueueTable.scala

Enqueues IDs into the queue table.

The queue table stores IDs in a TEXT column for portability across ID types. On PostgreSQL, numeric ID parameters are wrapped in CAST(... AS TEXT) so that they work against the TEXT column without an implicit-cast error.

Attributes

Source: QueueTable.scala

Installs capture triggers on sourceTable so that every INSERT, UPDATE, and DELETE upserts the affected key into the queue table within the writer's own transaction. This is the automatic counterpart to manual enqueue: once installed, producers write normally and dirty keys are captured without application-code changes.

The queue key column is the fixed id column (see QueueKeyColumn) — use the same codec for create/enqueue. sourceIdColumn names the primary key column on the source table and may differ from it.

Idempotent: safe to call again on restart (SQLite uses CREATE TRIGGER IF NOT EXISTS; PostgreSQL uses CREATE OR REPLACE TRIGGER, which requires PG 14+).

Do NOT enable capture on the same physical table the migrator writes to: the worker's own writes would re-enqueue processed keys forever.