ProjectionEngine

zio.blocks.projection.ProjectionEngine
See theProjectionEngine companion object
final class ProjectionEngine

Runs projections: it reads events from EventStores, applies each Projection's handlers, and keeps the results in ProjectionStores that can be queried.

A Projection only describes what should happen. The engine is the part that does it. Each projection gets a primary store, which is one SQLite file per projection on the JVM or an in-memory map in tests.

Lifecycle: start first compares each projection's stored schema hash with the current one, and brings stale stores up to date unless lazyRebuild is on. Then, for each source of each projection, it catches up on the events after the projection's saved position, reading them in batches with readFrom, and only then starts a fiber that follows the new events from the source's hub. start returns once every source has caught up. query first applies any pending lazy rebuild.

Routing: for each event the engine builds a ProjectionContext from the EventEnvelope, runs the matching handler, and applies the resulting ProjectionAction to the store chosen by the projection's ProjectionScope: the primary store for per-entity and global projections, and a shard per routing key for cross-entity ones.

Failures: an event whose handler throws, or that the store rejects, is logged as an error and skipped; an event that no handler matches is skipped too, and logged at debug level. Processing continues with the next event. The saved position is updated after each event that succeeds, so a skipped event is not retried once a later one succeeds.

Persistence: the stores that make, makeWithConfig and makeWithStoresVarargs create are SQLite files, so sqlite-jdbc must be on the runtime classpath and the folders of the files must exist. A store opens its file only when it is first used, so a missing driver or folder shows up as a failure then, not when the engine is created.

Create an engine with the factories on the companion object. make and makeWithConfig give it no event stores, so it has nothing to read; to give it event stores, create it with makeWithStores or makeWithStoresVarargs, or use TestEngine in tests.

Value parameters

cache

shared TransactorCache for SQLite connections

config

the settings the engine was created with

eventStores

per-source event stores keyed by source name

evolutionSem

semaphore guarding rebuilds

migrationRegistry

registered schema migrations

pendingRebuildRef

specs pending lazy rebuild

primaryStores

per-spec primary stores keyed by spec name

shardsRef

sharded stores for cross-entity routing

specs

list of projections this engine manages

Attributes

Companion
object
Source
ProjectionEngine.scala
Graph
Supertypes
class Object
trait Matchable
class Any

Members list

Value members

Concrete methods

def eventStoresMap: Map[String, EventStore[_]]

The event stores, keyed by source name.

The event stores, keyed by source name.

Attributes

Source
ProjectionEngine.scala
def getLastProcessedSeq[A](spec: Projection[A]): Task[Long]

The sequence number of the last event the projection has applied, which is where the engine resumes after a restart. It is kept in the projection's primary store, also for a cross-entity projection with shards.

The sequence number of the last event the projection has applied, which is where the engine resumes after a restart. It is kept in the projection's primary store, also for a cross-entity projection with shards.

Value parameters

spec

the projection

Attributes

Source
ProjectionEngine.scala
def query[A](spec: Projection[A], entityId: String): Task[Option[A]]

Queries the projection for the entity with the given id.

Queries the projection for the entity with the given id.

Ensures any pending lazy rebuild for spec is applied before reading. For cross-entity projections, also searches sharded stores if the primary has no match.

Type parameters

A

the row type of the projection

Value parameters

entityId

the entity id to look up

spec

the projection to query

Attributes

Returns

the row, or None if no store has it

Source
ProjectionEngine.scala
def queryByName[A](specName: String, entityId: String): Task[Option[A]]

Reads the row with the given ID from the projection with the given name, or None if there is no such projection or row.

Reads the row with the given ID from the projection with the given name, or None if there is no such projection or row.

Unlike query, it neither applies a pending lazy rebuild nor searches shards, and the caller chooses the result type A without it being checked.

Value parameters

entityId

the ID of the row

specName

the name of the projection

Attributes

Returns

the row, or None

Source
ProjectionEngine.scala
def registerMigration[Old, New](spec: Projection[New], migration: Migration[Old, New]): Task[Unit]

Registers how the row type of a projection changed, from Old to New, so that the change can be handled without a full rebuild.

Registers how the row type of a projection changed, from Old to New, so that the change can be handled without a full rebuild.

When the stored rows are out of date and every action of migration adds a top-level field, the engine adds the new columns with ALTER TABLE ADD COLUMN instead of replaying the events. The existing rows get no value in the new columns. For any other change, it rebuilds the rows from the events as usual.

Register it before start, which is when the schema is checked. That holds with lazyRebuild too: start ends by forking a background rebuild, which can run before a migration registered later. The migration is kept under the projection's name, so registering another one replaces it.

Type parameters

New

the current row type

Old

the previous row type

Value parameters

migration

the migration from Old to New

spec

the projection the migration applies to

Attributes

Source
ProjectionEngine.scala
def start: ZIO[Scope, Nothing, Unit]

Starts the engine.

Starts the engine.

It first checks the schema of each projection and, unless lazyRebuild is on, brings stale stores up to date, with the migration shortcut or a full rebuild. Then, for each source of each projection, it applies the events after the projection's saved position and starts a fiber that follows the new events. With lazyRebuild on, it also starts a fiber that rebuilds the pending projections in the background.

A source that fails is logged and skipped. A source without an event store, which only happens when the engine has no event stores at all, is skipped with a warning. The fibers run until the enclosing Scope closes; there is no separate stop method.

Attributes

Returns

a scoped task that completes once every source has caught up and its live fiber has been started

Source
ProjectionEngine.scala
def storesMap: Map[String, ProjectionStore[_]]

The primary store of each projection, keyed by projection name.

The primary store of each projection, keyed by projection name.

Attributes

Source
ProjectionEngine.scala

The TransactorCache that opens the engine's SQLite files: the one make and makeWithConfig created, or the one given to makeWithStores or makeWithStoresVarargs.

The TransactorCache that opens the engine's SQLite files: the one make and makeWithConfig created, or the one given to makeWithStores or makeWithStoresVarargs.

Exposed for tests and for code that needs a connection to the same files.

Attributes

Source
ProjectionEngine.scala

Concrete fields