Null engine

When writing to a Null table, data is ignored. When reading from a Null Data Source, the response is empty.

Why use the Null engine, then?

To create a Materialized View from a Null table that transforms the data as it's sent to the Null Data Source so the Target Data Source will receive the transformed data and the original will be discarded.

Example: for masking some PII data.

event.datasource
SCHEMA >
    `action` LowCardinality(String) `json:$.action`,
    `timestamp` DateTime64(3) `json:$.timestamp`,
    `version` LowCardinality(String) `json:$.version`,
    `userEmail` String `json:$.userEmail` DEFAULT '',
    `workspaceId` String `json:$.workspaceId` DEFAULT '',
    `payload` String `json:$.payload`

ENGINE Null
mask_events_mat.pipe
DESCRIPTION >
    Mask events for mat view

NODE maskEvents
SQL >
    SELECT
        action, timestamp, version, cityHash64(userEmail) as userEmailMasked, workspaceId, payload
    FROM event

TYPE Materialized
DATASOURCE event_masked
event_masked.datasource
SCHEMA >
    `action` LowCardinality(String),
    `timestamp` DateTime64(3),
    `version` LowCardinality(String),
    `userEmailMasked` UInt64,
    `workspaceId` String ,
    `payload` String

ENGINE MergeTree
ENGINE_PARTITION_KEY toYear(timestamp)
ENGINE_SORTING_KEY action, workspaceId, timestamp
Updated