Skip to main content
The ClickHouse destination writes structured pipeline output to a ClickHouse database using batch inserts. It supports configurable buffer sizes and flush intervals for throughput tuning.

​
Configuration

destination:
  type: "clickhouse"
  url: "clickhouse://host:8123/db"
  user: "${CH_USER}"
  password: "${CH_PASSWORD}"
  batch_size: 10000
  flush_interval_ms: 5000

​
Fields

FieldTypeRequiredDefaultDescription
urlstringyes—JDBC connection URL
userstringyes—Database user
passwordstringyes—Database password
batch_sizeintno10,000Rows per batch insert
flush_interval_mslongno5,000Max time (ms) between flushes

​
How It Works

  1. Rows are buffered in memory as they arrive from the pipeline.
  2. When the buffer reaches batch_size or flush_interval_ms elapses since the last flush, a batch INSERT executes against the target table.
  3. The INSERT SQL is dynamically generated from the field names present in each row.
  4. All column names are automatically quoted in the generated SQL to handle reserved words.

​
Type Coercion

The destination maps pipeline data types to ClickHouse column types as follows:
Data TypeDatabase Handling
StringString column
LongInt64 / UInt64
IntegerInt32 / UInt32
DoubleFloat64
BooleanBool / UInt8
byte[]Bytes
nullNull