Build applications that respond immediately to events. Craft materialized views over streams. Receive real-time push updates, or pull current state on demand.
Seamlessly leverage your existing Apache Kafka® infrastructure to deploy stream-processing workloads and bring powerful new capabilities to your applications.
Use a familiar, lightweight syntax to pack a powerful punch. Capture, process, and serve queries using only SQL. No other languages or services are required.
CREATE SOURCE CONNECTOR riders WITH (
'connector.class' = 'JdbcSourceConnector',
'connection.url' = 'jdbc:postgresql://...',
'topic.prefix' = 'rider',
'table.whitelist' = 'geoEvents, profiles',
'key' = 'profile_id',
...);
CREATE STREAM locations AS
SELECT rideId, latitude, longitude,
GEO_DISTANCE(latitude, longitude,
dstLatitude, dstLongitude
) AS kmToDst
FROM geoEvents
EMIT CHANGES;
CREATE TABLE activePromotions AS
SELECT rideId,
qualifyPromotion(kmToDst) AS promotion
FROM locations
GROUP BY rideId
EMIT CHANGES;
SELECT rideId, promotion
FROM activePromotions
WHERE ROWKEY = '6fd0fcdb';
ksqlDB enables you to build event streaming applications leveraging your familiarity with relational databases. Three categories are foundational to building an application: collections, stream processing, and queries.
Streams are immutable, append-only sequences of events. They're useful for representing a series of historical facts.
CREATE STREAM routeWaypoints (
vehicleId VARCHAR,
latitude DOUBLE(10, 2),
longitude DOUBLE(10, 2)
) WITH (
kafka_topic = 'locations',
partitions = 3,
key = 'vehicleId',
value_format = 'json'
);
Tables are mutable collections of events. They let you represent the latest version of each value per key.
CREATE TABLE currentCarLocations (
vehicleId VARCHAR,
latitude DOUBLE(10, 2),
longitude DOUBLE(10, 2)
) WITH (
kafka_topic = 'locations',
partitions = 3,
key = 'vehicleId',
value_format = 'json'
);
Stream processing enables you to execute continuous computations over unbounded streams of events, ad infinitum. Transform, filter, aggregate, and join collections together to derive new collections or materialized views that are incrementally updated in real-time as new events arrive.
Push queries let you subscribe to a query's result as it changes in real-time. When new events arrive, push queries emit refinements, which allow you to quickly react to new information. They’re a perfect fit for asynchronous application flows.
SELECT vehicleId,
latitude,
longitude
FROM currentCarLocations
WHERE ROWKEY = '6fd0fcdb'
EMIT CHANGES;
Pull queries allow you to fetch the current state of a materialized view. Because materialized views are incrementally updated as new events arrive, pull queries run with predictably low latency. They're a great match for request/response flows.
SELECT vehicleId,
latitude,
longitude
FROM currentCarLocations
WHERE ROWKEY = '6fd0fcdb';
Today, nearly all streaming architectures are complex, piecemeal solutions. They comprise multiple subsystems, each with its own mental model.
With a lightweight, familiar SQL syntax, ksqlDB presents a single mental model for working with event streams across your entire stack: event capture, continuous event transformations, aggregations, and serving materialized views.