- Posted on
- admin
- No Comments
What Is RisingWave? Complete Beginner's Guide
Batch analytics answers questions about yesterday. Plenty of products need answers about right now: which credit card just spent an unusual amount in the last minute, how many people are on the checkout page this second, which sensor reading crossed a threshold. Getting those answers has traditionally meant building a stream processing system, and that has traditionally meant a steep learning curve, a cluster to babysit, and a separate database to store whatever the stream produces.
RisingWave takes a different route. It’s a database you talk to with ordinary SQL, but instead of waiting for you to ask questions, it keeps the answers continuously up to date as new data arrives. This guide explains what RisingWave is, how its architecture works, which concepts a beginner needs, and walks through a hands-on Docker quickstart. If you’ve followed our ClickHouse vs PostgreSQL comparison, you’ve already met the change data capture pattern that RisingWave leans on heavily.
What Is RisingWave?
(cite index=”114-1″>RisingWave is a Postgres-compatible streaming database designed to be a simple, cost-efficient way to process, analyze, and manage real-time event streaming data. Let’s unpack each part of that description.
It’s a database. (cite index=”121-1″>RisingWave is an open-source distributed streaming database aimed at real-time analytics and event-driven applications, and it uses an incremental computation model to process streaming data with low latency. Unlike a pure processing framework, it stores data and serves queries itself.
It’s PostgreSQL compatible. (cite index=”118-1″>It’s wire compatible with PostgreSQL, meaning any tool, driver, or ORM that works with PostgreSQL works with RisingWave without modification. You connect with psql, write familiar SQL, and use the client libraries you already know.
It’s written in Rust and open source. (cite index=”117-1″>RisingWave was built from scratch in Rust and is released under the Apache 2.0 license.
Where It Came From
(cite index=”121-1″>RisingWave Labs, formerly known as Singularity Data, was founded in 2021 by former IBM researcher and Amazon Redshift engineer Yingjun Wu. The motivation is a useful way to remember what the product is for. (cite index=”121-1″>While working on Redshift, Wu saw that existing database systems couldn’t process streaming data efficiently, and that existing streaming systems were too complicated for most companies to use. RisingWave is an attempt to sit in the gap: database-style ease of use with streaming-style freshness.
Streaming Database vs Stream Processor
Beginners often ask why anyone would call this a “streaming database” instead of just another stream processor. The distinction is real, and it shapes how you use the tool.
(cite index=”115-1″>RisingWave describes itself as a streaming database, as opposed to a stream processing framework like Apache Flink. A framework transforms events in flight and hands the output somewhere else. A streaming database also keeps the results. (cite index=”115-1″>By defining materialized views, analytical results of input streams can be gathered and served directly inside RisingWave.
(cite index=”118-1″>A streaming database combines processing with storage and serving: it ingests events, computes streaming transformations, stores the results in materialized views, and lets applications query those views directly. That means your application can read fresh results with a normal SELECT, instead of the pipeline writing into yet another database that something else has to query.
The Core Idea: Incrementally Maintained Materialized Views
If you remember only one concept from this guide, make it this one.
In a traditional database, a materialized view is a saved query result. It goes stale, and you have to run a refresh to bring it up to date. RisingWave flips that. (cite index=”131-1″>A materialized view in RisingWave isn’t a static snapshot or a one-time query, it’s a continuously maintained result that updates automatically as new data arrives, like a live dashboard behind a SQL query. (cite index=”125-1″>These views don’t require a REFRESH command, because they’re continuously and incrementally maintained.
“Incrementally” is the key word. When a new row arrives, RisingWave doesn’t recompute a billion-row aggregation from scratch. It adjusts the existing result by the effect of that one row. (cite index=”121-1″>The system keeps user-defined materialized views up to date through a change propagation framework, so the work per event stays small.
You can also build on top of other views. (cite index=”116-1″>RisingWave reduces the complexity of building stream processing applications by letting developers express intricate logic through cascaded materialized views. One view cleans events, a second aggregates them, a third joins the aggregate to reference data, and each stays current automatically.
How RisingWave Works: The Architecture
Now for what’s happening under the hood. Understanding the pieces helps you operate it and explains its design choices.
Four Kinds of Nodes
(cite index=”137-1″>RisingWave is a distributed streaming database with a cloud-native architecture built from four node types:
- Serving nodes act as the PostgreSQL-compatible front door. (cite index=”139-1″>A frontend node is a stateless proxy that accepts user queries through the Postgres protocol.
- Streaming (compute) nodes do the continuous work. (cite index=”139-1″>A compute node executes the optimized query plans and handles data ingestion and output. (cite index=”135-1″>A compute node can run in a streaming role, a serving role, or both.
- Meta nodes coordinate everything. (cite index=”139-1″>The meta service is the control plane, and it also acts as a failure detector that sends heartbeats to the other nodes.
- Compactor nodes tidy storage. (cite index=”139-1″>A compactor is a stateless worker that compacts data for the storage engine.
There’s also an optional helper. (cite index=”139-1″>The connector node is a Java component that handles consuming CDC events from upstream systems and sinking data to downstream ones, and it’s enabled by default when you run RisingWave in Docker.
A useful operating rule of thumb comes from the deployment guidance. (cite index=”140-1″>The meta node owns the catalog and schedules barriers, the compute node runs streaming operators and is where to add memory as views grow, and the compactor scales with write volume. For learning, none of this matters, because (cite index=”131-1″>a single-node Docker command bundles everything together.
Barriers and Checkpoints: How It Stays Consistent
Streaming systems have to answer an awkward question: if a node crashes, how do you resume without losing or double-counting data? RisingWave uses periodic markers called barriers.
(cite index=”137-1″>A barrier is a periodic sync marker, and a checkpoint is a global consistent snapshot created from barriers. By default, RisingWave generates one barrier every second. (cite index=”137-1″>When something fails, the meta node recreates the affected actors on the streaming node and restarts from the last checkpoint snapshot to maintain consistency. That one-second cadence is also why the system advertises sub-second to low-second freshness rather than instant.
Hummock: State Lives in Object Storage
The most distinctive design choice is where the state lives. Streaming computations like aggregations and joins have to remember things, such as the running total per customer. In many systems that state sits on the machine’s local disk, which makes scaling and recovery painful.
(cite index=”119-1″>RisingWave stores the state for all its streaming operators in Hummock, a cloud-native, LSM-tree-based storage engine co-designed with the streaming engine and optimized for streaming workloads. (cite index=”115-1″>Hummock persists materialized views and internal operator state to S3-compatible cloud storage.
(cite index=”120-1″>Compute nodes are therefore stateless: they process events, push state to Hummock, and serve queries by reading from it. The practical payoffs follow directly. (cite index=”142-1″>Because storage is decoupled, there’s no data movement between an old compute node and its replacement after a failure, which speeds up recovery. And you can add or remove compute capacity without reshuffling state, much like the storage and compute separation you’ll see in Snowflake-style warehouses and in our ClickHouse architecture guide‘s discussion of SharedMergeTree.
What Happens When Things Fail
It’s worth knowing the failure behavior, since it informs production planning. (cite index=”142-1″>If a compute node fails, the meta node loses its heartbeat, tells all components to pause, and waits for the node to return, and on Kubernetes a new pod is started automatically. (cite index=”142-1″>If the meta node fails, the cluster stops all operations until it’s restored, so production setups pair it with a standby and a highly available metadata store. (cite index=”142-1″>A frontend failure doesn’t disrupt stream processing, though if it’s your only frontend you can’t issue SQL until it’s back, and a compactor failure has no immediate impact on jobs.
Key Concepts for Beginners
RisingWave’s vocabulary is small, and most of it is familiar SQL.
Source. (cite index=”138-1″>A source is a resource RisingWave can read data from, such as Kafka or Pulsar message brokers and databases like MySQL and PostgreSQL.
Table. There are two flavors, and the difference trips up newcomers. (cite index=”125-1″>A table created with a connector declares an external stream and also persists the data, while a table created without one is a standard mutable table for reference or dimension data you manage yourself.
Materialized view. The continuously maintained result described above.
Sink. (cite index=”128-1″>Data in tables and materialized views is stored in RisingWave, and you can sink it out to Kafka topics or databases; a sink can be created from a table, source, materialized view, or an ad hoc SELECT.
Subscription. (cite index=”125-1″>Subscriptions let applications receive a stream of changes from a materialized view, useful when an app needs to react to updates rather than poll.
Key Features
Rich ingestion, including CDC. (cite index=”141-1″>RisingWave ingests change data from PostgreSQL, MySQL, MongoDB, and SQL Server, and can also read from Iceberg tables. (cite index=”116-1″>That CDC support lets teams offload event-driven work such as materialized views and triggers from operational databases like PostgreSQL.
Many destinations. (cite index=”140-1″>Sinks go back out to Kafka, Iceberg, PostgreSQL, Redis, and object storage.
Time windows. Streaming analytics revolves around windows, such as “per minute” or “per hour.” (cite index=”125-1″>RisingWave supports tumbling windows through TUMBLE, and an EMIT ON WINDOW CLOSE option emits final results only after a window completes.
Iceberg integration. (cite index=”134-1″>An Iceberg serving engine operates when a table is declared with engine=iceberg, powered by the Apache Iceberg table format. If that format is new to you, our Apache Iceberg tutorial explains it from the ground up.
User-defined functions. (cite index=”123-1″>RisingWave offers UDFs as external functions in Python, and WASM can also be used to define them.
Hands-On Quickstart
Let’s run it. This walkthrough uses Docker and a few SQL statements, and it shows incremental maintenance happening in front of you.
Step 1: Start RisingWave
(cite index=”131-1″>The documented single-node command is:
docker run -it --pull=always -p 4566:4566 -p 5691:5691 \
risingwavelabs/risingwave:latest single_node
Port 4566 is the SQL endpoint, and port 5691 exposes the web dashboard. Wait for the log line indicating the node is ready. Single-node mode is for learning and testing, not production.
Step 2: Connect With psql
(cite index=”132-1″>Connect with the standard Postgres terminal, using the default database and user:
psql -h localhost -p 4566 -d dev -U root
Step 3: Create a Table and a Materialized View
CREATE TABLE orders (
order_id INT,
customer VARCHAR,
amount DECIMAL,
ts TIMESTAMP
);
CREATE MATERIALIZED VIEW revenue_by_customer AS
SELECT customer,
SUM(amount) AS total,
COUNT(*) AS orders
FROM orders
GROUP BY customer;
Step 4: Watch It Update
INSERT INTO orders VALUES (1, 'Alice', 100, NOW());
INSERT INTO orders VALUES (2, 'Bob', 200, NOW());
INSERT INTO orders VALUES (3, 'Alice', 150, NOW());
FLUSH;
SELECT * FROM revenue_by_customer;
You should see Alice at 250 across 2 orders, and Bob at 200 across 1. The FLUSH command asks RisingWave to wait until the pending changes have been applied, which makes results deterministic in a demo. Insert another row for Alice, flush, and select again. The totals move without you refreshing anything.
Step 5: Add a Time Window
Now compute revenue per minute with a tumbling window:
CREATE MATERIALIZED VIEW revenue_per_minute AS
SELECT window_start,
window_end,
SUM(amount) AS revenue,
COUNT(*) AS orders
FROM TUMBLE(orders, ts, INTERVAL '1 minute')
GROUP BY window_start, window_end;
SELECT * FROM revenue_per_minute ORDER BY window_start;
TUMBLE splits the stream into fixed, non-overlapping one-minute buckets based on the ts column. Each new order lands in the right bucket and updates that bucket’s totals. Append EMIT ON WINDOW CLOSE to the view definition if you want results only once a window has finished, rather than seeing partial values.
Step 6: Stack Views on Views
CREATE MATERIALIZED VIEW big_spenders AS
SELECT customer, total
FROM revenue_by_customer
WHERE total > 200;
This view reads from another view, and both stay current. That’s the cascading pattern from earlier, and it’s how larger pipelines get built from small, understandable pieces.
Step 7: Connect a Real Stream
The real value appears when the table is fed by Kafka or a database instead of manual inserts. A Kafka-backed definition follows the same shape:
CREATE TABLE website_visits (
page VARCHAR,
user_id VARCHAR,
ts TIMESTAMP
) WITH (
connector = 'kafka',
topic = 'website_visits',
properties.bootstrap.server = 'localhost:9092'
) FORMAT PLAIN ENCODE JSON;
Your materialized views then run against live events with no other changes. To push results outward, you’d add a CREATE SINK ... FROM your_view statement with the destination’s connector settings. Exact option names vary by connector and RisingWave version, so check the official docs for the one you’re using.
Common Beginner Mistakes to Avoid
Expecting to run REFRESH. Materialized views here are always live. If results look stale, check ingestion and barriers rather than looking for a refresh command.
Mixing up tables and sources. A table with a connector stores the data, while other source types behave differently. Decide whether you need RisingWave to persist the raw events or only to read them as they pass through.
Ignoring state growth. Joins and aggregations remember data. (cite index=”140-1″>Compute nodes are where you add memory as views grow, so watch memory when you build large joins or high-cardinality aggregations, and think about retention and windowing.
Forgetting the connector node. (cite index=”136-1″>The connector node is optional, but it’s required when creating certain external sources and sinks, and attempts to create them will fail if it isn’t running. Docker enables it by default, but a manual local install may not.
Treating single-node mode as production. It’s wonderful for learning. For real workloads, run separate node groups on Kubernetes with a highly available metadata store.
Treating it as a general OLTP database. It speaks PostgreSQL, but its purpose is streaming analytics and serving results. Keep your transactional system of record elsewhere.
RisingWave vs Flink and Other Streaming Tools
Comparisons come up immediately, so here’s an even-handed overview. (cite index=”115-1″>Flink is a stream processing framework and RisingWave is a streaming database; both are Apache 2.0 licensed and both guarantee consistency and completeness. The difference is in the experience. (cite index=”116-1″>RisingWave positions itself against Flink, Spark Streaming, and ksqlDB on ease of use and cost efficiency, through a PostgreSQL-style interaction experience and a Snowflake-like decoupled storage and compute design.
Take those positioning claims as the vendor’s view. Flink remains an extremely capable, battle-tested framework with a huge ecosystem, a strong choice when you need custom code in Java or Scala and fine control over operators. RisingWave’s appeal is SQL-first simplicity and built-in serving. Against Materialize, another streaming database, (cite index=”141-1″>RisingWave offers more deployment options, single binary, Kubernetes, or managed cloud, where Materialize is managed cloud only, and the two use different consistency models. Always check current documentation, since all of these products evolve quickly.
Who Should Use RisingWave?
RisingWave tends to be a good fit when:
- You want real-time analytics, alerting, or feature computation expressed in SQL.
- Your team already knows PostgreSQL and wants to avoid learning a separate streaming framework.
- You need to offload event-driven queries from an operational database through CDC.
- You want fresh results served directly to applications or dashboards through the Postgres protocol.
- You value storage and compute separation with state in object storage.
It’s a weaker fit when you need low-level custom operators in a general programming language, or when a nightly batch job is perfectly adequate for your needs.
Frequently Asked Questions
Is RisingWave free? Yes. The core is open source under the Apache 2.0 license, and you can self-host it. A managed cloud service also exists if you prefer not to operate it yourself.
What language is RisingWave written in? Rust. It was built from scratch in Rust rather than layered on an older framework.
Is RisingWave the same as PostgreSQL? No, but it speaks the PostgreSQL wire protocol and a mostly compatible SQL dialect, so standard clients and tools connect to it. Its purpose is streaming analytics, not transactional workloads.
What is a materialized view in RisingWave? A continuously and incrementally maintained query result. It updates automatically as new data arrives, with no manual refresh.
What is Hummock? RisingWave’s cloud-native, LSM-tree-based storage engine. It stores streaming state and materialized view data in S3-compatible object storage, which keeps compute nodes stateless.
How is RisingWave different from Apache Flink? Flink is a stream processing framework, while RisingWave is a streaming database that also stores and serves results. RisingWave emphasizes PostgreSQL-style SQL and decoupled storage, while Flink offers a very broad ecosystem and programmatic APIs.
Can RisingWave read from Kafka and databases? Yes. It reads from message brokers such as Kafka and Pulsar, and it supports CDC from PostgreSQL, MySQL, MongoDB, and SQL Server, and can read Iceberg tables.
How fresh are the results? Freshness is bounded by the checkpoint cadence and processing time. By default a barrier is generated every second, so results typically lag by around a second or a little more under normal load.
Final Thoughts
RisingWave’s pitch is easy to state: use the SQL you already know to build pipelines that never go stale. A materialized view that maintains itself, a database that serves it over the PostgreSQL protocol, and state kept in object storage together remove a lot of the machinery that makes stream processing intimidating.
The best next step is to run the quickstart, then replace the manual inserts with a Kafka topic or a PostgreSQL CDC source and watch live data flow through your views. For the authoritative reference on every connector and option, RisingWave’s official documentation is the place to go deeper.
For more breakdowns of open-source data infrastructure and real-time analytics tooling like this one, keep exploring the guides on CourseDrill.
Popular Courses
