feat: Allow ALTER TABLE to modify the skip_wal option dynamically by OmCheeLin · Pull Request #7550 · GreptimeTeam/greptimedb

OmCheeLin · 2026-01-09T03:48:22Z

I hereby agree to the terms of the GreptimeDB CLA.

Refer to a related PR or issue link (optional)

What's changed and what's your intention?

This PR adds support for dynamically modifying the skip_wal table option via ALTER TABLE SET/UNSET syntax, allowing users to toggle WAL on or off at runtime without recreating tables.

changes:

SQL Syntax: ALTER TABLE <table> SET 'skip_wal'='true'/'false' and ALTER TABLE <table> UNSET 'skip_wal'
Region-level: When skip_wal is altered, the change is immediately applied to all regions in memory via handle_alter_region_options_fast(), setting wal_options to Noop (disabled) or RaftEngine (enabled).
Metadata persistence: The change is persisted in both TableInfo and DatanodeTableValue entries in metasrv, ensuring consistency across restarts.
SHOW CREATE TABLE: Updated to display the skip_wal option in the WITH clause.

Limitations

In distributed mode, when enabling WAL (skip_wal=false), the current implementation uses RaftEngine as default. The correct WAL options (e.g., Kafka topic) allocated by metasrv are persisted in DatanodeTableValue and will be loaded correctly on region restart, but the in-memory version may temporarily show RaftEngine even if the actual WAL is Kafka. This is documented with a TODO for future improvement.

PR Checklist

I have written the necessary rustdoc comments.
I have added the necessary unit tests and integration tests.
This PR requires documentation updates.
API changes are backward compatible.
Schema or data changes are backward compatible.

Signed-off-by: OmCheeLin <19563671928@163.com>

Copilot

Pull request overview

This PR adds support for dynamically modifying the skip_wal table option via ALTER TABLE SET/UNSET syntax, enabling users to toggle WAL on or off at runtime without recreating tables.

Key changes:

Extended SQL syntax to support ALTER TABLE <table> SET 'skip_wal'='true'/'false' and ALTER TABLE <table> UNSET 'skip_wal'
Implemented immediate in-memory WAL option changes at the region level, setting wal_options to Noop (disabled) or RaftEngine/Kafka (enabled)
Added metadata persistence for skip_wal changes in both TableInfo and DatanodeTableValue, ensuring consistency across restarts

Reviewed changes

Copilot reviewed 15 out of 15 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
tests/cases/standalone/common/alter/alter_table_skip_wal.sql	Integration test verifying skip_wal can be set, unset, and persists across restarts
tests/cases/standalone/common/alter/alter_table_skip_wal.result	Expected output for skip_wal integration tests
src/store-api/src/region_request.rs	Added SkipWal variant to SetRegionOption and UnsetRegionOption enums with proper parsing
src/table/src/metadata.rs	Updated TableMeta to handle SetRegionOption::SkipWal in set_table_options method
src/mito2/src/worker/handle_alter.rs	Implemented in-memory WAL option switching in handle_alter_region_options_fast and new_region_options_on_empty_memtable
src/query/src/sql/show_create_table.rs	Modified SHOW CREATE TABLE to display skip_wal option in WITH clause
src/common/meta/src/key/datanode_table.rs	Enhanced build_update_table_options_txn to support updating region WAL options alongside region options
src/common/meta/src/key.rs	Updated update_table_info signature to accept optional new_region_wal_options parameter for persistence
src/common/meta/src/ddl/alter_table.rs	Added WAL options allocation logic when skip_wal changes, calling wal_options_allocator
src/common/meta/src/ddl/alter_table/executor.rs	Updated on_alter_metadata signature to pass new_region_wal_options through
src/common/meta/src/ddl/table_meta.rs	Exposed wal_options_allocator() method on TableMetadataAllocator
src/common/meta/src/ddl/alter_logical_tables/executor.rs	Updated update_table_info call with None for new_region_wal_options
src/common/meta/src/ddl/create_logical_tables/update_metadata.rs	Updated update_table_info call with None for new_region_wal_options
src/common/meta/src/reconciliation/reconcile_table/update_table_info.rs	Updated update_table_info call with None for new_region_wal_options
scripts/fetch-dashboard-assets.sh	Increased curl timeout values for improved download reliability (unrelated change)

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-01-09T09:28:59Z

tests/cases/standalone/common/alter/alter_table_skip_wal.sql

+CREATE TABLE skip_wal_test (
+    host STRING,
+    cpu_util DOUBLE,
+    ts TIMESTAMP DEFAULT CURRENT_TIMESTAMP(),
+    TIME INDEX(ts)
+);
+
+-- Test setting skip_wal to true
+ALTER TABLE skip_wal_test SET 'skip_wal'='true';
+
+SHOW CREATE TABLE skip_wal_test;
+
+-- Test setting skip_wal to false
+ALTER TABLE skip_wal_test SET 'skip_wal'='false';
+
+SHOW CREATE TABLE skip_wal_test;
+
+-- Test unsetting skip_wal (should set to false)
+ALTER TABLE skip_wal_test UNSET 'skip_wal';
+
+SHOW CREATE TABLE skip_wal_test;
+
+-- Test setting skip_wal to true again
+ALTER TABLE skip_wal_test SET 'skip_wal'='true';
+
+SHOW CREATE TABLE skip_wal_test;
+
+-- SQLNESS ARG restart=true
+-- Verify persistence after restart
+SHOW CREATE TABLE skip_wal_test;
+
+DROP TABLE skip_wal_test;
+


The test only verifies that the skip_wal option is correctly displayed in SHOW CREATE TABLE and persists across restarts. Consider adding tests that verify actual data write behavior when toggling skip_wal, such as:

Write data with skip_wal=true

Toggle to skip_wal=false

Write more data

Verify all data is retrievable after restart (data written with skip_wal=true should not be lost after toggle)

Copilot · 2026-01-09T09:28:59Z

scripts/fetch-dashboard-assets.sh

    local filename=$2

-    curl --connect-timeout 10 --retry 3 -fsSL $url --output $filename || {
+    curl --connect-timeout 30 --max-time 60 --retry 3 -fsSL $url --output $filename || {


This change increases curl timeouts from 10s connection timeout with 3 retries to 30s connection timeout, 60s max time, with 3 retries. While this may improve reliability, this change appears unrelated to the skip_wal feature. Consider moving this to a separate PR or explaining its relevance to this feature in the PR description.

Signed-off-by: OmCheeLin <19563671928@163.com>

v0y4g3r · 2026-01-12T08:03:29Z

src/mito2/src/worker/handle_alter.rs

+                        // The actual WAL options will be persisted in metasrv.
+                        // We should read the correct WAL options from DatanodeTableValue
+                        // or pass them through the AlterRegionRequest.
+                        current_options.wal_options = WalOptions::RaftEngine;


The WAL type could be Kafak

WenyXu

@OmCheeLin There are three things that need to be done:

Handle metadata changes, which you’ve almost done in this PR.
Modify the WAL provider (or set a flag to stop it from writing data into the WAL). Before doing this, make sure all data in memory has been flushed to the WAL.
Review the batch open logic for regions using Kafka options to make sure your changes work correctly with it.

You need to add new tests to cover these cases, make sure there are enough assertions so that all components behave as expected.

WenyXu · 2026-01-12T08:03:07Z

src/mito2/src/worker/handle_alter.rs

                        );
                    }
                }
+                SetRegionOption::SkipWal(skip_wal) => {


We can’t set the WAL options to RaftEngine; they could be Kafka. It would be better to let the MetaSrv send the detailed configuration.

WenyXu · 2026-01-12T08:11:00Z

src/mito2/src/worker/handle_alter.rs


                current_options.sst_format = Some(new_format);
            }
+            SetRegionOption::SkipWal(skip_wal) => {


If only the WAL options are configured, the underlying WAL provider is not affected, so the region continues writing data to the WAL.

WenyXu · 2026-01-12T08:12:44Z

src/common/meta/src/ddl/alter_table.rs


        // Safety: region distribution is set in `submit_alter_region_requests`.
+        // Check if skip_wal changed and update WAL options if needed
+        let current_skip_wal = table_info_value.table_info.meta.options.skip_wal;


I prefer not to reallocate the WAL options, since a region might start writing to a different Kafka topic after reallocation, which could cause problems.

OmCheeLin · 2026-01-13T08:01:39Z

Sorry, I accidentally closed the PR.

OmCheeLin · 2026-01-14T10:08:20Z

@WenyXu Should I wait until #7549 to be merged?

WenyXu · 2026-01-19T07:04:20Z

@WenyXu Should I wait until #7549 to be merged?

No, go ahead whenever you’re ready.

Signed-off-by: OmCheeLin <19563671928@163.com>

OmCheeLin · 2026-01-20T01:56:29Z

I have encountered some issues that need further confirmation at present. Should we modify the proto?

WenyXu · 2026-01-22T07:15:37Z

I have encountered some issues that need further confirmation at present. Should we modify the proto?

Could you explain the details of the issue?

OmCheeLin requested review from a team, MichaelScofield, WenyXu, discord9, evenyag, v0y4g3r and waynexia as code owners January 9, 2026 03:48

github-actions bot added size/S docs-not-required This change does not impact docs. labels Jan 9, 2026

OmCheeLin force-pushed the skip_wal branch from abed58a to 7256f36 Compare January 9, 2026 03:51

OmCheeLin added 3 commits January 9, 2026 14:26

init

57608d7

Signed-off-by: OmCheeLin <19563671928@163.com>

chore: skip wal

3de4ad8

Signed-off-by: OmCheeLin <19563671928@163.com>

chore: fix

353853f

Signed-off-by: OmCheeLin <19563671928@163.com>

OmCheeLin force-pushed the skip_wal branch from a2843d3 to 353853f Compare January 9, 2026 06:27

OmCheeLin added 2 commits January 9, 2026 14:53

Merge branch 'main' of github.com:OmCheeLin/greptimedb into skip_wal

93c37b3

chore: fix

d0c0ad0

Signed-off-by: OmCheeLin <19563671928@163.com>

killme2008 requested a review from Copilot January 9, 2026 09:22

Copilot started reviewing on behalf of killme2008 January 9, 2026 09:23 View session

Copilot AI reviewed Jan 9, 2026

View reviewed changes

OmCheeLin added 2 commits January 9, 2026 18:09

chore: fix lint

1432e2e

Signed-off-by: OmCheeLin <19563671928@163.com>

Merge branch 'main' of github.com:OmCheeLin/greptimedb into skip_wal

1a641b0

v0y4g3r reviewed Jan 12, 2026

View reviewed changes

WenyXu reviewed Jan 12, 2026

View reviewed changes

Merge branch 'main' of github.com:OmCheeLin/greptimedb into skip_wal

4f8ffde

OmCheeLin closed this Jan 13, 2026

OmCheeLin reopened this Jan 13, 2026

Merge branch 'main' of github.com:OmCheeLin/greptimedb into skip_wal

21fb8ce

github-actions bot added size/M and removed size/S labels Jan 20, 2026

OmCheeLin added 2 commits January 20, 2026 09:54

chore: fix

cd160e5

Signed-off-by: OmCheeLin <19563671928@163.com>

chore: fix

76b89c4

Signed-off-by: OmCheeLin <19563671928@163.com>

OmCheeLin force-pushed the skip_wal branch from 5e2aee0 to 76b89c4 Compare January 20, 2026 01:54

MichaelScofield marked this pull request as draft January 28, 2026 04:37

Conversation

OmCheeLin commented Jan 9, 2026

Refer to a related PR or issue link (optional)

What's changed and what's your intention?

PR Checklist

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Jan 9, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 9, 2026

Choose a reason for hiding this comment

Uh oh!

v0y4g3r Jan 12, 2026

Choose a reason for hiding this comment

Uh oh!

WenyXu left a comment

Choose a reason for hiding this comment

Uh oh!

WenyXu Jan 12, 2026

Choose a reason for hiding this comment

Uh oh!

WenyXu Jan 12, 2026

Choose a reason for hiding this comment

Uh oh!

WenyXu Jan 12, 2026

Choose a reason for hiding this comment

Uh oh!

OmCheeLin commented Jan 13, 2026

Uh oh!

OmCheeLin commented Jan 14, 2026

Uh oh!

WenyXu commented Jan 19, 2026

Uh oh!

OmCheeLin commented Jan 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

WenyXu commented Jan 22, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

OmCheeLin commented Jan 20, 2026 •

edited

Loading