feat support to set exclusive_manual_compaction when running compaction

[jinondo]

What type of PR is this?

• bug
• feature
• enhancement

What problem(s) does this PR solve?

Issue(s) number:

Fixes #6124

Description:

Currently, NebulaGraph doesn't support configuring RocksDB's exclusive_manual_compaction parameter. In production environments, when performing manual compaction to clean up expired historical data, users encounter the following pain points:

1. Automatic compaction blocked: When manual compaction is running, automatic compaction is completely stopped
2. SST file accumulation: Long-running manual compaction causes excessive SST files at the bottom level
3. Write performance degradation: Too many SST files significantly reduce write speed, impacting business operations
4. Low resource utilization: Manual compaction may take hours, during which automatic compaction cannot work at all

Core pain point: Users need to perform manual compaction to clean up expired data while maintaining normal automatic compaction to prevent write performance degradation. Currently, these two requirements cannot be satisfied simultaneously.

How do you solve it?

This PR solves the above problems by adding support for RocksDB's exclusive_manual_compaction parameter:

1. New Configuration Parameter

Added a new gflag configuration: FLAGS_rocksdb_compact_exclusive_manual_compaction

• Type: bool
• Default value: true (maintains backward compatibility and existing behavior)
• Purpose: Controls whether manual compaction blocks automatic compaction

2. Code Changes Details

File: src/kvstore/RocksEngineConfig.h

// Added declaration
DECLARE_bool(rocksdb_compact_exclusive_manual_compaction);

File: src/kvstore/RocksEngineConfig.cpp

// Added configuration definition
DEFINE_bool(rocksdb_compact_exclusive_manual_compaction,
            true,
            "If true, no other compaction will run when manual compaction is running. "
            "If false, automatic compactions can run concurrently with manual compaction.");

File: src/kvstore/RocksEngine.cpp

Applied the parameter in the compact() method:

nebula::cpp2::ErrorCode RocksEngine::compact() {
  rocksdb::CompactRangeOptions options;
  options.change_level = FLAGS_rocksdb_compact_change_level;
  options.target_level = FLAGS_rocksdb_compact_target_level;
  options.exclusive_manual_compaction = FLAGS_rocksdb_compact_exclusive_manual_compaction;  
  rocksdb::Status status = db_->CompactRange(options, nullptr, nullptr);
  if (status.ok()) {
    LOG(INFO) << "[RocksEngine] Manual compaction completed successfully";
    return nebula::cpp2::ErrorCode::SUCCEEDED;
  } else {
    LOG(WARNING) << "[RocksEngine] CompactAll Failed: " << status.ToString();
    return nebula::cpp2::ErrorCode::E_UNKNOWN;
  }
}

3. Configuration Methods

Users can configure this parameter in Configuration File:

--rocksdb_compact_exclusive_manual_compaction=false

Special notes for your reviewer, ex. impact of this fix, design document, etc:

Design Considerations

1. Backward Compatibility: Default value is true, fully maintains existing behavior, no impact on any existing deployments
2. Safety: No breaking changes, only adds an optional configuration item
3. Performance Impact: To Be Tested (TBT), as concurrent compaction may affect manual compaction speed, I need some advices from reviewers

Impact Analysis

Positive Impact:

• Users can optimize compaction strategy based on actual workload
• Maintains write performance during long-running manual compaction
• Provides finer-grained RocksDB control capability

Risk Assessment:

• Risk Level: TBT
• Impact Scope: Only affects manual compaction behavior, and requires explicit configuration
• Performance Impact: When set to false, manual compaction may be slightly slower, but prevents write performance degradation

Technical Details

RocksDB Behavior:

• exclusive_manual_compaction = true (default): Manual compaction monopolizes all compaction resources, other compactions are blocked
• exclusive_manual_compaction = false: Manual compaction and automatic compaction can execute concurrently

Testing Recommendations

1. Functional Testing:

   • Verify default behavior (true) is consistent with previous versions
   • Verify that when set to false, manual and automatic compaction can run concurrently
   • Verify dynamic configuration updates take effect

2. Performance Testing:

   • Compare compaction duration in both modes
   • Measure write performance in concurrent mode
   • Monitor SST file count changes

3. Stress Testing:

   • High-load write + manual compaction concurrent scenario
   • Long-running stability test

Checklist:

Tests:

• Unit test (positive and negative cases)
• Function test
• Performance test
• N/A

Affects:

• Documentation affected (Please add the label if documentation needs to be modified.)
• Incompatibility (If it breaks the compatibility, please describe it and add the label.)
• If it's needed to cherry-pick (If cherry-pick to some branches is required, please label the destination version(s).) Actually I am using v3.5.0 version
• Performance impacted: Consumes more CPU/Memory

Release notes:

Please confirm whether to be reflected in release notes and how to describe:

Feature Enhancement: Added support for RocksDB's exclusive_manual_compaction parameter. Through the new rocksdb_compact_exclusive_manual_compaction configuration option, users can control whether manual compaction blocks automatic compaction.

Key Features:

• New configuration option: rocksdb_compact_exclusive_manual_compaction (default: true)
• Supports both static configuration (config file) and dynamic configuration (nGQL/HTTP API)
• When set to false, manual compaction and automatic compaction can execute concurrently, preventing write performance degradation caused by SST file accumulation during long-running manual compaction
• Fully backward compatible, default behavior remains consistent with previous versions

Applicable Scenarios:

• Maintaining write performance during large-scale historical data cleanup
• Scheduled maintenance in high-throughput write scenarios
• Production environments requiring long-running manual compaction