Skip to content

Index Btree Destroy to support staging destruction #74

@yamingk

Description

@yamingk

Right now index btree destroy hold the cp guard and destroy btree in one shot.

This potential brings two considerable issues:

  1. The total memory that has to be pinned in one volume btree destroy when btree is large. And becomes more challenging when a bunch of volumes are being destroyed in near the same time (causing OOM?). This has to be quantified in with some testing data points.
  2. The CP guard won't be released until all btree nodes has been traversed, this potentially delay the cp flush times.

Solution:

  1. Do rate control for the destroy of the btree, e.g. Only destroy btree with preconfigured resource setting, say only do 64K number (configurable) of btree nodes in one CP. And slowly let future CPs to come and free the blocks in background.

This will be friendly to memmory usage as well as CP lock taken in mush short time.

Btree Destroy Crash Recovery:
This brings the crash recovery of the Btree to work because now btree destroy only can finish in multiple CP and if any crashes happens between CPs, the btree structure will be different and it has to garunteen there is no block leakage and no corruption after restart.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions