-
Notifications
You must be signed in to change notification settings - Fork 11
Open
Description
Right now index btree destroy hold the cp guard and destroy btree in one shot.
This potential brings two considerable issues:
- The total memory that has to be pinned in one volume btree destroy when btree is large. And becomes more challenging when a bunch of volumes are being destroyed in near the same time (causing OOM?). This has to be quantified in with some testing data points.
- The CP guard won't be released until all btree nodes has been traversed, this potentially delay the cp flush times.
Solution:
- Do rate control for the destroy of the btree, e.g. Only destroy btree with preconfigured resource setting, say only do 64K number (configurable) of btree nodes in one CP. And slowly let future CPs to come and free the blocks in background.
This will be friendly to memmory usage as well as CP lock taken in mush short time.
Btree Destroy Crash Recovery:
This brings the crash recovery of the Btree to work because now btree destroy only can finish in multiple CP and if any crashes happens between CPs, the btree structure will be different and it has to garunteen there is no block leakage and no corruption after restart.
Metadata
Metadata
Assignees
Labels
No labels