Skip to content

Limit memory usage for index vdev chunk management #784

@Besroy

Description

@Besroy

The number of index chunks currently scales with index vdev size, even when chunk_num is specified. This can lead to excessive memory usage by the allocator, with each chunk consuming around 7MB. In extreme cases, this can result in over 10GB of memory usage, increasing the risk of OOM.

For Example:
In some SH test cases, due to Tess scheduling a HDD to the index PVC, an index vdev size is calculated to be over 6TB, requiring 1.64K chunks, using more than 13GB of memory, and then causing OOM.

Proposal:

  • Limit chunk count for index vdev
  • Optimize fix allocator to reduce the memory required per chunk.

Metadata

Metadata

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions