Skip to content

Optimize: consistent hash algorithm#3918

Merged
szuecs merged 1 commit intomasterfrom
optimize/consistent-hash-linear-scan
Mar 16, 2026
Merged

Optimize: consistent hash algorithm#3918
szuecs merged 1 commit intomasterfrom
optimize/consistent-hash-linear-scan

Conversation

@szuecs
Copy link
Member

@szuecs szuecs commented Mar 13, 2026 •
edited
Loading

optimize: limit hash ring bucket size to max 10k
optimize: consistentHash linear scan by using binary search in case it is actually faster than linear scan

In practice we will see better results as the worst case will be in favor of the new code, because the linear scan had only to walk to index 332 instead of 5000 for example. I added some log during test to check it.

% benchstat old.txt new.txt
goos: linux
goarch: amd64
pkg: github.com/zalando/skipper/proxy
cpu: AMD Ryzen 7 PRO 4750U with Radeon Graphics
                                               │   old.txt    │               new.txt               │
                                               │    sec/op    │   sec/op     vs base                │
ConsistentHashSelectEndpoint/10_endpoints        195.8n ±  3%   182.7n ± 2%   -6.72% (p=0.001 n=10)
ConsistentHashSelectEndpoint/10_endpoints-2      243.4n ±  2%   231.6n ± 3%   -4.87% (p=0.001 n=10)
ConsistentHashSelectEndpoint/10_endpoints-4      235.2n ±  2%   218.1n ± 3%   -7.27% (p=0.000 n=10)
ConsistentHashSelectEndpoint/10_endpoints-8      240.6n ±  3%   232.3n ± 2%   -3.41% (p=0.000 n=10)
ConsistentHashSelectEndpoint/10_endpoints-16     239.9n ±  3%   231.1n ± 1%   -3.65% (p=0.000 n=10)
ConsistentHashSelectEndpoint/100_endpoints       339.0n ±  5%   334.4n ± 1%   -1.36% (p=0.011 n=10)
ConsistentHashSelectEndpoint/100_endpoints-2     376.8n ±  2%   360.0n ± 3%   -4.45% (p=0.000 n=10)
ConsistentHashSelectEndpoint/100_endpoints-4     390.5n ±  4%   356.1n ± 3%   -8.80% (p=0.000 n=10)
ConsistentHashSelectEndpoint/100_endpoints-8     394.8n ±  4%   370.8n ± 1%   -6.07% (p=0.000 n=10)
ConsistentHashSelectEndpoint/100_endpoints-16    391.8n ±  3%   368.4n ± 1%   -5.97% (p=0.000 n=10)
ConsistentHashSelectEndpoint/250_endpoints       276.8n ±  3%   286.2n ± 1%   +3.41% (p=0.003 n=10)
ConsistentHashSelectEndpoint/250_endpoints-2     296.0n ±  4%   303.2n ± 4%   +2.45% (p=0.043 n=10)
ConsistentHashSelectEndpoint/250_endpoints-4     283.3n ±  3%   303.1n ± 1%   +6.99% (p=0.000 n=10)
ConsistentHashSelectEndpoint/250_endpoints-8     303.9n ±  2%   318.3n ± 1%   +4.77% (p=0.000 n=10)
ConsistentHashSelectEndpoint/250_endpoints-16    302.1n ±  1%   314.4n ± 1%   +4.09% (p=0.000 n=10)
ConsistentHashSelectEndpoint/300_endpoints       272.9n ±  1%   288.4n ± 1%   +5.68% (p=0.000 n=10)
ConsistentHashSelectEndpoint/300_endpoints-2     277.3n ±  2%   304.7n ± 2%   +9.86% (p=0.000 n=10)
ConsistentHashSelectEndpoint/300_endpoints-4     271.9n ±  3%   296.1n ± 1%   +8.88% (p=0.000 n=10)
ConsistentHashSelectEndpoint/300_endpoints-8     276.0n ±  1%   307.1n ± 2%  +11.27% (p=0.000 n=10)
ConsistentHashSelectEndpoint/300_endpoints-16    284.3n ±  3%   304.2n ± 1%   +6.98% (p=0.000 n=10)
ConsistentHashSelectEndpoint/400_endpoints       968.4n ±  2%   710.4n ± 1%  -26.64% (p=0.000 n=10)
ConsistentHashSelectEndpoint/400_endpoints-2     976.3n ±  2%   698.5n ± 4%  -28.45% (p=0.000 n=10)
ConsistentHashSelectEndpoint/400_endpoints-4     975.3n ±  1%   690.0n ± 1%  -29.26% (p=0.000 n=10)
ConsistentHashSelectEndpoint/400_endpoints-8     981.5n ±  1%   701.9n ± 1%  -28.49% (p=0.000 n=10)
ConsistentHashSelectEndpoint/400_endpoints-16    985.3n ±  1%   700.5n ± 2%  -28.90% (p=0.000 n=10)
ConsistentHashSelectEndpoint/500_endpoints       987.3n ±  0%   725.2n ± 0%  -26.55% (p=0.000 n=10)
ConsistentHashSelectEndpoint/500_endpoints-2     960.5n ±  0%   702.5n ± 3%  -26.85% (p=0.000 n=10)
ConsistentHashSelectEndpoint/500_endpoints-4     968.0n ±  1%   698.5n ± 1%  -27.85% (p=0.000 n=10)
ConsistentHashSelectEndpoint/500_endpoints-8     975.9n ±  1%   715.6n ± 1%  -26.67% (p=0.000 n=10)
ConsistentHashSelectEndpoint/500_endpoints-16    977.6n ±  1%   711.1n ± 1%  -27.26% (p=0.000 n=10)
ConsistentHashSelectEndpoint/1000_endpoints      971.0n ±  0%   794.8n ± 2%  -18.15% (p=0.000 n=10)
ConsistentHashSelectEndpoint/1000_endpoints-2    955.0n ±  1%   748.6n ± 1%  -21.61% (p=0.000 n=10)
ConsistentHashSelectEndpoint/1000_endpoints-4    959.7n ±  1%   751.6n ± 1%  -21.68% (p=0.000 n=10)
ConsistentHashSelectEndpoint/1000_endpoints-8    968.8n ±  0%   755.1n ± 1%  -22.06% (p=0.000 n=10)
ConsistentHashSelectEndpoint/1000_endpoints-16   976.9n ± 11%   762.0n ± 1%  -22.00% (p=0.000 n=10)
ConsistentHashSelectEndpoint/5000_endpoints      957.2n ±  0%   879.9n ± 0%   -8.08% (p=0.000 n=10)
ConsistentHashSelectEndpoint/5000_endpoints-2    950.8n ±  6%   828.0n ± 0%  -12.92% (p=0.000 n=10)
ConsistentHashSelectEndpoint/5000_endpoints-4    951.0n ±  7%   834.9n ± 1%  -12.21% (p=0.000 n=10)
ConsistentHashSelectEndpoint/5000_endpoints-8    963.7n ±  9%   847.7n ± 1%  -12.04% (p=0.000 n=10)
ConsistentHashSelectEndpoint/5000_endpoints-16   962.4n ±  3%   852.9n ± 1%  -11.37% (p=0.000 n=10)
geomean                                          529.6n         467.9n       -11.65%

@szuecs szuecs added optimization major moderate risk, for example new API, small filter changes that have no risk like refactoring or logs labels Mar 13, 2026
@szuecs szuecs mentioned this pull request Mar 13, 2026
Closed
@szuecs szuecs force-pushed the optimize/consistent-hash-linear-scan branch 2 times, most recently from 6d09fb8 to 229bde1 Compare March 13, 2026 17:37
@szuecs
optimize: limit hash ring bucket size to max 10k
e381d2b 
optimize: consistentHash linear scan by using binary search in case it is actually faster than linear scan

% benchstat old.txt new.txt
goos: linux
goarch: amd64
pkg: github.com/zalando/skipper/proxy
cpu: AMD Ryzen 7 PRO 4750U with Radeon Graphics
                                               │   old.txt    │               new.txt               │
                                               │    sec/op    │   sec/op     vs base                │
ConsistentHashSelectEndpoint/10_endpoints        195.8n ±  3%   182.7n ± 2%   -6.72% (p=0.001 n=10)
ConsistentHashSelectEndpoint/10_endpoints-2      243.4n ±  2%   231.6n ± 3%   -4.87% (p=0.001 n=10)
ConsistentHashSelectEndpoint/10_endpoints-4      235.2n ±  2%   218.1n ± 3%   -7.27% (p=0.000 n=10)
ConsistentHashSelectEndpoint/10_endpoints-8      240.6n ±  3%   232.3n ± 2%   -3.41% (p=0.000 n=10)
ConsistentHashSelectEndpoint/10_endpoints-16     239.9n ±  3%   231.1n ± 1%   -3.65% (p=0.000 n=10)
ConsistentHashSelectEndpoint/100_endpoints       339.0n ±  5%   334.4n ± 1%   -1.36% (p=0.011 n=10)
ConsistentHashSelectEndpoint/100_endpoints-2     376.8n ±  2%   360.0n ± 3%   -4.45% (p=0.000 n=10)
ConsistentHashSelectEndpoint/100_endpoints-4     390.5n ±  4%   356.1n ± 3%   -8.80% (p=0.000 n=10)
ConsistentHashSelectEndpoint/100_endpoints-8     394.8n ±  4%   370.8n ± 1%   -6.07% (p=0.000 n=10)
ConsistentHashSelectEndpoint/100_endpoints-16    391.8n ±  3%   368.4n ± 1%   -5.97% (p=0.000 n=10)
ConsistentHashSelectEndpoint/250_endpoints       276.8n ±  3%   286.2n ± 1%   +3.41% (p=0.003 n=10)
ConsistentHashSelectEndpoint/250_endpoints-2     296.0n ±  4%   303.2n ± 4%   +2.45% (p=0.043 n=10)
ConsistentHashSelectEndpoint/250_endpoints-4     283.3n ±  3%   303.1n ± 1%   +6.99% (p=0.000 n=10)
ConsistentHashSelectEndpoint/250_endpoints-8     303.9n ±  2%   318.3n ± 1%   +4.77% (p=0.000 n=10)
ConsistentHashSelectEndpoint/250_endpoints-16    302.1n ±  1%   314.4n ± 1%   +4.09% (p=0.000 n=10)
ConsistentHashSelectEndpoint/300_endpoints       272.9n ±  1%   288.4n ± 1%   +5.68% (p=0.000 n=10)
ConsistentHashSelectEndpoint/300_endpoints-2     277.3n ±  2%   304.7n ± 2%   +9.86% (p=0.000 n=10)
ConsistentHashSelectEndpoint/300_endpoints-4     271.9n ±  3%   296.1n ± 1%   +8.88% (p=0.000 n=10)
ConsistentHashSelectEndpoint/300_endpoints-8     276.0n ±  1%   307.1n ± 2%  +11.27% (p=0.000 n=10)
ConsistentHashSelectEndpoint/300_endpoints-16    284.3n ±  3%   304.2n ± 1%   +6.98% (p=0.000 n=10)
ConsistentHashSelectEndpoint/400_endpoints       968.4n ±  2%   710.4n ± 1%  -26.64% (p=0.000 n=10)
ConsistentHashSelectEndpoint/400_endpoints-2     976.3n ±  2%   698.5n ± 4%  -28.45% (p=0.000 n=10)
ConsistentHashSelectEndpoint/400_endpoints-4     975.3n ±  1%   690.0n ± 1%  -29.26% (p=0.000 n=10)
ConsistentHashSelectEndpoint/400_endpoints-8     981.5n ±  1%   701.9n ± 1%  -28.49% (p=0.000 n=10)
ConsistentHashSelectEndpoint/400_endpoints-16    985.3n ±  1%   700.5n ± 2%  -28.90% (p=0.000 n=10)
ConsistentHashSelectEndpoint/500_endpoints       987.3n ±  0%   725.2n ± 0%  -26.55% (p=0.000 n=10)
ConsistentHashSelectEndpoint/500_endpoints-2     960.5n ±  0%   702.5n ± 3%  -26.85% (p=0.000 n=10)
ConsistentHashSelectEndpoint/500_endpoints-4     968.0n ±  1%   698.5n ± 1%  -27.85% (p=0.000 n=10)
ConsistentHashSelectEndpoint/500_endpoints-8     975.9n ±  1%   715.6n ± 1%  -26.67% (p=0.000 n=10)
ConsistentHashSelectEndpoint/500_endpoints-16    977.6n ±  1%   711.1n ± 1%  -27.26% (p=0.000 n=10)
ConsistentHashSelectEndpoint/1000_endpoints      971.0n ±  0%   794.8n ± 2%  -18.15% (p=0.000 n=10)
ConsistentHashSelectEndpoint/1000_endpoints-2    955.0n ±  1%   748.6n ± 1%  -21.61% (p=0.000 n=10)
ConsistentHashSelectEndpoint/1000_endpoints-4    959.7n ±  1%   751.6n ± 1%  -21.68% (p=0.000 n=10)
ConsistentHashSelectEndpoint/1000_endpoints-8    968.8n ±  0%   755.1n ± 1%  -22.06% (p=0.000 n=10)
ConsistentHashSelectEndpoint/1000_endpoints-16   976.9n ± 11%   762.0n ± 1%  -22.00% (p=0.000 n=10)
ConsistentHashSelectEndpoint/5000_endpoints      957.2n ±  0%   879.9n ± 0%   -8.08% (p=0.000 n=10)
ConsistentHashSelectEndpoint/5000_endpoints-2    950.8n ±  6%   828.0n ± 0%  -12.92% (p=0.000 n=10)
ConsistentHashSelectEndpoint/5000_endpoints-4    951.0n ±  7%   834.9n ± 1%  -12.21% (p=0.000 n=10)
ConsistentHashSelectEndpoint/5000_endpoints-8    963.7n ±  9%   847.7n ± 1%  -12.04% (p=0.000 n=10)
ConsistentHashSelectEndpoint/5000_endpoints-16   962.4n ±  3%   852.9n ± 1%  -11.37% (p=0.000 n=10)
geomean                                          529.6n         467.9n       -11.65%

Signed-off-by: Sandor Szücs <sandor.szuecs@zalando.de>
@szuecs szuecs force-pushed the optimize/consistent-hash-linear-scan branch from 229bde1 to e381d2b Compare March 13, 2026 17:52
@szuecs
Copy link
Member Author

szuecs commented Mar 13, 2026

👍

@ponimas
Copy link
Member

ponimas commented Mar 16, 2026

optimize: limit hash ring bucket size to max 10k

But why? How does that help?

@szuecs
Copy link
Member Author

szuecs commented Mar 16, 2026

optimize: limit hash ring bucket size to max 10k

But why? How does that help?

7d6a26a#diff-92db2045c41fcb6839064f0caac8129c14bf4aeedcc389a6ad161bafb7d8560a commit message shows that it is faster to limit the hash ring to a reasonable size:

% benchstat old.txt new.txt
goos: linux
goarch: amd64
pkg: github.com/zalando/skipper/loadbalancer
cpu: AMD Ryzen 7 PRO 4750U with Radeon Graphics
                                 │   old.txt    │               new.txt               │
                                 │    sec/op    │   sec/op     vs base                │
ConsistentHash/10_endpoints        219.3n ±  1%   211.6n ± 1%   -3.51% (p=0.001 n=10)
ConsistentHash/10_endpoints-2      224.6n ±  3%   210.4n ± 1%   -6.30% (p=0.000 n=10)
ConsistentHash/10_endpoints-4      229.1n ± 10%   207.7n ± 1%   -9.36% (p=0.000 n=10)
ConsistentHash/10_endpoints-8      222.5n ±  2%   211.8n ± 6%   -4.79% (p=0.035 n=10)
ConsistentHash/10_endpoints-16     220.8n ±  1%   215.4n ± 3%   -2.40% (p=0.010 n=10)
ConsistentHash/100_endpoints       767.9n ±  1%   783.9n ± 2%   +2.08% (p=0.004 n=10)
ConsistentHash/100_endpoints-2     778.0n ±  2%   794.5n ± 7%   +2.13% (p=0.004 n=10)
ConsistentHash/100_endpoints-4     782.8n ±  5%   772.2n ± 1%        ~ (p=0.159 n=10)
ConsistentHash/100_endpoints-8     777.2n ±  3%   784.0n ± 1%        ~ (p=0.631 n=10)
ConsistentHash/100_endpoints-16    772.5n ±  1%   775.6n ± 1%        ~ (p=0.065 n=10)
ConsistentHash/1000_endpoints      5.513µ ±  1%   4.261µ ± 1%  -22.70% (p=0.000 n=10)
ConsistentHash/1000_endpoints-2    5.573µ ±  4%   4.233µ ± 1%  -24.05% (p=0.000 n=10)
ConsistentHash/1000_endpoints-4    5.528µ ±  1%   4.242µ ± 1%  -23.26% (p=0.000 n=10)
ConsistentHash/1000_endpoints-8    5.512µ ±  2%   4.284µ ± 1%  -22.28% (p=0.000 n=10)
ConsistentHash/1000_endpoints-16   5.541µ ±  4%   4.290µ ± 3%  -22.58% (p=0.000 n=10)
geomean                            985.8n         889.8n        -9.74%

@ponimas
Copy link
Member

ponimas commented Mar 16, 2026

👍

@szuecs szuecs merged commit 85e3215 into master Mar 16, 2026
14 of 15 checks passed
@szuecs szuecs deleted the optimize/consistent-hash-linear-scan branch March 16, 2026 09:56
MustafaSaber
MustafaSaber reviewed Mar 16, 2026
View reviewed changes
MustafaSaber
MustafaSaber reviewed Mar 16, 2026
View reviewed changes
MustafaSaber
MustafaSaber reviewed Mar 16, 2026
View reviewed changes
loadbalancer/algorithm.go

func skipEndpoint(c *routing.LBContext, index int) bool {
host := c.Route.LBEndpoints[index].Host
if len(c.LBEndpoints) > 300 { // 300 see https://github.com/zalando/skipper/pull/3918/
Copy link
Member

@MustafaSaber MustafaSaber Mar 16, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's better to make this a constant with descriptive name

Copy link
Member Author

@szuecs szuecs Mar 16, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes true

This was referenced Mar 16, 2026
Open
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@MustafaSaber MustafaSaber MustafaSaber left review comments

Assignees

No one assigned

Labels

major moderate risk, for example new API, small filter changes that have no risk like refactoring or logs optimization

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@szuecs @ponimas @MustafaSaber