Skip to content

optimize: consistent hash lb aglortihm#3916

Closed
szuecs wants to merge 9 commits intomasterfrom
optimize/consistent-hash-lb-aglortihm
Closed

optimize: consistent hash lb aglortihm#3916
szuecs wants to merge 9 commits intomasterfrom
optimize/consistent-hash-lb-aglortihm

Conversation

@szuecs
Copy link
Member

@szuecs szuecs commented Mar 11, 2026 •
edited
Loading

optimize hash ring bucket size to max 10k
optimize skipEndpoint using sync.Map for lookup instead of a list

% benchstat old.txt new3.txt
goos: linux
goarch: amd64
pkg: github.com/zalando/skipper/loadbalancer
cpu: AMD Ryzen 7 PRO 4750U with Radeon Graphics
                                 │   old.txt    │               new3.txt               │
                                 │    sec/op    │    sec/op     vs base                │
ConsistentHash/10_endpoints        219.3n ±  1%   213.1n ± 48%        ~ (p=0.469 n=10)
ConsistentHash/10_endpoints-2      224.6n ±  3%   199.0n ±  2%  -11.40% (p=0.000 n=10)
ConsistentHash/10_endpoints-4      229.1n ± 10%   196.5n ±  2%  -14.25% (p=0.000 n=10)
ConsistentHash/10_endpoints-8      222.5n ±  2%   198.0n ±  3%  -10.99% (p=0.000 n=10)
ConsistentHash/10_endpoints-16     220.8n ±  1%   198.5n ±  1%  -10.10% (p=0.000 n=10)
ConsistentHash/100_endpoints       767.9n ±  1%   454.5n ±  4%  -40.81% (p=0.000 n=10)
ConsistentHash/100_endpoints-2     778.0n ±  2%   456.2n ±  1%  -41.37% (p=0.000 n=10)
ConsistentHash/100_endpoints-4     782.8n ±  5%   455.3n ±  1%  -41.83% (p=0.000 n=10)
ConsistentHash/100_endpoints-8     777.2n ±  3%   457.1n ±  1%  -41.20% (p=0.000 n=10)
ConsistentHash/100_endpoints-16    772.5n ±  1%   455.5n ±  1%  -41.04% (p=0.000 n=10)
ConsistentHash/1000_endpoints      5.513µ ±  1%   2.991µ ±  1%  -45.74% (p=0.000 n=10)
ConsistentHash/1000_endpoints-2    5.573µ ±  4%   2.994µ ±  2%  -46.29% (p=0.000 n=10)
ConsistentHash/1000_endpoints-4    5.528µ ±  1%   3.039µ ±  2%  -45.03% (p=0.000 n=10)
ConsistentHash/1000_endpoints-8    5.512µ ±  2%   3.180µ ±  5%  -42.30% (p=0.000 n=10)
ConsistentHash/1000_endpoints-16   5.541µ ±  4%   3.046µ ±  2%  -45.03% (p=0.000 n=10)
geomean                            985.8n         653.6n        -33.70%

another finding by polarsignals continuous profiling

@szuecs
reactor: renew some test code
7d6a26a 
test: create benchmark for consistentHash algorithm
optimize: limit hashring bucket size to a fixed number 10000 entries

>20% perf boost at higher number of endpoints

```
% benchstat old.txt new.txt
goos: linux
goarch: amd64
pkg: github.com/zalando/skipper/loadbalancer
cpu: AMD Ryzen 7 PRO 4750U with Radeon Graphics
                                 │   old.txt    │               new.txt               │
                                 │    sec/op    │   sec/op     vs base                │
ConsistentHash/10_endpoints        219.3n ±  1%   211.6n ± 1%   -3.51% (p=0.001 n=10)
ConsistentHash/10_endpoints-2      224.6n ±  3%   210.4n ± 1%   -6.30% (p=0.000 n=10)
ConsistentHash/10_endpoints-4      229.1n ± 10%   207.7n ± 1%   -9.36% (p=0.000 n=10)
ConsistentHash/10_endpoints-8      222.5n ±  2%   211.8n ± 6%   -4.79% (p=0.035 n=10)
ConsistentHash/10_endpoints-16     220.8n ±  1%   215.4n ± 3%   -2.40% (p=0.010 n=10)
ConsistentHash/100_endpoints       767.9n ±  1%   783.9n ± 2%   +2.08% (p=0.004 n=10)
ConsistentHash/100_endpoints-2     778.0n ±  2%   794.5n ± 7%   +2.13% (p=0.004 n=10)
ConsistentHash/100_endpoints-4     782.8n ±  5%   772.2n ± 1%        ~ (p=0.159 n=10)
ConsistentHash/100_endpoints-8     777.2n ±  3%   784.0n ± 1%        ~ (p=0.631 n=10)
ConsistentHash/100_endpoints-16    772.5n ±  1%   775.6n ± 1%        ~ (p=0.065 n=10)
ConsistentHash/1000_endpoints      5.513µ ±  1%   4.261µ ± 1%  -22.70% (p=0.000 n=10)
ConsistentHash/1000_endpoints-2    5.573µ ±  4%   4.233µ ± 1%  -24.05% (p=0.000 n=10)
ConsistentHash/1000_endpoints-4    5.528µ ±  1%   4.242µ ± 1%  -23.26% (p=0.000 n=10)
ConsistentHash/1000_endpoints-8    5.512µ ±  2%   4.284µ ± 1%  -22.28% (p=0.000 n=10)
ConsistentHash/1000_endpoints-16   5.541µ ±  4%   4.290µ ± 3%  -22.58% (p=0.000 n=10)
geomean                            985.8n         889.8n        -9.74%
```

Signed-off-by: Sandor Szücs <sandor.szuecs@zalando.de>
@szuecs szuecs added optimization major moderate risk, for example new API, small filter changes that have no risk like refactoring or logs labels Mar 11, 2026
szuecs added 2 commits March 11, 2026 16:10
@szuecs
optimize skipEndpoint
00844dc 
% benchstat old.txt new3.txt
goos: linux
goarch: amd64
pkg: github.com/zalando/skipper/loadbalancer
cpu: AMD Ryzen 7 PRO 4750U with Radeon Graphics
                                 │   old.txt    │               new3.txt               │
                                 │    sec/op    │    sec/op     vs base                │
ConsistentHash/10_endpoints        219.3n ±  1%   213.1n ± 48%        ~ (p=0.469 n=10)
ConsistentHash/10_endpoints-2      224.6n ±  3%   199.0n ±  2%  -11.40% (p=0.000 n=10)
ConsistentHash/10_endpoints-4      229.1n ± 10%   196.5n ±  2%  -14.25% (p=0.000 n=10)
ConsistentHash/10_endpoints-8      222.5n ±  2%   198.0n ±  3%  -10.99% (p=0.000 n=10)
ConsistentHash/10_endpoints-16     220.8n ±  1%   198.5n ±  1%  -10.10% (p=0.000 n=10)
ConsistentHash/100_endpoints       767.9n ±  1%   454.5n ±  4%  -40.81% (p=0.000 n=10)
ConsistentHash/100_endpoints-2     778.0n ±  2%   456.2n ±  1%  -41.37% (p=0.000 n=10)
ConsistentHash/100_endpoints-4     782.8n ±  5%   455.3n ±  1%  -41.83% (p=0.000 n=10)
ConsistentHash/100_endpoints-8     777.2n ±  3%   457.1n ±  1%  -41.20% (p=0.000 n=10)
ConsistentHash/100_endpoints-16    772.5n ±  1%   455.5n ±  1%  -41.04% (p=0.000 n=10)
ConsistentHash/1000_endpoints      5.513µ ±  1%   2.991µ ±  1%  -45.74% (p=0.000 n=10)
ConsistentHash/1000_endpoints-2    5.573µ ±  4%   2.994µ ±  2%  -46.29% (p=0.000 n=10)
ConsistentHash/1000_endpoints-4    5.528µ ±  1%   3.039µ ±  2%  -45.03% (p=0.000 n=10)
ConsistentHash/1000_endpoints-8    5.512µ ±  2%   3.180µ ±  5%  -42.30% (p=0.000 n=10)
ConsistentHash/1000_endpoints-16   5.541µ ±  4%   3.046µ ±  2%  -45.03% (p=0.000 n=10)
geomean                            985.8n         653.6n        -33.70%

Signed-off-by: Sandor Szücs <sandor.szuecs@zalando.de>
@szuecs
test: fix panics of unintialized hostmap for consistentHash lb tests
53b7060 
Signed-off-by: Sandor Szücs <sandor.szuecs@zalando.de>
@szuecs szuecs force-pushed the optimize/consistent-hash-lb-aglortihm branch from f1f3577 to 53b7060 Compare March 11, 2026 15:54
szuecs added 6 commits March 11, 2026 20:09
@szuecs
set HostMap at route creation
28b0d66 
Signed-off-by: Sandor Szücs <sandor.szuecs@zalando.de>
@szuecs
allocate hostmap during routing.Route creation
1d84893 
Signed-off-by: Sandor Szücs <sandor.szuecs@zalando.de>
@szuecs
refactor: simplify code and remove not required data structure
ba489e0 
Signed-off-by: Sandor Szücs <sandor.szuecs@zalando.de>
@szuecs
refactor: add line again
386cf1b 
Signed-off-by: Sandor Szücs <sandor.szuecs@zalando.de>
@szuecs
doc: fix test comment
6a52635 
Signed-off-by: Sandor Szücs <sandor.szuecs@zalando.de>
@szuecs
benchmark proxy.selectEndpoint shows that the current idea is not goo…
51224e6 
…d enough

                                               │      sec/op      │     sec/op       vs base                   │
ConsistentHashSelectEndpoint/10_endpoints            203.4n ± 56%     1042.0n ± 63%    +412.29% (p=0.000 n=10)
ConsistentHashSelectEndpoint/10_endpoints-2          237.2n ±  6%     1533.0n ±  6%    +546.43% (p=0.000 n=10)
ConsistentHashSelectEndpoint/10_endpoints-4          229.4n ±  4%     1449.5n ±  6%    +531.87% (p=0.000 n=10)
ConsistentHashSelectEndpoint/10_endpoints-8          243.1n ±  3%     1587.5n ±  3%    +553.16% (p=0.000 n=10)
ConsistentHashSelectEndpoint/10_endpoints-16         242.2n ±  2%     1571.0n ±  3%    +548.50% (p=0.000 n=10)
ConsistentHashSelectEndpoint/100_endpoints           337.9n ±  1%     8513.5n ±  1%   +2419.16% (p=0.000 n=10)
ConsistentHashSelectEndpoint/100_endpoints-2         372.4n ±  3%    12557.0n ±  4%   +3272.36% (p=0.000 n=10)
ConsistentHashSelectEndpoint/100_endpoints-4         370.1n ±  1%    12046.5n ±  4%   +3154.49% (p=0.000 n=10)
ConsistentHashSelectEndpoint/100_endpoints-8         382.5n ±  1%    12900.0n ±  2%   +3272.55% (p=0.000 n=10)
ConsistentHashSelectEndpoint/100_endpoints-16        384.8n ±  2%    12694.0n ±  3%   +3198.43% (p=0.000 n=10)
ConsistentHashSelectEndpoint/1000_endpoints          964.8n ±  1%   117206.5n ±  1%  +12048.27% (p=0.000 n=10)
ConsistentHashSelectEndpoint/1000_endpoints-2        956.2n ±  1%   133848.5n ±  4%  +13897.96% (p=0.000 n=10)
ConsistentHashSelectEndpoint/1000_endpoints-4        960.0n ±  1%   151291.0n ±  4%  +15659.48% (p=0.000 n=10)
ConsistentHashSelectEndpoint/1000_endpoints-8        971.2n ±  1%   168766.5n ±  2%  +17276.22% (p=0.000 n=10)
ConsistentHashSelectEndpoint/1000_endpoints-16       977.0n ±  1%   161262.5n ±  3%  +16405.89% (p=0.000 n=10)
geomean                                              434.8n            13.38µ         +2976.68%

                                               │ selectEP_old.txt │              selectEP_new.txt              │
                                               │       B/op       │      B/op       vs base                    │
ConsistentHashSelectEndpoint/10_endpoints              96.00 ± 0%      824.00 ± 0%     +758.33% (p=0.000 n=10)
ConsistentHashSelectEndpoint/10_endpoints-2            96.00 ± 0%      824.00 ± 0%     +758.33% (p=0.000 n=10)
ConsistentHashSelectEndpoint/10_endpoints-4            96.00 ± 0%      824.00 ± 0%     +758.33% (p=0.000 n=10)
ConsistentHashSelectEndpoint/10_endpoints-8            96.00 ± 0%      824.00 ± 0%     +758.33% (p=0.000 n=10)
ConsistentHashSelectEndpoint/10_endpoints-16           96.00 ± 0%      824.00 ± 0%     +758.33% (p=0.000 n=10)
ConsistentHashSelectEndpoint/100_endpoints             96.00 ± 0%     7064.00 ± 0%    +7258.33% (p=0.000 n=10)
ConsistentHashSelectEndpoint/100_endpoints-2           96.00 ± 0%     7064.00 ± 0%    +7258.33% (p=0.000 n=10)
ConsistentHashSelectEndpoint/100_endpoints-4           96.00 ± 0%     7064.00 ± 0%    +7258.33% (p=0.000 n=10)
ConsistentHashSelectEndpoint/100_endpoints-8           96.00 ± 0%     7064.00 ± 0%    +7258.33% (p=0.000 n=10)
ConsistentHashSelectEndpoint/100_endpoints-16          96.00 ± 0%     7064.00 ± 0%    +7258.33% (p=0.000 n=10)
ConsistentHashSelectEndpoint/1000_endpoints            96.00 ± 0%   109128.00 ± 0%  +113575.00% (p=0.000 n=10)
ConsistentHashSelectEndpoint/1000_endpoints-2          96.00 ± 0%   109128.00 ± 0%  +113575.00% (p=0.000 n=10)
ConsistentHashSelectEndpoint/1000_endpoints-4          96.00 ± 0%   109128.00 ± 0%  +113575.00% (p=0.000 n=10)
ConsistentHashSelectEndpoint/1000_endpoints-8          96.00 ± 0%   109128.00 ± 0%  +113575.00% (p=0.000 n=10)
ConsistentHashSelectEndpoint/1000_endpoints-16         96.00 ± 0%   109128.00 ± 0%  +113575.00% (p=0.000 n=10)
geomean                                                96.00          8.395Ki         +8854.34%

                                               │ selectEP_old.txt │           selectEP_new.txt            │
                                               │    allocs/op     │  allocs/op   vs base                  │
ConsistentHashSelectEndpoint/10_endpoints              2.000 ± 0%    7.000 ± 0%   +250.00% (p=0.000 n=10)
ConsistentHashSelectEndpoint/10_endpoints-2            2.000 ± 0%    7.000 ± 0%   +250.00% (p=0.000 n=10)
ConsistentHashSelectEndpoint/10_endpoints-4            2.000 ± 0%    7.000 ± 0%   +250.00% (p=0.000 n=10)
ConsistentHashSelectEndpoint/10_endpoints-8            2.000 ± 0%    7.000 ± 0%   +250.00% (p=0.000 n=10)
ConsistentHashSelectEndpoint/10_endpoints-16           2.000 ± 0%    7.000 ± 0%   +250.00% (p=0.000 n=10)
ConsistentHashSelectEndpoint/100_endpoints             2.000 ± 0%   13.000 ± 0%   +550.00% (p=0.000 n=10)
ConsistentHashSelectEndpoint/100_endpoints-2           2.000 ± 0%   13.000 ± 0%   +550.00% (p=0.000 n=10)
ConsistentHashSelectEndpoint/100_endpoints-4           2.000 ± 0%   13.000 ± 0%   +550.00% (p=0.000 n=10)
ConsistentHashSelectEndpoint/100_endpoints-8           2.000 ± 0%   13.000 ± 0%   +550.00% (p=0.000 n=10)
ConsistentHashSelectEndpoint/100_endpoints-16          2.000 ± 0%   13.000 ± 0%   +550.00% (p=0.000 n=10)
ConsistentHashSelectEndpoint/1000_endpoints            2.000 ± 0%   24.000 ± 0%  +1100.00% (p=0.000 n=10)
ConsistentHashSelectEndpoint/1000_endpoints-2          2.000 ± 0%   24.000 ± 0%  +1100.00% (p=0.000 n=10)
ConsistentHashSelectEndpoint/1000_endpoints-4          2.000 ± 0%   24.000 ± 0%  +1100.00% (p=0.000 n=10)
ConsistentHashSelectEndpoint/1000_endpoints-8          2.000 ± 0%   24.000 ± 0%  +1100.00% (p=0.000 n=10)
ConsistentHashSelectEndpoint/1000_endpoints-16         2.000 ± 0%   24.000 ± 0%  +1100.00% (p=0.000 n=10)
geomean                                                2.000         12.97        +548.72%

Signed-off-by: Sandor Szücs <sandor.szuecs@zalando.de>
@szuecs
Copy link
Member Author

szuecs commented Mar 13, 2026 •
edited
Loading

This PR has the weakness that BenchmarkConsistentHashSelectEndpoint shows that it's much slower than before.
#3918 makes it obsolete combining linear search and binary search for best result.

@szuecs szuecs closed this Mar 13, 2026
@szuecs szuecs deleted the optimize/consistent-hash-lb-aglortihm branch March 13, 2026 17:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

major moderate risk, for example new API, small filter changes that have no risk like refactoring or logs optimization

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@szuecs