Skip to content

feat(metrics-explorer): fast path for get treemap in case of no filter#11459

Open
nikhilmantri0902 wants to merge 2 commits into
metricsExplorer/get_stats_fast_pathfrom
metricsExplorer/get_treemap_fast_path
Open

feat(metrics-explorer): fast path for get treemap in case of no filter#11459
nikhilmantri0902 wants to merge 2 commits into
metricsExplorer/get_stats_fast_pathfrom
metricsExplorer/get_treemap_fast_path

Conversation

@nikhilmantri0902
Copy link
Copy Markdown
Contributor

Pull Request


📄 Summary

GetTreemap (samples mode) narrows the samples-table scan via fingerprint IN (huge set) regardless of filter presence. That set is millions of UInt64s.

When no user filter is present, the __metric_candidates IN-set is the only narrowing needed . The __filtered_fingerprints CTE and fingerprint IN (…) predicate become pure overhead.

This PR adds computeSamplesTreemapFastPath and dispatches to it from GetTreemap when req.Filter is empty. TimeSeries mode unchanged.

Follow-up to the GetStats fastpath PR (same pattern, different endpoint).

Issues closed by this PR

Part of https://github.com/SigNoz/engineering-pod/issues/5043


✅ Change Type

  • ♻️ Refactor (perf optimization, behavior-preserving)

🐛 Bug Context

Fix Strategy

Dispatch upstream from req.Filter in GetTreemap:

  • hasFilter := req.Filter != nil && strings.TrimSpace(req.Filter.Expression) != ""
  • Samples mode + !hasFilter → new computeSamplesTreemapFastPath (no filter param, no __filtered_fingerprints)
  • Samples mode + hasFilter → existing computeSamplesTreemap
  • TimeSeries mode → unchanged

Fastpath body matches slowpath exactly except: (1) no filterWhereClause.AddWhereClause on __metric_candidates, (2) no __filtered_fingerprints CTE, (3) no fingerprint IN (…) predicate. Both differences are gated on filter presence in the slowpath.


🧪 Testing Strategy

  • Tests added/updated: none (no existing unit coverage on this function; deferred).
  • Manual verification:
    • No-filter samples treemap → SQL contains metric_name GLOBAL IN (__metric_candidates) only; no __filtered_fingerprints.
    • With-filter samples treemap → existing slowpath SQL unchanged.
    • TimeSeries treemap → SQL unchanged either way.
    • Compared per-metric samples and percentage for metric_name = 'http.server.duration.bucket' between fastpath and slowpath responses — match exactly.
  • Edge cases covered:
    • req.Filter == nil → fastpath.
    • Empty/whitespace expression → fastpath.
    • Mode toggle (samples ↔ timeseries) preserved.
    • count(*) vs sum(count) table-aware via CountExpressionForSamplesTable.

⚠️ Risk & Impact Assessment

  • Blast radius: GetTreemap samples-mode no-filter calls only. TimeSeries mode and with-filter samples path untouched.
  • Potential regressions: Fastpath SQL is the slowpath SQL minus the two filter-conditional blocks — same __metric_candidates, same __total_samples, same __sample_counts (modulo fingerprint predicate). Semantic equivalence verified
    manually.
  • Rollback plan: revert this PR. Slowpath alone still handles all cases correctly (just slower for no-filter).

📝 Changelog

Field Value
Deployment Type Cloud / OSS / Enterprise
Change Type Maintenance (perf)
Description Faster metrics-treemap (samples mode) when no filter is applied

📋 Checklist

  • Tests added or explicitly not required (behavior-preserving; no existing coverage)
  • Manually tested
  • Breaking changes documented (none)
  • Backward compatibility considered (response shape unchanged)

👀 Notes for Reviewers

  • Dispatch logic mirrors the GetStats fastpath PR — same hasFilter check, same shape.
  • buildFilterClause is now called once per GetTreemap invocation (was already the case). When !hasFilter and samples mode, the returned empty WhereClause is discarded — buildFilterClause short-circuits on empty expression
    (module.go:932-934) so this is essentially free.
  • computeSamplesTreemap left intact (not renamed) — minimizes diff. Its if filterWhereClause != nil guards are now always true under the new dispatch, but harmless.

@nikhilmantri0902 nikhilmantri0902 changed the base branch from main to metricsExplorer/get_stats_fast_path May 26, 2026 13:27
@github-actions github-actions Bot added the enhancement New feature or request label May 26, 2026
@chatgpt-codex-connector
Copy link
Copy Markdown

💡 Codex Review

samplesSB.Where(samplesSB.Between("unix_milli", req.Start, req.End))
samplesSB.Where("NOT startsWith(metric_name, 'signoz')")
samplesSB.Where(fmt.Sprintf("metric_name IN (%s)", samplesSB.Var(metricNamesSB)))
samplesSB.GroupBy("metric_name")

P1 Badge Keep fingerprint filter in stats samples fast path

The new fetchMetricsStatsWithSamplesFastPath counts samples using only metric_name IN (...) from time_series, but it no longer restricts samples by the fingerprint set that is filtered with __normalized = false. In the previous path, samples were constrained via fingerprint IN (SELECT fingerprint FROM __filtered_fingerprints), so only the intended series contributed. With the new query, any extra fingerprints sharing the same metric name (for example normalized variants) are included, which inflates samples, changes ordering, and can change pagination totals for no-filter GetStats requests.


sampleCountsSB.Where(sampleCountsSB.Between("unix_milli", req.Start, req.End))
sampleCountsSB.Where("metric_name GLOBAL IN (SELECT metric_name FROM __metric_candidates)")
sampleCountsSB.GroupBy("metric_name")

P1 Badge Retain fingerprint scoping in treemap samples fast path

The no-filter treemap fast path drops the fingerprint predicate entirely and aggregates by metric name only. The previous implementation always scoped sample counts to __filtered_fingerprints (which carries the __normalized=false constraint from the TS table), so percentages were computed from the intended fingerprint population. Without that scoping, sample totals and percentages can be overstated whenever a metric name is shared by fingerprints outside that TS-filtered set.

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant