[ES|QL] Use correct timeFieldName for time brush filter #221322
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
🤖 Jobs for this PR can be triggered through checkboxes. 🚧
ℹ️ To trigger the CI, please tick the checkbox below 👇
|
markov00
added
bug
|
Pinging @elastic/kibana-visualizations (Team:Visualizations) |
💚 Build Succeeded
Metrics [docs]Async chunks
Page load bundle
History
|
dej611
approved these changes
May 26, 2025
Hey @jughosta Clicking on ESQL mode works only if you have a breakdown and you click on one the breakdown category. It doesn't work like the the standard Lens chart if you click on a bucket to filter by that bucket time window. |
|
Starting backport for target branches: 8.19 |
kibanamachine
pushed a commit
to kibanamachine/kibana
that referenced
this pull request
May 26, 2025
## Summary fix elastic#221241 ## Bug and fix descriptions The logic that applies a time filter by updating the time picker vs adding it as a filter pill in the filter bar is described here: https://github.com/elastic/kibana/blob/e877bafa3e4725d3e7cfd3b2ca0be4bc6d88a0b9/src/platform/plugins/shared/unified_search/public/actions/apply_filter_action/apply_filter_action.tsx#L107-L114 The `extractTimeFilter` function extracts the `timeRangeFilter` only if the `timeFieldName === keys(filter.query.range)[0]` . The problem was that the first key of `query.range` was different than the `timeFieldName`. That `timeFieldName` in the ESQL world was wrongly applied via the ` table.columns[xAxisColumnIndex].name` that doesn't reflect the actual column name for ES|QL, in fact that name reflect the visualization label name for the time dimension. In particular, before the fix, that timeFieldName was set as the axis name (e.g.`@timestamp every 5 minute` ) and cause the time range filter to fail to be extracted from the filters list. A combination of two PRs caused this to be now anymore the correct behaviour: - this PR elastic#196049 introduced the use of the `souceField` as field name for the creation of filter from a range - this PR elastic#217719 instead introduced the use of `sourceField` also for ESQL datasources. This field points to the actual column name described in the ESQL query. Both PR causes the `extractTimeFilter` to fail to extract the timeFilter, pushing the filter up to the filter pills. ### Side notes This could be probably fixed in other ways, like by avoiding using the `sourceField` in ESQL, or by using only the column.name in the filter creation, or by avoiding checking the `timeFieldName` against the `query.range` key (not really sure why this is required). In general the problem here is that there is a low confidence on what these fields/params are supposed to be and which is supposted to be the identifiers to use everywhere. For example the column ids reflects only a link between the rows and the column descriptions, the name is the associated label, but can we rely on that label for filtering? i believe we need a stronger connection with the data and the actual original source field or column identifier is a better choice. I believe a valid subsequent task is elastic#189044 (cherry picked from commit eb4ba96)
💚 All backports created successfully
Note: Successful backport PRs will be merged automatically after passing CI. Questions ?Please refer to the Backport tool documentation |
kibanamachine
added a commit
that referenced
this pull request
May 26, 2025
…) (#221524) # Backport This will backport the following commits from `main` to `8.19`: - [[ES|QL] Use correct timeFieldName for time brush filter (#221322)](#221322) <!--- Backport version: 9.6.6 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sorenlouv/backport) <!--BACKPORT [{"author":{"name":"Marco Vettorello","email":"[email protected]"},"sourceCommit":{"committedDate":"2025-05-26T14:41:06Z","message":"[ES|QL] Use correct timeFieldName for time brush filter (#221322)\n\n## Summary\n\nfix #221241\n\n## Bug and fix descriptions\n\nThe logic that applies a time filter by updating the time picker vs\nadding it as a filter pill in the filter bar is described here:\n\nhttps://github.com/elastic/kibana/blob/e877bafa3e4725d3e7cfd3b2ca0be4bc6d88a0b9/src/platform/plugins/shared/unified_search/public/actions/apply_filter_action/apply_filter_action.tsx#L107-L114\n\nThe `extractTimeFilter` function extracts the `timeRangeFilter` only if\nthe `timeFieldName === keys(filter.query.range)[0]` . The problem was\nthat the first key of `query.range` was different than the\n`timeFieldName`. That `timeFieldName` in the ESQL world was wrongly\napplied via the ` table.columns[xAxisColumnIndex].name` that doesn't\nreflect the actual column name for ES|QL, in fact that name reflect the\nvisualization label name for the time dimension. In particular, before\nthe fix, that timeFieldName was set as the axis name (e.g.`@timestamp\nevery 5 minute` ) and cause the time range filter to fail to be\nextracted from the filters list.\n\nA combination of two PRs caused this to be now anymore the correct\nbehaviour:\n- this PR #196049 introduced the\nuse of the `souceField` as field name for the creation of filter from a\nrange\n- this PR #217719 instead\nintroduced the use of `sourceField` also for ESQL datasources. This\nfield points to the actual column name described in the ESQL query.\n\nBoth PR causes the `extractTimeFilter` to fail to extract the\ntimeFilter, pushing the filter up to the filter pills.\n\n### Side notes\n\nThis could be probably fixed in other ways, like by avoiding using the\n`sourceField` in ESQL, or by using only the column.name in the filter\ncreation, or by avoiding checking the `timeFieldName` against the\n`query.range` key (not really sure why this is required). In general the\nproblem here is that there is a low confidence on what these\nfields/params are supposed to be and which is supposted to be the\nidentifiers to use everywhere. For example the column ids reflects only\na link between the rows and the column descriptions, the name is the\nassociated label, but can we rely on that label for filtering? i believe\nwe need a stronger connection with the data and the actual original\nsource field or column identifier is a better choice.\nI believe a valid subsequent task is\nhttps://github.com//issues/189044","sha":"eb4ba962ec29256f086ade49d04150c3aec1789b","branchLabelMapping":{"^v9.1.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["bug","Team:Visualizations","release_note:skip","Feature:ES|QL","backport:version","v9.1.0","v8.19.0"],"title":"[ES|QL] Use correct timeFieldName for time brush filter","number":221322,"url":"https://github.com/elastic/kibana/pull/221322","mergeCommit":{"message":"[ES|QL] Use correct timeFieldName for time brush filter (#221322)\n\n## Summary\n\nfix #221241\n\n## Bug and fix descriptions\n\nThe logic that applies a time filter by updating the time picker vs\nadding it as a filter pill in the filter bar is described here:\n\nhttps://github.com/elastic/kibana/blob/e877bafa3e4725d3e7cfd3b2ca0be4bc6d88a0b9/src/platform/plugins/shared/unified_search/public/actions/apply_filter_action/apply_filter_action.tsx#L107-L114\n\nThe `extractTimeFilter` function extracts the `timeRangeFilter` only if\nthe `timeFieldName === keys(filter.query.range)[0]` . The problem was\nthat the first key of `query.range` was different than the\n`timeFieldName`. That `timeFieldName` in the ESQL world was wrongly\napplied via the ` table.columns[xAxisColumnIndex].name` that doesn't\nreflect the actual column name for ES|QL, in fact that name reflect the\nvisualization label name for the time dimension. In particular, before\nthe fix, that timeFieldName was set as the axis name (e.g.`@timestamp\nevery 5 minute` ) and cause the time range filter to fail to be\nextracted from the filters list.\n\nA combination of two PRs caused this to be now anymore the correct\nbehaviour:\n- this PR #196049 introduced the\nuse of the `souceField` as field name for the creation of filter from a\nrange\n- this PR #217719 instead\nintroduced the use of `sourceField` also for ESQL datasources. This\nfield points to the actual column name described in the ESQL query.\n\nBoth PR causes the `extractTimeFilter` to fail to extract the\ntimeFilter, pushing the filter up to the filter pills.\n\n### Side notes\n\nThis could be probably fixed in other ways, like by avoiding using the\n`sourceField` in ESQL, or by using only the column.name in the filter\ncreation, or by avoiding checking the `timeFieldName` against the\n`query.range` key (not really sure why this is required). In general the\nproblem here is that there is a low confidence on what these\nfields/params are supposed to be and which is supposted to be the\nidentifiers to use everywhere. For example the column ids reflects only\na link between the rows and the column descriptions, the name is the\nassociated label, but can we rely on that label for filtering? i believe\nwe need a stronger connection with the data and the actual original\nsource field or column identifier is a better choice.\nI believe a valid subsequent task is\nhttps://github.com//issues/189044","sha":"eb4ba962ec29256f086ade49d04150c3aec1789b"}},"sourceBranch":"main","suggestedTargetBranches":["8.19"],"targetPullRequestStates":[{"branch":"main","label":"v9.1.0","branchLabelMappingKey":"^v9.1.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/221322","number":221322,"mergeCommit":{"message":"[ES|QL] Use correct timeFieldName for time brush filter (#221322)\n\n## Summary\n\nfix #221241\n\n## Bug and fix descriptions\n\nThe logic that applies a time filter by updating the time picker vs\nadding it as a filter pill in the filter bar is described here:\n\nhttps://github.com/elastic/kibana/blob/e877bafa3e4725d3e7cfd3b2ca0be4bc6d88a0b9/src/platform/plugins/shared/unified_search/public/actions/apply_filter_action/apply_filter_action.tsx#L107-L114\n\nThe `extractTimeFilter` function extracts the `timeRangeFilter` only if\nthe `timeFieldName === keys(filter.query.range)[0]` . The problem was\nthat the first key of `query.range` was different than the\n`timeFieldName`. That `timeFieldName` in the ESQL world was wrongly\napplied via the ` table.columns[xAxisColumnIndex].name` that doesn't\nreflect the actual column name for ES|QL, in fact that name reflect the\nvisualization label name for the time dimension. In particular, before\nthe fix, that timeFieldName was set as the axis name (e.g.`@timestamp\nevery 5 minute` ) and cause the time range filter to fail to be\nextracted from the filters list.\n\nA combination of two PRs caused this to be now anymore the correct\nbehaviour:\n- this PR #196049 introduced the\nuse of the `souceField` as field name for the creation of filter from a\nrange\n- this PR #217719 instead\nintroduced the use of `sourceField` also for ESQL datasources. This\nfield points to the actual column name described in the ESQL query.\n\nBoth PR causes the `extractTimeFilter` to fail to extract the\ntimeFilter, pushing the filter up to the filter pills.\n\n### Side notes\n\nThis could be probably fixed in other ways, like by avoiding using the\n`sourceField` in ESQL, or by using only the column.name in the filter\ncreation, or by avoiding checking the `timeFieldName` against the\n`query.range` key (not really sure why this is required). In general the\nproblem here is that there is a low confidence on what these\nfields/params are supposed to be and which is supposted to be the\nidentifiers to use everywhere. For example the column ids reflects only\na link between the rows and the column descriptions, the name is the\nassociated label, but can we rely on that label for filtering? i believe\nwe need a stronger connection with the data and the actual original\nsource field or column identifier is a better choice.\nI believe a valid subsequent task is\nhttps://github.com//issues/189044","sha":"eb4ba962ec29256f086ade49d04150c3aec1789b"}},{"branch":"8.19","label":"v8.19.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"}]}] BACKPORT--> Co-authored-by: Marco Vettorello <[email protected]>
akowalska622
pushed a commit
to akowalska622/kibana
that referenced
this pull request
May 29, 2025
## Summary fix elastic#221241 ## Bug and fix descriptions The logic that applies a time filter by updating the time picker vs adding it as a filter pill in the filter bar is described here: https://github.com/elastic/kibana/blob/e877bafa3e4725d3e7cfd3b2ca0be4bc6d88a0b9/src/platform/plugins/shared/unified_search/public/actions/apply_filter_action/apply_filter_action.tsx#L107-L114 The `extractTimeFilter` function extracts the `timeRangeFilter` only if the `timeFieldName === keys(filter.query.range)[0]` . The problem was that the first key of `query.range` was different than the `timeFieldName`. That `timeFieldName` in the ESQL world was wrongly applied via the ` table.columns[xAxisColumnIndex].name` that doesn't reflect the actual column name for ES|QL, in fact that name reflect the visualization label name for the time dimension. In particular, before the fix, that timeFieldName was set as the axis name (e.g.`@timestamp every 5 minute` ) and cause the time range filter to fail to be extracted from the filters list. A combination of two PRs caused this to be now anymore the correct behaviour: - this PR elastic#196049 introduced the use of the `souceField` as field name for the creation of filter from a range - this PR elastic#217719 instead introduced the use of `sourceField` also for ESQL datasources. This field points to the actual column name described in the ESQL query. Both PR causes the `extractTimeFilter` to fail to extract the timeFilter, pushing the filter up to the filter pills. ### Side notes This could be probably fixed in other ways, like by avoiding using the `sourceField` in ESQL, or by using only the column.name in the filter creation, or by avoiding checking the `timeFieldName` against the `query.range` key (not really sure why this is required). In general the problem here is that there is a low confidence on what these fields/params are supposed to be and which is supposted to be the identifiers to use everywhere. For example the column ids reflects only a link between the rows and the column descriptions, the name is the associated label, but can we rely on that label for filtering? i believe we need a stronger connection with the data and the actual original source field or column identifier is a better choice. I believe a valid subsequent task is elastic#189044
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
fix #221241
Bug and fix descriptions
The logic that applies a time filter by updating the time picker vs adding it as a filter pill in the filter bar is described here:
kibana/src/platform/plugins/shared/unified_search/public/actions/apply_filter_action/apply_filter_action.tsx
Lines 107 to 114 in e877baf
The
extractTimeFilterfunction extracts thetimeRangeFilteronly if thetimeFieldName === keys(filter.query.range)[0]. The problem was that the first key ofquery.rangewas different than thetimeFieldName. ThattimeFieldNamein the ESQL world was wrongly applied via thetable.columns[xAxisColumnIndex].namethat doesn't reflect the actual column name for ES|QL, in fact that name reflect the visualization label name for the time dimension. In particular, before the fix, that timeFieldName was set as the axis name (e.g.@timestamp every 5 minute) and cause the time range filter to fail to be extracted from the filters list.A combination of two PRs caused this to be now anymore the correct behaviour:
souceFieldas field name for the creation of filter from a rangesourceFieldalso for ESQL datasources. This field points to the actual column name described in the ESQL query.Both PR causes the
extractTimeFilterto fail to extract the timeFilter, pushing the filter up to the filter pills.Side notes
This could be probably fixed in other ways, like by avoiding using the
sourceFieldin ESQL, or by using only the column.name in the filter creation, or by avoiding checking thetimeFieldNameagainst thequery.rangekey (not really sure why this is required). In general the problem here is that there is a low confidence on what these fields/params are supposed to be and which is supposted to be the identifiers to use everywhere. For example the column ids reflects only a link between the rows and the column descriptions, the name is the associated label, but can we rely on that label for filtering? i believe we need a stronger connection with the data and the actual original source field or column identifier is a better choice.I believe a valid subsequent task is #189044