[Scout Reporter] Support scout_<custom> directories for test reporting
#248048
+16
−7
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
csr
added
release_note:skip
csr
commented
Jan 7, 2026
|
|
||
| private getScoutConfigCategory(configPath: string): ScoutTestRunConfigCategory { | ||
| const pattern = /scout\/(api|ui)\//; | ||
| // match scout or scout_* directory |
Contributor
Author
There was a problem hiding this comment.
Problem: previously, test config file paths such as x-pack/platform/plugins/shared/security/test/scout_uiam_local/api/parallel.playwright.config.ts were being reported with an unknown test config category.
Fix: we now correctly handle this custom Scout path so we report the test config category (api-test in this case).
csr
commented
Jan 7, 2026
| const pathSegments = configPath.split(path.sep); | ||
| const testDirIndex = pathSegments.indexOf('scout'); | ||
| const testDirIndex = pathSegments.findIndex( | ||
| (segment) => segment === 'scout' || segment.startsWith('scout_') |
Contributor
Author
There was a problem hiding this comment.
Problem: previously, the Scout Reporter wasn't able to infer the plugin manifest file path (this is important to derive info on the Kibana module type (e.g., plugin), group (e.g. platform), etc.
Fix: we now add support and the Kibana config file path is being reported correctly. We also no longer see errors like this one when we run tests (using npx playwright test --config ...):
Error in reporter Error: Invalid path: "scout" directory not found in /Users/cesaredecal/dev/kibana/x-pack/platform/plugins/shared/security/test/scout_uiam_local/api/parallel.playwright.config.ts.
Contributor
Author
|
To check before merging: the AppEx QA cluster has the correct test config category filled out for |
csr
changed the title
scout_<custom> directories for test reporting
dmlemeshko
approved these changes
Jan 7, 2026
Contributor
dmlemeshko
left a comment
dmlemeshko
left a comment
There was a problem hiding this comment.
LGTM, thanks a lot for the quick fix!
Contributor
|
Starting backport for target branches: 8.19, 9.1, 9.2, 9.3 |
kibanamachine
pushed a commit
to kibanamachine/kibana
that referenced
this pull request
Jan 7, 2026
…ing (elastic#248048) This PR updates the `kbn-scout-reporting` package to support Scout test file paths located in custom `scout_<custom>` directories (e.g., `scout_uiam_local`, `scout_cspm_agentless`). Example of a path the package now supports: ``` x-pack/platform/plugins/shared/security/test/scout_uiam_local/api/parallel.playwright.config.ts ``` With these fixes: * ✅ The test category is correctly parsed from the path and reported correctly * ✅ The plugin metadata is extracted from `kibana.jsonc` for failure reports (cherry picked from commit 28b7b03)
kibanamachine
pushed a commit
to kibanamachine/kibana
that referenced
this pull request
Jan 7, 2026
…ing (elastic#248048) This PR updates the `kbn-scout-reporting` package to support Scout test file paths located in custom `scout_<custom>` directories (e.g., `scout_uiam_local`, `scout_cspm_agentless`). Example of a path the package now supports: ``` x-pack/platform/plugins/shared/security/test/scout_uiam_local/api/parallel.playwright.config.ts ``` With these fixes: * ✅ The test category is correctly parsed from the path and reported correctly * ✅ The plugin metadata is extracted from `kibana.jsonc` for failure reports (cherry picked from commit 28b7b03)
kibanamachine
pushed a commit
to kibanamachine/kibana
that referenced
this pull request
Jan 7, 2026
…ing (elastic#248048) This PR updates the `kbn-scout-reporting` package to support Scout test file paths located in custom `scout_<custom>` directories (e.g., `scout_uiam_local`, `scout_cspm_agentless`). Example of a path the package now supports: ``` x-pack/platform/plugins/shared/security/test/scout_uiam_local/api/parallel.playwright.config.ts ``` With these fixes: * ✅ The test category is correctly parsed from the path and reported correctly * ✅ The plugin metadata is extracted from `kibana.jsonc` for failure reports (cherry picked from commit 28b7b03)
This was referenced Jan 7, 2026
Merged
kibanamachine
pushed a commit
to kibanamachine/kibana
that referenced
this pull request
Jan 7, 2026
…ing (elastic#248048) This PR updates the `kbn-scout-reporting` package to support Scout test file paths located in custom `scout_<custom>` directories (e.g., `scout_uiam_local`, `scout_cspm_agentless`). Example of a path the package now supports: ``` x-pack/platform/plugins/shared/security/test/scout_uiam_local/api/parallel.playwright.config.ts ``` With these fixes: * ✅ The test category is correctly parsed from the path and reported correctly * ✅ The plugin metadata is extracted from `kibana.jsonc` for failure reports (cherry picked from commit 28b7b03)
Contributor
💚 All backports created successfully
Note: Successful backport PRs will be merged automatically after passing CI. Questions ?Please refer to the Backport tool documentation |
kibanamachine
added a commit
that referenced
this pull request
Jan 7, 2026
…reporting (#248048) (#248172) # Backport This will backport the following commits from `main` to `9.2`: - [[Scout Reporter] Support `scout_<custom>` directories for test reporting (#248048)](#248048) <!--- Backport version: 9.6.6 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sorenlouv/backport) <!--BACKPORT [{"author":{"name":"Cesare de Cal","email":"[email protected]"},"sourceCommit":{"committedDate":"2026-01-07T18:57:39Z","message":"[Scout Reporter] Support `scout_<custom>` directories for test reporting (#248048)\n\nThis PR updates the `kbn-scout-reporting` package to support Scout test\nfile paths located in custom `scout_<custom>` directories (e.g.,\n`scout_uiam_local`, `scout_cspm_agentless`).\n\nExample of a path the package now supports:\n\n``` \nx-pack/platform/plugins/shared/security/test/scout_uiam_local/api/parallel.playwright.config.ts\n```\n\nWith these fixes:\n* ✅ The test category is correctly parsed from the path and reported\ncorrectly\n* ✅ The plugin metadata is extracted from `kibana.jsonc` for failure\nreports","sha":"28b7b03dbc1ce89f687081f090d9965817d45fd7","branchLabelMapping":{"^v9.4.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","backport:all-open","v9.4.0"],"title":"[Scout Reporter] Support `scout_<custom>` directories for test reporting","number":248048,"url":"https://github.com/elastic/kibana/pull/248048","mergeCommit":{"message":"[Scout Reporter] Support `scout_<custom>` directories for test reporting (#248048)\n\nThis PR updates the `kbn-scout-reporting` package to support Scout test\nfile paths located in custom `scout_<custom>` directories (e.g.,\n`scout_uiam_local`, `scout_cspm_agentless`).\n\nExample of a path the package now supports:\n\n``` \nx-pack/platform/plugins/shared/security/test/scout_uiam_local/api/parallel.playwright.config.ts\n```\n\nWith these fixes:\n* ✅ The test category is correctly parsed from the path and reported\ncorrectly\n* ✅ The plugin metadata is extracted from `kibana.jsonc` for failure\nreports","sha":"28b7b03dbc1ce89f687081f090d9965817d45fd7"}},"sourceBranch":"main","suggestedTargetBranches":[],"targetPullRequestStates":[{"branch":"main","label":"v9.4.0","branchLabelMappingKey":"^v9.4.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/248048","number":248048,"mergeCommit":{"message":"[Scout Reporter] Support `scout_<custom>` directories for test reporting (#248048)\n\nThis PR updates the `kbn-scout-reporting` package to support Scout test\nfile paths located in custom `scout_<custom>` directories (e.g.,\n`scout_uiam_local`, `scout_cspm_agentless`).\n\nExample of a path the package now supports:\n\n``` \nx-pack/platform/plugins/shared/security/test/scout_uiam_local/api/parallel.playwright.config.ts\n```\n\nWith these fixes:\n* ✅ The test category is correctly parsed from the path and reported\ncorrectly\n* ✅ The plugin metadata is extracted from `kibana.jsonc` for failure\nreports","sha":"28b7b03dbc1ce89f687081f090d9965817d45fd7"}}]}] BACKPORT--> Co-authored-by: Cesare de Cal <[email protected]>
kibanamachine
added a commit
that referenced
this pull request
Jan 7, 2026
… reporting (#248048) (#248170) # Backport This will backport the following commits from `main` to `8.19`: - [[Scout Reporter] Support `scout_<custom>` directories for test reporting (#248048)](#248048) <!--- Backport version: 9.6.6 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sorenlouv/backport) <!--BACKPORT [{"author":{"name":"Cesare de Cal","email":"[email protected]"},"sourceCommit":{"committedDate":"2026-01-07T18:57:39Z","message":"[Scout Reporter] Support `scout_<custom>` directories for test reporting (#248048)\n\nThis PR updates the `kbn-scout-reporting` package to support Scout test\nfile paths located in custom `scout_<custom>` directories (e.g.,\n`scout_uiam_local`, `scout_cspm_agentless`).\n\nExample of a path the package now supports:\n\n``` \nx-pack/platform/plugins/shared/security/test/scout_uiam_local/api/parallel.playwright.config.ts\n```\n\nWith these fixes:\n* ✅ The test category is correctly parsed from the path and reported\ncorrectly\n* ✅ The plugin metadata is extracted from `kibana.jsonc` for failure\nreports","sha":"28b7b03dbc1ce89f687081f090d9965817d45fd7","branchLabelMapping":{"^v9.4.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","backport:all-open","v9.4.0"],"title":"[Scout Reporter] Support `scout_<custom>` directories for test reporting","number":248048,"url":"https://github.com/elastic/kibana/pull/248048","mergeCommit":{"message":"[Scout Reporter] Support `scout_<custom>` directories for test reporting (#248048)\n\nThis PR updates the `kbn-scout-reporting` package to support Scout test\nfile paths located in custom `scout_<custom>` directories (e.g.,\n`scout_uiam_local`, `scout_cspm_agentless`).\n\nExample of a path the package now supports:\n\n``` \nx-pack/platform/plugins/shared/security/test/scout_uiam_local/api/parallel.playwright.config.ts\n```\n\nWith these fixes:\n* ✅ The test category is correctly parsed from the path and reported\ncorrectly\n* ✅ The plugin metadata is extracted from `kibana.jsonc` for failure\nreports","sha":"28b7b03dbc1ce89f687081f090d9965817d45fd7"}},"sourceBranch":"main","suggestedTargetBranches":[],"targetPullRequestStates":[{"branch":"main","label":"v9.4.0","branchLabelMappingKey":"^v9.4.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/248048","number":248048,"mergeCommit":{"message":"[Scout Reporter] Support `scout_<custom>` directories for test reporting (#248048)\n\nThis PR updates the `kbn-scout-reporting` package to support Scout test\nfile paths located in custom `scout_<custom>` directories (e.g.,\n`scout_uiam_local`, `scout_cspm_agentless`).\n\nExample of a path the package now supports:\n\n``` \nx-pack/platform/plugins/shared/security/test/scout_uiam_local/api/parallel.playwright.config.ts\n```\n\nWith these fixes:\n* ✅ The test category is correctly parsed from the path and reported\ncorrectly\n* ✅ The plugin metadata is extracted from `kibana.jsonc` for failure\nreports","sha":"28b7b03dbc1ce89f687081f090d9965817d45fd7"}}]}] BACKPORT--> Co-authored-by: Cesare de Cal <[email protected]>
kibanamachine
added a commit
that referenced
this pull request
Jan 7, 2026
…reporting (#248048) (#248171) # Backport This will backport the following commits from `main` to `9.1`: - [[Scout Reporter] Support `scout_<custom>` directories for test reporting (#248048)](#248048) <!--- Backport version: 9.6.6 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sorenlouv/backport) <!--BACKPORT [{"author":{"name":"Cesare de Cal","email":"[email protected]"},"sourceCommit":{"committedDate":"2026-01-07T18:57:39Z","message":"[Scout Reporter] Support `scout_<custom>` directories for test reporting (#248048)\n\nThis PR updates the `kbn-scout-reporting` package to support Scout test\nfile paths located in custom `scout_<custom>` directories (e.g.,\n`scout_uiam_local`, `scout_cspm_agentless`).\n\nExample of a path the package now supports:\n\n``` \nx-pack/platform/plugins/shared/security/test/scout_uiam_local/api/parallel.playwright.config.ts\n```\n\nWith these fixes:\n* ✅ The test category is correctly parsed from the path and reported\ncorrectly\n* ✅ The plugin metadata is extracted from `kibana.jsonc` for failure\nreports","sha":"28b7b03dbc1ce89f687081f090d9965817d45fd7","branchLabelMapping":{"^v9.4.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","backport:all-open","v9.4.0"],"title":"[Scout Reporter] Support `scout_<custom>` directories for test reporting","number":248048,"url":"https://github.com/elastic/kibana/pull/248048","mergeCommit":{"message":"[Scout Reporter] Support `scout_<custom>` directories for test reporting (#248048)\n\nThis PR updates the `kbn-scout-reporting` package to support Scout test\nfile paths located in custom `scout_<custom>` directories (e.g.,\n`scout_uiam_local`, `scout_cspm_agentless`).\n\nExample of a path the package now supports:\n\n``` \nx-pack/platform/plugins/shared/security/test/scout_uiam_local/api/parallel.playwright.config.ts\n```\n\nWith these fixes:\n* ✅ The test category is correctly parsed from the path and reported\ncorrectly\n* ✅ The plugin metadata is extracted from `kibana.jsonc` for failure\nreports","sha":"28b7b03dbc1ce89f687081f090d9965817d45fd7"}},"sourceBranch":"main","suggestedTargetBranches":[],"targetPullRequestStates":[{"branch":"main","label":"v9.4.0","branchLabelMappingKey":"^v9.4.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/248048","number":248048,"mergeCommit":{"message":"[Scout Reporter] Support `scout_<custom>` directories for test reporting (#248048)\n\nThis PR updates the `kbn-scout-reporting` package to support Scout test\nfile paths located in custom `scout_<custom>` directories (e.g.,\n`scout_uiam_local`, `scout_cspm_agentless`).\n\nExample of a path the package now supports:\n\n``` \nx-pack/platform/plugins/shared/security/test/scout_uiam_local/api/parallel.playwright.config.ts\n```\n\nWith these fixes:\n* ✅ The test category is correctly parsed from the path and reported\ncorrectly\n* ✅ The plugin metadata is extracted from `kibana.jsonc` for failure\nreports","sha":"28b7b03dbc1ce89f687081f090d9965817d45fd7"}}]}] BACKPORT--> Co-authored-by: Cesare de Cal <[email protected]>
kibanamachine
added a commit
that referenced
this pull request
Jan 8, 2026
…reporting (#248048) (#248173) # Backport This will backport the following commits from `main` to `9.3`: - [[Scout Reporter] Support `scout_<custom>` directories for test reporting (#248048)](#248048) <!--- Backport version: 9.6.6 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sorenlouv/backport) <!--BACKPORT [{"author":{"name":"Cesare de Cal","email":"[email protected]"},"sourceCommit":{"committedDate":"2026-01-07T18:57:39Z","message":"[Scout Reporter] Support `scout_<custom>` directories for test reporting (#248048)\n\nThis PR updates the `kbn-scout-reporting` package to support Scout test\nfile paths located in custom `scout_<custom>` directories (e.g.,\n`scout_uiam_local`, `scout_cspm_agentless`).\n\nExample of a path the package now supports:\n\n``` \nx-pack/platform/plugins/shared/security/test/scout_uiam_local/api/parallel.playwright.config.ts\n```\n\nWith these fixes:\n* ✅ The test category is correctly parsed from the path and reported\ncorrectly\n* ✅ The plugin metadata is extracted from `kibana.jsonc` for failure\nreports","sha":"28b7b03dbc1ce89f687081f090d9965817d45fd7","branchLabelMapping":{"^v9.4.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","backport:all-open","v9.4.0"],"title":"[Scout Reporter] Support `scout_<custom>` directories for test reporting","number":248048,"url":"https://github.com/elastic/kibana/pull/248048","mergeCommit":{"message":"[Scout Reporter] Support `scout_<custom>` directories for test reporting (#248048)\n\nThis PR updates the `kbn-scout-reporting` package to support Scout test\nfile paths located in custom `scout_<custom>` directories (e.g.,\n`scout_uiam_local`, `scout_cspm_agentless`).\n\nExample of a path the package now supports:\n\n``` \nx-pack/platform/plugins/shared/security/test/scout_uiam_local/api/parallel.playwright.config.ts\n```\n\nWith these fixes:\n* ✅ The test category is correctly parsed from the path and reported\ncorrectly\n* ✅ The plugin metadata is extracted from `kibana.jsonc` for failure\nreports","sha":"28b7b03dbc1ce89f687081f090d9965817d45fd7"}},"sourceBranch":"main","suggestedTargetBranches":[],"targetPullRequestStates":[{"branch":"main","label":"v9.4.0","branchLabelMappingKey":"^v9.4.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/248048","number":248048,"mergeCommit":{"message":"[Scout Reporter] Support `scout_<custom>` directories for test reporting (#248048)\n\nThis PR updates the `kbn-scout-reporting` package to support Scout test\nfile paths located in custom `scout_<custom>` directories (e.g.,\n`scout_uiam_local`, `scout_cspm_agentless`).\n\nExample of a path the package now supports:\n\n``` \nx-pack/platform/plugins/shared/security/test/scout_uiam_local/api/parallel.playwright.config.ts\n```\n\nWith these fixes:\n* ✅ The test category is correctly parsed from the path and reported\ncorrectly\n* ✅ The plugin metadata is extracted from `kibana.jsonc` for failure\nreports","sha":"28b7b03dbc1ce89f687081f090d9965817d45fd7"}}]}] BACKPORT--> Co-authored-by: Cesare de Cal <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR updates the
kbn-scout-reportingpackage to support Scout test file paths located in customscout_<custom>directories (e.g.,scout_uiam_local,scout_cspm_agentless).Example of a path the package now supports:
With these fixes:
kibana.jsoncfor failure reports