Skip to content

Commit 2a931cd

Browse files
Update fork-pr-testing doc
1 parent 56288bc commit 2a931cd

File tree

1 file changed

+17
-94
lines changed

1 file changed

+17
-94
lines changed

docs/fork-pr-testing.md

Lines changed: 17 additions & 94 deletions
Original file line numberDiff line numberDiff line change
@@ -1,109 +1,32 @@
11
# Fork PR Testing Guide
22

3-
This document explains how to test external pull requests using the dispatch action workflow system.
3+
This document explains how testing works for external pull requests from forks.
44

55
## Overview
66

7-
The testing system consists of three main workflows:
7+
The testing system consists of two main workflows:
88

9-
1. **E2E Tests** (`test-e2e.yml`) - Runs automatically for internal PRs, requires approval for external PRs
10-
2. **Ok To Test** (`ok-to-test.yml`) - Processes slash commands to trigger fork testing
11-
3. **E2E tests [fork]** (`test-e2e-fork.yml`) - Triggered manually via slash commands for external PRs
9+
1. **E2E Tests** (`test-e2e.yml`) - Runs automatically for internal PRs, need manual trigger on external PRs.
10+
2. **Ok To Test** (`ok-to-test.yml`) - Dispatches `repository_dispatch` event when maintainer put's `/ok-to-test sha=<commit hash>` comment in the forked PR thread.
1211

1312
## How It Works
1413

15-
### 1. Initial PR State
16-
When a PR is created or updated:
17-
- The `test-e2e.yml` workflow automatically runs
18-
- For **external PRs**: The `check-external-pr` job detects it's from a fork and **fails intentionally**
19-
- For **internal PRs**: The workflow proceeds normally with e2e tests
20-
- External PRs show ❌ for the check-external-pr job with a message asking for maintainer approval
14+
### 1. PR is created by maintainer:
2115

22-
### 2. Manual Approval Required
23-
**Important**: External PRs require manual approval before workflows can run.
16+
For the PR created by maintainer `E2E Test` workflow starts automatically. The PR check will reflect the status of the job.
2417

25-
**Steps for maintainers:**
26-
1. Go to the PR page
27-
2. Click **"Approve workflow to run"** button. For contributors who have made more than one contribution, workflows will start automatically after approval.
28-
3. The workflow will then execute
18+
### 2. PR is created by external contributor:
2919

30-
**Note**: The `test-e2e.yml` workflow will still fail for external PRs even after approval, as it's designed to prevent automatic execution. Need to use the slash command instead.
20+
For the PR created by external contributor `E2E Test` workflow **won't** start automatically.
21+
Maintainer should make a sanity check of the changes and run it manually by:
22+
1. Putting a comment `/ok-to-test sha=<latest commit hash>` in the PR thread.
23+
2. `E2E Test` workflow starts.
24+
3. After `E2E Test` workflow finishes, the commit with a link and workflow status will appear in the thread.
25+
4. Maintainer can merge PR or request the changes based on the `E2E Test` results.
3126

32-
### 3. Testing External PRs
33-
Once the initial checks have run (and failed), maintainers can test the PR using slash commands:
3427

35-
#### Step-by-Step Process:
28+
## Notes
3629

37-
1. **Navigate to the PR**
38-
- Go to the pull request you want to test
39-
40-
2. **Add a comment with slash command**
41-
```
42-
/ok-to-test sha=<commit-sha>
43-
```
44-
Replace `<commit-sha>` with the latest commit SHA from the PR.
45-
46-
**Note**: Use the short SHA.
47-
48-
3. **Dispatch Action Triggers**
49-
- The `ok-to-test.yml` workflow processes the slash command
50-
- It triggers a `repository_dispatch` event with type `ok-to-test-command`
51-
- The `test-e2e-fork.yml` workflow starts
52-
53-
4. **Workflow Execution**
54-
The fork workflow runs two jobs:
55-
56-
**a) `run-e2e-tests` job** (conditional)
57-
- Only runs if:
58-
- Event is `repository_dispatch`
59-
- SHA parameter is not empty
60-
- PR head SHA contains the provided SHA
61-
- Calls the reusable `run-e2e-tests.yml` workflow
62-
- Runs the actual E2E tests if conditions are met
63-
64-
**b) `update-check-status` job** (conditional)
65-
- Runs after `e2e-tests` completes
66-
- Updates the existing check for job named "run-e2e-tests" from 'test-e2e.yml' workflow
67-
- Sets the conclusion based on `run-e2e-tests` result:
68-
-**Success** if tests pass
69-
-**Failure** if tests fail
70-
71-
## Troubleshooting
72-
73-
### Workflow Not Running
74-
- **Check**: Ensure you've approved the workflow to run
75-
- **Action**: Click "Approve workflow to run" in the PR checks tab
76-
77-
### SHA Not Found
78-
- **Check**: Verify the SHA exists in the PR commits
79-
- **Action**: Use `git log --oneline` to find valid commit SHAs
80-
81-
### Dispatch Not Triggering
82-
- **Check**: Ensure the slash command format is correct
83-
- **Action**: Use exact format: `/ok-to-test sha=<sha>`
84-
85-
### Check Runs Not Updating
86-
- **Note**: The fork workflow updates the existing "E2E Tests [reusable]" check run
87-
- **Behavior**: The same check run is updated with new results rather than creating a new one
88-
89-
## Security Notes
90-
91-
- Only users with **write** permissions can trigger dispatch commands
92-
- The fork workflow runs in the main repository context, allowing it to update check runs
93-
- Manual approval is required for external PR workflows
94-
- External PRs are automatically detected and prevented from running tests automatically
95-
96-
## Workflow Files
97-
98-
- **`.github/workflows/ok-to-test.yml`** - Ok To Test - Slash command processor
99-
- **`.github/workflows/test-e2e.yml`** - E2E Tests - Initial PR checks
100-
- **`.github/workflows/test-e2e-fork.yml`** - E2E tests [fork] - Dispatch action handler
101-
- **`.github/workflows/e2e-tests.yml`** - E2E Tests [reusable] - Reusable workflow for actual testing
102-
103-
## Testing Checklist
104-
105-
- [ ] PR created with failing check-external-pr check (for external PRs)
106-
- [ ] Workflow approved to run (if required)
107-
- [ ] Slash command posted with valid SHA
108-
- [ ] Dispatch action triggered successfully
109-
- [ ] `check-external-pr` check run updated with correct conclusion
30+
- Only users with **write** permissions can trigger the `/ok-to-test` command.
31+
- External PRs are automatically detected and prevented from running e2e tests automatically.
32+
- Running e2e test on the external PR is optional. Maintainer can merge PR without running it. Maintainer decides whether it's needed to run an E2E test.

0 commit comments

Comments
 (0)