Add changelog check after a new release is created by isabelrios · Pull Request #321 · mozilla-mobile/testops-tools

isabelrios · 2026-04-22T11:07:39Z

The end goal is to automatically create targeted TestRail test runs based on what code changed between two release versions, added to the same milestone created by the existing iOS release automation.

In order to do this iteratively, let's just add the new jenkins file that will check the change log between current and previous release, nothing else for now. It will send a notification like:

Head tag from latest_tags.json: firefox-v150.1
 Base tag (previous version):    firefox-v149.3

 Comparing firefox-v149.3 → firefox-v150.1
 Total changed files (raw): 42
 After filtering ignored paths: 31

 Changed files (31):
   firefox-ios/Client/Frontend/Browser/Tabs/TabTray.swift
   firefox-ios/Client/Frontend/Settings/SettingsTableViewController.swift
   ... and 11 more

 Unmatched files: 3

isabelrios · 2026-04-22T11:08:40Z

+    environment {
+        PATH          = "/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/homebrew/bin:${env.PATH}"
+        GITHUB_TOKEN  = credentials('github-token')
+        SLACK_WEBHOOK = credentials('slack-mobile-alerts-tooling-webhook')


I may add the sandbox webhook instead to not make noise in channels

you can use slack-mobile-alerts-sandbox-webhook credentials

isabelrios · 2026-04-22T11:09:31Z

+            }
+            steps {
+                script {
+                    try {


Adding this so that if the new job fails, the current job will not be affected. The failure will be shown as a warning.

isabelrios · 2026-04-22T11:12:37Z

@@ -0,0 +1,36 @@
+name: Changelog Analysis


Adding this github action to be able to manually debug and verify that we get the correct changelog

mdotb-moz · 2026-04-24T11:26:53Z

+      - uses: actions/checkout@v4
+
+      - name: Install dependencies
+        run: pip install requests pyyaml


stage('Install Dependencies') {
steps {
dir('testrail') {
sh '''
set -e
python3 -m venv .venv
.venv/bin/pip install --no-cache-dir -r requirements.txt
'''
}
}
} Can we use venv?

here in the github action or you mean in the jenkinsfile @mdotb-moz ?

sorry in the jenkinsfile

AaronMT · 2026-04-28T14:46:34Z

+on:
+  push:
+    branches:
+      - irios-new-jenkins-file-verify-release-tags


Is this scoped just for testing on this branch currently?

Yes, this is for testing purposes, we would not need the github action since the Jenkins file with the script will be triggered as part of the Create milestone job. Once there is a new version detected, we run the changelog to know what has been affected by the new release

AaronMT · 2026-04-28T14:50:01Z

+
+
+def get_changed_files(owner: str, repo: str, base: str, head: str) -> List[str]:
+    url = f"https://api.github.com/repos/{owner}/{repo}/compare/{base}...{head}"


Should note here that this returns a maximum of 250 commits that will have to paginated.

When calling this endpoint without any paging parameter (per_page or page), the returned list is limited to 250 commits, and the last commit in the list is the most recent of the entire comparison.

For large releases, files beyond that limit are silently dropped here

oh that is a good catch thank you. I will add that

AaronMT · 2026-04-28T14:51:44Z

            testrail_project_id, milestone_name, milestone_description
        )

+        with open("milestone_id.txt", "w") as f:


This is written to root but in the Jenkins file it's read from testrail, I would write to a similar testrail dir

AaronMT · 2026-04-28T14:53:21Z

+                        passwordVariable: 'TESTRAIL_PASSWORD')]) {
+                        sh '''
+                            python3 run_release_selection.py \
+                                --head_tag   ${RELEASE_TAG} \


I only see a base_tag as currently accepted in the script

I am missing some changes here.. --head_tag would be given by RELEASE_TAG which is shared from a previous script.

create-milestone triggers create-smart-test-runs and passes RELEASE_TAG as a parameter, so the Jenkinsfile always calls the script with --head_tag

isabelrios commented Apr 22, 2026

View reviewed changes

isabelrios requested a review from mdotb-moz April 22, 2026 11:11

isabelrios commented Apr 22, 2026

View reviewed changes

isabelrios marked this pull request as ready for review April 23, 2026 08:11

mdotb-moz reviewed Apr 24, 2026

View reviewed changes

AaronMT reviewed Apr 28, 2026

View reviewed changes

isabelrios added 2 commits April 28, 2026 18:23

Add changelog check after a new release is created

b470ae2

Reviewers comments

3911547

isabelrios force-pushed the irios-new-jenkins-file-verify-release-tags branch from 0cfd35c to 3911547 Compare April 28, 2026 16:23

isabelrios added 5 commits April 28, 2026 18:28

fix looking for previous version

c116a18

fix looking for previous version

f25eda3

show number of commits

f880aaf

latest changes to github action

c1b468f

latest changes to github action

2961fa5



		def get_changed_files(owner: str, repo: str, base: str, head: str) -> List[str]:
		url = f"https://api.github.com/repos/{owner}/{repo}/compare/{base}...{head}"

Conversation

isabelrios commented Apr 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

isabelrios commented Apr 22, 2026 •

edited

Loading