chore: download war from mirrors unless we are publishing a release

[lemeurherve]

This PR helps reducing Artifactory bandwidth use for container builds by downloading the Jenkins war file from the nearest mirror site rather than downloading from repo.jenkins-ci.org when building without publishing.

Refs:

• Reduce artifactory bandwidth used by ci.jenkins.io jenkins-infra/helpdesk#4747
• Download war from mirrors unless we are publishing a release #2059

Testing done

Build Dockerfile directly without `make`, using get.jenkins.io

This build uses default JENKINS_VERSION & JENKINS_SHA from Dockerfile.

docker buildx bake --file docker-bake.hcl debian_jdk17 --progress=plain

Output snippet showing download from get.jenkins.io:

#56 [linux/ppc64le controller  6/12] RUN curl -fsSL https://get.jenkins.io/war/2.504/jenkins.war -o /usr/share/jenkins/jenkins.war   && echo "efc91d6be8d79dd078e7f930fc4a5f135602d0822a5efe9091808fdd74607d32  /usr/share/jenkins/jenkins.war" >/tmp/jenkins_sha   && sha256sum -c --strict /tmp/jenkins_sha   && rm -f /tmp/jenkins_sha

Build all images, no additional argument, using get.jenkins.io

This build uses default JENKINS_VERSION & JENKINS_SHA from Dockerfile.

make build

Output snippet showing download from get.jenkins.io:

#60 [rhel_ubi9_jdk17 controller  7/13] RUN curl -fsSL https://get.jenkins.io/war/2.504/jenkins.war -o /usr/share/jenkins/jenkins.war   && echo "efc91d6be8d79dd078e7f930fc4a5f135602d0822a5efe9091808fdd74607d32  /usr/share/jenkins/jenkins.war" >/tmp/jenkins_sha   && sha256sum -c --strict /tmp/jenkins_sha   && rm -f /tmp/jenkins_sha

Build all images specifying a Weekly `JENKINS_VERSION`, using get.jenkins.io with `war`

JENKINS_VERSION=2.528 JENKINS_SHA=0077a6377878eceb659fe0e4c5104fd4adbc36fa307a495eb2264fe4105ed88c make build

Output snippet showing download from get.jenkins.io:

#21 [rhel_ubi9_jdk21 controller  7/13] RUN curl -fsSL https://get.jenkins.io/war/2.528/jenkins.war -o /usr/share/jenkins/jenkins.war   && echo "0077a6377878eceb659fe0e4c5104fd4adbc36fa307a495eb2264fe4105ed88c  /usr/share/jenkins/jenkins.war" >/tmp/jenkins_sha   && sha256sum -c --strict /tmp/jenkins_sha   && rm -f /tmp/jenkins_sha

Build all images specifying a LTS `JENKINS_VERSION`, using get.jenkins.io with `war-stable`

JENKINS_VERSION=2.516.3 JENKINS_SHA=81b3abcc0f24cea48e74effe152f69dc5f0d880edc0c2737c61446b3c5992c00 make build

Output snippet showing download from get.jenkins.io:

#56 [debian_slim_jdk21 controller  6/12] RUN curl -fsSL https://get.jenkins.io/war-stable/2.516.3/jenkins.war -o /usr/share/jenkins/jenkins.war   && echo "81b3abcc0f24cea48e74effe152f69dc5f0d880edc0c2737c61446b3c5992c00  /usr/share/jenkins/jenkins.war" >/tmp/jenkins_sha   && sha256sum -c --strict /tmp/jenkins_sha   && rm -f /tmp/jenkins_sha

Build all images, no additional argument, using repo.jenkins-ci.org

WAR_URL=https://repo.jenkins-ci.org/public/org/jenkins-ci/main/jenkins-war/2.504/jenkins-war-2.504.war make build

Output snippet showing download from get.jenkins.io:

#56 [debian_slim_jdk21 controller  6/12] RUN curl -fsSL https://repo.jenkins-ci.org/public/org/jenkins-ci/main/jenkins-war/2.504/jenkins-war-2.504.war -o /usr/share/jenkins/jenkins.war   && echo "efc91d6be8d79dd078e7f930fc4a5f135602d0822a5efe9091808fdd74607d32  /usr/share/jenkins/jenkins.war" >/tmp/jenkins_sha   && sha256sum -c --strict /tmp/jenkins_sha   && rm -f /tmp/jenkins_sha

Build all images specifying a Weekly`JENKINS_VERSION`, using repo.jenkins-ci.org

JENKINS_VERSION=2.528 JENKINS_SHA=0077a6377878eceb659fe0e4c5104fd4adbc36fa307a495eb2264fe4105ed88c WAR_URL=https://repo.jenkins-ci.org/public/org/jenkins-ci/main/jenkins-war/2.528/jenkins-war-2.528.war make build

Output snippet showing download from get.jenkins.io:

#56 [debian_jdk17 controller  6/12] RUN curl -fsSL https://repo.jenkins-ci.org/public/org/jenkins-ci/main/jenkins-war/2.528/jenkins-war-2.528.war -o /usr/share/jenkins/jenkins.war   && echo "0077a6377878eceb659fe0e4c5104fd4adbc36fa307a495eb2264fe4105ed88c  /usr/share/jenkins/jenkins.war" >/tmp/jenkins_sha   && sha256sum -c --strict /tmp/jenkins_sha   && rm -f /tmp/jenkins_sha

Build all images specifying a LTS`JENKINS_VERSION`, repo.jenkins-ci.org

JENKINS_VERSION=2.516.3 JENKINS_SHA=81b3abcc0f24cea48e74effe152f69dc5f0d880edc0c2737c61446b3c5992c00 WAR_URL=https://repo.jenkins-ci.org/public/org/jenkins-ci/main/jenkins-war/2.516.3/jenkins-war-2.516.3.war make build

Output snippet showing download from get.jenkins.io:

#53 [debian_jdk17 controller  6/12] RUN curl -fsSL https://repo.jenkins-ci.org/public/org/jenkins-ci/main/jenkins-war/2.516.3/jenkins-war-2.516.3.war -o /usr/share/jenkins/jenkins.war   && echo "81b3abcc0f24cea48e74effe152f69dc5f0d880edc0c2737c61446b3c5992c00  /usr/share/jenkins/jenkins.war" >/tmp/jenkins_sha   && sha256sum -c --strict /tmp/jenkins_sha   && rm -f /tmp/jenkins_sha

Build Windows image specifying a LTS `JENKINS_VERSION`, using get.jenkins.io with `war-stable`

https://ci.jenkins.io/job/Packaging/job/docker/job/PR-2074/14/execution/node/388/log/?consoleFull

--- a/Jenkinsfile
+++ b/Jenkinsfile
@@ -17,7 +17,7 @@ def envVars = ['PUBLISH=true']
 // Set to true in a replay to simulate a LTS build on ci.jenkins.io
 // It will set the environment variables needed for a LTS
 // and disable images publication out of caution
-def SIMULATE_LTS_BUILD = false
+def SIMULATE_LTS_BUILD = true

Output snippet from https://ci.jenkins.io/job/Packaging/job/docker/job/PR-2074/14/execution/node/388/log/?consoleFull:

10:58:05  = PREPARE: List of jenkins4eval/jenkins images and tags to be processed:
10:58:11  name: packaging_docker_pr-2074
10:58:11  services:
10:58:11    jdk17:
10:58:11      build:
10:58:11        context: Z:\agent\workspace\Packaging_docker_PR-2074
10:58:11        dockerfile: ./windows/windowsservercore/hotspot/Dockerfile
10:58:11        args:
10:58:11          COMMIT_SHA: b735c051204815e7aa93a6d24d2efda12257625f
10:58:11          JAVA_HOME: C:/openjdk-17
10:58:11          JAVA_VERSION: 17.0.16_8
10:58:11          JENKINS_SHA: 3e53b52a816405e3b10ad07f1c48cd0cb5cb3f893207ef7f9de28415806b93c1
10:58:11          JENKINS_VERSION: 2.462.3
10:58:11          TOOLS_WINDOWS_VERSION: "1809"
10:58:11          WAR_URL: https://get.jenkins.io/war-stable/2.462.3/jenkins.war
10:58:11          WINDOWS_VERSION: ltsc2019

Submitter checklist

• Make sure you are opening from a topic/feature/bugfix branch (right side) and not your main branch!
• Ensure that the pull request title represents the desired changelog entry
• Please describe what you did
• Link to relevant issues in GitHub or Jira
• Link to relevant pull requests, esp. upstream and downstream changes
• Ensure you have provided tests that demonstrate the feature works or the issue is fixed

[lemeurherve]

I forgot Jenkins is setting its own JENKINS_URL environment variable value 🤦

https://ci.jenkins.io/job/Packaging/job/docker/job/PR-2074/2/pipeline-overview/?start-byte=0&selected-node=194#log-194-186

[controller 5/11] RUN curl -fsSL https://ci.jenkins.io/ -o /usr/share/jenkins/jenkins.war && echo "efc91d6be8d79dd078e7f930fc4a5f135602d0822a5efe9091808fdd74607d32 /usr/share/jenkins/jenkins.war" >/tmp/jenkins_sha && sha256sum -c --strict /tmp/jenkins_sha && rm -f /tmp/jenkins_sha:

In draft while I rework that point and check the Windows build.

[dduportal]

I forgot Jenkins is setting its own JENKINS_URL environment variable value 🤦

Good catch! That's a tricky one 😅

[timja]

I forgot Jenkins is setting its own JENKINS_URL environment variable value 🤦

Good catch! That's a tricky one 😅

Just pick a different var ^.^.

I was surprised JENKINS_URL was used as that is a standard Jenkin env var.

[lemeurherve]

Just pick a different var ^.^.

aa634ff 😉

I was surprised JENKINS_URL was used as that is a standard Jenkin env var.

Same ^^

[lemeurherve]

With the help of temporary debug statements in fd32886, I found out why my builds were failing on Windows on the SHA256 checksum:

• Checksum in the Windows Dockerfile:

  docker/windows/windowsservercore/hotspot/Dockerfile

  Lines 107 to 108 in 80da2e1

  	RUN Invoke-WebRequest -Uri "$env:WAR_URL" -OutFile C:/ProgramData/Jenkins/jenkins.war ; `
  	if ((Get-FileHash C:/ProgramData/Jenkins/jenkins.war -Algorithm SHA256).Hash -ne $env:WAR_SHA) {exit 1}

• (Get-FileHash C:/ProgramData/Jenkins/jenkins.war -Algorithm SHA256).Hash returns only the hash (efc91d6be8d79dd078e7f930fc4a5f135602d0822a5efe9091808fdd74607d32)
• The content of https://get.jenkins.io/war/2.504/jenkins.war.sha256 is efc91d6be8d79dd078e7f930fc4a5f135602d0822a5efe9091808fdd74607d32 jenkins.war
• Reported initially in uppercase:
  • https://ci.jenkins.io/job/Packaging/job/docker/job/PR-2074/10/pipeline-overview/?start-byte=0&selected-node=172#log-172-1
  • https://ci.jenkins.io/job/Packaging/job/docker/job/PR-2074/10/pipeline-overview/?start-byte=0&selected-node=172#log-172-15
  • EFC91D6BE8D79DD078E7F930FC4A5F135602D0822A5EFE9091808FDD74607D32 JENKINS.WAR
• Result after the fix:
  • https://ci.jenkins.io/job/Packaging/job/docker/job/PR-2074/11/pipeline-overview/?start-byte=0&selected-node=183#log-183-1
  • https://ci.jenkins.io/job/Packaging/job/docker/job/PR-2074/11/pipeline-overview/?start-byte=0&selected-node=183#log-183-15
  • efc91d6be8d79dd078e7f930fc4a5f135602d0822a5efe9091808fdd74607d32

[MarkEWaite]

Change passed my tests and it passes CI tests. Code review looks good as well. Thanks!

Let's use it for the weekly build of 2.529.

[lemeurherve]

Confirmed working as intended with the 2.529 weekly release by checking logs on trusted.ci.jenkins.io in pair with @dduportal