Upgrading to Jenkins LTS 2.73.x

Each section covers the upgrade from the previous LTS release, the section on 2.73.1 covers the upgrade from 2.60.3.

Upgrading to Jenkins 2.73.3

Changes to on-disk storage layout for user records

The fix for SECURITY-499 renames the directories containing user records in JENKINS_HOME/users/: Directories with unsafe names, like reserved names such as NUL (Windows) or using unsafe characters such as \ will be renamed to filesystem-safe representations of those user names, with problematic characters or sequences escaped.

Downgrading Jenkins to versions older than 2.73.3 would result in a mismatch between the name widely used to reference a user in Jenkins (such as assigned permissions), and what user name is associated with the user’s metadata (password record for Jenkins user database, SSH public key, and various preferences) — as the latter might change to the filesystem-safe representation of the intended user name — and is therefore discouraged.

Upgrading to Jenkins 2.73.2

Low-privilege users may not be able to reconfigure some agents anymore

The launch method Launch agent via execution of command on master did not properly check the permission to run commands of the configuring user. This has been fixed, and this launch method can now only be configured by Jenkins administrators (or, more specifically, users with the Run Scripts permission).

A known limitation of this fix is that users without the Run Scripts permission (typically everyone who isn’t an administrator) are no longer able to configure agents with this launch method at all, even if the launch method and the script remain unchanged.

Restrictions to remote API output

Multiple remote API endpoints had the content they provide restricted. More detailed information can be found in the security advisory linked above.

  • All user properties got removed from /user/(username)/api/ and other APIs returning users except when accessed by administrators or the users themselves.

  • APIs returning queue items and tasks (builds) now only return them when requested by users with read permission for them.

  • The /queue/item/(ID)/api/ is now only accessible by users with read permission for the specified item.

Fixes for regressions in 2.73.1

The regressions in 2.73.1 listed below have been fixed in 2.73.2.

Upgrading to Jenkins 2.73.1

Known Issue: Agent connection failures involving Jenkins controllers with undefined or non-writable home directory

A regression involving agent connections has been identified. If the user home directory (HOME, not JENKINS_HOME) on the Jenkins controller is undefined, or not writable, no remoting connections can be established.

The following features are known to impacted:

  1. Agent connections

  2. Remoting-based CLI connections (deprecated since 2.46.2)

  3. Building Maven projects (Maven Integration Plugin)

The error shown will include the path to a directory .jenkins/cache/jars inside the home directory, and its stack trace will reference hudson.remoting.FileSystemJarCache.

The Jenkins Docker image has been changed to have a defined, writable home directory for the Jenkins user. Typical web application container (e.g. Tomcat) setups may not have a writable home directory for the user the container is running as, and may need to be adapted to work around this issue.

Possible workarounds:

  1. Make the specified home directory writable to the Jenkins user.

  2. Define a home directory that is writable for the user Jenkins is running as.

  3. Override it for the process running Jenkins, for example using the -Duser.home=/path/to/directory parameter to the java command, pointing to a writable directory.

Known Issue: Randomly occurring connection failures with passphrase-protected ed25519 SSH keys

A regression involving authentication using passphrase-protected ed25519 SSH keys has been identified in 2.73. It randomly causes failures in features using these keys, such as:

  1. SSH Slaves Plugin connections

  2. Accessing Git repositories with the JGit implementation

  3. Accessing Subversion repositories (unconfirmed)

Workarounds for this issue:

  1. Do not use ed25519 for SSH keys in Jenkins

  2. Do not protect ed25519 SSH keys in Jenkins with a passphrase

  3. In the case of Git Plugin, switch to the Git CLI implementation

This Groovy Script, when executed in the "Script Console" (located at /script) can be used to determine if the instance contains any passphrase-proteced ed25519 SSH keys before upgrading.

Groovy 2.4.11 Upgrade

Groovy was updated from 2.4.8 to 2.4.11. This may resolve existing Groovy-related memory problems, as 2.4.9 contains the fix for GROOVY-8067. It may no longer be necessary to apply the groovy.use.classvalue workaround.

Winstone 4.1 Upgrade

The embedded Jetty container has been updated from version 9.2.15 to version 9.4.5. For a detailed list of changes, see the Jetty changelog.

In this update, support for the optional --spdy parameter has been dropped. Jenkins will now refuse to start if the ---spdy parameter is specified. It needs to be removed from any Jenkins init scripts.

Additionally, support for all TLS versions before 1.2 has been dropped. If you’re using the embedded Jetty container’s HTTPS functionality, it will no longer accept TLS 1.0 or 1.1 connections.

Build Authorization

This change affects only users of Authorize Project Plugin.

Previous releases of Jenkins implemented a special permission fallback when Authorize Project Plugin did not specify a global default configuration: For the purposes of Build other projects and Build after other projects are built, if Authorize Project configuration did not specify a build authorization for the job in question, it fell back to acting with the permissions of the anonymous user. This ensured a secure default, but only for these trigger-related permission checks.

This behavior has been changed, and Jenkins will now perform the permission check as SYSTEM (i.e. with full permissions) to determine whether a project should be built.

To restore the previous behavior, configure a global Project default Build Authorization setting the default authorization to that of the anonymous user. This feature has been implemented in Authorize Project Plugin version 1.2.0.

Remoting Work Directories

The embedded Jenkins Remoting version has been updated from 3.7 to 3.10. It introduces support of work directories, which may be used by Remoting to store caches, logs and other metadata.

Once work directory mode is enabled, Jenkins agents start writing logs to the disk and change the default destination of the filesystem JAR Cache. In Remoting this opt-in feature can be enabled using the -workDir=${ROOT_DIR} command-line option, but the Jenkins defines custom behavior for some agent launchers:

  • Java Web Start Launcher (aka JNLP agent)

    • Old agents: Work directory needs to be enabled manually

    • New agents created from Web UI: Work directory is enabled by default, work directory points to Remote root directory of the agent.

    • New agents created from CLI/API: Behavior depends on the passed configuration file, work directory is disabled by default

  • Command Launcher

    • No changes, work directory should be manually enabled in launch settings if required

  • Other Launcher types (e.g. SSH Launcher)

    • The behavior is defined in plugins, which have independent release cycle

    • Follow updates in tickets linked to JENKINS-44108

You can find more information, examples and upgrade guides in Jenkins Remoting documentation.