Daily Life as a Jenkins Administrator

Hi, my name is Mark. I am a Jenkins administrator, and I want to share some of my daily experiences using and maintaining Jenkins. I hope you enjoy it.

We have two Jenkins servers, production and quality. Both run on a commercially supported Linux distribution, normally on the latest LTS version, with around 200 plugins installed. Production runs more than 6,000 builds every day and supports more than 1,000 users and 200 agents running Windows and Linux. On this note, most of the agents are maintained by the departments themselves. We offer guidance and also (limited) support.

In addition to the Jenkins servers, we provide every Jenkins user with access to Kubernetes agents. These agents use a special build image that we maintain. The image includes and configures various tools (Java, .NET, Python, Node.js) and services (proxies and certificates). Users can therefore get started with their multibranch pipelines quickly.

We also provide template repositories for different languages, including examples of how to build and deploy an application to a Kubernetes cluster. Users can use these repositories to get started quickly.

Finally, we have a shared library with common methods for accessing our tools and setting up tool-specific steps.

Our maintenance is announced one week in advance. Between the announcement and the date, we test the changes on quality with different jobs and coordinate with a few departments on additional testing. Security advisories with an impact on our environment are addressed within 36 hours. Breaking changes in Jenkins or its plugins, such as a Java 21 upgrade, are announced further in advance because of their complexity:

We have a ticketing system for our users to report issues or request support. Most tickets actually concern access. We have guides for these requests, but it seems they are hidden well. The remaining tickets involve pipeline support, debugging connections to other tools, evaluating new plugins, and supporting user-managed agents.

Believe me or not but I have content for a few more posts, so I hope you had fun and will have fun in the future. I plan to post a new blog every two weeks, so stay tuned for more content.

If you have feedback or questions, please use the comment section below.

Greetings, Mark