Jenkins is a CI/CD solution and as such, it is critical that the open source plugins that constitute an integral part of it don’t expose the systems they are used on to any security risks and vulnerabilities. It is in that context that we worked as an audit/code review team to track and report such flaws and problematic practices.
We worked in collaboration with Jenkins Security team member Wadeck Follonier, as part of an end-of-study project during our last year of the Master’s Degree - Reliability and IT Security in the university of Aix-Marseille.
The main goal of this project was to allow us to discover and work on diverse security vulnerabilities in the context of a widely-used software solution, and in order to achieve that we had to separate the project in multiple other goals:
Learning about some of the most common vulnerabilities and the form in which they can be found on a Jenkins Instance and its plugins.
Performing code review and technical audit on the application, and compiling our results as part of a security team.
Reporting our findings to the Jenkins team and the plugin maintainers, while sometimes helping the latter to correct these vulnerabilities.
At the beginning of the project, we set up communication channels with our mentor through Slack and Google Meets, and agreed to schedule weekly sessions with him. The goal of these sessions was both to teach us more about the functionalities of Jenkins and the types of vulnerabilities we would encounter, and to allow us to ask more specific questions regarding our findings.
Thanks to our mentor developing a mock plugin compiling a variety of classic vulnerabilities and several of their implementations, including server side request forgery (SSRF), cross site scripting (XSS), and XML external entity (XXE) attacks, we have been able to learn through practice. It allowed us to analyze the context of the code and the different ways the Jelly framework can be used to display information, expanding our payload options and giving us a clearer view of the patterns to look for during our code reviews.
We have also had the opportunity to learn about the process used to report the vulnerabilities to the maintainers through Jira issues, and some ways we could correct them or provide steps to do so.
At first, we decided to work as a pretty loose team, with each member working on a different plugin and regrouping our findings to confirm or reject them, while staying in constant contact to ask each other questions. This allowed us to broaden the scope of our searches, and is the reason why we have been able to find a larger number of vulnerabilities, in plugins that differed widely in popularity, than we would have working together from the start.
We used a single file to compile the plugins we audited and our findings, making it easier for our mentor to review them and give us feedback. Pinpointing the specific portion of code causing the issue and providing reproduction steps as clear as we could proved useful for the reporting process, thus making the approval and correction faster.
During the last third of this project, we began to work together on bigger plugins, in order to have more points of view reflecting on the same problem. With different analyses, we were able to come up with new payloads, and sometimes with new vulnerabilities where we only found one separately.
All of the reporting was done through Jira issues, which allows the Jenkins team to centralize and triage the vulnerabilities. Once we provided the necessary information, along with the reproduction steps we had, a member of the Jenkins security team contacted the plugin maintainer and guided them through the next steps of the process, with hope that they would answer.
We have also tried to make the maintainers' job easier, working on some fixes. To achieve this, we delved not only into the functionalities of the vulnerable plugins, but also into some mitigation processes that we found either in the Jenkins documentation, or with the help of our mentor.
Each one of our modifications has been tested locally, in order to assess whether the vulnerability was still present, and whether no function had been altered. However, some of the plugins we audited demanded more complex fixes, due to their intrinsic logic, or the thought process of their developer, which led to us being unable to provide a clear fix.
Considering this, the fixes we have been able to bring into light were only suggestions to the maintainers, for them to use as inspiration or template, in order not to interfere with the plugin logic.
Through this project, we have been able to work as a team, delving into some of the different issues that security engineers are bound to face, and the ways they have at their disposal to mitigate them. This has allowed us to complement our studies with a more practical aspect, that we couldn’t have had otherwise, and to transition into the companies we are now interns in. This experience has strongly encouraged us to improve in and document ourselves on this branch of cybersecurity, which will have a significant impact on our professional future.
I didn’t expect to have four students with a so deep desire to learn new things, new tricks. Their curiosity helped them to find numerous vulnerabilities that already led to 14 published CVEs. The experience was great and I wish them all the best for their professional career and their never ending quest for knowledge.
If you are student, intern, or just someone really interested in security and Jenkins in particularly, please reach out to us to see if there is a possibility to organize something together. Mailing list: firstname.lastname@example.org