Goal: Extend Jenkins to support storing artifact usage history in external databases
File fingerprinting is a way to track which version of a file is being used by a job/build, making dependency tracking easy. The fingerprinting engine of Jenkins can track usages of artifacts, credentials, files, etc. within the system. It does this by maintaining a local XML-based database. This led to dependence on the physical disk of the Jenkins controller.
This project extended Jenkins core to support storing of fingerprints in external storages. External storage plugins can then be used to configure connection to the storage. Once configured, the fingerprints are stored and loaded by Jenkins core via these plugins in a seamless manner. The project led to the creation of two plugins, backed by Redis and PostgreSQL. The demo shows the Redis fingerprint storage plugin in action.
All the links mentioned below correspond to code which has been merged. There is no unmerged code.
Redis fingerprint storage plugin repository was created for the purpose of this project and the project README contains installation and usage instructions.
PostgreSQL fingerprint storage plugin repository was also created for the purpose of this project and README contains installation and usage instructions.
The external fingerprint storage API (JEP-226) was created in Jenkins core. Following are the relevant PRs:
Introduced External Fingerprint storage API in Jenkins core.
Released in Jenkins-2.242 🚀
Introduced fingerprint cleanup functionality.
Released in Jenkins-2.248 🚀
Introduced fingerprint migration.
Released in Jenkins-2.251 🚀
FingerprintStorageDescriptor to offer a unified interface for configuring any external fingerprint
Released in Jenkins-2.248 🚀
Exposes fingerprint methods for plugins
Released in Jenkins-2.253 🚀
This section explains the external fingerprint storage API that was created in Jenkins core. For further details, please refer to JEP-226 which explains the design decisions in detail.
We created the
FingerprintStorage class which defines the API for allowing building of custom storage plugins.
We defined the following methods in the API for plugin developers, which the plugins need to implement:
Saves the given Fingerprint in the storage.
Fingerprint load(String id)
Returns the Fingerprint with the given unique ID. The unique ID for a fingerprint is defined by
void delete(String id)
Deletes the Fingerprint with the given unique ID.
Returns true if there is some data in the fingerprint database corresponding to the particular Jenkins instance.
For more details, please refer to the Javadoc:
Fingerprint cleanup thread works by periodically iterating over the fingerprints and editing the job and build information of the ones based on whether they are still present in the system. It also deletes the fingerprints which do not have any build or job associated with them.
We extend this fingerprint cleanup functionality to be supported by external storages. Fingerprint cleanup support for external storage plugins was implemented in Jenkins-2.248. FingerprintStorage API was extended with the following methods:
Plugins can implement this method (which is called by Jenkins core periodically) to iterate and cleanup the fingerprints. The reason to design it this way, and not to iterate all the fingerprints via core, is because external storages may be able to implement more efficient traversal strategies on their own.
boolean cleanFingerprint(Fingerprint fingerprint, TaskListener taskListener)
This provides a reference implementation of cleanup, which external storages can use to cleanup a fingerprint. They may use this, or extend it to provide custom implementations.
This allows the plugins to implement their own cleanup strategies in efficient ways. For example, the Redis plugin uses cursors to traverse and cleanup the fingerprints.
Finally, we introduced the option to turn off fingerprint cleanup. This was done because it may be the case that storing extra data may be cheaper than performing cleanups, especially with external storages.
We implemented a lazy migration strategy to transfer the fingerprints from local storage to the newly configured external storage. Once an external fingerprint storage is configured, the new fingerprints are stored directly in the new storage engine. However, the old fingerprints present on the disk storage are migrated as and when they are used.
This allows the fingerprints to be migrated gradually from the local storage to the external storage and prevent huge migrations in one go. One caveat is that in case the fingerprint cleanup is turned on, the fingerprints will get transferred whenever cleanup is triggered.
Migration was introduced as part of this project in Jenkins-2.251. Both, the Redis and PostgreSQL, fingerprint storage plugins support migration.
The Redis fingerprint storage plugin uses the external fingerprint storage API to store the fingerprints as blobs inside Redis instances.
The plugin can be installed using the Jenkins Update Center.
Follow along the following steps after running Jenkins to download and install the plugin:
Redis Fingerprint Storage Plugin and check the box beside it.
Install without restart
The plugin should now be installed on your system.
Once the plugin has been installed, you can configure the Redis server details by following the steps below:
Scroll to the section
Redis Fingerprint Storage Configuration and fill in the required details:
Host - Enter hostname where Redis is running
Port - Specify the port on which Redis is running
SSL - Click if SSL is enabled
Database - Redis supports integer indexed databases, which can be specified here.
Connection Timeout - Set the connection timeout duration in milliseconds.
Socket Timeout - Set the socket timeout duration in milliseconds.
Credentials - Configure authentication using username and password to the Redis instance.
Test Redis Connection to verify that the details are correct and Jenkins is able to connect to the Redis
Now, all the fingerprints produced by this Jenkins instance should be saved in the configured Redis server!
The PostgreSQL fingerprint storage plugin defines a relational structure for storing the fingerprints, and allows fingerprint metadata to be easily queried. Installing and using the plugin is very similar to the Redis fingerprint storage plugin. The usage is not explained here for the sake of brevity. The project README and phase-3 post have more information about this plugin.
The phase wise progress can be found in the following posts:
If you are a Jenkins user, consider trying out the Redis Fingerprint Storage Plugin and the PostgreSQL Fingerprint Storage Plugin. We appreciate you trying out the plugins, and welcome any suggestions, feature requests, bug reports, etc.
The relational structure of the plugin allows some performance improvements that can be made when implementing cleanup,
as well as improving the performance of
Fingerprint#add(String job, int buildNumber).
These designs were discussed and are a scope of future improvement.
The current external fingerprint storage API supports configuring multiple Jenkins instances to a single storage. This opens up the possibility of developing traceability plugins which can track fingerprints across Jenkins instances.
Please consider reaching out to us if you feel any of the use cases would benefit you, or if you would like to share some new use cases.
Special thanks to Oleg Nenashev, Andrey Falko, Mike Cirioli, Tim Jacomb, Jesse Glick and the entire Jenkins community for all the contribution to this project.
Feel free to reach out to us for any questions, feedback, etc. on the project’s
Gitter Channel or the
Jenkins Developer Mailing list.
We use Jenkins Jira to track issues.
Feel free to file issues under