Domain-specific LLM based on actual Jenkins usage using ci.jenkins.io data

Goal: To develop a web app using an existing open-source LLM model with Jenkins usage data collected for domain-specific Jenkins knowledge to be fine-tuned

Status: Selected

Team

Details

Abstract

This full-stack project focuses on a proof-of-concept (PoC) idea to fine-tune an existing open-source LLM model (to be determined) with domain-specific Jenkins data to be compiled, wrangled, and processed by the contributor as a part of an AI-driven application, to develop a minimalistic UI for the user to interact with the LLM as a complete end-to-end product. Our focus will be on offering some much-needed visualizations, search functionality, and LLM-driven infrastructure assistance tools using the ci.jenkins.io data.

Rationale

The main source of raw data will be the publicly available ci.jenkins.io datasets. It is hoped that by providing some LLM-driven visualizations and search functionality, Infra team users will be able to better understand the Jenkins usage data and gain insights into how Jenkins is being used in real-world scenarios. The LLM-driven infrastructure assistance tools will help users understand and troubleshoot issues related to Jenkins infrastructure, such as configuration issues, build failures, test failures, and other common problems.

Implementation

The project will involve the following key steps:

  • Data Collection: Collect and analyze the ci.jenkins.io data to identify patterns and trends in Jenkins usage.

  • Data Wrangling: Process the collected data to prepare it for fine-tuning the LLM model.

  • LLM Fine-tuning: Fine-tune an existing open-source LLM model (such as Llama 2) with the processed Jenkins data to create a domain-specific model.

  • Dashboard Development: Develop a dashboard using React.js to visualize the Jenkins usage data and provide insights into Jenkins usage patterns.

  • LLM Integration: Integrate the fine-tuned LLM model into the dashboard to provide users with AI-driven assistance for infrastructure-related tasks.

  • Search Functionality: Implement a search feature that allows users to query the LLM for specific Jenkins-related information.

  • UI/UX Design: Design a clean and intuitive user interface for the web application to enhance user experience.

Office hours

  • Weekly project office hours: Tuesdays 12:45 UTC on Google Meet (Video call link)

Links