Machine Learning Plugins for Data Science

Goal: Create a new plugin for integrating Jenkins with one of Machine Learning tools (e.g. Jupyter Python, TensorBoard, or Sacred)

Status: Completed

Team

Details

The main goal of this project is integrating Machine Learning workflow including Data preprocessing, Model Training, Evaluation and Prediction with Jenkins build tasks. This plugin will be capable of executing code fragments via IPython kernel as currently supported by Jupyter Notebook. Version control will be handled as an added advantage of this project.

Deliverables

  • An IPython plugin with pipeline compatible

  • Improved config.jelly for the plugin

  • File parsers for ipynb, py and text(containing python code) files

  • Code segment extractor

  • Implement a code editor

  • Output console with static visual components

Community Bonding

The community bonding period is all about, well, community bonding. Rather than jumping straight into coding, I’ve got some time to learn about Jenkins’s processes - release and otherwise - developer interactions, codes of conduct, etc.

Phase 1

  • Implement the configuration part of Ipython kernel interpreter

  • Implement file parser

  • Improve current config.jelly

  • Improve form validations

  • Increase the test coverage and documentation

  • Blog post

  • Change log

Phase 2

Phase 3

  • Design and implement code extractor

  • Multiple language kernel support

  • Implement Visual generator for the results

  • Integrate whole previous work and testing

  • Git integration for code segments (optional)

  • User documentation and examples

  • Blog post

  • Change log

Future Improvements

  • Improving performance of the plugin

  • Try to implement JENKINS-63377

  • Support parameterized definitions in Notebooks JENKINS-63478

  • Increasing testing code coverage

  • Implementing Interactive visualizer for data sets and results.

Final Demo Presentation Slides and Video

Final Machine Learning Plugin slides

Final GSoC Demo

Articles

Jupyter Notebook Integration

Weekly Meeting Sync

  • Monday

  • Time - 03:00 am (UTC)

Github
Plugin page
Meeting Notes
GSoC Proposal
Timeline
Design Document

Links