Build a Python app with PyInstaller

This tutorial shows you how to use Jenkins to orchestrate building a simple Python application with PyInstaller.

If you are a Python developer who is new to CI/CD concepts, or you might be familiar with these concepts but don’t know how to implement building your application using Jenkins, then this tutorial is for you.

The simple Python application (which you’ll obtain from a sample repository on GitHub) is a command line tool "add2vals" that outputs the addition of two values. If at least one of the values is a string, "add2vals" treats both values as a string and instead concatenates the values. The "add2" function in the "calc" library (which "add2vals" imports) is accompanied by a set of unit tests. These are tested with pytest to check that this function works as expected and the results are saved to a JUnit XML report.

The delivery of the "add2vals" tool through PyInstaller converts this tool into a standalone executable file for Linux, which you can download through Jenkins and execute at the command line on Linux machines without Python.

Note: Unlike the other tutorials in this documentation, this tutorial requires approximately 500 MB more Docker image data to be downloaded.

Duration: This tutorial takes 20-40 minutes to complete (assuming you’ve already met the prerequisites below). The exact duration will depend on the speed of your machine and whether or not you’ve already run Jenkins in Docker from another tutorial.

You can stop this tutorial at any point in time and continue from where you left off.

If you’ve already run though another tutorial, you can skip the Prerequisites and Run Jenkins in Docker sections below and proceed on to forking the sample repository. (Just ensure you have Git installed locally.) If you need to restart Jenkins, simply follow the restart instructions in Stopping and restarting Jenkins and then proceed on.

Prerequisites

For this tutorial, you will require:

  • A macOS, Linux or Windows machine with:

    • 256 MB of RAM, although more than 512MB is recommended.

    • 10 GB of drive space for Jenkins and your Docker images and containers.

  • The following software installed:

    • Docker - Read more about installing Docker in the Installing Docker section of the Installing Jenkins page.
      Note: If you use Linux, this tutorial assumes that you are not running Docker commands as the root user, but instead with a single user account that also has access to the other tools used throughout this tutorial.

    • Git and optionally GitHub Desktop

Run Jenkins in Docker

In this tutorial, you’ll be running Jenkins as a Docker container from the jenkins/jenkins Docker image.

To run Jenkins in Docker, follow the relevant instructions below for either macOS and Linux or Windows.

You can read more about Docker container and image concepts in the Docker section of the Installing Jenkins page.

On macOS and Linux

  1. Open up a terminal window.

  2. Customise official Jenkins Docker image, by executing below two steps:

    1. Create Dockerfile with the folloing content:

      FROM jenkins/jenkins:2.249.1-lts
      USER root
      RUN apt-get update && apt-get install -y \
             apt-transport-https \
             ca-certificates \
             curl \
             gnupg2 \
             software-properties-common
      RUN curl -fsSL https://download.docker.com/linux/debian/gpg | apt-key add -
      RUN apt-key fingerprint 0EBFCD88
      RUN add-apt-repository \
             "deb [arch=amd64] https://download.docker.com/linux/debian \
             $(lsb_release -cs) \
             stable"
      RUN apt-get update && apt-get install -y docker-ce-cli
      USER jenkins
      ENTRYPOINT ["/sbin/tini", "--", "/usr/local/bin/jenkins.sh"]
    2. Build a new docker image from this Dockerfile and assign the image meaningful name, e.g. "myjenkins:1.0":

      docker build -t "myjenkins:1.0" .

      Keep in mind that described above process will automatically download official Jenkins Docker image if this hasn’t been done before.

  3. Run your own myjenkins:1.0 image as a container in Docker using the following docker run command:

    docker run --rm -u root --name jenkins-tutorial \
      --volume jenkins-data:/var/jenkins_home \ (1)
      --volume "$HOME":/home \ (2)
      --volume /var/run/docker.sock:/var/run/docker.sock \ (3)
      --publish 8080:8080 myjenkins:1.0
    1 Maps the /var/jenkins_home directory in the container to the Docker volume with the name jenkins-data.
    2 Maps the $HOME directory on the host (i.e. your local) machine (usually the /Users/<your-username> directory) to the /home directory in the container.
    3 Enables socket communication for the docker image.

    Note: If copying and pasting the command snippet above doesn’t work, try copying and pasting this annotation-free version here:

    docker run --rm -u root --name jenkins-tutorial \
      --volume jenkins-data:/var/jenkins_home \
      --volume /var/run/docker.sock:/var/run/docker.sock \
      --volume "$HOME":/home --publish 8080:8080 myjenkins:1.0
  4. Proceed to the Setup wizard.

On Windows

The Jenkins project provides a Linux container image, not a Windows container image. Be sure that your Docker for Windows installation is configured to run Linux Containers rather than Windows Containers. See the Docker documentation for instructions to switch to Linux containers. Once configured to run Linux Containers, the steps are:

  1. Open up a command prompt window.

  2. Customise official Jenkins Docker image using the macOS and Linux instructions above.

  3. Run your own myjenkins:1.0 image as a container in Docker using the following docker run command:

    docker run --rm -u root --name jenkins-tutorial ^
      --volume jenkins-data:/var/jenkins_home ^
      --volume "%HOMEDRIVE%%HOMEPATH%":/home ^
      --volume /var/run/docker.sock:/var/run/docker.sock ^
      --publish 8080:8080 myjenkins:1.0
  4. Proceed to the Setup wizard.

Accessing the Docker container

If you have some experience with Docker and you wish or need to access your Docker container through a terminal/command prompt using the docker exec command, you can add an option like --name jenkins-tutorial to the docker exec command. That will access the Jenkins Docker container named "jenkins-tutorial".

This means you could access your docker container (through a separate terminal/command prompt window) with a docker exec command like:

docker exec -it jenkins-tutorial bash

Setup wizard

Before you can access Jenkins, there are a few quick "one-off" steps you’ll need to perform.

Unlocking Jenkins

When you first access a new Jenkins instance, you are asked to unlock it using an automatically-generated password.

  1. After the 2 sets of asterisks appear in the terminal/command prompt window, browse to http://localhost:8080 and wait until the Unlock Jenkins page appears.

    Unlock Jenkins page

  2. Display the Jenkins console log with the command:

    docker logs jenkins-tutorial
  3. From your terminal/command prompt window again, copy the automatically-generated alphanumeric password (between the 2 sets of asterisks).

    Copying initial admin password

  4. On the Unlock Jenkins page, paste this password into the Administrator password field and click Continue.

Customizing Jenkins with plugins

After unlocking Jenkins, the Customize Jenkins page appears.

On this page, click Install suggested plugins.

The setup wizard shows the progression of Jenkins being configured and the suggested plugins being installed. This process may take a few minutes.

Creating the first administrator user

Finally, Jenkins asks you to create your first administrator user.

  1. When the Create First Admin User page appears, specify your details in the respective fields and click Save and Finish.

  2. When the Jenkins is ready page appears, click Start using Jenkins.
    Notes:

    • This page may indicate Jenkins is almost ready! instead and if so, click Restart.

    • If the page doesn’t automatically refresh after a minute, use your web browser to refresh the page manually.

  3. If required, log in to Jenkins with the credentials of the user you just created and you’re ready to start using Jenkins!

Stopping and restarting Jenkins

Throughout the remainder of this tutorial, you can stop your Docker container by running docker stop jenkins jenkins-docker.

To restart your Docker container:

  1. Run the same docker run …​ command you ran for macOS, Linux or Windows above.
    Note: This process also updates your Docker image, if an updated one is available.

  2. Browse to http://localhost:8080.

  3. Wait until the log in page appears and log in.

Fork and clone the sample repository on GitHub

Obtain the simple "add" Python application from GitHub, by forking the sample repository of the application’s source code into your own GitHub account and then cloning this fork locally.

  1. Ensure you are signed in to your GitHub account. If you don’t yet have a GitHub account, sign up for a free one on the GitHub website.

  2. Fork the simple-python-pyinstaller-app on GitHub into your local GitHub account. If you need help with this process, refer to the Fork A Repo documentation on the GitHub website for more information.

  3. Clone your forked simple-python-pyinstaller-app repository (on GitHub) locally to your machine. To begin this process, do either of the following (where <your-username> is the name of your user account on your operating system):

    • If you have the GitHub Desktop app installed on your machine:

      1. In GitHub, click the green Clone or download button on your forked repository, then Open in Desktop.

      2. In GitHub Desktop, before clicking Clone on the Clone a Repository dialog box, ensure Local Path for:

        • macOS is /Users/<your-username>/Documents/GitHub/simple-python-pyinstaller-app

        • Linux is /home/<your-username>/GitHub/simple-python-pyinstaller-app

        • Windows is C:\Users\<your-username>\Documents\GitHub\simple-python-pyinstaller-app

    • Otherwise:

      1. Open up a terminal/command line prompt and cd to the appropriate directory on:

        • macOS - /Users/<your-username>/Documents/GitHub/

        • Linux - /home/<your-username>/GitHub/

        • Windows - C:\Users\<your-username>\Documents\GitHub\ (although use a Git bash command line window as opposed to the usual Microsoft command prompt)

      2. Run the following command to continue/complete cloning your forked repo:
        git clone https://github.com/YOUR-GITHUB-ACCOUNT-NAME/simple-python-pyinstaller-app
        where YOUR-GITHUB-ACCOUNT-NAME is the name of your GitHub account.

Create your Pipeline project in Jenkins

  1. Go back to Jenkins, log in again if necessary and click create new jobs under Welcome to Jenkins!
    Note: If you don’t see this, click New Item at the top left.

  2. In the Enter an item name field, specify the name for your new Pipeline project (e.g. simple-python-pyinstaller-app).

  3. Scroll down and click Pipeline, then click OK at the end of the page.

  4. ( Optional ) On the next page, specify a brief description for your Pipeline in the Description field (e.g. An entry-level Pipeline demonstrating how to use Jenkins to build a simple Python application with PyInstaller.)

  5. Click the Pipeline tab at the top of the page to scroll down to the Pipeline section.

  6. From the Definition field, choose the Pipeline script from SCM option. This option instructs Jenkins to obtain your Pipeline from Source Control Management (SCM), which will be your locally cloned Git repository.

  7. From the SCM field, choose Git.

  8. In the Repository URL field, specify the directory path of your locally cloned repository above, which is from your user account/home directory on your host machine, mapped to the /home directory of the Jenkins container - i.e.

    • For macOS - /home/Documents/GitHub/simple-python-pyinstaller-app

    • For Linux - /home/GitHub/simple-python-pyinstaller-app

    • For Windows - /home/Documents/GitHub/simple-python-pyinstaller-app

  9. Click Save to save your new Pipeline project. You’re now ready to begin creating your Jenkinsfile, which you’ll be checking into your locally cloned Git repository.

Create your initial Pipeline as a Jenkinsfile

You’re now ready to create your Pipeline that will automate building your Python application with PyInstaller in Jenkins. Your Pipeline will be created as a Jenkinsfile, which will be committed to your locally cloned Git repository (simple-python-pyinstaller-app).

This is the foundation of "Pipeline-as-Code", which treats the continuous delivery pipeline a part of the application to be versioned and reviewed like any other code. Read more about Pipeline and what a Jenkinsfile is in the Pipeline and Using a Jenkinsfile sections of the User Handbook.

First, create an initial Pipeline with a "Build" stage that executes the first part of the entire production process for your application. This "Build" stage downloads a Python Docker image and runs it as a Docker container, which in turn compiles your simple Python application into byte code.

  1. Using your favorite text editor or IDE, create and save new text file with the name Jenkinsfile at the root of your local simple-python-pyinstaller-app Git repository.

  2. Copy the following Declarative Pipeline code and paste it into your empty Jenkinsfile:

    pipeline {
        agent none (1)
        stages {
            stage('Build') { (2)
                agent {
                    docker {
                        image 'python:2-alpine' (3)
                    }
                }
                steps {
                    sh 'python -m py_compile sources/add2vals.py sources/calc.py' (4)
                    stash(name: 'compiled-results', includes: 'sources/*.py*') (5)
                }
            }
        }
    }
    1 The agent section with the none parameter specified at the top of this Pipeline code block means that no global agent will be allocated for the entire Pipeline’s execution and that each stage directive must specify its own agent section.
    2 Defines a stage (directive) called Build that appears on the Jenkins UI.
    3 This image parameter (of the agent section’s docker parameter) downloads the python:2-alpine Docker image (if it’s not already available on your machine) and runs this image as a separate container. This means that:
    • You’ll have separate Jenkins and Python containers running locally in Docker.

    • The Python container becomes the agent that Jenkins uses to run the Build stage of your Pipeline project. However, this container is short-lived - its lifespan is only that of the duration of your Build stage’s execution.

    4 This sh step (of the steps section) runs the Python command to compile your application and its calc library into byte code files (each with .pyc extension), which are placed into the sources workspace directory (within the /var/jenkins_home/workspace/simple-python-pyinstaller-app directory in the Jenkins container).
    5 This stash step (of the basic steps section) saves the Python source code and compiled byte code files (with .pyc extension) from the sources workspace directory for use in later stages.
  3. Save your edited Jenkinsfile and commit it to your local simple-python-pyinstaller-app Git repository. E.g. Within the simple-python-pyinstaller-app directory, run the commands:
    git add .
    then
    git commit -m "Add initial Jenkinsfile"

  4. Go back to Jenkins again, log in again if necessary and click Open Blue Ocean on the left to access Jenkins’s Blue Ocean interface.

  5. In the This job has not been run message box, click Run, then quickly click the OPEN link which appears briefly at the lower-right to see Jenkins running your Pipeline project. If you weren’t able to click the OPEN link, click the row on the main Blue Ocean interface to access this feature.
    Note: You may need to wait a few minutes for this first run to complete. After making a clone of your local simple-python-pyinstaller-app Git repository itself, Jenkins:

    1. Initially queues the project to be run on the agent.

    2. Runs the Build stage (defined in the Jenkinsfile) on the Python container. During this time, Python uses the py_compile module to compile the code of your Python application and its calc library into byte code, which are stored in the sources workspace directory (within the Jenkins home directory).

    The Blue Ocean interface turns green if Jenkins compiled your Python application successfully.

    Initial Pipeline runs successfully

  6. Click the X at the top-right to return to the main Blue Ocean interface.

    Main Blue Ocean interface

Add a test stage to your Pipeline

  1. Go back to your text editor/IDE and ensure your Jenkinsfile is open.

  2. Copy and paste the following Declarative Pipeline syntax immediately under the Build stage of your Jenkinsfile:

            stage('Test') {
                agent {
                    docker {
                        image 'qnib/pytest'
                    }
                }
                steps {
                    sh 'py.test --junit-xml test-reports/results.xml sources/test_calc.py'
                }
                post {
                    always {
                        junit 'test-reports/results.xml'
                    }
                }
            }

    so that you end up with:

    pipeline {
        agent none
        stages {
            stage('Build') {
                agent {
                    docker {
                        image 'python:2-alpine'
                    }
                }
                steps {
                    sh 'python -m py_compile sources/add2vals.py sources/calc.py'
                    stash(name: 'compiled-results', includes: 'sources/*.py*')
                }
            }
            stage('Test') { (1)
                agent {
                    docker {
                        image 'qnib/pytest' (2)
                    }
                }
                steps {
                    sh 'py.test --junit-xml test-reports/results.xml sources/test_calc.py' (3)
                }
                post {
                    always {
                        junit 'test-reports/results.xml' (4)
                    }
                }
            }
        }
    }
    1 Defines a stage (directive) called Test that appears on the Jenkins UI.
    2 This image parameter (of the agent section’s docker parameter) downloads the qnib:pytest Docker image (if it’s not already available on your machine) and runs this image as a separate container. This means that:
    • You’ll have separate Jenkins and pytest containers running locally in Docker.

    • The pytest container becomes the agent that Jenkins uses to run the Test stage of your Pipeline project. This container’s lifespan lasts the duration of your Test stage’s execution.

    3 This sh step (of the steps section) executes pytest’s py.test command on sources/test_calc.py, which runs a set of unit tests (defined in test_calc.py) on the "calc" library’s add2 function (used by your simple Python application add2vals). The:
    • --junit-xml test-reports/results.xml option makes py.test generate a JUnit XML report, which is saved to test-reports/results.xml (within the /var/jenkins_home/workspace/simple-python-pyinstaller-app directory in the Jenkins container).

    4 This junit step (provided by the JUnit Plugin) archives the JUnit XML report (generated by the py.test command above) and exposes the results through the Jenkins interface. In Blue Ocean, the results are accessible through the Tests page of a Pipeline run. The post section’s always condition that contains this junit step ensures that the step is always executed at the completion of the Test stage, regardless of the stage’s outcome.
  3. Save your edited Jenkinsfile and commit it to your local simple-python-pyinstaller-app Git repository. E.g. Within the simple-python-pyinstaller-app directory, run the commands:
    git stage .
    then
    git commit -m "Add 'Test' stage"

  4. Go back to Jenkins again, log in again if necessary and ensure you’ve accessed Jenkins’s Blue Ocean interface.

  5. Click Run at the top left, then quickly click the OPEN link which appears briefly at the lower-right to see Jenkins running your amended Pipeline project. If you weren’t able to click the OPEN link, click the top row on the Blue Ocean interface to access this feature.
    Note: It may take a few minutes for the qnib:pytest Docker image to download (if this hasn’t already been done).
    If your amended Pipeline ran successfully, here’s what the Blue Ocean interface should look like. Notice the additional "Test" stage. You can click on the previous "Build" stage circle to access the output from that stage.

    Test stage runs successfully (with output)

  6. Click the X at the top-right to return to the main Blue Ocean interface.

Add a final deliver stage to your Pipeline

  1. Go back to your text editor/IDE and ensure your Jenkinsfile is open.

  2. Copy and paste the following Declarative Pipeline syntax immediately under the Test stage of your Jenkinsfile:

            stage('Deliver') {
                agent any
                environment {
                    VOLUME = '$(pwd)/sources:/src'
                    IMAGE = 'cdrx/pyinstaller-linux:python2'
                }
                steps {
                    dir(path: env.BUILD_ID) {
                        unstash(name: 'compiled-results')
                        sh "docker run --rm -v ${VOLUME} ${IMAGE} 'pyinstaller -F add2vals.py'"
                    }
                }
                post {
                    success {
                        archiveArtifacts "${env.BUILD_ID}/sources/dist/add2vals"
                        sh "docker run --rm -v ${VOLUME} ${IMAGE} 'rm -rf build dist'"
                    }
                }
            }

    and add a skipStagesAfterUnstable option so that you end up with:

    pipeline {
        agent none
        options {
            skipStagesAfterUnstable()
        }
        stages {
            stage('Build') {
                agent {
                    docker {
                        image 'python:2-alpine'
                    }
                }
                steps {
                    sh 'python -m py_compile sources/add2vals.py sources/calc.py'
                    stash(name: 'compiled-results', includes: 'sources/*.py*')
                }
            }
            stage('Test') {
                agent {
                    docker {
                        image 'qnib/pytest'
                    }
                }
                steps {
                    sh 'py.test --junit-xml test-reports/results.xml sources/test_calc.py'
                }
                post {
                    always {
                        junit 'test-reports/results.xml'
                    }
                }
            }
            stage('Deliver') { (1)
                agent any
                environment { (2)
                    VOLUME = '$(pwd)/sources:/src'
                    IMAGE = 'cdrx/pyinstaller-linux:python2'
                }
                steps {
                    dir(path: env.BUILD_ID) { (3)
                        unstash(name: 'compiled-results') (4)
                        sh "docker run --rm -v ${VOLUME} ${IMAGE} 'pyinstaller -F add2vals.py'" (5)
                    }
                }
                post {
                    success {
                        archiveArtifacts "${env.BUILD_ID}/sources/dist/add2vals" (6)
                        sh "docker run --rm -v ${VOLUME} ${IMAGE} 'rm -rf build dist'"
                    }
                }
            }
        }
    }
    1 Defines a stage (directive) called Deliver that appears on the Jenkins UI.
    2 This environment block defines two variables which will be used later in the 'Deliver' stage.
    3 This dir step (of the basic steps section) creates a new subdirectory named by the build number. The final program will be created in that directory by pyinstaller. BUILD_ID is one of the pre-defined Jenkins environment variables and is available in all jobs.
    4 This unstash step (of the basic steps section) restores the Python source code and compiled byte code files (with .pyc extension) from the previously saved stash. image] (if it’s not already available on your machine) and runs this image as a separate container. This means that:
    • You’ll have separate Jenkins and PyInstaller (for Linux) containers running locally in Docker.

    • The PyInstaller container becomes the agent that Jenkins uses to run the Deliver stage of your Pipeline project. This container’s lifespan lasts the duration of your Deliver stage’s execution.

    5 This sh step (of the steps section) executes the pyinstaller command (in the PyInstaller container) on your simple Python application. This bundles your add2vals.py Python application into a single standalone executable file (via the --onefile option) and outputs the this file to the dist workspace directory (within the Jenkins home directory). Although this step consists of a single command, as a general principle, it’s a good idea to keep your Pipeline code (i.e. the Jenkinsfile) as tidy as possible and place more complex build steps (particularly for stages consisting of 2 or more steps) into separate shell script files like the deliver.sh file. This ultimately makes maintaining your Pipeline code easier, especially if your Pipeline gains more complexity.
    6 This archiveArtifacts step (provided as part of Jenkins core) archives the standalone executable file (generated by the pyinstaller command above at dist/add2vals within the Jenkins home’s workspace directory) and exposes this file through the Jenkins interface. In Blue Ocean, archived artifacts like these are accessible through the Artifacts page of a Pipeline run. The post section’s success condition that contains this archiveArtifacts step ensures that the step is executed at the completion of the Deliver stage only if this stage completed successfully.
  3. Save your edited Jenkinsfile and commit it to your local simple-python-pyinstaller-app Git repository. E.g. Within the simple-python-pyinstaller-app directory, run the commands:
    git stage .
    then
    git commit -m "Add 'Deliver' stage"

  4. Go back to Jenkins again, log in again if necessary and ensure you’ve accessed Jenkins’s Blue Ocean interface.

  5. Click Run at the top left, then quickly click the OPEN link which appears briefly at the lower-right to see Jenkins running your amended Pipeline project. If you weren’t able to click the OPEN link, click the top row on the Blue Ocean interface to access this feature.
    Note: It may take a few minutes for the cdrx/pyinstaller-linux Docker image to download (if this hasn’t already been done).
    If your amended Pipeline ran successfully, here’s what the Blue Ocean interface should look like. Notice the additional "Deliver" stage. Click on the previous "Test" and "Build" stage circles to access the outputs from those stages.

    Deliver stage runs successfully

    Here’s what the output of the "Deliver" stage should look like, showing you the results of PyInstaller bundling your Python application into a single standalone executable file.

    Deliver stage output only

  6. Click the X at the top-right to return to the main Blue Ocean interface, which lists your previous Pipeline runs in reverse chronological order.

    Main Blue Ocean interface with all previous runs displayed

Follow up (optional)

If you use Linux, you can try running the standalone add2vals application you generated with PyInstaller locally on your machine. To do this:

  1. From the main Blue Ocean interface, access your last Pipeline run you performed above. To do this, click the top row (representing the most recent Pipeline run) on the main Blue Ocean’s Activity page.

    Main Blue Ocean interface with all previous runs displayed

  2. On the results page of the Pipeline run, click Artifacts at the top right to access the Artifacts page.

    Deliver stage runs successfully

  3. In the list of artifacts, click the down-arrow icon at the far right of the dist/add2vals artifact item to download the standalone executable file to your browser’s "Downloads" directory.

    Deliver stage Artifacts page

  4. Back in your operating system’s terminal prompt, cd to your browser’s "Downloads" directory.

  5. Make the add2vals file executable - i.e. chmod a+x add2vals

  6. Run the command ./add2vals and follow the instructions provided by your app.

Wrapping up

Well done! You’ve just used Jenkins to build a simple Python application!

The "Build", "Test" and "Deliver" stages you created above are the basis for building more complex Python applications in Jenkins, as well as Python applications that integrate with other technology stacks.

Because Jenkins is extremely extensible, it can be modified and configured to handle practically any aspect of build orchestration and automation.

To learn more about what Jenkins can do, check out:


Was this page helpful?

Please submit your feedback about this page through this quick form.

Alternatively, if you don't wish to complete the quick form, you can simply indicate if you found this page helpful?

    


See existing feedback here.