05-ci.adoc

Continuous Integration with Jenkins

aka "CI"

CI: Why ?

Continuous Integration doesn’t get rid of bugs, but it does make them dramatically easier to find and remove.

— Martin Fowler

CI: What ?

Continuous Integration is a software development practice where members of a team integrate their work frequently, usually each person integrates at least daily, leading to multiple integrations per day.

— Martin Fowler - Continuous Integration

CI: How ?

• Each integration is verified by an automated build (including test)

• Integrate code often, at least daily, to make integration a non-event

• Continuously build and integrate, with a feedback loop

Continuous Integration with Jenkins

CI: Accessing Jenkins

• Access your Jenkins instance:

  • {jenkins-url}

  • Log in as the user butler (password is the same)

  • It is the "Legacy UI"

CI: Jenkins BlueOcean

• Switch to BlueOcean, the new UI

  • Direct link: {jenkins-url}/blue

  • Or click on the top button "Open Blue Ocean"

CI: Creating Project

• Create your 1st Pipeline:

  • Stored in Git

  • Fetch URL from the Gitserver

    • Direct link: {gitserver-url}/butler/demoapp.git

  • Add a User/password credential (butler / butler)

  • Pipeline is empty (for now): no Jenkinsfile

CI: Fast feedback with Webhooks

• We want Fast feedback !

  • Pushed code to repository ? Tell Jenkins to build it now

• Let’s use Webhook to the repository

  • HTTP request Gitserver → Jenkins

CI: Add a Gogs Webhooks

• From repo. in Gitserver → Settings → Webhooks

  • Direct link: {gitserver-url}/butler/demoapp/settings/hooks

• Add a Gogs webhook:

  • Payload URL: {jenkins-url}/job/demoapp/build?delay=0

  • When should this webhook be triggered?: I need everything

CI: Starting with Pipelines

• Pipeline-as-code: We need a Jenkinsfile

  • Declarative or Scripted ?

• Where to start ?

  • Documentation: https://jenkins.io/doc/pipeline

  • Getting Started: https://jenkins.io/doc/pipeline/tour/hello-world/

  • Syntax Reference: https://jenkins.io/doc/book/pipeline/syntax/

CI: BlueOcean Pipeline Editor

• Provides the full round trip with SCM

• No Pipeline ? Follow the wizard (not Gandalf fool !)

• Already have a Pipeline ? Edit, commit, run it

• Needs a compliant SCM

  • Only Github with BO 1.0.1

  • Interested ? Open-Source: Contribute !

CI: Use the Pipeline Editor

• Let’s hack: open the BlueOcean Pipeline Editor

  • Direct (hidden) URL: {jenkins-url}/blue/organizations/jenkins/pipeline-editor/

  • Use CTRL + S (On Mac: CMD +S) to switch to textual version

• Also, use the "Legacy" Pipeline Syntax Snippet Generator:

  • {jenkins-url}/job/demoapp/pipeline-syntax/

CI: Exercise - Your First Pipeline

• Use the BO editor and Gitserver

• Create a Pipeline that have a single stage "Hello"

• This stage have 1 step that prints the message "Hello World"

• Copy/Paste this Pipeline in a new file Jenkinsfile on the repository root

• A build will kick off immediately:

  • demoapp Activity Dashboard

CI: Solution - Your first pipeline

pipeline {
  agent any
  stages {
    stage('Build') {
      steps {
        echo 'Hello World !'
      }
    }
  }
}

CI: Exercise - Simple Build Pipeline

• Exercise: Implement a simple build pipeline demoapp

• We want 4 stages, for the 4 Maven goals:

  • clean compile, test, package, verify

• We need to build on the maven agent

CI: Solution - Simple Build Pipeline

pipeline {
  agent {
    node {
      label 'maven'
    }
  }
  stages {
    stage('Compile') {
      steps {
        sh 'mvn compile'
      }
    }
    stage('Unit Tests') {
      steps {
        sh 'mvn test'
      }
    }
    stage('Build') {
      steps {
        sh 'mvn package'
      }
    }
    stage('Verify') {
      steps {
        sh 'mvn verify'
      }
    }
  }
}

CI: Exercise - Artifacts

• We want to simplify to 2 stages, based on Unit Tests definition:

  • Build: compile, unit test and package the application

  • Verify: Run Integration Tests

• We also want to archive the generated jar file

  • Only if the build is in sucess

• Clues: Keywords post + success (not in Editor), and archiveArtifacts

CI: Solution - Artifacts

pipeline {
  agent {
    node {
      label 'maven'
    }
  }
  stages {
    stage('Build') {
      steps {
        sh 'mvn clean compile test package'
      }
    }
    stage('Verify') {
      steps {
        sh 'mvn verify'
      }
    }
  }
  post {
    success {
      archiveArtifacts 'target/demoapp.jar'
    }
  }
}

CI: Exercise - Integration Tests Reports

• We want the integration test reports to be published to Jenkins

  • Better feedback loop

• If Integration Tests are failing, do NOT fail the build

  • Make it UNSTABLE instead

• Clues:

  • Maven flag -fn ("Fails Never")

  • keyword junit (Pipeline keyword)

CI: Solution - Integration Tests Reports

pipeline {
  agent {
    node {
      label 'maven'
    }
  }
  stages {
    stage('Build') {
      steps {
        sh 'mvn clean compile test package'
      }
    }
    stage('Verify') {
      steps {
        sh 'mvn verify -fn'
        junit '**/target/failsafe-reports/*.xml'
      }
    }
  }
  post {
    success {
      archiveArtifacts 'target/demoapp.jar'
    }
  }
}

CI: Exercise - All Tests Reports

• We now want all test reports published

  • Problem: how to handle Unit test failure ?

• We also want to archive artifacts if build is unstable only due to the Verify stage

• Clues: post can be used per stage

CI: Solution - All Tests Reports

pipeline {
  agent {
    node {
      label 'maven'
    }
  }
  stages {
    stage('Build') {
      steps {
        sh 'mvn clean compile test package'
      }
      post {
        always {
          junit '**/target/surefire-reports/*.xml'
        }
      }
    }
    stage('Verify') {
      steps {
        sh 'mvn verify -fn'
        junit '**/target/failsafe-reports/*.xml'
      }
      post {
        unstable {
          archiveArtifacts 'target/demoapp.jar'
        }
      }
    }
  }
  post {
    success {
      archiveArtifacts 'target/demoapp.jar'
    }
  }
}

CI: Failing Tests

• Validate your changes by making your tests fails.

• Edit each one and uncomment the failing block:

  • Integration: src/master/src/test/java/hello/ApplicationIT.java

  • Unit Tests: src/master/src/test/java/hello/ApplicationTest.java

• Browse the top-level items "Changes", "Tests" and "Artifacts"

• Do NOT forget to correct your tests at the end

That’s all folks !