EDP Pipeline Framework

This chapter provides detailed information about the EDP pipeline framework parts as well as the accurate data about the Code Review, Build and Deploy pipelines with the respective stages.

Overview

The general EDP Pipeline Framework consists of several parts:

Jenkinsfile - a text file that keeps the definition of a Jenkins Pipeline and is checked into source control. Every Job has its Jenkinsfile that is stored in the specific application repository and in Jenkins as the plain text.
Loading Shared Libraries - a part where every job loads libraries with the help of the shared libraries mechanism for Jenkins that allows to create reproducible pipelines, write them uniformly, and manage the update process. There are two main libraries: EDP Pipelines with the common logic described for the main pipelines Code Review, Build, Deploy pipelines and EDP Stages library that keeps the description of the stages for every pipeline.
Run Stages - a part where the predefined default stages are launched.

NOTE: The whole logic is applied to Jenkins as it is the main tool for the CI/CD processes organization.

EDP Pipelines and Stages

Get acquainted with the sections below to get detailed information about the EDP pipelines and their stages:

1. Code Review Pipeline

1.1 Code Review Pipeline. EDP Library Pipelines Description

1.2 Code Review Pipeline Stages

1.3 Code Review Pipeline. How to Redefine or Extend the EDP Pipeline Stages Library

1.4 Code Review Pipeline. Using EDP Library Stages in the Pipeline

2. Build Pipeline

2.1 Build Pipeline. EDP Library Pipelines Description

2.2 Build Pipeline Stages

2.3 Build Pipeline. How to Redefine or Extend EDP Pipeline Stages Library

2.4 Build Pipeline. Using EDP Library Stages in the Pipeline

3. EDP Stages Library Description

5. Deploy Pipeline

5.1 Deploy Pipeline. EDP Library Pipelines Description

5.2 Deploy Pipeline. EDP Library Stages Description

5.3 Deploy Pipeline Stages

5.4 Deploy Pipeline. How to Redefine or Extend EDP Pipeline Stages Library

5.5 Deploy Pipeline. Using EDP Library Stages in the Pipeline

1. Code Review Pipeline

CodeReview() – a function that allows using the EDP implementation for the Code Review pipeline.

NOTE: All values of different parameters that are used during the pipeline execution are stored in Map "context".

The Code Review pipeline consists of several steps:

On the master:

Initialization of all objects (Platform, Job, Gerrit, Nexus, Sonar, Application, StageFactory) and loading of the default implementations of EDP stages.

On a particular slave that depends on the build tool:

Creating workdir for application sources
Loading build tool implementation for a particular application
Run in a loop all stages (From) and run them either in parallel or one by one

1.1 Code Review Pipeline. EDP Pipelines Library Description

Using in pipelines - @Library(['edp-library-pipelines@version']) _

The corresponding enums, interfaces, classes, and their methods can be used separately from the EDP Pipelines library function (please refer to Table 1 and Table 2).

Table 1. Enums and Interfaces with the respective properties, methods, and examples.

Enums Interfaces

PlatformType:

- OPENSHIFT
- KUBERNETES

JobType:

- CODEREVIEW
- BUILD
- DEPLOY

BuildToolType:

- MAVEN
- GRADLE
- NPM
- DOTNET Platform() - contains methods for working with platform CLI.
At the moment only OpenShift is supported.

Properties:

Script script - Object with type script, in most cases if class
created from Jenkins pipelines it is "this".

Methods:

getJsonPathValue(String k8s_kind, String k8s_kind_name,
String jsonPath): return String value of specific parameter
of particular object using jsonPath utility.

Example:

context.platform.getJsonPathValue(''cm'',
''project-settings'', ''.data.username'').

BuildTool() - contains methods for working with different
buildTool from ENUM BuildToolType.
Should be invoked on slave build agents.

Properties:

Script script - Object with type script, in most cases if class
created from Jenkins pipelines it is "this".
Nexus object - Object of class Nexus.

Methods:

init: return parameters of buildTool that are needed
for running stages.

Example:

context.buildTool = new BuildToolFactory().
getBuildToolImpl
(context.application.config.build_tool, this,
context.nexus) context.buildTool.init().

Table 2. Classes with the respective properties, methods, and examples.

Classes	Description (properties, methods, and examples)
PlatformFactory() - Class that contains methods getting an implementation of CLI of the platform. At the moment OpenShift and Kubernetes are supported.	Methods: `getPlatformImpl(PlatformType platform, Script script): return Class Platform`. Example: `context.platform = new PlatformFactory().getPlatformImpl(PlatformType.OPENSHIFT, this)`.
Application(String name, Platform platform, Script script) - Class that describes the application object.	Properties: Script script - Object with type script, in most cases if class created from Jenkins pipelines it is "this". Platform platform - Object of a class Platform(). String name - Name for the application for creating an object. Map config - Map of configuration settings for the particular application that is loaded from config map project-settings. String version - Application version, initially empty. Is set on the get-version step. String deployableModule - The name of the deployable module for multi-module applications, initially empty. String buildVersion - Version of the built artifact, contains build number of Job initially empty. String deployableModuleDir - The name of deployable module directory for multi-module applications, initially empty. Array imageBuildArgs - List of arguments for building an application Docker image. Methods: setConfig(String gerrit_autouser, String gerrit_host, String gerrit_sshPort, String gerrit_project): set the config property with values from config map. Example: `context.application = new Application(context.job, context.gerrit.project, context.platform, this) context.application.setConfig(context.gerrit.autouser, context.gerrit.host, context.gerrit.sshPort, context.gerrit.project)`
Job(type: JobType.value, platform: Platform, script: Script) - Class that describes the Gerrit tool.	Properties: Script script - Object with type script, in most cases if class created from Jenkins pipelines it is "this". Platform platform - Object of a class Platform(). JobType.value type. String deployTemplatesDirectory - The name of the directory in application repository where deploy templates are located. It can be set for a particular Job through DEPLOY_TEMPLATES_DIRECTORY parameter. String edpName - The name of the EDP Project. Map stages - Contains all stages in JSON format that is retrieved from Jenkins job env variable. String envToPromote - The name of the environment for promoting images. Boolean promoteImages - Defines whether images should be promoted or not. Methods: getParameterValue(String parameter, String defaultValue = null): return parameter of ENV variable of Jenkins job. init(): set all the properties of the Job object. setDisplayName(String displayName): set display name of the Jenkins job. setDescription(String description, Boolean addDescription = false): set new or add to the existing description of the Jenkins job. printDebugInfo(Map context): print context info to the log of Jenkins' job. runStage(String stage_name, Map context): run the particular stage according to its name. Example: `context.job = new Job(JobType.CODEREVIEW.value,` `context.platform, this)` `context.job.init()` `context.job.printDebugInfo(context)` `context.job.setDisplayName("test")` `context.job.setDescription("Name: ${context.application.config.name}")`
Gerrit(Job job, Platform platform, Script script) - Class that describes the Gerrit tool.	Properties: Script script - Object with type script, in most cases if class created from Jenkins pipelines it is "this". Platform platform - Object of a class Platform(). Job job - Object of a class Job(). String credentialsId - Credential Id in Jenkins for Gerrit. String autouser - Username of an auto user in Gerrit for integration with Jenkins. String host - Gerrit host. String project - the project name of the built application. String branch - branch to build the application from. String changeNumber - change number of Gerrit commit. String changeName - change name of Gerrit commit. String refspecName - refspecName of Gerrit commit. String sshPort - Gerrit ssh port number. String patchsetNumber - patchsetNumber of Gerrit commit. Methods: init(): set all the properties of Gerrit object. Example: `context.gerrit = new Gerrit(context.job, context.platform, this)` `context.gerrit.init()`
Nexus(Job job, Platform platform, Script script) - Class that describes the Nexus tool.	Properties: Script script - Object with type script, in most cases if class created from Jenkins pipelines it is "this". Platform platform - Object of a class Platform(). Job job - Object of a class Job(). String autouser - Username of an auto user in Nexus for integration with Jenkins. String credentialsId - Credential Id in Jenkins for Nexus. String host - Nexus host. String port - Nexus http(s) port. String repositoriesUrl - Base URL of repositories in Nexus. String restUrl - URL of Rest API. Methods: init(): set all the properties of Nexus object Example: `context.nexus = new Nexus(context.job, context.platform, this)` `context.nexus.init()`
Sonar(Job job, Platform platform, Script script) - Class that describes the Sonar tool.	Properties: Script script - Object with type script, in most cases if class created from Jenkins pipelines it is "this". Platform platform - Object of a class Platform(). Job job - Object of a class Job(). String route - External route of the sonar application. Methods: init(): set all the properties of Sonar object Example: `context.sonar = new Sonar(context.job, context.platform, this)` `context.sonar.init()`

1.2 Code Review Pipeline Stages

Each EDP stage implementation has run method that is as input parameter required to pass a context map with different keys. Some stages can implement the logic for several build tools and application types, some of them are specific.

The Code Review pipeline includes the following default stages: Checkout → Gerrit Checkout → Compile → Tests → Sonar.

INFO: To get the full description of every stage, please refer to the EDP Stages Framework section.

1.3 Code Review Pipeline. How to Redefine or Extend the EDP Pipeline Stages Library

Inspect the points below to redefine or extend the EDP Pipeline Stages Library:

Create “stage” folder in your App repository.
Create a Groovy file with a meaningful name for the custom stage description. For instance – CustomBuildMavenApplication.groovy.
Describe the stage logic.

Redefinition:

import com.epam.edp.stages.ProjectType
import com.epam.edp.stages.Stage
@Stage(name = "compile", buildTool = "maven", type = ProjectType.APPLICATION)
class CustomBuildMavenApplication {
    Script script
    void run(context) {
        script.sh "echo 'Your custom logic of the stage'"
    }
}
return CustomBuildMavenApplication

Extension:

import com.epam.edp.stages.ProjectType
import com.epam.edp.stages.Stage
@Stage(name = "new-stage", buildTool = "maven", type = ProjectType.APPLICATION)
class NewStageMavenApplication {
    Script script
    void run(context) {
        script.sh "echo 'Your custom logic of the stage'"
    }
}
return NewStageMavenApplication

1.4 Code Review Pipeline. Using EDP Stages Library in the Pipeline

In order to use the EDP stages, the created pipeline should fit some requirements, that`s why a developer has to do the following:

import library - @Library(['edp-library-stages']) _
import StageFactory class - import com.epam.edp.stages.StageFactory
define context Map – context = [:]
define stagesFactory instance and load EDP stages: context.factory = new StageFactory(script: this) context.factory.loadEdpStages().each() { context.factory.add(it) }

After that, there is the ability to run any EDP stage beforehand by defining a necessary context: context.factory.getStage("checkout","maven","application").run(context)

For instance, the pipeline can look like:

@Library(['edp-library-stages']) _
 
import com.epam.edp.stages.StageFactory
import org.apache.commons.lang.RandomStringUtils
 
context = [:]
 
node('maven') {
    context.workDir = new File("/tmp/${RandomStringUtils.random(10, true, true)}")
    context.workDir.deleteDir()
 
    context.factory = new StageFactory(script: this)
    context.factory.loadEdpStages().each() { context.factory.add(it) }
   
    context.gerrit = [:]
    context.application = [:]
    context.application.config = [:]
    context.buildTool = [:]
    context.nexus = [:]
   
 
   
    stage("checkout") {
        context.gerrit.branch = "master"
        context.gerrit.credentialsId = "jenkins"
        context.application.config.cloneUrl = "ssh://jenkins@gerrit:32092/sit-718-cloned-java-maven-project"
        context.factory.getStage("checkout","maven","application").run(context)
    }
 
 
    stage("compile") {
        context.buildTool.command = "mvn"
        context.nexus.credentialsId = "nexus"
        context.factory.getStage("compile","maven","application").run(context)
    }
}

Or in a declarative way:

@Library(['edp-library-stages']) _
 
import com.epam.edp.stages.StageFactory
import org.apache.commons.lang.RandomStringUtils
 
context = [:]
 
pipeline {
    agent { label 'maven' }
    stages {
        stage('Init'){
            steps {
                script {
                    context.workDir = new File("/tmp/${RandomStringUtils.random(10, true, true)}")
                    context.workDir.deleteDir()
                    context.factory = new StageFactory(script: this)
                    context.factory.loadEdpStages().each() { context.factory.add(it) }
                                                           
                    context.gerrit = [:]
                    context.application = [:]
                    context.application.config = [:]
                    context.buildTool = [:]
                    context.nexus = [:]
                }
            }
        }
                       
        stage("Checkout") {
            steps {
                 script {
                    context.gerrit.branch = "master"
                    context.gerrit.credentialsId = "jenkins"
                    context.application.config.cloneUrl = "ssh://jenkins@gerrit:32092/sit-718-cloned-java-maven-project"
                    context.factory.getStage("checkout","maven","application").run(context)
                    }
            }
        }
                       
        stage('Compile') {
            steps {
                 script {
                    context.buildTool.command = "mvn"
                    context.nexus.credentialsId = "nexus"
                    context.factory.getStage("compile","maven","application").run(context)
                    }
            }
        }
    }
}

2. Build Pipeline

Build() – a function that allows using the EDP implementation for the Build pipeline. All values of different parameters that are used during the pipeline execution are stored in Map "context".

The Build pipeline consists of several steps:

On the master:

Initialization of all objects (Platform, Job, Gerrit, Nexus, Sonar, Application, StageFactory) and loading default implementations of EDP stages.

On a particular slave that depends on the build tool:

Creating workdir for application sources;
Loading build tool implementation for a particular application;
Run in a loop all stages (From) and run them either in parallel or one by one.

2.1 Build Pipeline. EDP Pipelines Library Description

Using in pipelines - @Library(['edp-library-pipelines@version'])_

The corresponding enums, interfaces, classes, and their methods can be used separately from the EDP Pipelines library function (please refer to Table 3 and Table 4).

Table 3. Enums and Interfaces with the respective properties, methods, and examples.

Enums Interfaces

PlatformType:

- OPENSHIFT
- KUBERNETES

JobType:

- CODEREVIEW
- BUILD
- DEPLOY

BuildToolType:

- MAVEN
- GRADLE
- NPM
- DOTNET Platform() - contains methods for working with platform CLI.
At the moment only OpenShift is supported.

Properties:

Script script - Object with type script, in most cases if class
created from Jenkins pipelines it is "this".

Methods:

getJsonPathValue(String k8s_kind, String k8s_kind_name,
String jsonPath): return String value of specific parameter
of particular object using jsonPath utility.

Example:

context.platform.getJsonPathValue("cm","project-settings",
".data.username")

BuildTool() - contains methods for working with different
buildTool from ENUM BuildToolType.
Should be invoked on slave build agents.

Properties:

Script script - Object with type script, in most cases if class
created from Jenkins pipelines it is "this".
Nexus object - Object of class Nexus. See description below:

Methods:

init: return parameters of buildTool that are needed for
running stages.

Example:

context.buildTool = new BuildToolFactory().getBuildToolImpl
(context.application.config.build_tool, this,
context.nexus)context.buildTool.init()

Table 4. Classes with the respective properties, methods, and examples.

Classes	Description (properties, methods, and examples)
PlatformFactory() - Class that contains methods getting an implementation of CLI of the platform. At the moment OpenShift and Kubernetes are supported.	Methods: getPlatformImpl(PlatformType platform, Script script): return Class Platform Example: `context.platform = new PlatformFactory().getPlatformImpl(PlatformType.OPENSHIFT, this)`
Application(String name, Platform platform, Script script) - Class that describes the application object.	Properties: Script script - Object with type script, in most cases if class created from Jenkins pipelines it is "this". Platform platform - Object of a class Platform(). String name - Name for the application for creating an object. Map config - Map of configuration settings for the particular application that is loaded from config map project-settings. String version - Application version, initially empty. Is set on the get-version step. String deployableModule - The name of the deployable module for multi-module applications, initially empty. String buildVersion - Version of the built artifact, contains build number of Job initially empty. String deployableModuleDir - The name of deployable module directory for multi-module applications, initially empty. Array imageBuildArgs - List of arguments for building the application Docker image. Methods: setConfig(String gerrit_autouser, String gerrit_host, String gerrit_sshPort, String gerrit_project): set the config property with values from config map. Example: `context.application = new Application(context.job, context.gerrit.project, context.platform, this) context.application.setConfig(context.gerrit.autouser, context.gerrit.host, context.gerrit.sshPort, context.gerrit.project)`
Job(type: JobType.value, platform: Platform, script: Script) - Class that describes the Gerrit tool.	Properties: Script script - Object with type script, in most cases if class created from Jenkins pipelines it is "this". Platform platform - Object of a class Platform(). JobType.value type. String deployTemplatesDirectory - The name of the directory in application repository, where deploy templates are located. It can be set for a particular Job through DEPLOY_TEMPLATES_DIRECTORY parameter. String edpName - The name of the EDP Project. Map stages - Contains all stages in JSON format that is retrieved from Jenkins job env variable. String envToPromote - The name of the environment for promoting images. Boolean promoteImages - Defines whether images should be promoted or not. Methods: getParameterValue(String parameter, String defaultValue = null): return parameter of ENV variable of Jenkins job. init(): set all the properties of the Job object. setDisplayName(String displayName): set display name of the Jenkins job. setDescription(String description, Boolean addDescription = false): set new or add to the existing description of the Jenkins job. printDebugInfo(Map context): print context info to the log of Jenkins' job. runStage(String stage_name, Map context): run the particular stage according to its name. Example: `context.job = new Job(JobType.CODEREVIEW.value, context.platform, this) context.job.init() context.job.printDebugInfo(context) context.job.setDisplayName("test") context.job.setDescription("Name: ${context.application.config.name}")`
Gerrit(Job job, Platform platform, Script script) - Class that describes the Gerrit tool.	Properties: Script script - Object with type script, in most cases if class created from Jenkins pipelines it is "this". Platform platform - Object of a class Platform(). Job job - Object of a class Job(). String credentialsId - Credentials Id in Jenkins for Gerrit. String autouser - Username of an auto user in Gerrit for integration with Jenkins. String host - Gerrit host. String project - the project name of the built application. String branch - branch to build an application from. String changeNumber - change number of Gerrit commit. String changeName - change name of Gerrit commit. String refspecName - refspecName of Gerrit commit. String sshPort - Gerrit ssh port number. String patchsetNumber - patchsetNumber of Gerrit commit. Methods: init(): set all the properties of Gerrit object Example: `context.gerrit = new Gerrit(context.job, context.platform, this) context.gerrit.init()`
Nexus(Job job, Platform platform, Script script) - Class that describes the Nexus tool.	Properties: Script script - Object with type script, in most cases if class created from Jenkins pipelines it is "this". Platform platform - Object of a class Platform(). Job job - Object of a class Job(). String autouser - Username of an auto user in Nexus for integration with Jenkins. String credentialsId - Credentials Id in Jenkins for Nexus. String host - Nexus host. String port - Nexus http(s) port. String repositoriesUrl - Base URL of repositories in Nexus. String restUrl - URL of Rest API. Methods: init(): set all the properties of the Nexus object. Example: `context.nexus = new Nexus(context.job, context.platform, this) context.nexus.init()`
Sonar(Job job, Platform platform, Script script) - Class that describes the Sonar tool.	Properties: Script script - Object with type script, in most cases if class created from Jenkins pipelines it is "this". Platform platform - Object of a class Platform(). Job job - Object of a class Job(). String route - External route of the sonar application. Methods: init(): set all the properties of Sonar object. Example: `context.sonar = new Sonar(context.job, context.platform, this) context.sonar.init()`

2.2 Build Pipeline Stages

Each EDP stage implementation has run method that is as input parameter required to pass a context map with different keys. Some stages can implement the logic for several build tools and application types, some of them are specific.

The Build pipeline includes the following default stages: Checkout → Gerrit Checkout → Compile → Get version → Tests → Sonar → Build → Build Docker Image → Push → Git tag.

INFO: To get the full description of every stage, please refer to the EDP Stages Framework section.

2.3 Build Pipeline. How to Redefine or Extend EDP Pipeline Stages Library

Inspect the points below to redefine or extend the EDP Pipeline Stages Library:

Create a “stage” folder in the App repository.
Create a Groovy file with a meaningful name for the custom stage description. For instance – CustomBuildMavenApplication.groovy
Describe stage logic.

Redefinition:

import com.epam.edp.stages.ProjectType
import com.epam.edp.stages.Stage
@Stage(name = "compile", buildTool = "maven", type = ProjectType.APPLICATION)
class CustomBuildMavenApplication {
    Script script
    void run(context) {
        script.sh "echo 'Your custom logic of the stage'"
    }
}
return CustomBuildMavenApplication

Extension:

import com.epam.edp.stages.ProjectType
import com.epam.edp.stages.Stage
@Stage(name = "new-stage", buildTool = "maven", type = ProjectType.APPLICATION)
class NewStageMavenApplication {
    Script script
    void run(context) {
        script.sh "echo 'Your custom logic of the stage'"
    }
}
return NewStageMavenApplication

2.4 Build Pipeline. Using EDP Stages Library in the Pipeline

In order to use the EDP stages, the created pipeline should fit some requirements, that`s why a developer has to do the following:

import library - @Library(['edp-library-stages']) _
import StageFactory class - import com.epam.edp.stages.StageFactory
define context Map – context = [:]
define stagesFactory instance and load EDP stages: context.factory = new StageFactory(script: this) context.factory.loadEdpStages().each() { context.factory.add(it) }

After that, there is the ability to run any EDP stage beforehand by defining a requirement context context.factory.getStage("checkout","maven","application").run(context)

For instance, the pipeline can look like:

``@Library(['edp-library-stages']) _
 
import com.epam.edp.stages.StageFactory
import org.apache.commons.lang.RandomStringUtils
 
context = [:]
 
node('maven') {
    context.workDir = new File("/tmp/${RandomStringUtils.random(10, true, true)}")
    context.workDir.deleteDir()
 
    context.factory = new StageFactory(script: this)
    context.factory.loadEdpStages().each() { context.factory.add(it) }
   
    context.gerrit = [:]
    context.application = [:]
    context.application.config = [:]
    context.buildTool = [:]
    context.nexus = [:]
   
 
   
    stage("checkout") {
        context.gerrit.branch = "master"
                context.gerrit.credentialsId = "jenkins"
                context.application.config.cloneUrl = "ssh://jenkins@gerrit:32092/sit-718-cloned-java-maven-project"
                context.factory.getStage("checkout","maven","application").run(context)
    }
 
 
    stage("compile") {
        context.buildTool.command = "mvn"
                context.nexus.credentialsId = "nexus"
                context.factory.getStage("compile","maven","application").run(context)
    }
}

Or in a declarative way:

``@Library(['edp-library-stages']) _
 
import com.epam.edp.stages.StageFactory
import org.apache.commons.lang.RandomStringUtils
 
context = [:]
 
pipeline {
    agent { label 'maven' }
    stages {
        stage('Init'){
            steps {
                script {
                    context.workDir = new File("/tmp/${RandomStringUtils.random(10, true, true)}")
                    context.workDir.deleteDir()
                    context.factory = new StageFactory(script: this)
                    context.factory.loadEdpStages().each() { context.factory.add(it) }
                                                           
                    context.gerrit = [:]
                    context.application = [:]
                    context.application.config = [:]
                    context.buildTool = [:]
                    context.nexus = [:]
                }
            }
        }
                       
        stage("Checkout") {
            steps {
                 script {
                    context.gerrit.branch = "master"
                    context.gerrit.credentialsId = "jenkins"
                    context.application.config.cloneUrl = "ssh://jenkins@gerrit:32092/sit-718-cloned-java-maven-project"
                    context.factory.getStage("checkout","maven","application").run(context)
                    }
            }
        }
                       
        stage('Compile') {
            steps {
                 script {
                    context.buildTool.command = "mvn"
                    context.nexus.credentialsId = "nexus"
                    context.factory.getStage("compile","maven","application").run(context)
                    }
            }
        }
    }
}

3. EDP Library Stages Description

Using in pipelines - @Library(['edp-library-stages@version']) _

The corresponding enums, classes, interfaces and their methods can be used separately from the EDP Stages library function (please refer to Table 5).

Table 5. Enums and Classes with the respective properties, methods, and examples.

Enums Classes

ProjectType:

- APPLICATION
- AUTOTESTS
- LIBRARY StageFactory() - Class that contains methods getting an implementation of the particular stage either EDP
from shared library or custom from application repository.

Properties:

Script script - Object with type script, in most cases if class created from Jenkins pipelines it is "this".

Map stages - Map of stages implementations.

Methods:

loadEdpStages(): return a list of Classes that describes EDP stages implementations.

loadCustomStages(String directory): return a list of Classes that describes EDP custom stages from application
repository from "directory". The "directory" should have an absolute path to files with classes of custom stages
implementations. Should be run from slave agent.

add(Class clazz): register class for some particular stage in stages map of StageFactory class.

getStage(String name, String buildTool, String type): return an object of the class for a particular stage
from stages property based on stage name and buildTool, type of application.

Example:

context.factory = new StageFactory(script: this)
context.factory.loadEdpStages().each() { context.factory.add(it) }
context.factory.loadCustomStages("${context.workDir}/stages").each() { context.factory.add(it) }
context.factory.getStage(stageName.toLowerCase(),context.application.config.build_tool.toLowerCase(),
context.application.config.type).run(context)

4. EDP Stages Framework

Each EDP stage implementation has run method that is as input parameter required to pass a context map with different keys. Some stages can implement the logic for several build tools and application types, some of them are specific.

Inspect the Table 6 and Table 7 that contain the full description of every stage that can be included in Code Review and Build pipelines: Checkout → Gerrit Checkout → Compile → Get version → Tests → Sonar → Build → Build Docker Image → Push → Git tag.

Table 6. The Checkout, Gerrit Checkout, Compile, Get version, and Tests stages description.

Checkout	Gerrit Checkout	Compile	Get version	Tests
name = "checkout", buildTool = ["maven", "npm", "dotnet","gradle"] type = [ProjectType.APPLICATION] context required: - String context.workDir - StageFactory context.factory - String context.gerrit.branch - String context.gerrit.credentialsId - String context.application.config.cloneUrl	name = "gerrit-checkout", buildTool = ["maven", "npm", "dotnet","gradle"] type = [ProjectType.APPLICATION, ProjectType.AUTOTESTS, ProjectType.LIBRARY] context required: - String context.workDir - StageFactory context.factory - String context.gerrit.changeName - String context.gerrit.refspecName - String context.gerrit.credentialsId - String context.application.config.cloneUrl	name = "compile" buildTool = ["dotnet"] type = [ProjectType.APPLICATION] context required: - String context.workDir - String context.buildTool.sln_filename output: - String context.buildTool.sln_filename buildTool = ["gradle"] type = [ProjectType.APPLICATION] context required: - String context.workDir - String context.buildTool.command - String context.nexus.credentialsId buildTool = ["maven"] type = [ProjectType.APPLICATION] context required: - String context.workDir - String context.buildTool.command - String context.nexus.credentialsId buildTool = ["npm"] type = [ProjectType.APPLICATION] context required: - String context.workDir - String context.nexus.credentialsId - String context.buildTool.groupRepository	name = "get-version" buildTool = ["dotnet"] type = [ProjectType.APPLICATION] context required: - String context.workDir - Map(empty) context.application - String context.gerrit.branch - Job context.job output: -String context.application.deplyableModule - String context.application.deplyableModuleDir - String context.application.version - String context.application.buildVersion buildTool = ["gradle"] type = [ProjectType.APPLICATION] context required: - String context.workDir - String context.nexus.credentialsId - String context.buildTool.command - Job context.job - String context.gerrit.branch output: - String context.application.deplyableModuleDir - String context.application.version - String context.application.buildVersion buildTool = ["maven"] type = [ProjectType.APPLICATION] context required: - String context.workDir - String context.nexus.credentialsId - String context.buildTool.command - Job context.job - String context.gerrit.branch output: - String context.application.deplyableModule - String context.application.deplyableModuleDir - String context.application.version - String context.application.buildVersion buildTool = ["npm"] type = [ProjectType.APPLICATION] context required: - String context.workDir - Job context.job - String context.gerrit.branch output: - String context.application.deplyableModuleDir - String context.application.version - String context.application.buildVersion	name = "tests" buildTool = ["dotnet"] type = [ProjectType.APPLICATION] context required: - String context.workDir buildTool = ["gradle"] type = [ProjectType.APPLICATION] context required: - String context.workDir - String context.nexus.credentialsId - String context.buildTool.command buildTool = ["maven"] type = [ProjectType.APPLICATION] context required: - String context.workDir - String context.buildTool.command type = [ProjectType.AUTOTESTS] context required: - String context.workDir - String context.buildTool.command - String context.application.config.report_framework buildTool = ["npm"] type = [ProjectType.APPLICATION] context required: - String context.workDir

Table 7. The Sonar, Build, Build Docker Image, Push, and Git tag stages description.

Sonar	Build	Build Docker Image	Push	Git tag
name = "sonar" buildTool = ["dotnet"] type = [ProjectType.APPLICATION] context required: - String context.workDir - String context.job.type - String context.application.name - String context.buildTool.sln_filename - String context.sonar.route - String context.gerrit.changeName(Only for codereview pipeline) - String context.gerrit.branch(Only for build pipeline) buildTool = ["gradle"] type = [ProjectType.APPLICATION] context required: - String context.workDir - String context.job.type - String context.nexus.credentialsId - String context.buildTool.command - String context.application.name - String context.sonarRoute - String context.gerrit.changeName(Only for codereview pipeline) - String context.gerrit.branch(Only for build pipeline) buildTool = ["maven"] type = [ProjectType.APPLICATION, ProjectType.AUTOTESTS, ProjectType.LIBRARY] context required: - String context.workDir - String context.job.type - String context.nexus.credentialsId - String context.application.name - String context.buildTool.command - String context.sonar.route - String context.gerrit.changeName(Only for codereview pipeline) - String context.gerrit.branch(Only for build pipeline) buildTool = ["npm"] type = [ProjectType.APPLICATION] context required: - String context.workDir - String context.job.type - String context.sonar.route - String context.application.name - String context.gerrit.changeName(Only for codereview pipeline) - String context.gerrit.branch(Only for build pipeline)	name = "build" buildTool = ["gradle"] type = [ProjectType.APPLICATION] context required: - String context.workDir - String context.buildTool.command - String context.nexus.credentialsId buildTool = ["maven"] type = [ProjectType.APPLICATION] context required: - String context.workDir - String context.buildTool.command - String context.nexus.credentialsId buildTool = ["npm"] type = [ProjectType.APPLICATION] context required: - String context.workDir - String context.nexus.credentialsId - String context.buildTool.groupRepository	name = "build-image" buildTool = ["dotnet"] type = [ProjectType.APPLICATION] context required: - String context.workDir - String context.application.deployableModule - String context.application.deployableModuleDir - String context.application.name - String context.application.config.language - String context.application.buildVersion - Boolean context.job.promoteImages - String context.job.envToPromote buildTool = ["gradle"] type = [ProjectType.APPLICATION] context required: - String context.workDir - String context.application.deployableModule - String context.application.deployableModuleDir - String context.application.name - String context.application.config.language - String context.application.buildVersion - Boolean context.job.promoteImages - String context.job.envToPromote buildTool = ["maven"] type = [ProjectType.APPLICATION] context required: - String context.workDir - String context.application.deployableModule - String context.application.deployableModuleDir - String context.application.name - String context.application.config.language - String context.application.buildVersion - Boolean context.job.promoteImages - String context.job.envToPromote buildTool = ["npm"] type = [ProjectType.APPLICATION] context required: - String context.workDir - String context.application.deployableModule - String context.application.deployableModuleDir - String context.application.name - String context.application.config.language - String context.application.buildVersion - Boolean context.job.promoteImages - String context.job.envToPromote	name = "push" buildTool = ["dotnet"] type = [ProjectType.APPLICATION] context required: - String context.workDir - String context.gerrit.project - String context.buildTool.sln_filename - String context.buildTool.snugetApiKey - String context.buildTool.hostedRepository buildTool = ["gradle"] type = [ProjectType.APPLICATION] context required: - String context.workDir - String context.nexus.credentialsId - String context.application.version - String context.buildTool.hostedRepository - String context. buildTool.settings buildTool = ["maven"] type = [ProjectType.APPLICATION] context required: - String context.workDir - String context.nexus.credentialsId - String context.application.version - String context.buildTool.hostedRepository - String context.buildTool.command buildTool = ["npm"] type = [ProjectType.APPLICATION] context required: - String context.workDir - String context.nexus.credentialsId - String context.buildTool.hostedRepository - String context.gerrit.autouser	name = "git-tag" buildTool = ["maven", "npm", "dotnet","gradle"] type = [ProjectType.APPLICATION] context required: - String context.workDir - String context.gerrit.credentialsId - String context.gerrit.sshPort - String context.gerrit.host - String context.gerrit.autouser - String context.application.buildVersion

5. Deploy Pipeline

Deploy() – a function that allows using the EDP implementation for the deploy pipeline. All values of different parameters that are used during the pipeline execution are stored in Map "context".

The deploy pipeline consists of several steps:

On the master:

Initialization of all objects (Platform, Job, Gerrit, Nexus, StageFactory) and loading the default implementations of EDP stages
Creating an environment if it doesn`t exist
Deploying the last versions of the applications
Run predefined manual gates

On a particular autotest slave that depends on the build tool:

Creating workdir for autotest sources
Run predefined autotests

5.1 Deploy Pipeline. EDP Library Pipelines Description

Using in pipelines - @Library(['edp-library-pipelines@version']) _

The corresponding enums and interfaces with their methods can be used separately from the EDP Pipelines library function (please refer to Table 8 and Table 9).

Table 8. Enums and Interfaces with the respective properties, methods, and examples.

Enums Interfaces

PlatformType:

- OPENSHIFT
- KUBERNETES

JobType:
- CODEREVIEW
- BUILD
- DEPLOY

BuildToolType:
- MAVEN
- GRADLE
- NPM
- DOTNET Platform() - contains methods for working with platform CLI.
At the moment only OpenShift is supported.

Properties:

Script script - Object with type script, in most cases if class
created from Jenkins pipelines it is "this".

Methods:

getJsonPathValue(String k8s_kind, String k8s_kind_name,
String jsonPath): return String value of specific parameter
of particular object using jsonPath utility.

Example:

context.platform.getJsonPathValue("cm","project-settings",
".data.username")

BuildTool() - contains methods for working with different buildTool
from ENUM BuildToolType.
(Should be invoked on slave build agents)

Properties:

Script script - Object with type script, in most cases if class
created from Jenkins pipelines it is "this".

Nexus object - Object of class Nexus.

Methods:

init: return parameters of buildTool that are needed for
running stages.

Example:

context.buildTool = new BuildToolFactory().getBuildToolImpl
(context.application.config.build_tool, this, context.nexus)
context.buildTool.init()

Table 9. Classes with the respective properties, methods, and examples.

Classes	Description (properties, methods, and examples)
PlatformFactory() - Class that contains methods getting implementation of CLI of platform. At the moment OpenShift and Kubernetes are supported.	Methods: getPlatformImpl(PlatformType platform, Script script): return Class Platform Example: `context.platform = new PlatformFactory().getPlatformImpl(PlatformType.OPENSHIFT, this)`
Application(String name, Platform platform, Script script) - Class that describe the application object.	Properties: Script script - Object with type script, in most cases if class created from Jenkins pipelines it is "this". Platform platform - Object of a class Platform() String name - Name for the application for creating object Map config - Map of configuration settings for particular application that is loaded from config map project-settings String version - Application version, initially empty. Is set on get-version step. String deployableModule - The name of deployable module for multi module applications, initially empty. String buildVersion - Version of built artifact, contains build number of Job initially empty String deployableModuleDir - The name of deployable module directory for multi module applications, initially empty. Array imageBuildArgs - List of arguments for building application Docker image Methods: setConfig(String gerrit_autouser, String gerrit_host, String gerrit_sshPort, String gerrit_project): set the config property with values from config map Example: `context.application = new Application(context.job, context.gerrit.project, context.platform, this)` `context.application.setConfig(context.gerrit.autouser, context.gerrit.host, context.gerrit.sshPort, context.gerrit.project)`
Job(type: JobType.value, platform: Platform, script: Script) - Class that describe the Gerrit tool.	Properties: Script script - Object with type script, in most cases if class created from Jenkins pipelines it is "this" Platform platform - Object of a class Platform(). JobType.value type. String deployTemplatesDirectory - The name of the directory in application repository, where deploy templates are located. Can be set for particular Job through DEPLOY_TEMPLATES_DIRECTORY parameter. String edpName - The name of the EDP Project. Map stages - Contains all stages in JSON format that is retrieved from Jenkins job env variable. String envToPromote - The name of the environment for promoting images. Boolean promoteImages - Defines whether images should be promoted or not. Methods: getParameterValue(String parameter, String defaultValue = null): return parameter of ENV variable of Jenkins job. init(): set all the properties of Job object. setDisplayName(String displayName): set display name of the Jenkins job. setDescription(String description, Boolean addDescription = false): set new or add to existing description of the Jenkins job. printDebugInfo(Map context): print context info to log of Jenkins job. runStage(String stage_name, Map context): run the particular stage according to its name. Example: `context.job = new Job(JobType.DEPLOY.value, context.platform, this)` `context.job.init()` `context.job.printDebugInfo(context)` `context.job.setDisplayName("test")` `context.job.setDescription("Name: ${context.application.config.name}")`
Gerrit(Job job, Platform platform, Script script) - Class that describe the Gerrit tool.	Properties: Script script - Object with type script, in most cases if class created from Jenkins pipelines it is "this". Platform platform - Object of a class Platform(). Job job - Object of a class Job(). String credentialsId - Credential Id in Jenkins for Gerrit. String autouser - Username of autouser in Gerrit for integration with Jenkins. String host - Gerrit host. String project - project name of built application. String branch - branch to build application from. String changeNumber - change number of Gerrit commit. String changeName - change name of Gerrit commit. String refspecName - refspecName of Gerrit commit. String sshPort - gerrit ssh port number. String patchsetNumber - patchsetNumber of Gerrit commit. Methods: init(): set all the properties of Gerrit object. Example: `context.gerrit = new Gerrit(context.job, context.platform, this)` `context.gerrit.init()`.
Nexus(Job job, Platform platform, Script script) - Class that describe the Nexus tool.	Properties: Script script - Object with type script, in most cases if class created from Jenkins pipelines it is "this". Platform platform - Object of a class Platform(). Job job - Object of a class Job(). String autouser - Username of autouser in Nexus for integration with Jenkins. String credentialsId - Credential Id in Jenkins for Nexus. String host - Nexus host. String port - Nexus http(s) port. String repositoriesUrl - Base URL of repositories in Nexus. String restUrl - URL of Rest API. Methods: init(): set all the properties of Nexus object. Example: `context.nexus = new Nexus(context.job, context.platform, this)` `context.nexus.init()`.

5.2 Deploy Pipeline. EDP Library Stages Description

Using in pipelines - @Library(['edp-library-stages@version']) _

The corresponding classes with methods can be used separately from the EDP Pipelines library function (please refer to Table 10).

Table 10. Classes with the respective properties, methods, and examples.

Classes Description (properties, methods, and examples)

StageFactory() - Class that contains methods getting implementation of particular stage either EDP from shared library or custom from application repository. Properties:

Script script - Object with type script, in most cases if class created from Jenkins pipelines it is "this"

Map stages - Map of stages implementations

Methods:

loadEdpStages(): return list of Classes that describes EDP stages implementations

loadCustomStages(String directory): return list of Classes that describes EDP custom stages from
application repository from "directory". The "directory" should be absolute path to files
with classes of custom stages implementations. Should be run from slave agent.

add(Class clazz): register class for some particular stage in stages map of StageFactory class

getStage(String name, String buildTool, String type): return object of the class for particular stage
from stages property based on stage name and buildTool, type of application

Example:

context.factory = new StageFactory(script: this)
context.factory.loadEdpStages().each() { context.factory.add(it) }
context.factory.loadCustomStages("${context.workDir}/stages").each() { context.factory.add(it) }
context.factory.getStage(stageName.toLowerCase(),context.application.config.build_tool.toLowerCase(),
context.application.config.type).run(context).

5.3 Deploy Pipeline Stages

Each EDP stage implementation has run method that is as input parameter required to pass a context map with different keys. Some stages can implement the logic for several build tools and application types, some of them are specific.

The stages for the deploy pipeline are independent of the build tool and application type. Find below (see Table 11 ) the full description of every stage: Deploy → Automated tests → Promote Images.

Table 11. The Deploy, Automated tests, and Promote Images stages description.

Deploy	Automated tests	Promote Images
name = "deploy" buildTool = null type = null context required: • String context.workDir • StageFactory context.factory • String context.gerrit.autouser • String context.gerrit.host • String context.application.config.cloneUrl • String context.jenkins.token • String context.job.edpName • String context.job.buildUrl • String context.job.jenkinsUrl • String context.job.metaProject • List context.job.applicationsList [['name':'application1_name','version':'application1_version],...] • String context.job.deployTemplatesDirectory output: • List context.job.updatedApplicaions [['name':'application1_name','version':'application1_version],...]	name = "automation-tests", buildTool = null, type = null context required: - String context.workDir - StageFactory context.factory - String context.gerrit.credentialsId - String context.autotest.config.cloneUrl - String context.autotest.name - String context.job.stageWithoutPrefixName - String context.buildTool.settings - String context.autotest.config.report_framework	name = "promote-images" buildTool = null type = null context required: - String context.workDir - String context.buildTool.sln_filename - List context.job.updatedApplicaions [['name':'application1_name','version':'application1_version],...]

5.4 Deploy Pipeline. How to Redefine or Extend EDP Pipeline Stages Library

INFO: Currently, the redefinition of Deploy pipeline stages is prohibited.

5.5 Deploy Pipeline. Using EDP Library Stages in the Pipeline

In order to use the EDP stages, the created pipeline should fit some requirements, that`s why a developer has to do the following:

import libraries - @Library(['edp-library-stages', 'edp-library-pipelines']) _
import reference EDP classes(See example below)
define context Map – context = [:]
define reference "init" stage

After that, there is the ability to run any EDP stage beforehand by defining requirement context context.job.runStage("Deploy", context).

For instance, the pipeline can look like:

@Library(['edp-library-stages', 'edp-library-pipelines']) _

@Library(['edp-library-stages', 'edp-library-pipelines']) _

import com.epam.edp.stages.StageFactory
import com.epam.edp.platform.PlatformFactory
import com.epam.edp.platform.PlatformType
import com.epam.edp.JobType

context = [:]

node('master') {
	stage("Init") {
		context.platform = new PlatformFactory().getPlatformImpl(PlatformType.OPENSHIFT, this)
		context.job = new com.epam.edp.Job(JobType.DEPLOY.value, context.platform, this)
		context.job.init()
		context.job.initDeployJob()
		println("[JENKINS][DEBUG] Created object job with type - ${context.job.type}")

		context.nexus = new com.epam.edp.Nexus(context.job, context.platform, this)
		context.nexus.init()

		context.jenkins = new com.epam.edp.Jenkins(context.job, context.platform, this)
		context.jenkins.init()

		context.gerrit = new com.epam.edp.Gerrit(context.job, context.platform, this)
		context.gerrit.init()

		context.factory = new StageFactory(script: this)
		context.factory.loadEdpStages().each() { context.factory.add(it) }

		context.environment = new com.epam.edp.Environment(context.job.deployProject, context.platform, this)
		context.job.printDebugInfo(context)
		context.job.setDisplayName("${currentBuild.displayName}-${context.job.deployProject}")

		context.job.generateInputDataForDeployJob()
	}

	stage("Pre Deploy Custom stage") {
		println("Some custom pre deploy logic")
	}

	context.job.runStage("Deploy", context)

	stage("Post Deploy Custom stage") {
		println("Some custom post deploy logic")
	}
}

Or in a declarative way:

@Library(['edp-library-stages', 'edp-library-pipelines']) _

import com.epam.edp.stages.StageFactory
import com.epam.edp.platform.PlatformFactory
import com.epam.edp.platform.PlatformType
import com.epam.edp.JobType

context = [:]

pipeline {
	agent { label 'master'}
	stages {
		stage('Init') {
			steps {
				script {
					context.platform = new PlatformFactory().getPlatformImpl(PlatformType.OPENSHIFT, this)
					context.job = new com.epam.edp.Job(JobType.DEPLOY.value, context.platform, this)
					context.job.init()
					context.job.initDeployJob()
					println("[JENKINS][DEBUG] Created object job with type - ${context.job.type}")

					context.nexus = new com.epam.edp.Nexus(context.job, context.platform, this)
					context.nexus.init()

					context.jenkins = new com.epam.edp.Jenkins(context.job, context.platform, this)
					context.jenkins.init()

					context.gerrit = new com.epam.edp.Gerrit(context.job, context.platform, this)
					context.gerrit.init()

					context.factory = new StageFactory(script: this)
					context.factory.loadEdpStages().each() { context.factory.add(it) }

					context.environment = new com.epam.edp.Environment(context.job.deployProject, context.platform, this)
					context.job.printDebugInfo(context)
					context.job.setDisplayName("${currentBuild.displayName}-${context.job.deployProject}")

					context.job.generateInputDataForDeployJob()
				}
			}
		}
		stage('Deploy') {
			steps {
				script {
					context.factory.getStage("deploy").run(context)
				}
			}
		}

		stage('Custom stage') {
			steps {
				println("Some custom logic")
			}
		}
	}
}

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

edp_pipeline_framework.md

edp_pipeline_framework.md

EDP Pipeline Framework

Overview

EDP Pipelines and Stages

1. Code Review Pipeline

2. Build Pipeline

3. EDP Stages Library Description

4. EDP Stages Framework

5. Deploy Pipeline

1. Code Review Pipeline

1.1 Code Review Pipeline. EDP Pipelines Library Description

1.2 Code Review Pipeline Stages

1.3 Code Review Pipeline. How to Redefine or Extend the EDP Pipeline Stages Library

1.4 Code Review Pipeline. Using EDP Stages Library in the Pipeline

2. Build Pipeline

2.1 Build Pipeline. EDP Pipelines Library Description

2.2 Build Pipeline Stages

2.3 Build Pipeline. How to Redefine or Extend EDP Pipeline Stages Library

2.4 Build Pipeline. Using EDP Stages Library in the Pipeline

3. EDP Library Stages Description

4. EDP Stages Framework

5. Deploy Pipeline

5.1 Deploy Pipeline. EDP Library Pipelines Description

5.2 Deploy Pipeline. EDP Library Stages Description

5.3 Deploy Pipeline Stages

5.4 Deploy Pipeline. How to Redefine or Extend EDP Pipeline Stages Library

5.5 Deploy Pipeline. Using EDP Library Stages in the Pipeline

Files

edp_pipeline_framework.md

Latest commit

History

edp_pipeline_framework.md

File metadata and controls

EDP Pipeline Framework

Overview

EDP Pipelines and Stages

1. Code Review Pipeline

2. Build Pipeline

3. EDP Stages Library Description

4. EDP Stages Framework

5. Deploy Pipeline

1. Code Review Pipeline

1.1 Code Review Pipeline. EDP Pipelines Library Description

1.2 Code Review Pipeline Stages

1.3 Code Review Pipeline. How to Redefine or Extend the EDP Pipeline Stages Library

1.4 Code Review Pipeline. Using EDP Stages Library in the Pipeline

2. Build Pipeline

2.1 Build Pipeline. EDP Pipelines Library Description

2.2 Build Pipeline Stages

2.3 Build Pipeline. How to Redefine or Extend EDP Pipeline Stages Library

2.4 Build Pipeline. Using EDP Stages Library in the Pipeline

3. EDP Library Stages Description

4. EDP Stages Framework

5. Deploy Pipeline

5.1 Deploy Pipeline. EDP Library Pipelines Description

5.2 Deploy Pipeline. EDP Library Stages Description

5.3 Deploy Pipeline Stages

5.4 Deploy Pipeline. How to Redefine or Extend EDP Pipeline Stages Library

5.5 Deploy Pipeline. Using EDP Library Stages in the Pipeline