Jenkins: building a CI/CD system – part1

This first post will cover the basics of how to connect Jenkins with Github and how to configure both to auto trigger a build upon a pull request (PR) and post back results to GitHub.
While we chose Jenkins and Github, all of the concepts you are going to read here are quite similar between the different tools (bitbucket, TeamCity, GitLab, etc), we only chose Jenkins as it is still being used by most users out there.

Let’s jump right in!

Step 1 – configuring our GitHub

Our first step in connecting our Jenkins instance to our Github repository is to create a form of communication between them.
Github provides an API for developers so we will leverage that for our Jenkins automation.
Since the API requires a username and a password in order to work, our first step will be to create a personal access token.

In our GitHub account, choose the “Settings” option:

Github-settings-menu

From there, on the left pane menu choose “Developer settings”

Github-developer-settings

And then finally “Personal access tokens“.

Github-personal-access-tokens

From here, press the “Generate new token” button and check:
1. repo.repo:status + repo.repo_deployment + repo.public_repo
2. admin:org_hook
3. admin:repo_hook

Then “Generate token“.

Screen Shot 2018-05-05 at 17.27.02

Note: The “repo:status | deployment | public_repo” is needed if you want Jenkins to be able to post back to GitHub the job run status. If you only need the trigger for the job, you can leave this unchecked.

Once generated, make sure to copy it and keep it in a safe place.

Github-access-token

Next, let’s create a webhook in GitHub.
A webhook is the Jenkins URL endpoint that GitHub should use to deliver the payload once an event is happening (pull request, push, etc).
Let’s go back to our repository and go into the “Settings” tab. on the left side menu press “Webhooks“:

Screen Shot 2018-04-28 at 16.25.09.png

From here, press the “Add webhook” on the upper right side.
Here, you will have to choose between three options:
1. Just the push event – this webhook will only trigger on push events
2. Send me everything – this webhook will trigger on every event in the repo
3. Let me select individual events – customize which event you want to trigger on
The choice here depends largely on what type of jobs you are planning to build. There is no “right or wrong” here, so for the sake of simplicity, I will choose to trigger on everything. (The common use case will be to only choose “pull requests”)

Screen Shot 2018-04-28 at 16.36.44

Step 2 – configuring our Jenkins

Now that we have an access token, let’s update our Jenkins with it.

In the main Jenkins screen, on the left side menu go to “Credentials

Screen Shot 2018-04-28 at 14.52.39

Still, on the left pane go into “System

Screen Shot 2018-04-28 at 14.53.09

And from there into the “Global credentials (unrestricted)

Screen Shot 2018-04-28 at 14.53.32

You should see the following screen:

Screen Shot 2018-04-28 at 14.53.50

Now, let’s tell Jenkins about the credentials it can use to communicate with GitHub.

Let’s start with our personal access token.
On the left side menu, press the “Add Credentials“, and choose “Username with Password“.

Sope should be “Global( Jenkins, nodes, items, all child items, etc)
Username should be the user you created the access token for.
Password should be the access token you copied from GitHub.
ID should be a unique id (you will reference this from jobs in the future)
And lastly, add some description

Screen Shot 2018-05-05 at 16.26.35

If you followed the instructions above, you should see a new entry in Jenkins.

Screen Shot 2018-04-28 at 14.56.45

In order for our Jenkins to communicate with Github, it uses a plugin. Let’s make sure we have this plugin installed.
From the main Jenkins screen, press the “Manage Jenkins” on the left side menu

Screen Shot 2018-04-28 at 14.52.39

Here, look for the “Manage plugins” button and press it.

Screen Shot 2018-04-28 at 15.49.40

Under the “Installed” tab, search for “GitHub Plugin

Screen Shot 2018-04-28 at 15.51.59

If you can’t find it, go into the “Available” tab, search for it and install it.

We have finished connecting our Jenkins and Github account such that they can communicate with one another.
In other words, once we do anything on GitHub, an event will be created and pushed to our Jenkins which in turn will process the event and act on it.

Problem is.. what should the Jenkins do when it receives the notification from GitHub?
For that, we will need to form some kind of a “Job definition” for Jenkins to work with.
In this post, we will work with Jenkins pipelines.

In Jenkins, create a new job by pressing the “New item” button on the upper left menu.
Enter the name of the repository you wish Jenkins to analyze events from and choose the “Multibranch Pipeline” type.

Screen Shot 2018-05-05 at 16.54.43

On the next screen, under “Branch Sources” press the “Add source” and from the drop-down menu choose “Github

Screen Shot 2018-05-05 at 16.56.58

On the new form that will show, choose the credentials you created in the previous step.
Owner should be the name of the GitHub account/organization your repository is under (e.g: given our git url: https://github.com/DevInPractice/learn-jenkins –>
Owner: DevInPractice
Repository: learn-jenkins)
For “Behaviors”, it depends on what you want Jenkins to trigger on.
You can trigger on any branch a developer is pushing.
You can trigger only if a developer has created a pull request (what I marked)
You can trigger on pull requests from forks of the repo

Screen Shot 2018-05-05 at 16.59.35
For the sake of keeping this simple, we will choose to only trigger on pull requests + merge the pull request into the target branch beforehand.
Why do we want to do that? again, it’s a matter of taste, in our company we trust the developers to run all their tests locally and only submitting a pull request once they are “ready for merge”. if we will keep the “Discover branches” then it will trigger for any new branch created for our repository, which might lead to jobs being run even before a developer has finished working on his/her branch.
The second option of “Merging the pull request with the current target branch revision” is checked because we would only want to trigger a build in case there are no conflicts with the target branch (target == the branch we created a pull request to).

Lastly, you will see that the build configuration is being taken from something called a Jenkinsfile.  A Jenkinsfile contains the job definition in a DSL format, we will have a detailed post about Jenkinsfile usage in a future post.

Screen Shot 2018-05-05 at 17.11.34

Note: we will use the “scripted pipeline”, which is a little different than the structured one you see in the documentation, but that is simply a matter of taste, you can easily transform what we write here to the structured version.

That’s it! press “save” and you should see something like:

Screen Shot 2018-05-05 at 17.14.15

Step 3 – Bringing it all together

Let’s test our CI system! given we have never used Jenkins before, let’s create a basic Jenkinsfile and push a new branch called “our-first-PR” to our GitHub repository.

// dev-in-practice/learn-jenkins/Jenkinsfile
node {
stage(‘checkout’) {
checkout scm
}
stage(‘test’) {
println(“Here we will run our unit tests”)
}
}

We will not explain each stage here as this will come in more details later on a dedicated blog post.
Let’s head to our GitHub repository and create a new pull request

Screen Shot 2018-05-05 at 17.22.12

Screen Shot 2018-05-05 at 17.22.35.png

Once you press the “Create pull request” head quickly to our Jenkins and you might notice the job that was triggered:

Screen Shot 2018-05-05 at 17.22.58.png

Screen Shot 2018-05-05 at 17.23.09.png

And if you head back to the pull request page on GitHub, you will see that Jenkins has posted a “SUCCESS” link on the pull request, marking it as safe for merge

Screen Shot 2018-05-05 at 17.26.20.png

Step 4 – Conclusion

Congratulations! you have now built your first CI system 🙂
To summarize the tutorial, we have learned that:
1. In Github, you will need two things: a personal access token and a webhook
2. In Jenkins, you will need two things: Github credentials (username + token) + pipeline job that will monitor your github repository for new pull requests
3. In your source code, you will need to create a Jenkinsfile with a job definition
4. See it all in action 🙂

We hope you enjoyed this guide and that it will be a good source for you in the future.
In our next blog post, we will dive a little more into an actual build of an example project, from building the artifacts, running the unit tests and publishing them to an artifact store.

Stay tuned!

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s