Jenkins: building a CI/CD system – part1

This first post will cover the basics of how to connect Jenkins with Github and how to configure both to auto trigger a build upon a pull request (PR) and post back results to GitHub.
While we chose Jenkins and Github, all of the concepts you are going to read here are quite similar between the different tools (bitbucket, TeamCity, GitLab, etc), we only chose Jenkins as it is still being used by most users out there.

Let’s jump right in! Continue reading “Jenkins: building a CI/CD system – part1”

Jenkins: New CLI

Hello. It’s been ages since my last post, sorry for that. Looks like my work and studies took over most of my time. But let’s not dwell in the past and move to the purpose of this post!

I have quite a lot of interaction with Jenkins lately and to be honest, I really don’t like using  the Jenkins web interface. I’m always in favor of using good working CLI.

Unfortunately I couldn’t find any client that was good enough for what I’ve been doing with Jenkins. My requirements are pretty basic – 1. it should work, 2. it should cover enough of the different tasks I’m doing on Jenkins. I have been trying couple of clients, but each was either too basic, missing a lot of commands or either not working at some point.

Continue reading “Jenkins: New CLI”

OpenStack Infra: Jenkins Jobs

A few days ago, while adding a new job to OpenStack Infra, I realized how difficult it must be for newcomers ( to OpenStack) to understand how OpenStack CI works and make new changes. The OpenStack Infra documentation coverage of each project is great and very detailed , but connecting the dots, which  assembles the complete work-flow can be a complex task for anyone.

Hopefully this post can help for those who unfamiliar with OpenStack Infra. This is written in a form of ‘Q & A’. If you read this and find yourself still wondering about additional subjects, please let me know and I’ll make sure to add it here.

Continue reading “OpenStack Infra: Jenkins Jobs”

Jenkins & Gerrit: trigger build on added comment

Sometimes when you submit patch to gerrit, you may have one or several gates running against the patch, verifying it has no issues, so it’s safe to merge it.

But what would you do if want to re-run all gates? usually I see developers using one of the following ways:

  • rebase
  • go to the actual build page in jenkins and click on ‘retrigger all’
  • comment in gerrit system with specific string

Rebase would work only if there is what to rebase on and using directly the build page on jenkins server will only work if you have account with permissions to access this jenkins server – so those two methods would be less convenient to use.

Continue reading “Jenkins & Gerrit: trigger build on added comment”

Tests: How to convert Subnit stream to junitXML

Subunit is a streaming protocol for test results. It is a binary encoding generated while you are running tests. It is also widely used in the Openstack project.

Subunit comes with a lot of useful filters. Some of them are:

  • subunit2gtk
  • subunit2pyunit
  • tap2subunit

I will focus on subunit2junitxml which converts a Subunit stream into a junitXML representation. I recently had to use it for CI as the Jenkins plugin I’m using doesn’t support publishing tests results using pure subunit stream.

First let’s start with installing the needed packages.

For RHEL/CentOS/Fedora:

yum install -y subunit-filters python-junitxml

Now take your subunit stream ( if you don’t know where it is, look in “Q&A” section ) and run the following command:

cat my_subunit_stream | subunit2junitxml > tests_results.xml

This command will use the subunit filters to convert the stream into junitXML.

In order to verify it worked, read the file and see if it’s in the desired format.

You can also simply run:  grep “<testcase classname=”  tests_results.xml

If it didn’t work, you may need first to convert the stream into the newer protocol version 2.  In such case simply run:

cat my_subunit_stream | subunit-1to2 > my_subunit_stream_v2
cat my_subunit_stream_v2 | subunit2junitxml > tests_results.xml

That’s it. Now you have it in junitXML format and if you are using jenkins, you can also use it to publish your tests results.


Q&A

Q: Where can I find the subunit stream files?

A: It may depend on how you run the tests. If you are using ‘tox’ you could find the files in .testrepository directory. The name of the files would be numbered. Those numbers are actually the tests run order. 1 for example, would be the second time you ran the tests.

Q: In jenkins, How do you publish tests results in Junit format?

A: I’ve installed the Junit plugin and in ‘Post-build Actions’ I’m using ‘Publish Junit test result report’