Jenkins is the way to test software packages overnight
Technology Package Testing with Jenkins
Parallel execution helped reduce build and testing time from 3 days to overnight.
Background: We had been testing technology package code but it was done manually and took around 2 to 3 days to achieve. We searched for the best technology to automate the continuous Integration Testing, we came across Jenkins.
The challenges we faced were many. First, the project was quite helpful for Jenkins users to test their technology package being developed, but there was a need to improve on several aspects due to the utility code written in Python for creating a new job or updating an existing job. These are cited below:
- Performance - Overnight results were expected from Continuous Integration setup using Jenkins
- Reliability - Results should be reliable enough to take further decisions on the package being tested
- Stability - Execution status should be stable between different runs
- Duration - We wanted testing to do more in less time, which was achieved using the Jenkins parallelization concept
- Confidence - Last but not least, we were hoping to build confidence in using Jenkins for management, users, and other stakeholders
Goals: The main goal of this project was to test every new technology package automatically overnight. But we were also interested in:
Users and stakeholders were not initially interested in using Jenkins due to utility code implementation required to create and update test cases. But by using the Jenkins parallelization concept, we were able to achieve faster releases with no performance, reliability, or stability issues. Management then took the decision to rollout Jenkins in a big way across the teams.
- time efficiency
- easy-to-trace build and debug code issues
- less effort and complexity
- easy-to-add test cases in one place
- easy-to-manage project configurations
Solution & Results: Using Jenkins, we build a parametrized project that can be used by variant teams. They specify the version of the application they want to develop, and Jenkins takes responsibility to create the technology package code and does the necessary configurations automatically. The customer receives a notification when the new fix has been released and can provide testing without human interaction.
Prior to using Jenkins, testing was carried manually by individual teams. After incorporating Jenkins, we have a centralized place to trigger and monitor the results. And Jenkins maintenance is done by the software 'teams.
Out of all the great results we've seen, these are the top four:
- Our build and testing time is reduced from 3 days to overnight
- We achieved automated continuous integration of technology package testing
- We achieved parallel execution with the Jenkins Parallellization concept
- We achieved stability between different releases