Context: The API test suite had been running well on Jenkins, suddenly one day the Jenkins build was failed and report was not sending out. It turned out that one of framework dependency, in particular, “azure” had been released new version and it is not compatible with the current test script. That is the time I realized how important it is to control the dependencies version. If we don’t control them, we will pay for it later.
In this post, I would like to share some ideas how to control the version of dependencies properly.
I. Select dependencies version when creating an Automation Framework
There are many things you need to consider when creating a new automation framework from scratch: selecting the language binding, the framework type, the tools/libraries combination in the framework… and selecting the specific version for each dependency is also very important part that need to be considered carefully. A different version usually has a different set of features. It will impact the capability of the framework or may change the way you write tests.
For example, when I was creating Cucumber framework, I selected Cucumber JVM version 4.2.3 and TestNG 6.14.3. Because this combination allows running test in parallel with scenario level. Older Cucumber JVM does not have this ability. I see many people still keep using Cucumber JVM 1.2.5 which released in 2016 and I don’t really know what is the reason. But I suggest you use recent versions which has more features, enhancement and better performance.
II. Specify the dependencies version and limit the highest version framework can download.
This is lesson learn I got from my pain that I experienced. Please DO specify the exact version that dependency can be downloaded, or at least give the range of version number that you know it is safe to use. For example, If you are sure that your script works well with Cucumber version between 4.2.3 and 4.4.7, you can define it like below if using Maven POM file :
<dependency>
<groupId>io.cucumber</groupId>
<artifactId>cucumber-java8</artifactId>
<version>[4.2.3,4.4.7]</version>
</dependency>
DO NOT allow the dependency management download the latest version like this:
The reason is that if you don’t limit the version, it will always try to get the new available version. You may get a new major release which very likely will no longer compatible with your scripts and cause all your test broken. You will not know when new major version is released and when your test will be impacted. It is out of your control.
III. Be careful when upgrading dependencies to new version
Like I mentioned, each version of a dependency will have different set of features. And a major release usually has some breaking changes that not compatible with previous version. If you upgrade to a major version, it is very likely that all your tests will be impacted.
I was trying to upgrade Cucumber from 4.2.3 to 4.5.x and It made all my existing tests broken. I thought 4.5 is just a minor release, 5.0 would be next major release and 4.5 would not break anything. However, I was wrong. In version 4.5.0 they refactored some core API, moving them to other package and my test scripts are not compatible with this release.
Therefore, before deciding to upgrade the framework dependencies, you should check the release note, read the change log carefully and try to run all tests with the new version to make sure those changes of new version will not impact anything.
Step Up Automation 