One of the main roles of Automation testing is to give a fast feedback to the team for the quality of the software. The Automation team can catch failures in unit test, smoke test, regression test, performance test or any automation test execution by monitoring the execution reports and then analyse it to check if any failures, are they software defects or test script issues? This process is not easy and it actually requires a lot of effort.
I have been working in many projects and many times I have seen automation team lost the focus, forgot to check the report, or they did check the report but didn’t analyse the failures properly led to missing the chance to report the software defects to the team early.
Adding Report Pass Rate in automation test report notification is good practice to help the team notice the quality of the software easily and then take actions quickly.
Here is a report notification for nightly regression without the Pass Rate. Automation members then need to open the link, login Jenkins to check the status of regression report. If you do this everyday, you might expect the report is good and ignore it.
With the Pass Rate included, it immediately catch the attention from the team if the pass rate is lower and they know something happened with the build yesterday (nightly regression). Here is how it looks like:
In this post I will share how to configure Teams channel Web Hook and update jenkinsfile to push Allure Report and Email notifications with Pass Rate to that channel.
I. Create incoming Web Hook for Teams channel
- Open the channel in which you want to add the webhook and select ••• from the upper-right corner.
- Select Connectors from the dropdown menu.
- Search for Incoming Webhook and select Add.
- Select Configure, provide a name, and upload an image for your webhook if necessary.
- Copy and save the unique webhook URL present in the dialog. The URL maps to the channel and you can use it to send information to Teams.
- Select Done.
For more details and screenshots, you can find here.
II. Update jenkinsfile
The stage “Publish Allure Report” will create folder named allure-report, inside this folder we can find the summary.json which contains the number of tests passed, failed, total to get the Pass Rate.
Then in post section , we add the Pass Rate into Teams webhook message and email-ext subject.
Done. Next time your automation pipeline is run, it should send the notifications with the Automation Execution Pass Rate.
If you want to push to a Slack channel, it would be done by similar steps using Jenkins Slack Notification Plugin.
- Go to Slack App Directory and search for Jenkins CI
- Click on the Add to Slack button. Select the channel and click on the Add Jenkins CI Integration. click Save setting
- Note down Team Subdomain and Integration Token Credential ID
- Configure Slack notification plugin configuration on Jenkins.
- configure slack notification step in jenkinsfile
Please refer to these links:
Step Up Automation 