Soft Assertions in Selenium using TestNG

Assertions are used to perform various kinds of validations in the tests and help us to decide whether the test has passed or failed. There are two types of assertions in Selenium that we can place in our test scripts using TestNG:

1. Hard Assertions
2. Soft Assertions

Hard Assertions: As the name suggests, these assertions put a strict restriction on the test script in which it is placed. When using hard assertions in the test scripts, your test script will stop executing when the assertion fails, and the test will be failed in TestNG report.

We will discuss following scenarios to understand hard assertions:

Scenario 1: Let’s assume you have a test case with multiple validations using hard assertions and all of them are passed. Here is the code:

Now, let us analyze the output of the above code:

As seen above, the console output contains the text “Line executed after assertion passed” after all the assertions have passed and the test is marked as passed as expected.

Scenario 2: Now let’s analyze the scenario when one of the hard assertions fails:

As seen above, the text is not printed, and the code following the assertion is never executed after assertion failure.

Soft Assertions: If you need to execute the remaining test case even after an assertion fails, and you also want to report assertion and test case failure in TestNG report, TestNG provides soft assertions for this. Unlike hard assertions, they do not bring the entire program to a halt, i.e. a test script does not stop running even if an assertion fails, but the test itself is failed to indicate the right result.

The softAssert.assertAll() method does the trick. This method collates all the failures and decides whether to pass the test case or not at the end. TestNG library itself offers the facility to perform Soft Assertions in your test without writing any custom logic. We will take help to understand how soft assertions are used in TestNG with the code snippets mentioned below:

Scenario 1: When one of the soft assertions fails in a single test script:

As you can see, the test is failed, and the text is successfully printed on console even after a previous assertion has failed thus ensuring complete test execution.

Scenario 2: Soft Assertions with multiple test cases. When using soft assertions with multiple test cases the result is not comprehensive if you end up using the same object of Soft Assert class in multiple test cases as shown below:

The same object ‘softAssert’ is used with multiple tests in the above case. The expected output is ‘SoftAssertionTest’ test should pass (as all assertions have passed) and ‘SoftAssertionSecondTest’ test should fail as one of its assertions has failed.

But since the object is same in both test cases, both test cases will fail. WHY? The assertAll() method when collating assertion failures uses the Soft Assert class object. It will collate all the assertion failures for the same object at a single time.

In the above case, since the same object is used in different test cases when the control executes assertAll () method of the ‘SoftAssertionSecondTest’ test case, it will evaluate all the above four assertions (even if they are in different test cases) at the same time. As a result, all the test cases using this same object of Soft Assert class will fail as shown below:

Solution: Use different objects of ‘Soft Assert’ class in different test cases (best practice) as shown below:

This coding pattern will restrict assertAll() method assertions failure collations to only current test method thus giving expected result as below:

Using soft assertions in your test scripts is a good practice and an effective way of handling your test execution. However, if you are writing test cases for config, then placing hard assertions should be the way. I hope this blog provides you a solution.

That’s it folks Cheerio! J