Playwright (Nodejs) Custom Reports: A Deep Dive into Test Pass/Fail Analytics for Test Outcomes

We’ll explore Playwright with Node.js Custom Test Reports and, with the help of sample code, demonstrate how these reports can be your key to unlocking valuable insights from test results, ultimately enhancing the quality of your testing outcomes.

Why Test Pass/Fail Analytics Matter

Before we dive into the practical aspects, let’s understand the significance of test pass/fail analytics. In automated testing, a test’s outcome is categorized as either “pass” (successful execution) or “fail” (an issue that occurred during execution and can be timeout or skipped ). These verdicts offer vital insights that drive software quality improvement.

Introduction to Playwright Custom Reports

Playwright, a versatile and developer-friendly tool, provides custom reporting capabilities that allow you to extract and analyze test data in a structured manner. These reports are much more than just summaries; they are your secret weapon for informed decision-making.

Key Features of Playwright Custom Reports

Detailed Test Logs: Playwright custom reports offer exhaustive logs for every test run, pinpointing the exact step where a failure occurred.
Custom Metrics: Define custom metrics relevant to your application to track specific key performance indicators (KPIs) during testing.

Analysing Test Outcomes with Sample Code

Let’s dive into the practical aspect of using Playwright Custom Reports with some sample code. We’ll outline a simple test scenario to demonstrate the power of these reports.

Create a js file with name - 'customerReportFilePath.js'

"use strict";
Object.defineProperty(exports, "__esModule", { value: true })
import { writeFileSync } from "fs";
var JSONSummaryReporter = /** @class */ (function () {
class JSONSummaryReporter {
           constructor () {
           this.executionDurationInMS = -1;
           this.testStartedAt = 0;
           this.passTests = [];
           this.skipTests = [];
           this.failTests = [];
           this.interruptedTests = [];
           this.timedOutTests = [];
           this.testStatus = 'unknown';
           this.warned = [];
}
onBegin() {
           this.testStartedAt = Date.now();
}
onTestEnd(test, result) {
     let testTitle = [];
     let testFileName = [];
     let clean = true;
         for (let index = 0, a = test.titlePath(); index < a.length; index++) {
            let fileName = a[index];
              if (fileName === '' && clean)
               continue;
              clean = false;
           testTitle.push(fileName);
              if (fileName.includes('spec.js')) {
                  testFileName.push(fileName);
               }
           }
// This will publish the file name + test title + status + + line number test begins on and column
       let testDetails = "".concat(testFileName[0], " : ").concat(test.title, " : ", result.status, " :                               ").concat(test.location.line, " : ").concat(test.location.column)
// Using the t variable in the push will push a full test test name + test description
      let t = testTitle.join(' > ');
      let status = !['passed', 'skipped', 'interrupted'].includes(result.status) && t.includes('@warn')
      ? 'warned'
      : result.status;
      this[status].push(testDetails);
  }
onEnd(result) {
    var _this = this;
    this.executionDurationInMS = Date.now() - this.testStartedAt;
    this.testStatus = result.status;
    // removing duplicate tests from pass tests array
    this.passTests = this.passTests.filter(function (element, index) {
    return _this.passTests.indexOf(element) === index;
});
// removing duplicate and flakey (pass on a retry) tests from the fail tests array
   this.failTests = this.failTests.filter(function (element, index) {
     if (!_this.passTests.includes(element))
        return _this.failTests.indexOf(element) === index;
     });
     writeFileSync(`./summary.json`, JSON.stringify(this, null, ' '));
    }
  }
   return JSONSummaryReporter;
}());
const _default = JSONSummaryReporter;
export { _default as default };

In playright.config file add below line of code  :

         reporter :['./customerReportFilepath.js'] // js file path 

Once the test execution ends, it will generate a json file ‘summary.json’ with the execution results, test pass/fail, and execution duration analytics. Similarly, we can get more test details based on the application requirements.

Result in 'summary.json' file :

{
     "executionDurationInMS": 4157,
      "passTests": [
      "DemoPassTest.spec.js : Demo test : passed : 20 : 9"
       ],
      "skipTests": [
       ],
       "failTests": [
        "DemoFailTest.spec.js : Demo Fail test : failed : 10 : 5"
        ],
       "interruptedTests": [
        ],
        "warned": [],
        "timedOutTests": [],
        "testStatus": "failed",
        "testStartedAt": 169467477831
     }

Conclusion

Playwright Custom Reports provides test outcomes of automated testing. They illuminate the path to test pass/fail insights, helping you navigate toward enhanced testing results. By grasping the importance of test execution analytics and harnessing the capabilities of Playwright Custom Reports, you can elevate your testing strategy.