Dev-Test-Ops: The Way Forward for Continuous Integration & Continuous Delivery (CI/CD)

Competition has grown manifold in the last decade with a constant need for faster time to market, superior customer experience and continuous evolution. WIth the modern software economy becoming biased towards innovative products that keep pace with accelerating market demands, technology-led products appeal to early adopters and niche markets who seek the latest technological developments.  Owing to this, many companies adopted the fail first and fast attitude as a part of their software development process. Consequently, the sequential waterfall model gave way to Agile enabling quick turnaround time incorporating iterations at any stage of the project.

Continuous feedback and Continuous Iterations gave birth to Continuous Integration and Continuous Deployment. With CI/CD led processes, companies leveraged:

• Shortened development cycles
• Faster release cycles
• Continuous feedback incorporation at each stage
• Frequent releases
• Accelerated time to market

While development and operations are key stages of a product development process, another significant stage is testing. Earlier, companies followed the building of manual test suites for each feature post development and integrated changes, in case of bugs/ errors. This took about 5-6 days, varying in each case, after the development of a particular feature and then it would be integrated into the build. Testing is done on one side, and once complete, the testers effectively hand the delivery over the fence to the operations team who deploy it into production.

Dev-Test-Ops – What is it?

With the increased adoption of CI/CD, another iterative stage has emerged called Continuous Validation. This stage is focused on executing automated test suites as part of the software delivery pipeline to get immediate feedback on the gaps, bugs, and risks associated with a software release and whether or not it is a showstopper for the entire product. This means involving testing at all the possible stages of the development process and assessing risks and recording the feedback in the software.Continuous testing integrates continuous ability to test the code, which means bridging the gap between the Test teams, Dev teams, and Ops team. Dev-Test-Ops is crucial for a successful CI delivery model, in the following manner:

Continuous Integration: Continuous Integration is a widely adopted model when it comes to software product development and delivery with its primary aim being to minimize integration time to accelerate time-to-market. It can enable delivery of a high-quality software quicker, however, without appropriate automated tests that run fast, have good coverage, and no erroneous results, it’s magnified impact cannot be achieved and there can be no successful Continuous Integration or Continuous Delivery.

Continuous Testing/ Validation: Continuous testing integrates automated test suites for, in most cases, sanity and regression testing and generating automated extent reports for each feature. The extent report, in case of success, enables further integration and validates a successful build. In other cases, it brings out the gaps/bugs/errors or flaws and helps determine if there is a blocker or showstopper for the feature in the build. Automated feedback at each checkpoint acts as an automatic trigger for the next step in the delivery pipeline to initiate if the feedback is to move forward. If the feedback is to not move forward, the development is immediately paused, corrective measures taken, and then the development progresses. Continuous Integration (CI) tools, such as, Jenkins (which executes automated tests upon completion of successful build) enable integration of automated unit, regression and functional tests to run for every successful build.

As and when a feature is added successfully to the build, their checks get added into the test automation suites as a part of the extent report.  

Continuous Delivery: It aims to deliver successful builds into pre-decided target environments (such as QA, UAT, and production) by leveraging automated approach. The deployments can be pre-configured for a predefined time or on-demand as a best practice for agile delivery.

Why is Continuous Testing important in Dev-test-Ops?

Continuous testing reduces bottlenecks by reducing the cost of provisioning and streamlining the process of test environment creation. Implementing continuous testing also helps development balance quality and speed. It contributes a great value to operations as well by enabling them to visualize the created environment functions and how they would support the application, giving the team necessary heads-up to make any required changes. Software products are visible to production-like environments at an earlier stage, preferably at the beginning of the development process allowing for testing and validation of products, earlier, by using a repeatable delivery process.

Dev-Test-Ops ensures that the development or product team involved has access to the required test infrastructure, platforms and frameworks without having to spend a great deal of time on configuration before proceeding with testing. The essential value to be derived from the CI system will be lost if the QA process takes days to complete owing to setup and configuration on the part of the QA team.

Continuous Testing should be integrated into the delivery pipeline using tools like Jenkins, TeamCity, etc. Let’s understand considerations before making a foray into the Dev-Test-Ops Journey:

• Automate the Test Cycle by getting all possible tests automated
• Work with the development team to set up  automated tests (maybe gate test) as criteria to promote from Development to Test
• Work with the development team to set up automated build and deploy processes
• Define and automate the hand-offs between the development and the QA team
• Create test environment profiles in preparation for automated environment provisioning
• Leverage Cloud or other virtual technology to set up testing environments on demand
• Integrate the automated testing capability with the development team’s continuous integration capability to establish continuous testing

How will Dev-Test-Ops impact Agile Companies?

For Agile companies, it is all about continuous planning, continuous testing, continuous integration with ‘continuous’ being the key theme. A Dev-Test-Ops model is critical to Agile companies as it ensures that the focus of the whole team is on quality and all developments undertaken need to be seamless and reliable. Further benefits  that this model brings to Agile companies are:

• Enhanced, bug-free build and software product –  The bug-free policy essentially means that any bug raised on production systems at any point during a team sprint must be evaluated, worked on and closed within a minimum possible timeframe without having an impact on the customer or at the expense of a less valuable feature.
• Continuous feedback will ensure integration of real-time and run-time business changes
• The connection from a build system to a test system are automated and ready for testing. No human interaction to start up functional tests is required.
• Changes adopted as per advancement of technology trends in market even during development process
• Accelerated time to market due to iterative approach
• Reduction in costs due to faster release and no post-deployment changes

Just bringing the three main stages of a software lifecycle together will not resolve all the challenges associated with the typical development or may not optimize development cycles. Significant optimization in development cycles will occur by implementing best practices in automation processes that are incorporated through shared knowledge and cohesive teamwork. Hence, it is easy to say that adopting Dev-Test-Ops will result in accelerated software delivery, however, the larger picture starts from different teams working together to understanding project requirements to delivering short-term releases with almost no bugs.

While Agile companies will benefit a great scale in terms of products and their delivery, a different perspective will come in on the functioning of the team members, with the adoption of Dev-Test-Ops. For some roles, the lines may start blurring and for some, there may be add-ons, but, for all roles, there will be an evolution to broaden their knowledge and skills.