Cloud Manager Blue-Green Deployment model in AEM-Adobe Managed Service (AMS)

Adobe Cloud Manager for Adobe Experience Manager (AEM) is a cloud service that allows customers to build, test and deploy Adobe Experience Manager applications hosted by Adobe Managed Services. It enables organizations to self-manage Adobe Experience Manager environments in the cloud, including both AMS and AEMaaCS.

AMS introduced the Blue-Green deployment model in Cloud Manager. Blue Green deployment will aid in the reduction of downtime caused by Cloud Manager deployment on production.

What is the Blue-Green Deployment Model?

A blue-green deployment will create a Replica of the existing publish/dispatcher tiers.

Blue Green is a Cloud Manager CICD pipeline add-on that creates two distinct but identical environments (Publish-Dispatcher leg). One environment (blue) runs the current application version, while the other (green) runs the new application version(idle).

How does the Blue/Green Model work?

• A blue/green deployment strategy improves application availability while lowering deployment risk and downtime. Once the green environment has been thoroughly tested, live application traffic is directed there, and the blue environment is deprecated.
• Blue-Green instances are treated as transient in this implementation, and each iteration of a blue-green pipeline generates a new set of publish and dispatcher servers.
• With Blue Green Deployment, existing servers (Dispatchers and Publishers) will be terminated after the deployment is complete, and a new Publish/Dispatcher will be created for use.

Diagrammatic Representation

What is in scope?
Only the production publish and dispatcher instances.

What is not in scope?
Author Instance

What is the cost?
There are no fees for customers who use Blue/Green deployments. It is a benefit of being an AMS customer and using Cloud Manager.

Who is eligible?
All AMS customers who have completed horizontal scaling (Autoscaling) readiness and are using Cloud Manager for production deployments are eligible.

How long can we keep the green servers?
The green servers will be available for 24 hours after being created. The pipeline fails and rollback occurs if they are not added to the production load balancer within 24 hours.

Deployment Duration
Stage deployment – 30 mins
Production deployment – 3 to 4 hours

Here are the detailed steps for a Blue/Green Deployment

1. Production deployment deploys the changes to Stage instances again, followed by approval to Production instances.
2. The following deployment steps will be applicable when the mentioned configurations are updated in the pipeline:

• Deployment Trigger is “Manual” in Stage
• Deployment Option is “Approve After Stage” in Stage
• Deployment Option is “Use Go-Live Approval” in Production
• Other deployment options, such as “scheduled” or “use CSE Oversight,” are available

** The time durations listed above will not apply to Dispatcher deployments.

How to Validate Changes Using a Load Balancer Following a Blue-Green Deployment Model

1. Modify your local hosts file [1] at that time to point the domain to the Application Gateway IP address.

2. The load balancers’ ip’s are listed below. These ip’s will not change and will remain constant.

Blue (Live) Application Gateway Static Public IP – xx.xxx.xxx.xx

Green (New) Application Gateway Static Public IP – yy.yyy.yyy.yyy

Example entry in hosts file – yy.yyy.yyy.yyy       www.domain.com

3. Following the above host entry, when you access www.domain.com in your browser, you will be directed to the green load balancer where the new code is deployed.

4. You can remove the entry from the host file once you have tested your changes in the load balancer. After that, when you visit www.domain.com, you will be directed to the CDN.

5. After a successful production Blue/Green deployment, the ip’s of the production publish, and the dispatcher will change. You must obtain the new ip’s from the Adobe Runbook or contact Adobe CSE.

How to Rollback

Option 1: Rollback the necessary changes in the Git repository, push them to the Cloud Manager Git repository, and rerun the pipeline for production deployment (Recommended)

Option 2: Rollback from a data volume backup that was taken prior to the CM deployment. This step needs to be performed by Adobe CSE.

We will discuss the Pro’s and Con’s of using the Blue-Green model in Cloud Manager:

Pro’s

1. It is simple to clear the dispatcher cache via the pipeline after deployment.
2. Dispatcher syntax can be validated during the security check stage.
3. It’s easy to use, and updating the release branch details in the pipeline is simple.
4. There is no downtime for author/live websites during deployment.
5. Handle the Production deployment without the assistance of Adobe CSE unless there are any pipeline issues.
6. Before making the code/configuration changes live, the Green instance can be thoroughly tested.

Con’s

1. There will be no security or performance checks in the non-production pipeline.
2. Since there’s no separate pipeline for dispatcher changes, we may have to wait more than 2 hours to implement the changes, even though single-line dispatcher configuration changes take the same time.
3. The Blue-Green deployment model is time-consuming because it takes 3 to 4 hours to complete the Production deployment.
4. Data inconsistency is a significant concern when content authors replicate any pages/assets during deployment, and those published contents/assets are not in sync with the Green instance. If customers report the problem, we may need to republish those pages/assets.
5. We will be unable to examine if any deployment stage in Cloud Manager fails since there are no cloud manager logs available in the deployment console, and we will need to contact Adobe CSE to share the logs.
6. Rollbacking to a previous code version/dispatcher changes via Cloud Manager may take 2 to 3 hours.
7. Publisher/Dispatcher IP addresses do not remain consistent after each Production deployment.
8. If any stage fails, we must restart the Production pipeline from the beginning.