Unlocking Keycloak’s Potential: Building High Availability and Scalability with Distributed Cache over TCP
Introduction
Keycloak is a powerful open-source identity and access management solution that offers a wide range of features for authentication and authorization. In modern web applications, high availability and scalability are crucial for uninterrupted service. To achieve this, setting up a Keycloak cluster is essential.
Objective
In this comprehensive guide, we will explore the role of infinispan and JGroups in enabling a smooth setup and communication for the Keycloak cluster.
What is Keycloak
Keycloak is an open-source Identity and Management system that provides robust authentication and authorization capabilities. It allows developers to secure their applications by managing user identities, protecting APIs, and enabling single sign-on. Keycloak offers features such as user registration, role-based access control, social login integration, and more. By using Keycloak, developers can streamline the authentication process and focus on building core application functionality.
What is Infinispan
Infinispan is an open-source in-memory data grid that offers flexible deployment options and robust capabilities for storing, managing, and processing data. Infinispan provides a key/value data store that can hold all types of data, from Java objects to plain text. Infinispan distributes your data across elastically scalable clusters to guarantee high availability and fault tolerance, whether you use Infinispan as a volatile cache or a persistent data store.
Typical use-cases for Infinispan include:
- Distributed cache, often in front of a database
- Storage for temporal data, like web sessions
- In-memory data processing and analytics
- Cross-JVM communication and shared storage
- MapReduce Implementation in the In-Memory Data Grid.
What is JGroups
JGroups is a framework used for reliable group communication in Keycloak clustering. It enables Keycloak instances to communicate and coordinate with each other in a cluster. JGroups uses various protocols for communication, including TCP, UDP, and multicast, allowing developers to choose the most suitable option based on their network environment. JGroups also provides dynamic group formation, which allows Keycloak clusters to adapt to changes in the cluster membership dynamically. With JGroups, developers can build scalable and resilient cluster setups for Keycloak.
Step-by-Step Guide to Setting up a Keycloak Cluster
Now, let’s dive into the practical aspect of setting up a Keycloak cluster. We will break it down into several steps to ensure a comprehensive process understanding.
Preparing the Environment
Before setting up a Keycloak cluster, it is crucial to meet certain requirements. This includes having multiple servers for Keycloak instances, ensuring proper network connectivity between the servers, and allocating sufficient resources for optimal cluster performance. Installing and configuring Keycloak instances is the next step, which involves downloading the Keycloak distribution, creating a domain, and configuring the instances for clustering.
Configuring Keycloak for Production Mode
To enable Keycloak in Production mode, we need to define some additional configurations like
- TLS{Self Signed in our case}
- Cache type{ispn }
- Proxy type{ edge}
- Hostname Configuration {load balancer IP in our case}
Append the following configuration in the File path: /opt/keycloak/conf/keycloak.conf
#CACHE & Proxy cache=ispn proxy=edge HTTPS or TLS configuration # The file path to a server certificate or certificate chain in PEM format. https-certificate-file=<path for PEM file> # The file path to a private key in PEM format. https-certificate-key-file=<path for Key file> # Hostname for the Keycloak server. hostname-admin=<Load Balancer’s IP> hostname-strict-backchannel=true hostname-strict=false hostname-url=http://<Load Balancer’s IP>
Configuring Infinispan for Clustered Caching
To enable clustered caching in Keycloak, we need to configure infinispan. In this step, we will explore various infinispan configurations, including cache modes, eviction policies, consistency modes, and data persistence options. These configurations play a crucial role in determining the behavior of the distributed cache in a Keycloak cluster.
Append following block File Path <Keycloak-Path>/conf/ispan.xml
<infinispan> <jgroups> <stack-file name="<stack-name>" path="<Keycloak-Path>/conf/jgroups-tcp.xml" /> </jgroups> <cache-container name=keycloak> <transport lock-timeout="60000" initial-cluster-size="1" cluster="infinispan-cluster" stack="<stack-name>" node-name="keycloak1" machine="<machine-id>" /> </cache-container> </infinispan>
Note: For cache configuration, we are keeping the default configuration.
Establishing Communication with JGroups
JGroups is responsible for inter-cluster communication in Keycloak. In this step, we will define JGroups stack configurations, which specify how Keycloak instances communicate with each other in a cluster. We will cover the basics of JChannel, protocols, and decision strategies in JGroups. Configuration changes required in Keycloak to enable JGroups for inter-cluster communication will also be discussed in detail.
Create New File /opt/keycloak/conf/jgroups-tcp.xml
<!--
TCP based stack, with flow control and message bundling. This is usually used when IP
multicasting cannot be used in a network, e.g. because it is disabled (routers discard multicast).
Note that TCP.bind_addr and TCPPING.initial_hosts should be set, possibly via system properties, e.g.
-Djgroups.bind_addr=192.168.5.2 and -Djgroups.tcpping.initial_hosts=192.168.5.2[7800]
author: Bela Ban
-->
<config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns="urn:org:jgroups"
xsi:schemaLocation="urn:org:jgroups http://www.jgroups.org/schema/jgroups.xsd"
>
<TCP bind_addr="<Instance-IP>"
port_range="100"
bind_port="${jgroups.bind_port:7800}"/>
<RED/>
<TCPPING async_discovery="true"
initial_hosts="${jgroups.tcpping.initial_hosts:<instance1-ip>[7800],<instance2-ip>[7800]}"
send_cache_on_join="true"
return_entire_cache="${jgroups.tcpping.return_entire_cache:true}"
num_discovery_runs="5"
port_range="0"
ergonomics="true"/>
<MERGE3 min_interval="10000"
max_interval="30000"/>
<FD_SOCK2/>
<FD_ALL3 timeout="40000" interval="5000" />
<VERIFY_SUSPECT2 timeout="1500" />
<BARRIER />
<pbcast.NAKACK2 use_mcast_xmit="false" />
<UNICAST3 />
<pbcast.STABLE desired_avg_gossip="50000"
max_bytes="4M"/>
<pbcast.GMS ergonomics="true" print_local_addr="true" join_timeout="2000"/>
<UFC max_credits="2M"
min_threshold="0.4"/>
<MFC max_credits="2M"
min_threshold="0.4"/>
<FRAG2 frag_size="60K" />
<!--RSVP resend_interval="2000" timeout="10000"/-->
<pbcast.STATE_TRANSFER/>
</config>
Make changes for Logs File and Monitoring
# changes in /opt/keycloak/conf/keycloak.conf # If the server should expose healthcheck endpoints. health-enabled=true # If the server should expose metrics endpoints. metrics-enabled=true While starting keycloak: /opt/keycloak/bin/kc.sh start --log="console,file" --log-file="<logs-file-path>"
NOTE: Create an empty file first on path <logs-file-path>
Apply the configuration
To Apply the configuration, we first need to build keycloak and then start keycloak in production mode with the following commands:
/opt/keycloak/bin/kc.sh build /opt/keycloak/bin/kc.sh start --log="console,file" --log-file="<logs-file-path>"
Testing and Verifying the Cluster Setup
After starting all the cluster instances, check and run the following command
cat <keycloak-logs-file> | grep ‘topology id’
NOTE: By Default, keycloak doesn’t create any Logs file.
Summary
Setting up a Keycloak cluster is crucial for achieving scalability and high availability in modern web applications. In this blog post, we explored the importance of Keycloak clustering and its advantages in handling increased traffic. We discussed infinispan as the underlying distributed cache for Keycloak, highlighting its performance and data consistency benefits. JGroups was introduced as a framework for reliable group communication in Keycloak clustering, emphasizing its role in scalable messaging and dynamic group formation. By following the step-by-step guide, developers can set up a robust Keycloak cluster and ensure a seamless user authentication experience.
If you still have questions, comment and join the discussion.
