Keycloak on ROSA Benchmark Key Results
This summarizes a benchmark run with Keycloak 25 release candidate build performed in May 2024. Use this as a starting point to calculate the requirements of a Keycloak environment. Use them to perform a load testing in your environment.
Collecting the CPU usage for refreshing a token is currently performed manually and is expected to be automated in the near future (keycloak/keycloak-benchmark#517). |
Setup
This setup is run daily on a GitHub action schedule:
-
OpenShift 4.15.x deployed on AWS via ROSA with two AWS availability zones in AWS one region.
-
Machinepool with
m7g.2xlarge
instances. -
Keycloak 25 release candidate build deployed with Operator and 3 pods in each site as an active/passive setup, and Infinispan connecting the two sites.
-
Default user password hashing with Argon2 and 5 hash iterations and minimum memory size 7 MiB as recommended by OWASP.
-
Database seeded with 100,000 users and 100,000 clients.
-
Infinispan caches at default of 10,000 entries, so not all clients and users fit into the cache, and some requests will need to fetch the data from the database.
-
All sessions in distributed caches as per default, with two owners per entries, allowing one failing pod without losing data.
-
Database Amazon Aurora PostgreSQL in a multi-AZ setup, with the writer instance in the availability zone of the primary site.
Installation
.env
file# no KC_CPU_LIMITS set for this scenario KC_CPU_REQUESTS=6 KC_INSTANCES=3 KC_DISABLE_STICKY_SESSION=true KC_MEMORY_REQUESTS_MB=3000 KC_MEMORY_LIMITS_MB=4000 KC_DB_POOL_INITIAL_SIZE=30 KC_DB_POOL_MAX_SIZE=30 KC_DB_POOL_MIN_SIZE=30
Results
Refer to concepts-memory-and-cpu-sizing in main Keycloak docs for results on the latest release, calculation example, observations and recommendations. Updated information about the upcoming release are available in the main branch of the GitHub repository.
Tests performed
Each test ran for 10 minutes.
-
Setup ROSA cluster as default.
-
Deploy Keycloak and Monitoring
cd provision/openshift task task monitoring
-
Create dataset
task dataset-import -- -a create-realms -u 100000 # wait for first task to complete task dataset-import -- -a create-clients -c 100000 -n realm-0
-
Prepare environment for running the benchmark via Ansible
See Running benchmarks via Ansible and EC2 for details.
Contents ofenv.yml
used herecluster_size: 5 instance_type: t4g.small instance_volume_size: 30 kcb_zip: ../benchmark/target/keycloak-benchmark-0.15-SNAPSHOT.zip kcb_heap_size: 1G
-
Create load runners
cd ../../ansible ./aws_ec2.sh start <region of ROSA cluster>
-
Run different load tests
-
Testing memory for creating sessions
./benchmark.sh eu-west-1 \ --scenario=keycloak.scenario.authentication.AuthorizationCode \ --server-url=${KEYCLOAK_URL} \ --realm-name=realm-0 \ --users-per-sec=<number of users per second> \ --ramp-up=20 \ --logout-percentage=0 \ --measurement=600 \ --users-per-realm=100000 \ --log-http-on-failure
-
Testing CPU usage for user logins
./benchmark.sh eu-west-1 \ --scenario=keycloak.scenario.authentication.AuthorizationCode \ --server-url=${KEYCLOAK_URL} \ --realm-name=realm-0 \ --users-per-sec=<number of users per second> \ --ramp-up=20 \ --logout-percentage=100 \ --measurement=600 \ --users-per-realm=100000 \ --log-http-on-failure
-
Testing CPU usage for logins and refreshing tokens with a ratio of 10 refreshes per one login. Use the previous test to deduct the CPU usage of logins only to get the CPU usage of token refreshes.
./benchmark.sh eu-west-1 \ --scenario=keycloak.scenario.authentication.AuthorizationCode \ --server-url=${KEYCLOAK_URL} \ --realm-name=realm-0 \ --users-per-sec=<number of users per second> \ --ramp-up=20 \ --logout-percentage=100 \ --refresh-token-count=10 \ --measurement=600 \ --users-per-realm=100000 \ --log-http-on-failure
-
Testing CPU usage for client credential grants
./benchmark.sh eu-west-1 \ --scenario=keycloak.scenario.authentication.ClientSecret \ --server-url=${KEYCLOAK_URL} \ --realm-name=realm-0 \ --users-per-sec=<number of clients per second> \ --ramp-up=20 \ --logout-percentage=100 \ --measurement=600 \ --users-per-realm=100000 \ --log-http-on-failure
-