Running benchmarks from the CLI

Generic command line to run the benchmarks from any host against an already provisioned Keycloak instance.

Use this to run a Keycloak benchmark against a locally running Keycloak instance. When running more than 300 new users per second against a remote instance of Keycloak, the load generation might stall due to a lot of network connections in TIME_WAIT mode. In those cases, use the Running benchmarks via Ansible and EC2 or a similar mechanism to split the load generation across multiple nodes.

Prerequisites

  1. Keycloak URL from Preparing Keycloak for testing

  2. Either download the benchmark module or build it locally.

    To run the tests as is without local modifications, downloading it is the recommended option.

  3. Java 21 installed with the java executable available on the path.

Running the test from the CLI

Preparations

Running the tests

To start running tests, execute:

./kcb.sh

By default, tests expect Keycloak to run on http://0.0.0.0:8080, and the default scenario is keycloak.scenarion.authentication.ClientSecret.

To use a different server URL, realm and scenario:

./kcb.sh --scenario=keycloak.scenario.authentication.AuthorizationCode --server-url=https://keycloak-keycloak.192.168.12.345.nip.io --realm-name=test-realm

See Scenario overview for an overview over all available scenarios and their configuration options.

See Configuring benchmarks for setting different parameters.

Single vs. incremental run mode

In the single run mode, which is the default, the script performs one run and prints the result on the console.

In the incremental mode, which is triggered by the CLI option --increment, the script runs the benchmark once as a warm-up, and then again, and if successful repeatedly with an ever-increasing workload as given in the parameter. Once the benchmark fails after one of the increments, the script performs additional runs with smaller increments to find the maximum possible successful workload and prints this information on the console.

Report

For the default gatling generated reports, check reports at the result directory. For other reports, see Reports for Keycloak Benchmark.