Configuring benchmarks

The benchmark tests with Gatling can be configured with various parameters.

This describes the command line options passed to the kcb.sh shell script. To run the benchmarks, refer to Running benchmarks from the CLI.

Configuration Overview

The configuration allows running a benchmark against one or more Keycloak servers, realms, users and clients. All those entities need to exist before the test runs. See Preparing Keycloak for testing on how to do that.

If a configuration parameter allows multiple values (like, for example, the Keycloak server URL) or resolves to a range of values (like, for example, the realm name), those values are randomly assigned to each execution within a scenario.

General parameters

CLI option Default Notes

--server-url

(not set)

The URL of the Keycloak server.

Specify multiple URLs separated with a blank to run the test against multiple server URLs:

--server-url="http://host1:8080/ http://host2:8080/"

--share-connections

false

Enable sharing of connections among concurrent users within a scenario. This can be useful if the load generator host cannot handle as many TCP connections as are necessary. Note, using this option will produce less load on the target system than real world. Note, only scenarios that pause between steps allow sharing of connections.

--share-connections=true

--use-all-local-addresses

false

Since the test may involve tens of thousands of connections from a single testing system to a single host:port on a server, we may need to add additional addresses to increase the number of TCP connections we can make and this enables the use of those. Should not be used when the target is localhost, as using external IP addresses doesn’t work well with local addresses.

--use-all-local-addresses=true

--scenario

ClientSecret scenario

The full classname of the scenario to run, for example keycloak.scenario.authentication.AuthorizationCode. See Scenario overview for a list of all scenarios.

--increment

(not set)

A user can provide an increment value to run the incremental benchmark to find the limit of a given system under test for the assertions set within the simulation. If the CLI option is set without a parameter, it defaults to an increment value of 32. See Single vs. incremental run mode for more insight into how the incremental scalability benchmark works.

--debug

(not set)

If Java remote debugging should be enabled for the Gatling process. If the CLI option is set without a parameter, it defaults to port 8787. Add a different port number as the argument to this CLI parameter to change the default port.

--users-per-sec

1

Open workload model: Numbers of users per second to be added to the simulation. Each user will execute the simulation once.

Either --users-per-sec or --concurrent-users should be specified. If none is specified, defaults to --users-per-sec set to 1.

--concurrent-users

(not set)

Close workload model: Number of users executing the same scenario multiple times.

Either --users-per-sec or --concurrent-users should be specified. If none is specified, defaults to --users-per-sec set to 1.

--ramp-up

5

Ramp up time to reach the full load.

--measurement

30

Measurement time of the test in seconds.

--user-think-time

0

User think time in seconds used in some scenarios.

--log-http-on-failure

(not set)

If set, will log all failed HTTP requests to the console.

Enable this during testing and when analyzing problems. Never use it during a load test as it slows down Gatling.

--log-http-always

(not set)

If set, will log all HTTP requests to the console.

Will log a lot of information. Use this only during development. Never use it during a load test as it slows down Gatling.

--http-proxy

(not set)

If set, will proxy all HTTP requests to a proxy tool of your choice and the current default proxy host and port are 127.0.0.1, 8888 respectively. These are currently not configurable.

Never use it during a load test as it slows down Gatling to be able to proxy the requests to a proxy client and comes with the usual proxy nuances that we have to be careful, in general.

--sla-error-percentage

0

Maximum percentage of requests to fail during a run.

Running across multiple realms, users and clients

CLI option Default Notes

--realms

1

The number of realms available. Runs the test scenarios using realms from 0 to --realms minus 1.

With the default of 1 and --realm-prefix not set, this will result in only realm-0 being used.

--users-per-realm

1

The number of users available in each realm. Runs the test scenarios using users from 0 to --users-per-realm minus 1.

With the default of 1 this will result in only user-0 being used.

--clients-per-realm

1

The number of clients available in each realm. Runs the test scenarios using clients from 0 to --users-per-realm minus 1.

With the default of 1 this will result in only client-0 being used.

When running the CLI with the following example parameters:

--realm=10 --users-per-realm=10 --client-per-realm=10

This results in the following values being used in the scenarios:

Entity Values

Realm

realm-0 ... realm-9

Username

user-0 ... user-9

User Password

user-0-password ... user-9-password

Client ID

client-0 ... client-9

Client Secret

client-0-secret ... client-9-secret

Client Redirect URL

The account console of the respective realm:

http://localhost:8080/realms/realm-0/account ... http://localhost:8080/realms/realm-9/account

Overriding default values for realms, users and clients

CLI option Default Notes

--realm-prefix

realm-

Sets the prefix for the realm name. With a value of myrealm it uses myrealm-0, myrealm-1, …​.

If not set on the CLI, realm names use the realm- prefix. For example: realm-0, realm-1 …​.

--realm-name

(not set)

If set, tests run using a single realm set in this parameter.

If not set and assuming that --realm-prefix is not set, the realm names are numbered starting with realm-0.

--username

(not set)

If set, tests run using a single username set in this parameter.

If not set, the usernames are numbered starting with user-0.

--user-password

(not set)

If set, tests run using a single password set in this parameter.

If not set, the passwords are numbered starting with user-0-password.

--client-id

(not set)

If set, tests run using a single client ID set in this parameter.

If not set, the client IDs are numbered starting with client-0.

--client-secret

(not set)

If set, tests run using a single client secret set in this parameter.

If not set, the client IDs are numbered starting with client-0-secret.

--client-redirect-uri

(not set)

If set, tests will run using a single client-redirect-URI set in this parameter.

If not set, the redirect URL is the account console of the respective realm.

--scope

(not set)

A comma-separated list of scopes to be set when making authentication requests.

If not set, the default scopes are openid profile.

Additional settings

These settings are only necessary if the scenario requires the settings. Please refer to the scenario overview to review the scenarios.

CLI option Default Notes

--admin-username

(not set)

Username of the admin user in the master realm.

--admin-password

(not set)

Password of the admin user in the master realm.

--logout-percentage

100

Percentage of users to log out at the end of the scenario. Used in Scenario AuthorizationCode.

--refresh-token-period

0

Period to wait between token refreshes. Used in Scenario AuthorizationCode.

--refresh-token-count

0

Number of token refreshes after logging in. Used in Scenario AuthorizationCode.

--refresh-close-http-connection

true

Close the HTTP connection after a token refresh. Used in Scenario AuthorizationCode.

--basic-url

(not set)

URL to be called in the Scenario Get.

Further reading

For a complete list of the available options, see Config.java.