Generating datasets

This script contains a subset of the operations described in the next sections around realms, users and clients.

The script assumes that the dataset provider is installed as described in Installing on minikube setup and that it is available at
https://keycloak-keycloak.$(minikube ip).nip.io/realms/master/dataset/.

Run the following command to receive a help description:

You will see these options:

Example of creating 10 new realms:

Alternatively, the user could also run the Taskfile from within the provision/minikube module to execute the dataset-import task, to seamlessly execute the automation for Kubernetes style deployments.

The dataset-import task re-uses the command line arguments defined above to make it convenient for the user. Note the two dashes (--) that separate the task name from the command line options passed to the script.

To learn more about the tool, see Automation using the tool task for details.

You need to call this HTTP REST requests. This request is useful for create 10 realms. Each realm will contain a set of roles, clients, groups and users:

This is a request to create 200 new clients in the realm realm-5.

Each client will have service account enabled and secret like <client_id>-secret (For example client-156-secret in the case of the client client-156):

You can also configure the access-type (bearer-only, confidential or public) and whether the client should be a service-account-client with these two parameters:

This is a request to create 1000 new users in the realm realm-5:

Each user will have the specified amount of roles, client roles and groups, which were already created by the create-realms endpoint.

Each user will be assigned a password in the format <username>-password. For example user-156 will have the password user-156-password.

When creating users, the bottleneck is CPU time to hash the passwords that are by default unique for each user.

To reduce total time to create a large number of users, configure the dataset to only create a limited pool of hashed passwords and reuse these credentials when creating users. Specify the unique-credential-count parameter in order to limit the number of passwords created.

When limiting the total number of passwords, individual passwords for a given user are calculated as "password-" + i where i is the user’s number % unique-credential-count.

For example, after executing:

the user user-156 will have the password password-6 as we calculate i = 156 % 10.

Groups are created as part of the realm creation. The number of groups and the structure of the created groups can be managed by using the following parameters:

With the default values, only top-level groups are created. With groups-with-hierarchy set to true, the groups-per-realm parameter is ignored and the group tree structure is created as defined by the other parameters. groups-count-each-levelgroups-hierarchy-depth will be the total number of groups created. The hierarchical groups implementation honors groups-per-transaction. The adopted subgroup naming convention uses a dot (.) in the group names which allows finding the parent group even if it was created in a previous transaction.

You can also create groups in an existing realm by invoking the create-groups endpoint and setting the realm-name parameter:

This is a request to create 10M new events in the available realms with prefix realm-. For example if we have 100 realms like realm-0, realm-1, …​ realm-99, it will create 10M events in realm-99.

This is a request to create 10M new sessions in the available realms with prefix realm-. For example if we have 100 realms like realm-0, realm-1, … realm-99, it will create 10M sessions in realm-99 distributed across all users and clients of the realm as defined by clients-per-realm and users-per-realm.

You can also specify the optional parameter session-expiration-interval to stagger the expiration times of the created sessions. The interval is specified in seconds, and the sessions will be randomly distributed in the interval. The below example creates 100 sessions, with sessions expiring in a 3600-second interval:

This is a request to create 10M new offline sessions in the available realms with prefix realm-. For example if we have 100 realms like realm-0, realm-1, … realm-99, it will create 10M offline sessions in realm-99 for user user-0.

To remove all realms with the default realm prefix realm

You can use realm-prefix to change the default realm prefix. You can use parameters to remove all realms for example just from foorealm5 to foorealm15

For change the parameters, take a look at DataSetConfig class to see available parameters and default values and which endpoint the particular parameter is applicable. For example to create realms with prefix foo and with just 1000 hash iterations (with the default hashing algorithm) used for the password policy you can use these parameters:

Another example would be to specify a particular hashing algorithm in combination with the hashing iterations with the below parameters:

The configuration is written to the server log when HTTP endpoint is triggered, so you can monitor the progress and what parameters were effectively applied.

Note that creation of new objects will automatically start from the next available index. For example when you trigger endpoint above for creation many clients, and you already had 230 clients in your DB (client-0, client-1, …​ client-229), then your HTTP request will start creating clients from client-230 .

Call the following URL using the GET method:

If there is a task still running, it returns information like the following:

If there is no task running, it returns information like the following:

Call the following URL using the GET method:

If there is a task still running, it returns information like the following:

If there is no completed task, it returns this information with status code 404:

Call the following URL using the DELETE method:

To see the last created realm index

To see the last created client in given realm

To see the last created user in a given realm

Before provisioning organizations, make sure to manually create or provision a realm. For example, provision a realm-0 realm as follows:

As a result, you have a realm realm-0 with 5k users in it.

This is the request to create 1000 organizations in a realm with prefix org-:

Alternatively, you can create a single organization with a given name:

You can also specify how many managed and unmanaged members and how many identity providers should be linked to each organization created:

As a result, 1k organizations with the following configuration:

It is also possible to specify a number of identity provider mappers per each identity provider:

This creates 1k organizations with 500 unmanaged members each, 10 identity providers and with identity provider having 3 identity provider mappers.

You can also provision data to a specific organization. For instance, to provision more identity providers to a specific organization:

Optionally, it’s possible to specify a number of identity provider mappers per each identity provider

Or to provision more unmanaged members to a specific organization:

Or to provision more managed members to a specific organization:

When provisioning members, make sure you have created enough users in the realm. For managed members, you also need at least a single identity provider linked to an organization.

If you want to remove an organization: