A/B Testing

Plan Availability:

Overview

The A/B Testing feature allows organizations to compare different process alternatives enabling process owners to make informed decisions based on data, leading to improved efficiency and effectiveness of business processes.

By using A/B Testing, organizations can achieve the following benefits:

• Improved Process Performance: Identify process alternatives that deliver better results, such as reduced cycle time or increased productivity.

• Optimal Resource Allocation: Determine the most efficient allocation of resources to maximize process outcomes.

• Continuous Process Improvement: Utilize insights gained from A/B testing to continually optimize and refine business processes.

• Enhanced Decision-making: Make data-driven decisions based on objective evidence rather than subjective opinions.

Features

• Alternative Creation: Users can create and maintain two independent variants of a process model, enabling testing and comparison against different scenarios.

• A/B Testing: Designers can activate both alternatives for live execution and define rules for distribution, ensuring reliable and unbiased results.

• Data Collection & Analysis: Process metrics and user feedback are collected and compared against live alternatives, offering quantitative insights for informed decision-making.

A/B Testing addresses the challenges organizations face in identifying and improving their business processes efficiently. This feature provides concrete evidence to support decision-making by collecting and analyzing data from different process alternatives. It tracks and measures key performance indicators (KPIs) for each alternative, providing objective proof of which performs best.

Watch the following product tour to get a quick overview of how A/B testing works.

Enable A/B Testing

The first step to enable A/B Testing is to add an alternate path that a process can take.

Permissions

Your user account or group membership must have the following permissions unless your user account has the Make this user a Super Admin setting selected:

• Processes: Edit Processes

• Processes: View Processes

See the Process permissions or ask your Administrator for assistance.

Follow the steps below to create an alternative for a process model:

1. Edit an existing process or create a new one in the Modeler.

2. From the top left of the Modeler, click the + Create the Alternative B icon to add an alternate flow for the process.
3. When creating an alternative, ProcessMaker will clone the existing model (Alternative A) into a separate tab (Alternative B). Subsequent edits to either alternative will be saved independently.

4. Edit the two alternative as needed and click Publish to publish the alternatives.

5. The Start Event for both alternatives must be identical to enable A/B Testing. If the events differ, A/B Testing cannot be enabled, and the following message will be displayed.

   

Troubleshooting "AB Testing is now disabled" message

Check the following settings in the Configuration Panel of the start events for Alternative A and B to ensure they are identical.

1. Properties: Names must be identical for both alternatives.

2. Start Permissions: Same user/group must be selected for both alternatives.

3. Web Entry: Web entry configuration must be identical.

4. Advanced: The Node Identifier property must be identical.

• When publishing the process, Alternative A will be the default version, however, Designers can choose to publish either Alternative A, or B, or both.

• Simultaneously publishing both alternatives enables the A/B Testing configuration, while publishing just one will disable it.

• The active alternative will be highlighted in green, while the inactive alternative will be displayed in gray, indicating its disabled status.

Simple Testing Configuration

Follow these steps for configuring basic A/B Testing settings:

1. From the Publish New Version screen, select the A+B Alternatives tab.

   
2. Click the A/B Settings button to configure test settings.

3. For simple testing configuration, select the Simple option and adjust the slider to set the percentage distribution of incoming requests between the two alternatives. By default, requests are evenly distributed between the two alternatives.

   
4. Click Save and Publish to publish the two alternatives with basic A/B Testing configuration.

5. Click the Version Info button to go back to the alternative selection screen.

6. Click See Process A/B Testing Configuration to view tests results and summary.

7. For advanced configuration, see Advanced Testing Configuration.

• If no changes are made to the A/B Testing configuration, requests are distributed equally by default, with 50% allocated to Alternative A and 50% to Alternative B.

• Both alternatives must have the same start event for A/B testing to work correctly.

Advanced Testing Configuration

For advanced testing configuration, choose the Advanced option in A/B settings when publishing the alternatives and follow these instructions:

1. From A/B Settings, choose the Advanced option and input an expression to set the rules for activating either of the two alternatives.

   

   For more information about expressions, see FEEL Expression Syntax.

Here are a few examples of how to enter expressions in the advanced settings:

• When routing based on the value of a Request Variable, an expression like this can be used: loanApproved == "Yes"

• If loanApproved variable stores a Boolean value, the expression can be written simply as: loanApproved

• It is also possible to combine expressions such as: (loanApproved) or (loanRating > 500)

3. When the entered expression evaluates to TRUE, requests will follow the process model in Alternative B.

4. When the entered expression evaluates to FALSE, requests will be allocated between Alternative A and B based on the percentages selected in the slider. Following table provides some examples of how advanced A/B Testing configuration works:

Expression	Alternative %	Request Distribution
TRUE	N/A	Alternative A = 0% , Alternative B=100%
FALSE	A=50% , B=50%	Alternative A = 50% , Alternative B=50%
FALSE	A=100% , B=0%	Alternative A = 100% , Alternative B=0%
FALSE	A=75%, B=25%	Alternative A = 75% , Alternative B=25%

4. Click Save and Publish to publish the process.

5. Click the Version Info button to go back to the alternative selection screen.

6. Click See Process A/B Testing Configuration to view tests results and summary.

Watch the following product tour to learn how to setup advanced configuration for A/B Testing.

Republish Alternatives

After the initial configuration, you can edit and republish one or both alternatives.

Permissions

Your user account or group membership must have the following permissions unless your user account has the Make this user a Super Admin setting selected:

• Processes: Edit Processes

• Processes: View Processes

See the Process permissions or ask your Administrator for assistance.

Follow these steps to edit and republish the alternatives:

1. Edit an existing process that has A/B Testing enabled. The two alternatives will display in their respective tabs.

2. Edit one or both alternatives as needed and click the Publish button to publish them.

3. To individually publish the alternatives, select the tab associated with that alternative.

   
4. Optionally, enter a Version Name and Description and click Save and Publish.

• Publishing only one alternative will disable the other, hence disabling the A/B Testing configuration. In this scenario, all requests will be routed using the published alternative only.

• The active alternative will be highlighted in green, while the inactive alternative will be displayed in gray, indicating its disabled status.

5. To publish both alternatives, select the A+B Alternatives tab. This action enables the A/B Testing configuration.

6. Optionally, click the A/B Settings button to adjust the configuration for A/B Testing.

7. Click Save and Publish when done.

Testing Results and Summary

Run an A/B Test from the Modeler

Switch and Copy Alternatives