Maestro and Composer Forms A/B Testing

   Journey Manager (JM) The transaction engine for the platform.  |    System Manager / DevOps  |  18.05 This feature was introduced in 18.05.

First thing to do is to determine a need to do A/B Testing at all. Avoka Insights does an excellent job of highlighting problems in the User Experience that a change in Form design may address. Or, more proactively, investigating a new Form feature can involve A/B Testing, to determine that the new additions won't stop end users from submitting.

Maestro

To test multiple versions of a Form, the versions must of course first be created. As an example, a Form named aBtesting was made. It is a default Maguire Template form, with two Text Fields added.

In this example, the Text Fields are arranged horizontally in the first version, and vertically in the second. This is similar to A/B Testings intended use case how does a possibly small change to a form impact the user experience, as seen in completion rates.

Figure 1 - the "A" design	Figure 2- the "B" design

Build and deploy both versions to Manager as you normally would.

Composer

The same concepts apply when using Composer as the form designer. Build the first design, the A version, and publish it to a ZIP file. Then use Composers Form Revisions feature to create the B version. Publish the B version to a ZIP file as well.

Import both ZIP files to Manager using the Import Composer Package feature. You will create a new form for the first ZIP file imported, and a new version of that form when you import the second ZIP file.

Manager

Once deployed to Transact, the Form dashboard would look like shown below .

This shows that both Form versions are deployed. For A/B testing, it does not matter which version is marked as Current Version. In fact, neither needs to be current. The Form Version Selector Service will chose which version to render.
The next step is to then add a Form Version Selector Service to the Form. Switch to the Details tab and add one.

The Form Version Selector Service shown above is a default service. In practice, a copy of this Service should be made and that copy used for this Form's testing. This needs to be done to allow the Service to be configured with the Form Version numbers specific to this Form. Click the Edit link to setup the Form Version Selector Service.

On this screen you add the Form version numbers, separated by commas, that need to be tested. Note that A/B Testing can actually be used against three or more different Form Versions!
The Form Version Selector Service will randomly choose from the supplied list of versions, evenly distributing the selections. So in our example, version 1.0-develop will be selected 50% of the time, and version 1.0 will be selected 50% of the time.
If one of the versions needs to be presented more often than the other, include it in the list more than once. Using 1.0-develop,1.0.1,1.0.1 will result in version 1.0.1 being selected 2/3 of the times.
That's all that's needed. The A/B Testing is now live and functioning. Accessing the Form's base url will trigger the Form Version Selector Service, and it will return a randomly selected version. Normally the base url (like https://training.avoka.com/web-plugin/servlet/SmartForm.html?formCode=abtesting) would return the Form Version marked Current.

Performing the Tests, Reviewing Results

Now that the A/B Test is live and running, let it collect data. There is nothing additional to do to perform the test other than let the system run. After a suitable amount of time (to be determined by each project team for each form Forms under heavy use may only need a day's data, others may need a week), review the results and turn off the test.

The Analytics menu item on Transact's main menu contains the Form A/B Testing link. This page will show a graph of the relative performance of the entire Form, one graph for each version tested (as shown above). Of course, Insights can also be used to review the results of A/B Testing. And with Insights, you can see differences between the two Form Version's performance on a much more granular level.
After reviewing the results, the A/B Test should be shut off. Go back to the Form Dashboard tab and mark the proper Form Version as Current. Then on the Details tab, clear the Form Version Selector setting.

What is implied above is that this is a production server test. Live Forms being used by end users. So all versions tested this way must go through all of the normal development processes full testing on lower environments, etc. After all, the desired goal is to find a new production Form design, that removes friction from the user experience.

Next, learn about form A/B Testing analytics and reporting in Journey Analytics.