Transact Insights 17.10

Overview

Transact Insights 17.10.0 is the first major release after Insights 5.1 was made Generally-Available in August 2017. In the past few months, several improvements have been implemented in Insights. These improvements primarily revolve around Insights data collection, Insights-Maestro integration, Insights-Composer integration, and Insights-TM integration, and are included as part of the 17.10.0 release, which is very promising in terms of stability and data quality.

The Insights drop-off view has been redesigned to improve the focus on comparative and relative analysis of abandonment hot-spots in Transact Applications. This release also features the new Milestone filter which helps in filtering transactions with custom-defined milestones and performing comparative analyses based on users reaching specific milestones in the application.

For more information see the Transact Insights documentation.

Key Features and Enhancements

Milestones Filter

A milestone is a key event in a transaction that marks the transaction with a specific milestone name. There are standard/pre-defined milestones in Insights such as Opened, Bounced, Started, Saved, Resumed, Abandoned, Completed etc, which are all used to track the status of a transaction. Using this feature, customers can define custom milestones to be sent within the context of a transaction, typically, to mark a business significance for that transaction. Documentation on how to send custom milestone events from Maestro, Composer, and TM can be found at the following links:

• Send Milestone from a Maestro Form
• Send Milestone from a Composer Form
• Send Milestones from Transact Manager

Key benefits of this feature -

• Define your own milestones in the form relevant to your businesses and perform comparative analysis based on users reaching specific milestones in the form.
• Measure the engagement of exchange components, their impact on UX and conversion rates.
• Create a universal form used for different purposes, yet analyze and optimize conversion for each use-case independently.
• Send a milestone event for a submission object from Transact Manager even after the form is submitted by the user.

There are many ways custom milestones can be used to analyze data in Insights. At present, Insights presents this as part of its global filter set. One or more milestone filters can be applied and only transactions that have at least one of the selected milestone filters will be presented in all Insights views. In the future releases, there are plans to present milestone data in its own Analytics view.

Improved Drop off view

The drop-off view has been revamped to enhance the readability of data and present conceptually different data in separate UI components to assist users in focusing on the right metrics at transaction and section level accordingly.

A new Transaction Metrics component has been added at the top of the view which captures transaction level data, their statuses, conversion, and completion rate. Contextualual help has been included so that users can quickly understand what each metric represents and how they are calculated.

A key change has been made in the way onboarding metrics are calculated such as Completion, Bounced and Abandonment rates. Prior to 17.10, all transactions, including the ones that are currently in open/user-saved status were being considered as part of that calculation. This was negatively skewing the completion rates due to in-progress transactions. Additionally, section level data from these transactions were affecting the drop-off chart leading to confusions. As it may take a few hours for Insights data to be ready for consumption, and because events may not be processed in the order they occurred, partial data from in-progress transactions were impacting section level drop-off data. To address this issue, in 17.10, all in-progress (Open/User-saved) transactions are excluded from completion and section level analysis. This means, only transactions that have reached a final status (Bounced, Abandoned, Completed), will be considered for calculations. This affects the way Completion, Bounced and Abandonment rates are calculated.

A new metric, Conversion Rate. has been added to the drop-off chart. This metric which computes the rate calculation for all finalized transactions (Bounced, Abandoned and Completed) and is different from the Completion rate calculation in that it also considers Bounced transactions.

The new drop-off chart only provides data for section level completion and has been renamed to the Section Level Completion chart. Additionally, all absolute numbers have been removed from section level completion chart and instead all metrics are presented with rates. The doughnut chart in each section provides Bounce, Abandonment and Completion rates for that section.

• Bounce in a section means a user visited the page/section but did not interact with any fields in that section and abandoned the transaction.
• Abandonment in a section means a user visited the page/section, interacted with at least one field and then abandoned the transaction.
• Completion in a section means a user visited the page/section, filled all mandatory fields in that section and completed that section and proceeded to the following section.

Hovering over the corresponding wedges of the doughnut shows the corresponding metric at the center of it. The completion rate of the section is displayed by default. The total fields metrics that was shown in the previous version of the Insights drop-off view has been removed. These relative metrics replace the Visits, Started and Completed metrics that were shown in the previous version of Insights. The total field count in the form can still be calculated by adding up field counts in each section. The Saved count at each section has also been removed from the view. This metric will be added to the Field level view in a future release.

A new progressive completion rate metric is shown at the bottom of each section. This metric shows the percentage of users who completed that section compared to the number of users who started the application. This metric essentially captures the numerical value of the green portion (completion) of the chart with reference to the vertical axis. Note the Completion rate and Progressive Completion rate for the "Processing Team" section in the chart below. Although the Completion rate for this section is quite high (96.1%, indicating most people who visited this section completed the section), the Progressive Completion rate for this section is quite low (28.4%). This is due to a dip in the number of users visiting this section, indicating that this is an optional section and that not all users journey through. So, compared to the users who started the form, the users completing this section is quite low.

Architectural enhancements

Improved data ingestion at Scale

Event processing in Insights depends on corresponding form request events from TM and form session events form the browser. When either of these events are unavailable after 5 retries (typically 1.25 hours), the events are dropped as the system assumes that data will never be available. Based on the study of empirical data, processing of a form session event before processing a form request event is highly likely. When this happens, form session events are held back in the processing queue as the system wouldn't have the knowledge of the corresponding TM event. These form session events are processed by a dedicated Insights service that handles untagged events. Based on the improved logging in the past few releases, we were able to see that other events from the browsers had the form request data available to them and hence were tagged with the TM ID. The design creates one dedicated service per TM to process corresponding tagged events. We observed that due to the volume of TMs and traffic, untagged processors would never get the processing time required, as there is a limit of 50 concurrent processing entities on GCP. However, the tagged events were processed 5 times and the events were being dropped due to the lack of form session data.

This work improves the strategy used in ingesting data when there is a large number of TMs with a large number of transaction events being processed. It prioritizes processing of untagged events which do not have TM details for them to be processed. With this approach, form session events would be processed successfully faster. This enables the system to process all other events successfully and improve the data collection quality. Since this enhancement was deployed on 21st Dec, the event drop rate has significantly reduced, as can be seen in the screenshot from the data ingestion tracking system we have in place.

Handling missing events from the browser

Form Session event:

As events from browsers are not processed if Insights doesn't have the knowledge of form sessions, it is critical for us to ensure an architecture that improves reliably of recording form session in Insights. We have seen that much of the cases where form sessions are not received in Insights are for bounced transactions. In many cases, users land on a form and close the browser too quick for the Insights code running in the browser to send a form session event to the backend. These scenarios end up as bounced eventually in TM. However, when we receive Bounced milestone events from TM, due to the lack of form session events, the Bounced milestone is dropped. These transactions would not be tracked in Insights leading to a discrepancy in the number of Bounced transactions between TM and Insights.

This work ensures a default form session event is registered for Bounced transactions where we haven't received a form session event from the browser. All fields of the form session data besides the TM, Form, Version, Space and the Org details, are set to be unknown (ex: device type, browser type etc.). In Insights, when the Bounced timeline view is split by the browser or device type, these transactions appear in the Unknown category.

Submit event from the browser:

In some cases, where the user submits the form but closes the browser before for the Insights code running in the browser sends the Submitted milestone event to Insights backend, Insights would end up considering them as open. Whereas it would appear as Completed in TM. This would lead to discrepancies in Completed count and Completion rate between TM and Insights.

This work leverages on the backup event Insights receives from TM called Completed milestone which makes the Completion metrics shown in Insights closer to the Completion metrics TM reports, and hence improves the quality of Insights data analysis.

Handling Invalidated form sessions from TM

Due to the difference in treatment of transaction's request/open time in TM and Insights, we observed some transactions being anchored to a different date in Insights compared to TM. This difference would lead those transactions to be reported in different dates in TM and Insights, leading to discrepancies. This specifically happens for transactions arising from anonymous tasks part of a collaboration job in TM. TM replaces the transaction's request/open time when they do not have a form start event and they are arising from anonymous tasks. So, those transactions are reported on the replaced data/time. Whereas, Insights continues to report those transactions on the transaction's original request/open time.

This work implements a new session invalidation event to be sent from TM when TM replaces a transaction's request/open time. Insights uses this event to select the oldest non-invalidated session open time to report the transaction. This improves the consistency of Insights data with TM reports.

Release Details