| Version | Date Released | Features and Enhancements | Resolved Issues |
|---|---|---|---|
| 18.05.1 | June 1, 2018 | ✓ | |
| 18.05.0 | June 1, 2018 | ✓ | ✓ |
Overview
Transact Insights 18.05.0 is a major release of Insights introducing a powerful new analytics visualization called User Journeys which provides insights into how users are progressing through key milestones in their on-boarding journey, what paths users take and how it impacts on-boarding experience and conversion. It further adds the capability to segment data and understand how various user segments experience the on-boarding process allowing you to focus on specific problematic journeys.
This release also includes a major architectural enhancement in Insights integration with Maestro by the introduction of a new module called Event Consumer Client Library (ECCL). 18.05.0 adds many other architectural changes that enhance security in data collection, improve scalability and performance of data ingestion, and improve quality of data collected in Insights.
Key Features
User Journeys
The User Journeys feature offers key Insights about your customer journeys in the on-boarding process. It allows you to study customer Journey paths, identify bottlenecks in the Transact Application and the workflow around it, and optimize the on-boarding process by using metrics such as:
- % of users reaching a milestone.
- Time to reach a milestone.
- Completion rates for different journeys.
User journeys are made up of custom Milestone events. You can send application/business specific events that represent key milestones in the on-boarding journey either from a Transact Application or from a Fluent/Groovy service on TM. More on how to send custom Milestones are documented here:
- Sending Milestones from a Maestro Application.
- Sending Milestones from a Composer Application.
- Sending Milestones from Transact Manager.
Segmentation
In the User Journeys view, you can add any number of Segmentation charts on any Milestone presented in the User Journeys graph. Adding a Segmentation chart on a Milestone simply means, you are Segmenting transactions that have reached the corresponding Milestone by a specific Segmentation Category (eg: Age, Gender). Segmenting by Age on the Started milestone may present a different segmentation chart compared to Segmenting by Age on the LinkedIn prefill milestone. This allows great flexibility in the Analytics reports you want to generate and also gives great insights at various levels with a click of a button.
You can send Segmentation events from a Transact Application or from a Fluent/Groovy service on TM. Additionally, you can also send Segmentation data via URL parameter of the Transact Application.
ECCL - A new Client-side Insights integration strategy
Only Available with Transact Maestro 18.05 and Transact Manager 18.05. Also note that, Insights support for Transact Maestro 18.05 Applications is only available with Transact Manager 18.05.
Prior to 18.05, the business logic of Insights that generates Insights events was implemented in the Maestro code base. This presented multiple challenges in terms of; the dependency on the Maestro release cycle, and the customer's schedule to upgrade the Transact Application, test and republish. It would take months to deliver a fix to the customer. With ECCL, most of Insights business logic has been migrated to an intermediary module which gets injected by a script via Transact Manager when a Transact Application is rendered on the browser. That means, if we fix something or introduce a new capability in data collection today and deploy it on the ECCL hosted server, all subsequent Application renders will automatically get the latest code without having to republish the Transact Application. This saves enormous amounts of time, effort and money for our customers. Most importantly, our customers get the latest Insights client capabilities very quickly.
The following figure explains the old and the new implementation.
ECCL support for Composer is yet to be planned in future releases.
Enhancements
Improved Data Security
Only Available with Transact Manager 18.05.
Collecting events from Insights enabled forms requires Transact Manager to perform a number of initialization steps prior to the Application being rendered. This entailed a trade off between Application render speed and event collection reliability & latency. Prior to 18.05, in favor of Application render speed, Insights took the approach of deferring the initialization from Transact Manager by creating an object called a FormRequest in the Insights back-end that contained the required information to authorize Insights events and map them to a data-set that belonged to the corresponding Transact Manager instance. As this approach did not block the Application rendering, there was no impact on performance. However, this approach leads to a race between the time FormRequest is available to Insights and the time Session and Analytics events comes in. Although we mitigated the race by retrying FormRequest tasks from Transact Manager, in some circumstances, it was possible for events to be dropped if their deferred initialization from TM failed repeatedly.
To address this, a new approach has been implemented that satisfies both constraints; Application render time is minimized, and it is no longer possible for legitimate & authorized events to be dropped. All information from the FormRequest is now included in the authentication token (JWT format) injected into Insights-enabled Applications. This eliminates the need for Transact Manager to communicate with the Insights back-end to populate this information. The information in the JWT is non-sensitive, and cryptographically signed, ensuring that it cannot be altered by the user; the Insights back-end will only accept tokens with a valid signature from Transact Manager.
The following figure explains the old and the new implementation.
Improved Insights Data Quality
Only Available with Transact Manager 18.05.
There are two changes which will improve the quality of the data collected in Insights:
- ECCL implementation adopts the latest browser technologies such as sendBeacon provided by most modern browsers. This capability allows the browser to send queued events even after the browser is closed abruptly. A key scenario where we believe this will help is for Bounced transactions for which the user closes the browser too quickly. Presently, multiple customers have seen a huge number of bounced transacts categorized with an Unknown browser. That is because Insights has not received a Session event from the browser. With ECCL, even if the user closes the browser too quickly, the Insights back-end will still get the session event. Additionally, ECCL implements Priority event queues. Both a critical queue, for events that need to be sent immediately, for example: Transact status events or Section level events, and a batch processed queue, which will hold lower priority events which are periodically processed at 5 second intervals. This strategy will also contribute to improved data quality.
- The new authorization strategy for Insights event collection makes the authorization stateless and hence removes all inter-event dependencies that existed prior to 18.05. Previously, if for some reason, there was a delay in receiving a form_request from TM, and we started receiving Analytics events from the browser, we would retry for 5 times with a gap of 15 mins and if the form_request was still unavailable at the 5th attempt, we would drop all the events for that transaction. A similar dependency existed between Session event and Analytics event from the browser itself. With the new stateless authorization model, we were able to remove all dependency checks in the data processing pipeline. This new data processing pipeline is also highly scalable with improved performance compared to its predecessor. This also allows Avoka to keep the costs of Insights operations lower.
Release Details
18.05.1
Resolved Issues
You need to login to access this content. If you still don't have access after logging in, you can request it by posting a new question and selecting the access you need in the Type dropdown.
18.05.0
Features and Enhancements
| Reference | Summary | Description | Feature |
|---|---|---|---|
| TID-1184 | Implement EventConsumer client library | This implements a standard client library for EventConsumer, to be served from GCP and versioned along with all GAE modules, and consumed by both Composer and Maestro. | ECCL |
| TID-1228 | Maestro - Implement new EventConsumer library | Replace existing Maestro insights client integration to make use of common Insights client API. | ECCL |
| TID-1231 | EC Client Lib: Build setup |
Add a new webpack configuration that builds the Insights client library. In particular:
|
ECCL |
| TID-1240 | EC backend changes to support client library | Implement a new endpoint to receive sendBeacon requests, accepting the auth token as a URL parameter. Generate milestone opened upon storing a session payload, if the payload contains a flag indicating it came from the new client library. | ECCL |
| TID-1258 | publicPath for ECCL | Determine if we need publicPath set on output for ECCL when loading the library from a CDN. When setting up maestro integrations have only been able to succesfully load additional chunks required for ECCL when public path was set. (can likely just use an arg for our webpack build that corresponds to the deployment environement e.g. DEV, DEV1, ... , PROD, NONE (for no publicPath) etc.) | ECCL |
| TID-1260 | ECCL integration infrastructure | Add package.json scripts to run integration tests and generate separate coverage report | ECCL |
| TID-1276 | Implement ECCL build-payloads module | Implement the build-payloads module of ECCL which is currently just a stub. | ECCL |
| TID-1341 | Polyfill Promise for Webpack machinery without using script-loader | Previously, we used Webpack's script-loader to evaluate promise-polyfill in the global scope, as Webpack requires Promise support for chunk loading. IE11 lacks native Promise support. While this approach worked fine during initial development where ECCL was run in a standalone test page, this cannot be used with TM-rendered forms, as script-loader internally uses eval() which is not permitted by TM's CSP, nor should we relax the CSP to permit the 'unsafe-eval' rule as this has security implications. This work implements an alternative approach that does not use script-loader, so that ECCL is usable in IE11. |
ECCL |
| TID-1856 | Flush non-critical queue on section completion | Flush the non-critical queue on section completion. Investigate if we are flushing the queues when form is submitted. | ECCL |
| TID-1752 | ECCL - Use hashed requestLogKey in all payloads | Update ECCL to use the sha256 hash of the requestLogKey throughout, to match TM changes done for TID-1630 | ECCL |
| TID-1287 | User journeys and segmentation | This feature allows customers to analyse user journeys through a form and do segmentation analysis at each milestone so that they can optimize forms for specific user journeys and for specific segment of users. | User journeys and segmentation |
| TID-857 | Segmentation with form URL | This allows customers to send Segments via URL parameter of the form. | User journeys and segmentation |
| TID-1473 | Milestone view front end implementation | Implements the front end for the user journey graph. This includes ability to select milestones, viewing key metrics for each milestone and links between them and ability to add a segment graph in the view, view the segment graphs added for a milestone and remove it from the view. | User journeys and segmentation |
| TID-1485 | User journey graph backend | Backend implementation of View and Queries for user journey graph | User journeys and segmentation |
| TID-1486 | Milestone:Segmentation pair analyses backend |
Implement the backend required to present the segmentation data. Basically, it presents segmented data for a group of transactions that have reached a specific milestone. There will be one chart component for each segmentation:milestone pair selected by the user. The data includes
|
User journeys and segmentation |
| TID-1751 | PII deterrence warning message | A note has been added in the segment whitelist UI advising customers to make sure the segment whitelist doesnt not allow PII from being stored in Insights. | User journeys and segmentation |
| TID-1771 | Segment charts front end implementation | Ability to add a segment graph in the user journey view, view the segment graphs added for a milestone and remove it from the view | User journeys and segmentation |
| TID-1800 | User journey graph UI enhancements | UI enhancements to improve readability of user journey graph and fix UX issues | User journeys and segmentation |
| TID-1857 | User journey graph UI enhancements | Inverse the zoom logic for user journey | User journeys and segmentation |
| TID-1356 | Improve data ingestion design and implementation | This improves data ingestion strategy so that, there is minimal data loss and Insights data is more reliable | Event data ingestion optimizations |
| TID-1496 | Streamline event ingestion | As a result of the changes made in TID-1407 we now have a clear separation between ingested data and analysed data. This presents an opportunity to begin simplifying our event ingestion model. Details of implementation to be supplied in sub tasks. | Event data ingestion optimizations |
| TID-1538 | Remove dependency on ti-eventconsumer client library from TM | This enhancement allows TM to remove all Insights specific JAR dependencies from TM. This amends InsightsFormHelper to construct appropriate POST bodies and send these via whichever HTTP client library is preferred to the existing EventConsumer endpoints, rather than using the generated EventConsumer client library. | Event data ingestion optimizations |
| TID-1353 | Improve transaction metadata ingestion | With the new authorization model, ingesting metadata of a transaction (FormRequest) is made stateless which improves both data security, and the reliability of storing FormRequest, FormSession and Analytics events successfully. |
Event data ingestion optimizations |
| TID-1304 | Data quality improvements | Improves Ingestion mechanism for FormMetadata and FormSession and removes dependencies when processing different kinds of Insights events. This avoids event drops and hence improves data quality. |
|
Resolved Issues
You need to login to access this content. If you still don't have access after logging in, you can request it by posting a new question and selecting the access you need in the Type dropdown.