Data Retention Management v2 Service

Journey Manager (JM) The transaction engine for the platform. | System Manager / DevOps | 24.10This feature was updated in 24.10

Manager comes with the Data Retention Management v2 service, which extends the Data Retention Management service and provides performance improvements, such as reduced memory footprint and more efficient database queries for the Transaction Processor service. Enabling and running the Data Retention Management v2 servicee will help clear any data backlog and gradually improve system performance.

Important

When you install or upgrade Manager, the Data Retention Management v2 service is not set as the default service for the Data Retention Management type - the Data Retention Management service is set by default. However, as with any other services, you can change it by following these steps:

Apply any existing Data Retention Management service settings to the Data Retention Management v2 service. This is particularly important as every client has their own customized settings which are aligned with particular business requirements and legal obligations to keep and purge PII data.
Set the Data Retention Management v2 service as default by clicking Make Default or selecting the Service Type Default checkbox.
Test the Data Retention Management v2 service in your lower environments, such as DEV and TEST.
Confirm successful results before moving to the next environment.
Enable it in your production environment when you are completely satisfied with test results.

To configure the Data Retention Management v2 service:

Select Services > Core Global Services.
Select the Data Retention Management from the Type dropdown list to show all services of this type.
Locate the Data Retention Management v2 service and click Edit.
Check and update the standard service settings.
Select the Parameters Edit tab to view and customize the service configuration.
Note
For explanation of the parameters, see Data Retention Management Service.
Some of the service's settings duplicate the Global Data Retention Configuration, so you can override them.
Warning
This service was introduced in Journey Manager 24.04 and then backported to all supported releases. However, there are some difference in the following versions:
- Prior to 24.04, not all of these settings are available.
- Prior to 23.10, some of these settings, such as Fetch Limit For Transaction Purging and Max Data Retention Period are only configured in the Transaction Processor service.
Select a size of a commit batch, which is used for transaction purging, from the Commit Batch dropdown list. The value must be within the fetch limit specified below. The default is 1000.

Select the Log to Event Log checkbox to log the results of applying the data retention policy to the event log, so you can see what data was deleted from the system. The example of a message recorded in the even logs is shown bellow:

Data Retention Management (v2.0.1) applied, 493 total records deleted in ~634 ms

Processing 1 of 2 - Transactions PII Data purged 1 records in ~18 ms:

"AWS Submission Data Storage" removed 1 records in 0 ms
"File Upload Batch Update" updated 0 records in 0 ms
"Submission History Batch Update" updated 0 records in 0 ms
"Async Msg and Body PII" removed 0 records in 0 ms
"Extracts and Search PII" removed 0 records in 3 ms
"File Uploads and Data PII" removed 0 records in 0 ms
"Groovy Service Logs and Data PII" removed 0 records in 1 ms
"Submission History Data PII" removed 0 records in 2 ms
"Job Properties and Search PII" removed 0 records in 2 ms
"Submission Error Logs" removed 0 records in 0 ms
"Submission Properties and Search PII" removed 0 records in 2 ms
"Submission Data PII" removed 0 records in 0 ms

Processing 2 of 2 - Transactions and Related Data purged 492 records in ~615 ms:

"Audit Log(s)" removed 0 records in 0 ms
"Email Queue(s)" removed 456 records in 42 ms
"Error Log(s)" removed 0 records in 18 ms
"Event Log(s)" removed 2 records in 14 ms
"Groovy Service Log(s)" removed 0 records in 13 ms
"Scheduled Job History(s)" removed 33 records in 15 ms
"Security Manager Log(s)" removed 0 records in 13 ms
"Submission(s)" removed 1 records in 453 ms
"Transaction History(s)" removed 0 records in 13 ms
"User Authorized Event(s)" removed 0 records in 14 ms
"Composer Package Queue(s)" removed 0 records in 15 ms
"Import Action(s)" removed 0 records in 2 ms
"Job(s)" removed 0 records in 1 ms
"Job Group(s)" removed 0 records in 1 ms
"Request Log(s)" removed 0 records in 0 ms
"Submission Group(s)" removed 0 records in 1 ms

Then you can search for particular retention event in the event logs using the following criteria: Submission Data Pii as a message. This is illustrated below.

You can further narrow the search down by using an event ID, for example, 748285, as shown below:

Select the age in days at which billing records become eligible for purging from the Max Billing Record Age Days dropdown list. Default is 60 days. | 24.10 This feature was introduced in 24.10
Select the age in days at which Event Inbox records (both Delivered and Failed) become eligible for purging from the Max Event Inbox Age Days dropdown list. Default is 3 days. | 24.10 This feature was introduced in 24.10
Select the age in days at which Event Outbox records (both Delivered and Failed) become eligible for purging from the Max Event Outbox Age Days dropdown list. Default is 3 days. | 24.10 This feature was introduced in 24.10
Select the Purge Submission Error Log With PII checkbox to delete all PIIPersonally Identifiable Information (PII) is information about an individual that can be used to distinguish or trace an individual‘s identity, such as name, social security number, date and place of birth, mother‘s maiden name, or biometric records; and any other information that is linked to an individual. In Europe, PII is known as personal data. data of a submission from the error log.
Click Save to update the changes.

Next, learn how to view all core global services.

Topics

Data Retention Management v2 Service

In this topic

Need More Help?