Data Retention Management Service

   Journey Manager (JM) The transaction engine for the platform. |   System Manager / DevOps |  24.04This feature was updated in 24.04

Manager comes with the Data Retention Management service that performs the following critical steps in the transaction life cycle:

• Task expiry
• Abandonment
• Data purging

This service is periodically called by the Data Retention Job and the Transaction Processor service, if its configuration has the Include Data Retention checkbox selected.

To configure the Data Retention Management service:

1. Select Services > Core Global Services.
2. Select the Data Retention Management from the Type dropdown list to show all services of this type.
3. Locate the Data Retention Management service and click Edit.
4. Check the standard service settings.
5. Select the Parameters Edit tab to view and customize the service configuration.

   

   Note

   For explanation of the parameters, see Global Data Retention Configuration.

   Some of the service's settings duplicate the global data retention configuration settings, so you can override them.

6. Select the Enforce Global Threshold checkbox to ensure that data retention policies can't be longer than that of global system or organization settings.
7. Edit the maximum number of items to be purged per each category: submissions, collaboration jobs, transaction history, error logs, and so on in the Fetch Limit For Transaction Purging field, which is. The default is 500. For more information, see Transaction Processor service.
8. Select the Log to Event Log checkbox to log the results of applying the data retention policy to the event log, so you can see what data was deleted from the system. The example of a message recorded in the even logs is shown bellow:

   Data Retention Management applied, 0 records deleted in 3 sec.
   
   "Table","Rows","Time (ms)","Percentage"
   "Delete Submission Data",0,77,2.0
   "Job",0,7,0.2
   "Submission",0,10,0.3
   "Transaction History",0,4,0.1
   "Audit Log",0,4,0.1
   "User Auth Event",0,3,0.1
   "Error Log",0,13,0.3
   "Event Log",0,3784,96.2
   "Groovy Service Log",0,3,0.1
   "Offline Client System Log",0,3,0.1
   "Offline Sync Call",0,4,0.1
   "Offline Sync Log",0,3,0.1
   "Scheduled Job History",0,3,0.1
   "Security Manager Log",0,3,0.1
   "Email Queue",0,4,0.1
   "Composer Package Queue",0,3,0.1
   "Import Action",0,5,0.1

   Then you can search for particular retention event in the event logs using the following criteria:

   

9. Select the age in days at which an audit log entry becomes eligible for purging from the Max Audit Log Age Days dropdown list.
10. Edit the number of days in the Max Collaboration Job Abandon Age Days field to specify when a collaboration job and all its transactions become eligible for abandonment.
    Note

    You can disable collaboration job abandonment by setting the Max Collaboration Job Abandon Age Days to 0.

    Note

    Check collaboration job abandonment configuration in Global Data Retention Configuration and Configure Organization Data Retention.

11. Edit the age in days at which a collaboration job and its transactions become eligible for abandonment in the Max Collaboration Job Age Days field. The enforced range of this value is:
    • Min: 1 day
    • Recommended: 30 days
    • Max (Strict mode): 730 days
    • Max (Relaxed mode): 730 days
    Note

    Manager enforces the maximum limits of data retentions to provide required system stability.

12. Edit the age in days at which a collaboration job data become eligible for purging in the Max Collaboration Job Data Age Days field.
13. Edit the maximum period of time, in seconds, allocated for data retention purging in the Max Data Retention Period field. The default is 120. For more information, see Transaction Processor Service.
14. Edit the age in days at which delivered transaction data (XML, PDFs, file attachments) become eligible for purging in the Max Delivered Submission Data Age Days field. The default is 10.
15. Select the age in days at which an email queue entry becomes eligible for purging from the Max Email Queue Age Days dropdown list. The default is 30 days.
16. Select the age in days at which an error log entry becomes eligible for purging from the Max Error Log Age Days dropdown list. The default is 30 days.
17. Select the age in days at which an event log entry becomes eligible for purging from the Max Event Log Age Days dropdown list. The default is 30 days.
18. Select the age in days at which a Groovy service log entry becomes eligible for purging from the Max Groovy Service Log Age Days dropdown list. The default is 30 days.
19. Select the age in days at which a T.Field App client and sync log entry become eligible for purging from the Max Offline Sync Log Age Days dropdown list. The default is 30 days.
20. Edit the age in days after which saved transactions will be abandoned in the Max Saved Submission Age Days field. The default is 30 days.
21. Select the age in days at which a scheduled job history entry becomes eligible for purging from the Max Scheduled Job History Age Days dropdown list. The default is 30 days.
22. Select the age in days at which a security manager log entry becomes eligible for purging from the Max Security Manager Log Age Days dropdown list. The default is 30 days.
23. Edit the age in days after which a transaction record becomes eligible for purging in the Max Submission Log Age Days field. The enforced range of this value is:
    • Min: 1 day
    • Recommended: 100 days
    • Max (Strict mode): 730 days
    • Max (Relaxed mode): 1825 days
24. Select the age in days at which a transaction history entry becomes eligible for purging from the Max Transaction History Age Days dropdown list. The default is 180 days.
25. Select the age in days at which a user authentication event entry becomes eligible for purging from the Max User Auth Event Age Days dropdown list. The default is 180 days.
26. Select the required purge mode from the PII Search Purge Mode dropdown list to specify when PII searching metadata, such as data extracts and transactions properties, will be deleted:
    • Form Completed: the smallest set of data to be deleted.
    • Delivery Completed: a bigger set of data to be deleted.
    • PII Purge Time: the biggest set of data to be deleted and it takes the longest time to do it.
27. Click Save to update the changes.

Next, learn how to view all core global services.