Retention Policies in Paragon Automation

Use this topic to learn about the different retention policies used in Paragon Automation.

A retention policy is the period of time for which data is retained in the various databases before being purged. Retention policies in Paragon Automation vary by use case components and are dependent on the database and services used by the features. The following sections list the retention policies for the different use cases and components in Paragon Automation.

Observability

Table 1: Retention Policy for Observability Features
Feature	Retention Policy
KPIs and events	Stored for seven days.
Device logs	Stored for seven days.
Device alerts	Purged two hours after the alert is closed.
Device alarms	Stored in OpenSearch indefinitely or until the data is manually deleted.

Service Orchestration

Table 2 lists the retention policies used in service orchestration.

Table 2: Retention Policy for Service Orchestration
Feature	Retention Policy
Service instance	Stored until the service is deleted.
Service order history	Stored until the service is deleted.
Service order modifications	Stored in the audit log.
Device configuration modifications	Stored in the audit log
Workflow logs in Airflow	Stored in the PVC at /opt/airflow/mount/logs, with a default storage capacity of 10-GB. A DAG, `airflow_logs_cleanup.py`, is scheduled to run every 6 hours to trim the oldest logs when the PVC disk usage exceeds 80%. To retain logs for a longer period, increase the PVC size to allocate more storage space.

Active Assurance

Table 3 lists information on the retention policy for metrics related to a stream.

Table 3: Retention Policy for Active Assurance
Metrics for a Stream	Retention Policy
Raw Data	3 days Note: Active Assurance does not store the raw data forever. As the data gets older, the data is stored at a lower resolution (concise version of a data). This process is called as data rollup. Data can be rolled up for a period of 1 minute, 5 minutes, 30 minutes, and 1 hour.
1 minute data rollup	7 days
5 minutes data rollup	30 days
30 minutes data rollup	180 days

Administration

Table 4 lists the retention policies used by the administration features.

Table 4: Retention Policy for Administration Features
Data	Retention Policy
Audit logs	3 months
System logs	30 days
Device backups	2 months

Infrastructure

Table 5 lists the retention policies used by the Paragon Automation infrastructure.

Table 5: Retention Policy for the Infrastructure Databases
Data	Retention Policy
OpenSearch data	Indices with -_yyyymm or __yyyymm or _yyyymm-xxxxxxx names are stored for the last three months. Indices with -_yyyymmdd or *__yyyymmdd names are stored for the last 30 days. An automatic cleanup script runs every day at midnight to clean up indices that have exceeded their retention period. However, if the corresponding application cleans up the application data earlier, the application cleanup task will override the OpenSearch retention policy.