NorthStar Analytics Data Retention Policy

The two parameters described in Table 1 work together to control how long collection logs remain in the elasticsearch database. Both parameters are located in /opt/northstar/data/northstar.cfg, and both are user-configurable.

Table 1: Data Retention Policy Parameters
Parameter	Description
collection_cleanup_task_interval	Controls how often the collector-utils.py utility is called upon to clean up old logs. The default is one day (1d). The collector-utils.py utility runs at approximately 1:00 AM, NorthStar server time. Units can be hours (h), days (d), or weeks (w).
es_log_retention_days	Defines what is considered an “old log”. The default is 90 days. This can be expressed only in days, so no unit designation is required.

The collector-utils.py utility uses the elaticsearch APIs to clean up logs older than the value of the es_log_retention_days parameter. The cleanup task is called from the NorthStar server.

To modify the collection_cleanup_task_interval or es_log_retention_days parameter, use a text editing tool such as vi and modify the value of the parameter. For example:

In this example, logs older than 30 days are purged every seven days.