Help us improve your experience.

Let us know what you think.

Do you have time for a two-minute survey?

 
ON THIS PAGE
 

Monitoring the System

This section points to parameters that are useful for monitoring the health of the Paragon Active Assurance system.

System Parameters

Standard system parameters:

  • CPU utilization
    • Should stay below 80%
  • Memory utilization, excluding cache and buffers
    • Should stay below 80%
  • Disk utilization
    • Should stay below 80%

Application Parameters

For Control Center, run the command

The following parameters are of particular interest:

  • test_agent_appliance_online: This indicates how many Test Agent Appliances are currently logged in to the server. The desirable numbers here are of course dependent on how you deploy the Test Agents.
  • scheduled_call_latency: This indicates how far behind schedule the background job processing is. Among other things, this processing collects results from Test Agents and generates periodic reports. The latency should stay below 10 s.

Licenses

The expiry date for the license activated on the server can be inspected with the command

In the output from this command, look for end_date.

Processes

The following ncc processes should normally be running:

  • netrounds-callexecuter
  • netrounds-confd
  • netrounds-plugin
  • netrounds-probe-login
  • apache2
  • kafka
  • openvpn@netrounds
  • zookeeper
  • netrounds-metrics (if Streaming API or TimescaleDB is enabled)
  • netrounds-timescaledb (if Streaming API or TimescaleDB is enabled)

Logs

Webapp Logs

The file /var/log/apache/netrounds_access.log contains all HTTP requests made to the Control Center web GUI.

The file /var/log/apache/netrounds_error.log contains all errors reported by Apache for HTTP requests towards the Control Center web GUI. Console output from the Control Center back-end is also in this file; by default, all logging is done by the console.

The configuration of the logging is done using the Python logging module configuration schema (docs.python.org/2/library/logging.config.html#configuration-dictionary-schema).

Call Executer Logs

callexecuter is a service that executes background tasks such as configuring Test Agents or fetching metrics.

To see the logs for this service, you can use journalctl, for example to look at the logs from the last hour and then follow all incoming logs:

Plugin Service Logs

For the plugin system you can look at the logs using:

Metrics Service Logs

Note:

These logs are only relevant if you have this feature enabled, which is the case if you are using the Streaming API or TimescaleDB.

The metrics service is responsible for ingesting metrics data into TimescaleDB and to push such data to the Streaming API, insofar as these are enabled.

TimescaleDB

To view TimescaleDB logs, run:

Kafka

To view Kafka logs, run:

Zookeeper

Zookeeper is relied on by Kafka.

To view Zookeeper logs, run:

Custom Formatters

Paragon Active Assurance provides two custom formatters that yield additional information about log entries and thus should be favored over Python's default formatters:

  • The formatter netrounds.utils.loggers.ContextFormatter provides record_tags and context_tags that provide additional information about the context of the log entry, for example what Test Agent the log entry is about. Colors are also supported.
  • The formatter netrounds.thirdparty.logstash_formatter.ContextLogstashFormatterV1 outputs a JSON format compatible with Logstash.

The custom formatters are added by default in:

  • LOGGING['formatters']['context']
  • LOGGING['formatters']['context_color']
  • LOGGING['formatters']['logstash']