AppFormix with Kafka

Set Up Kafka

1. Create a Docker network for this Kafka container and its dependencies to be connected to by running the following command:
2. Next, bring up Zookeeper for Kafka to work:
3. Bring up the Kafka container by running the following. The variable ip_address must be specified appropriately.
4. If Kafka with SSL is required, then additional parameters are required:

   The secret_files_path should be replaced where all of the keystore and truststore files are present. For an example of how to create the above keystores and truststores, reference: https://github.com/confluentinc/cp-docker-images/blob/5.2.1-post/examples/kafka-cluster-ssl/secrets/create-certs.sh.

   Note

   If Kafka with SSL is required, then all of the hosts monitored by Appformix must have at least Python version of 2.7.9. It is also required that the Certificate Authority (CA) used for the certificates for the Kafka broker(s) be a trusted CA on all of the hosts monitored by AppFormix. In order for AppFormix containers to communicate with the Kafka broker(s), the CA file must be set as a group_vars/all variable appformix_kafka_ssl_ca at installation time.

   Now that Kafka is set up, next you can configure AppFormix with Kafka.

Set Up AppFormix with Kafka

To configure AppFormix with Kafka, a POST request must be sent to an AppFormix Controller API:

The following fields must be sent in this request:

1. Run the POST body request, which is similar to the following:
2. If Kafka has been set up with SSL, then an additional field is needed:

1. Select Settings in the upper right corner, then select AppFormix Settings > Kafka. Next, click + Add Config.

   Figure 1: AppFormix Settings for Kafka Page

   

2. Enter a name for the Kafka configuration and list the BootstrapServers as a comma separated list of strings with each string in the host:port format.

   Figure 2: Enter Information for Kafka Configuration Name and Bootstrap Servers

   

3. Click Setup after the fields have been populated. You will then see the Kafka configuration as shown in Figure 3.

   Figure 3: Completed AppFormix Settings for Kafka Configuration

   

   Note

   The following steps are for streaming network telemetry data to Kafka. All AppFormix alarms are automatically sent to Kafka once Kafka has been configured as stated in the earlier procedures above. There are no additional steps needed for alarms. See AppFormix Alarms With Kafka.

4. Click + Add Subscription to create a subscription.

   Figure 4: Add Subscription for Streaming Network Telemetry Data to Kafka

   

5. Next create a Topic, select devices, and then select which Sensors/MIBs you want sent to Kafka. The specified data will then be sent to Kafka under the specified topic. Click Create Subscription after the fields are populated.

   Figure 5: Create Topic, Select Devices and Sensors/MIBs to Send to Kafka

   

   You can see all of your successfully added Subscriptions in the table.

   Figure 6: Successfully Added Subscription for Streaming Network Telemetry Data to Kafka

   

Messages from AppFormix to Kafka

After configuration, messages from AppFormix are received by an appropriate Kafka consumer.

In the following command, bootstrap_server is one of the bootstrap servers specified in the BootstrapServers variable above and topic is the topic that was specified in the subscription created. :

This command outputs messages to standard output. Output for topic grpc-components with sensor /components/ selected will look something like:

AppFormix Alarms With Kafka

AppFormix Alarms are configured to automatically send alerts to Kafka, if Kafka has been configured in AppFormix. See Set Up AppFormix with Kafka. AppFormix sends alarms with the topic as the alarm's name. For example, Alarm name host_cpu is sent to Kafka with topic host_cpu.