Endpoint Monitoring with Service Groups
Service group is a logical collection of URL endpoints that can be monitored as a single entity. This feature performs continuous monitoring of all endpoints, monitors their reachability, and collects corresponding latency metrics. Contrail Insights supports charting of metrics collected for each endpoint.
There are two ways to add Service Groups in Contrail Insights:
Add service groups using the Contrail Insights Dashboard.
Add service groups using Ansible.
Based on your system setup the REST calls being made can
take more time. Use appformix_agent_timeout_rest_client
in the group_vars/all
file to set a timeout
value suitable to your system. The default value is two (2) seconds.
Example:
appformix_agent_timeout_rest_client: 5
Monitor Service Groups on User-Specified Agents
By default, service groups are monitored from the Contrail Insights Platform nodes. You can monitor service groups on specific Contrail Insights Agent by specifying them in the Ansible inventory.
Example:
In the Ansible inventory directory, edit the inventory/hosts
file to add the following:
[appformix_service_group_agents] 172.16.70.220 172.16.70.221
Now service groups are monitored on the Agents specified below
the appformix_service_group_agents
tag.
Add Service Groups Using Contrail Insights Dashboard
To add a service group to the Dashboard:
Select Settings in the top right of the Dashboard as shown in Figure 1.
Figure 1: Settings in DashboardSelect Services Settings.
Figure 2: Services Settings for Service GroupsIn the Services Settings panel, select the Service Groups tab and complete the necessary fields.
Figure 3: Service Groups TabContrail Insights supports monitoring of OpenStack URLs, regular URLs, and ICMP (Internet Control Message Protocol) URLs.
To monitor ICMP endpoint URLs:
Click Add Service Group.
Figure 4: Add Service Group to Monitor ICMP Endpoint URLsComplete the indicated fields, as shown in Figure 5.
For Protocol, select ICMP Ping from the drop-down list.
Figure 5: Add Service Group Details for ICMPEnter the ICMP endpoint you want to monitor in the URL field and the interval at which it needs to be monitored.
Click Add Endpoint when done configuring the endpoint. Multiple endpoints can be monitored under a single service group.
When done adding endpoints for this specific Service Group, click Setup.
To monitor OpenStack endpoint URLs:
Click Add Openstack Service Group.
Figure 6: Add OpenStack Service GroupFrom the drop-down list, select the type of OpenStack service endpoint that needs to be monitored. This will autogenerate a service group name, which you can modify, if needed.
Figure 7: Autogenerated Service Group NameEnter the Username and Password for the Contrail Insights Credentials.
Enter the OpenStack endpoint you want to monitor in the URL field, the interval at which it needs to be monitored, and the type of REST call that needs to be made.
Click Add Endpoint when done configuring the endpoint. Multiple endpoints can be monitored under a single service group.
To monitor regular endpoint URLs:
Click Add Service Group.
Figure 8: Add Service Group to Monitor Regular Endpoint URLsComplete the indicated fields, as shown in Figure 9. Default Protocol is HTTP/HTTPS.
Figure 9: Add Service Group Endpoint DetailsTo add more endpoints, click Add Endpoint. Following is an example where three endpoints are configured for one service group.
Figure 10: Add Three Endpoints for One Service GroupWhen done adding endpoints for this specific Service Group, click Setup.
The Service Group will show as successfully added. It can be deleted by clicking the Trash icon.
Figure 11: Add Service Group Successful
Add Service Groups Using Ansible
Profile Overview—In the directory agent/tools/ansible/profiles/, there are five profiles each pertaining to an OpenStack service. The prefix of each file is the name of the OpenStack service; either cinder, glance, keystone, neutron, or nova. The suffix is *_default_service_profile.json.j2. For example, the profile for the OpenStack service Glance is named glance_default_service_profile.json.j2.
The default layout of the Glance profile is shown in the following example. The other profiles have an identical layout, just with the corresponding OpenStack service listed.
Glance Profile Example:
{ "ServiceGroupName": "AppformixGlanceServiceGroup", "Protocol": "http_or_https", "Endpoints": [ { "Url": "{{ glance_url }}", "EndpointName": "glanceEndpoint", "Method": "GET", "Interval": 2 } ], "ServiceGroupId": "GlanceServiceGroupId", "RefreshTokenData": { "RefreshToken": "False", "Username": "admin", "AuthType": "openstack", "Password": "", "AuthUrl": "", "Project": "" } }
ICMP Profile Example:
{ "ServiceGroupId": "ICMP_service_group_id", "Protocol": "icmp_ping", "ServiceGroupName": "ICMP_test", "Endpoints": [{ "EndpointName": "icmp", "Url": "127.0.0.1", "Interval": 2 }] }
Add an HTTP Profile—Profiles support unauthenticated and authenticated endpoints.
Unauthenticated Endpoint
To add an unauthenticated endpoint:
Add the variable that the
Url
key is mapped to to yourgroup_vars/all
file.Confirm this variable is mapped to a working endpoint.
For example: In the
group_vars/all
, if you are using the Glance profile, add theglance_url
variable as shown here:glance_url: "http://0.0.0.0:9292"
Authenticated Endpoint
To add an endpoint that needs authentication, a RefreshToken is required. A RefreshToken enables access to endpoints that require authentication, as well as keeps that access by getting a new token when the current one is about to expire.
To obtain a refresh token:
Set the
RefreshToken
field in theRefreshTokenData
dictionary to beTrue
.Then provide
Username
,Password
, andAuthUrl
in the sameRefreshTokenData
.
GET and POST Examples for Refresh Token
GET Example:
{ "ServiceGroupName": "AppformixGlanceServiceGroup", "Protocol": "http_or_https", "Endpoints": [ { "Url": "{{ glance_url }}", "EndpointName": "glanceEndpoint", "Method": "GET", "Interval": 2 } ], "ServiceGroupId": "GlanceServiceGroupId", "RefreshTokenData": { "RefreshToken": "True", "Username": "admin", "AuthType": "openstack", "Password": "password", "AuthUrl": "auth_url", "Project": "" } }
POST Example:
{ "ServiceGroupName": "AppformixGlanceServiceGroup", "Protocol": "http_or_https", "Endpoints": [ { "Url": "{{ glance_url }}", "EndpointName": "glanceEndpoint", "Method": "POST", "Interval": 2, "Data": "{\"AuthType\":\"openstack\", \"UserName\": \"admin\", \"Password\": \"password\"}" } ], "ServiceGroupId": "GlanceServiceGroupId", "RefreshTokenData": { "RefreshToken": "True", "Username": "admin", "AuthType": "openstack", "Password": "password", "AuthUrl": "auth_url", "Project": "" } }
Add a Profile or Multiple Profiles to Ansible
Using Ansible, a profile corresponding to a service group can be added to the Contrail Insights Dashboard during the installation.
To add a profile to the Contrail Insights Dashboard:
Add the variable
appformix_service_connectivity_profiles
to yourgroup_vars/all
file.Map the variable to a list of dictionaries. Each dictionary in the list should only contain one key and one value.
The key should always be
connectivity_profiles
.The value should be the path of the profile you want added to the Contrail Insights Dashboard during installation. An example follows:
appformix_service_connectivity_profiles: [{ connectivity_profiles: 'profiles/glance_default_service_profile.json.j2' }]
To add multiple profiles, repeat these steps for as many profiles as needed.
View Service Groups
Successfully added service groups are viewable from the Dashboard.
To view service groups from the Dashboard:
Select Infrastructure > Service Groups.
Select the Service Group you want to view.
Click Charts to view data being collected for this Service Group's endpoints.
Select Charts to view the Charts display and endpoint details.
Create Alarms for Service Groups
To create alarms for service groups:
After a service group is created, navigate to the Alarms page and click Add Rule.
Figure 12: Creating an Alarm for Service GroupFor Scope, select Service Group and for Metric, select service_group.heartbeat (default). For Alarm Rule Type, both static and dynamic alarms are supported for service groups.
Figure 13: Adding Alarm Rules for Service GroupComplete any further details and click Save to confirm.
Figure 14: Saving Alarm Rules for Service GroupAfter the alarms are triggered, they are visible on the Dashboard as active or inactive based on the rules set.
Figure 15: Service Group Triggered Alarm Visible on Dashboard
Health and Risk SLA for Service Groups
To create health and risk service-level agreements (SLAs) for service groups:
Select Settings in the top right of the Dashboard.
In Settings, select SLA Settings.
In Health Profile, click the Service Group tab.
Figure 16: Health Profile Service Group TabBy default, Contrail Insights has a Health/Risk profile created for all the service groups. Click Delete Profile to add new profiles and set up a new SLA.
Figure 17: Delete Profile in Service Groups to Add New Profile or New SLAWhen a service group is in bad health, it is reflected on the service groups Dashboard view based on the profile.
Figure 18: Example Service Group Alert for Missed Heartbeat on Dashboard
Service Group Configuration Examples
Following are service group configuration examples.
OpenStack Service Group configuration example:
{ "ServiceGroupName": "AppformixGlanceServiceGroup", "Protocol": "http_or_https", "Endpoints": [ { "Url": "glance_url", "EndpointName": "glanceEndpoint", "Method": "GET", "Interval": 2 } ], "ServiceGroupId": "GlanceServiceGroupId", "RefreshTokenData": { "RefreshToken": "False", "Username": "openstack_admin", "AuthType": "openstack", "Password": "openstack_password", } }
ICMP Service Group configuration example:
{ "ServiceGroupId": "ICMP_service_group_id", "Protocol": "icmp_ping", "ServiceGroupName": "ICMP_test", "Endpoints": [{ "EndpointName": "icmp", "Url": "127.0.0.1", "Interval": 2 }] }
Regular Service Group configuration example:
{ "ServiceGroupName": "ServiceGroup", "Protocol": "http_or_https", "Endpoints": [ { "Url": "url", "EndpointName": "endpoint", "Method": "GET", "Interval": 2 } ], "ServiceGroupId": "ServiceGroupId" }
Service Group Alarm configuration example:
Service Group alarms have their own scope service_group
.
{ "Severity": "none", "IntervalDuration": "60s", "Module": "alarms", "ServiceGroupId": "Service_Group_Id", "ComputeMultipleBaselines": false, "IntervalCount": 1, "EventRuleType": "static", "IntervalsWithException": 1, "Name": "alarm_name", "LearningPeriodStart": 0, "ComparisonFunction": "above", "EventRuleScope": "service_group", "AggregationFunction": "max", "Sensitivity": "", "DisplayEvent": true, "MetricType": "service_group.heartbeat", "Threshold": 0, "Mode": "alert" }