Help us improve your experience.

Let us know what you think.

Do you have time for a two-minute survey?

Monitoring and Troubleshooting Puppet for Junos OS

 

Reporting for Puppet Agents Running Junos OS

You can require a Puppet agent to compile reports containing the log messages and metrics that are generated during configuration updates. To require that the Puppet agent report to the server after each transaction, you must set the agent report value to true in the puppet.conf file. If you enable reporting, by default, the agent node sends a YAML-formatted transaction report to the same server from which it downloads its configuration.

Puppet log messages can identify the source, severity level, and timestamp of the message, information about the operation or error that generated the message, and any tags associated with that operation or error. The Puppet agent always generates log messages with a severity level of notice, info, or err as part of a normal update. To generate log messages with a severity level of debug, you must specify the --debug option when you run the Puppet agent.

The Junos OS provider code for the netdev_stdlib_junos module designates log entries specific to Junos OS processing with source: JUNOS. Table 1 describes the Puppet agent reporting logs generated for Junos OS operations.

Table 1: Puppet Agent Reporting Logs for Devices Running Junos OS

Severity Level

Operation

Message Content

Tags

debug

configuration changes

Junos OS configuration changes in XML format.

debug, config, changes

debug

operational updates

Information concerning the operation, for example: "Opening a local connection: jex.example.com".

debug

err

commit operation failed

Reason for failed commit.

config, fail

info

commit operation requested

Number of configuration changes.

config, commit

notice

configuration changes

Junos OS configuration changes in a diff format.

config, changes

notice

commit operation successful

Commit success message.

config, success

The following examples show sample log messages generated by a Puppet agent while performing a configuration update.

  • The following sample log message shows that the Puppet agent requested a commit operation involving one change to the configuration:

  • The following sample log message shows that the Puppet agent requested the specified update to the configuration. The message only displays the configuration differences.

  • The following sample debug log message shows that the Puppet agent requested the specified update to the configuration. This is the same configuration request as in the previous example, but in this case, the message displays the configuration data using XML format. To generate log messages with a severity level of debug, you must specify the --debug option when you run the Puppet agent.

  • The following sample log message shows a successful commit operation on the agent node:

Troubleshooting Puppet for Junos OS Errors

The following sections outline errors that you might encounter when using Puppet to manage devices running Junos OS. These sections also present potential causes and solutions for each error.

Troubleshooting Junos OS Configuration Exclusive Lock Errors

Problem

Description: The Puppet agent cannot obtain an exclusive lock on the configuration. Thus, the dependency on the netdev_device fails causing the Puppet agent to skip configuration updates for all netdev resources.

Cause

Another user currently has the exclusive lock on the candidate configuration or is modifying the configuration.

The following sample error output indicates that the configuration database is locked by another user:

The following sample error output indicates that the configuration database has modifications in progress:

Solution

Wait until the lock on the configuration is released. When the Puppet agent retrieves the configuration and can obtain an exclusive lock on the configuration database, the agent updates the system configuration accordingly.

Troubleshooting Junos OS Configuration Load Errors

Problem

Description: The Puppet agent is unable to load the requested configuration changes into the candidate configuration.

Cause

The configuration change might contain invalid syntax, elements, or values.

The following sample error output indicates that the Puppet agent attempted to set the VLAN ID to 9999, which is out of the accepted range of 1 through 4094:

Solution

Correct the corresponding Puppet manifest file so that it contains valid configuration changes for the agent node.

Troubleshooting Junos OS Configuration Commit Errors

Problem

Description: The Puppet agent is unable to commit the requested configuration changes.

Cause

The configuration change might contain invalid syntax, elements, or values.

The following sample error output indicates that the Puppet agent attempted to associate an interface with a nonexistent VLAN:

Solution

Correct the corresponding Puppet manifest file so that it contains valid configuration changes for the agent node.

Troubleshooting Junos OS Configuration Errors

Problem

Description: The log files indicate that the Puppet agent successfully committed the configuration, but the agent node does not reflect the desired configuration changes.

Cause

There can be multiple reasons why the agent node does not reflect the correct configuration.

  • The Puppet manifest contains incorrect configuration information.

  • The Puppet agent has not yet performed the configuration update for the latest catalog.

    To verify that the Puppet agent has downloaded and committed a specific catalog, issue the show system commit operational mode command on the agent node running Junos OS to view the commit history and catalog versions.

    root@jd.example.com> show system commit

Solution

If the Puppet manifest file contains incorrect configuration changes, correct the file to include the desired configuration changes for the agent node.

If the Puppet agent has not yet installed the changes in the latest catalog, wait until the update is made and then verify the configuration.

Troubleshooting Agent Errors on an EX4300 Switch

Problem

Description: On an EX4300 switch, the Puppet agent reports errors during a run which involves configuring a large number of number of VLANs. For example, you might see a “Could not send report” or "Could not run: failed to allocate memory" message.

Cause

Memory limitation on EX4300 devices.

Solution

Divide the VLAN configuration across multiple manifest files and apply each manifest file in a separate Puppet agent run.

For example, suppose you have 1024 VLANs. You can split the VLAN configuration across four manifest files (vlan1.pp, vlan2.pp, vlan3.pp, and vlan4.pp) so that each manifest file contains configuration for 256 VLANs. Then run the Puppet agent four times, changing the node definition in the main manifest file as follows on each agent run:

  • First agent run:

  • Second agent run:

  • Third agent run:

  • Fourth agent run:

Troubleshooting Connection and Certificate Errors on Puppet Clients

The following sections outline errors that you might encounter on Puppet clients running Junos OS. These sections also present potential causes and solutions for each error.

Puppet Client Request Certificate Error

Problem

Description: The Puppet client generates an error that it cannot request a certificate from the Puppet master.

Cause

The Puppet master might not be running an instance of the puppet master process.

On the Puppet master, review the list of active processes to determine whether the puppet master process is running. The output should include the puppet process if it is already running.

[root@puppet-master ~]# ps aux | grep puppet

Alternatively, on the Puppet client, telnet to the Puppet master on port 8140. If the puppet master process is not running, the connection fails.

% telnet puppet-master.example.com 8140

Solution

If the Puppet master is not running an instance of the puppet master process, start the process by issuing the puppet master command with any required options. Then verify that the process is running.

[root@puppet-master ~]# puppet master options
[root@puppet-master ~]# ps aux | grep puppet

Puppet Client No Certificate Found Error

Problem

Description: The Puppet client generates a no certificate found error and fails to download the catalog from the Puppet master.

Cause

The error might indicate that the certificate for the Puppet client is not signed.

Solution

On the Puppet master, sign outstanding client certificate requests using the puppet cert sign command. For example:

[root@puppet-master]# puppet cert sign puppet-client.example.com

See the official Puppet documentation for detailed information about Puppet commands.