Help us improve your experience.

Let us know what you think.

Do you have time for a two-minute survey?

 

Import Traffic

 

The Import Traffic Wizard is accessed via Traffic > Import Traffic. This wizard performs three functions which can be executed independently of each other:

  • Convert third party traffic data to IP/MPLSView traffic data.

  • Aggregate IP/MPLSView traffic data from multiple sessions to a single aggregated IP/MPLSView data set.

  • Convert aggregated IP/MPLSView data to a 24 period trafficload file.

Figure 110: Traffic Data Import Wizard
Traffic Data Import Wizard


Third Party Traffic Data > IP/MPLSView Traffic Data



Note

This feature requires a license. Please contact your Juniper Networks representative for more details on this feature.

The Import Traffic feature supports the following types of data files:

  • MRTG (Multi-Router Traffic Grapher)

  • Arbor Networks (XML)

  • Juniper Networks LDP Statistics Import

Note that, for popular traffic sources, Juniper Networks may offer customized utilities to perform the conversion from other traffic formats into IP/MPLSView’s required input format. Please contact Juniper Networks support to learn if IP/MPLSView supports conversion of your traffic measurement data type.

Figure 111: Select Import Directory, Import Type, and Output Directory
Select Import Directory, Import Type, and Output Directory

Field

Description

Select Import Type

Click to see the drop-down selection box to select the appropriate data type for import.

Select Import Directory

The directory shown here displays the current import directory. This field should specify the directory in which traffic data files are stored. Click on the Browse button to change this setting.

Select Parser Configuration File

This file contains configuration settings specific to the type of Import Type being performed.

Select Output Directory

The directory shown here displays the current output directory. This field should specify the directory in which output files are saved. Click on the Browse button to change this setting.

Depending on which data type is being imported, the next pages of the wizard will show different tabs and required parameters to be entered.

To import third party traffic data, select one of either MRTG, InfoVista, or Concord from the Import Type drop-down menu.

In order to import third party traffic sources, you must configure import options to perform pattern matching on filenames, indicators, and other fields. This requires knowledge of regular expressions. Examples are provided in this document. For information on regular expressions, please visit http://www.regular-expressions.info.



MRTG (Multi-Router Traffic Grapher)



Note

This feature requires a license. Please contact your Juniper Networks representative for more details on this feature.

Figure 112: Selecting MRTG as the Import Type
Selecting MRTG as the Import Type

This option imports traffic collected from an MRTG, or Multi-Router Traffic Grapher. The grapher generally provides traffic log files, whose filename contains delimiters that need to be specified in the import wizard. The format of a filename might look like “<nodename>-<interface>.log”, where the dash (“-”) is the delimiter, and “log” is the filename extension.

After you have selected the import directory in which these log files are located, the next page in the wizard prompts you for instructions on how to convert MRTG log file names to the correct node and interface names. In the example below, the goal is to use the Transform Pattern to convert filenames in the Text Source directory, which are displayed in the Text column, to the format specified by the Transform Template. Prior to applying the Transform Pattern, the Text Transform Substitutions are applied.

Figure 113: MRTG Filename Parsing
MRTG Filename Parsing

In the previous example, the Text Transform Substitution replaces the underscore with the forward slash. The Transform Pattern then matches the first word to the node name, uses the first forward slash as a delimiter, then matches the remaining word (up to “.old”) to the interface name. Thus, ahdbccr1_fxp0.old gets transformed to AHDBCCR1 FXP0.

Figure 114: MRTG Options
MRTG Options

The MRTG Options tab lists several options related to MRTG parsing, as explained in the following table.

Field

Description

Traffic Type

This is the type of traffic being collected. Choices are Interface, Tunnel, and PVC. For MRTG, this is usually Interface.

Data Type

This is the data type being collected. Choices are traffic delta, raw traffic count, traffic rate, and aggregate traffic rate. For MRTG, this should be traffic rate.

Data Cache Limit

This sets how much data to store in memory before flushing it out to a file. On most systems, setting this to 0 (no limit) should be fine. However, if the traffic source is extraordinarily large, it might be prudent to set a data cache limit.

Overwrite IP/MPLSView traffic data

This option indicates whether or not to overwrite existing traffic data in the output directory specified in the first step. If there is overlapping data in the output directory (identical interface and days), then if this option is checked, that data is overwritten. If this option is not checked, that data is averaged with the new data.

Conversion from traffic rate to bits

This is the rate used to convert the traffic unit in the source data to bits. Since MRTG reports traffic in bytes, a value of eight should be used here.

Sample Interval (minutes)

This is the sample interval to be used when collecting data from the traffic source.

Sample Interval Limit (minutes)

Data with sample intervals larger than the sample interval limit specified here are ignored.

Start Time

All data between the start time and the present time are extracted.

Interval (minutes)

This is the interval to be used in the 24 period trafficload file, which are generated at the completion of the Import Traffic Wizard. A typical value for the interval is 60 minutes.

The MRTG Config Directory is the location of the MRTG configuration files, which are used to determine the device/interface names. These config files can be created by using a perl script from the MRTG package which performs an snmp collection on all interface indices on each device. To create the MRTG configs, SNMP access to network devices and a list of the devices is necessary. Please ask your MRTG administrator for the location of MRTG configuration files.

At this point, clicking Next will start the conversion of MRTG data to IP/MPLSView traffic data. Once this conversion is complete, the next step is to convert IP/MPLSView traffic data to aggregate IP/MPLSView traffic data. Please skip ahead to that section, “IP/MPLSView Traffic Data > Aggregate IP/MPLSView Traffic Data” for more information.



Arbor Networks (XML)



Note

This feature requires a license. Please contact your Juniper Networks representative for more details on this feature.

Figure 115: Selecting Arbor XML as the Import Type
Selecting Arbor XML as the Import Type

Arbor traffic import can be used to import xml files with the end-to-end flow traffic at the router level. The following is a sample input file format:

To import the xml files, select the Import Directory (where the data is stored) and the Output Directory (where output files are written), and click Next. The automatic import process will then begin, which will convert Arbor traffic data to IP/MPLSView traffic data.

Once this conversion is complete, the next step is to convert IP/MPLSView traffic data to aggregate IP/MPLSView traffic data. Please skip ahead to that section, “IP/MPLSView Traffic Data > Aggregate IP/MPLSView Traffic Data” for more information.



Juniper Networks LDP Statistics Import



Note

This feature requires a license. Please contact your Juniper Networks representative for more details on this feature.

This option imports Juniper Networks LDP traffic statistics data files obtained by FTP from Juniper Networks devices (routers and switches) specifically configured to collect traffic statistics. Juniper Networks LDP traffic statistics can report the amount of traffic destined to each forwarding equivalence class (FEC) from a Juniper Networks device running LDP.

This is a different method than that employed by the LDP Traffic Collection Task, as described in the “Reference” chapter of the IP/MPLSView Java-Based Management and Monitoring Guide, which uses the CLI show command to collect Juniper Networks LDP traffic statistics. Both methods can be used to generate a demand and trafficload file. However, the Traffic Import method in this chapter assumes that you have already downloaded LDP traffic statistics data from the devices, whereas the Traffic Collection Task employs an online, automated approach, via CLI.



Prerequisites



To obtain Juniper Networks LDP traffic statistics data files for the traffic import, you must configure the traffic-statistics statement at the [edit protocols ldp] hierarchy level. This configuration causes the router to automatically gather the LDP traffic statistics and periodically write the statistics to a file. The relevant syntax is as follows:

where:

  • file filename—Name of the file to receive the output of the LDP statistics operation. Enclose the name within quotation marks. All files are placed in the directory /var/log.

  • files number—(Optional) Maximum number of LDP statistics files. When a statistics file named ldp-stat reaches its maximum size, it is renamed ldp-stat.0, then ldp-stat.1, and so on, until the maximum number of LDP statistics files is reached. Then the oldest file is overwritten.

    Range: 2 through 1000

    Default: 2 files

    If you specify a maximum number of files, you also must include the size statement to specify the maximum file size.

  • interval seconds—(Optional) The interval, in seconds, at which the statistics are polled and written to the file.

    Default: 300 seconds (5 minutes)

  • no-penultimate-hop—(Optional) Do not collect traffic statistics on the penultimate hop router.

  • no-world-readable—(Optional) Prevent all users from reading the log file.

  • size size—(Optional) Maximum size of each statistics file, in kilobytes (KB), megabytes (MB), or gigabytes (GB). When a statistics file named ldp-stat reaches this size, it is renamed ldp-stat.0. When ldp-stat again reaches this size, ldp-stat.0 is renamed ldp-stat.1 and ldp-stat is renamed ldp-stat.0. This renaming scheme continues until the maximum number of statistics files is reached. Then the oldest statistics file is overwritten.

    Syntax: xk to specify KB, xm to specify MB, or xg to specify GB

    Range: 10 KB through the maximum file size supported on your system

    Default: 1 MB

    If you specify a maximum file size, you also must also include the files statement to specify the maximum number of files.

  • world-readable—(Optional) Enable log file access for all users.

See the JUNOS® OS MPLS Applications Configuration Guide for more detailed LDP traffic statistics configuration details.

For instance, the following figure shows the traffic-statistics statement configured on a router called SFO.

Figure 116: LDP Stats Configuration
LDP Stats Configuration

The router SFO will then automatically collect LDP traffic statistics into the file /var/log/LDP_Stats every 300sec.

You can then use a utility such as SCP or SFTP to copy the compressed or uncompressed versions of the /var/log/LDP_Stats files to the IP/MPLSView server. For compressed (*.gz) files, be sure to manually uncompress the file before copying it to the server.

When you use the Import Traffic Wizard to import the LDP_stats.file, IP/MPLSView does not perform any integrity checks to validate whether the collected LDP statistics are complete. Incomplete LDP statistics can occur, for example, if there are days for which the LDP statistics are not fully collected on all Juniper Networks devices. Running network simulations using these partial LDP statistics as input can cause erroneous results.

To ensure that the LDP_Stats files are complete, we recommend that you create a preprocessing script that verifies the LDP statistics files from all imported devices, checks the included timestamps, and determines for which days the LDP stats are complete for all devices. You can then safely import the LDP_Stats files using the Import Traffic Wizard.

Note

When the ldp-stat file on a Juniper Networks device reaches the maximum configured size, it is closed and a new file is opened. In this case, an FEC statistics record might get split across two files. The IP/MPLSView Import Traffic Wizard cannot process an FEC statistics record that happens to get split across two files. To enable The IP/MPLSView Import Traffic Wizard to process the FEC statistics record, you must merge the two files.

For the Import Traffic Wizard to correlate the device with its LDP statistics data file, name the files using the format {router-node-ID}.LDP-STATS, irrespective of the LDP statistics filename configured on the router itself. For example, if the router hostname is SFO-re0 or SFO-re1 (depending on which Routing Engine is momentarily active) and the configured LDP statistics file on that router is named LDP.stats, then the LDP statistics data file should be named SFO.LDP-STATS.

Best Practice

IP/MPLSView does not create LDP statistics-based demands (traffic) bound for unknown destinations (that is, IP addresses that are not attached to any device in your IP/MPLS View topology), and does not generate an error log for such discarded demands. To obtain realistic simulation results, we recommend that you create a script to verify that all LDP destinations collected in LDP statistics files are actually present in the imported IP/MPLS View network topology.

To use the Juniper Networks LDP Stat traffic import feature, first start the Import Traffic Wizard by selecting Traffic>Import Traffic. In the Traffic Import Options window, select Select Import Type>Juniper LDP Stat.

In the Select Import Directory pane, click Browse and select the directory where the Juniper Networks LDP Statistics data files are located. Under the Select Output Directory pane, click Browse and select the directory where IP/MPLSView traffic data object files (created in the background by the conviptraf program) are to be stored.

Finally, select Use destination node IP to include the destination node’s IP address in the demand file that is created. Click Next and the Juniper Networks LDP Statistics data file (by conviptraf) is parsed.

Figure 117: Selecting Juniper Networks LDP Stat as the Import Type
Selecting Juniper Networks LDP Stat as the Import Type

After a period of time parsing/processing the data files, the next stage is the traffic data aggregation stage, in which data points from multiple days are aggregated together for each hour. IP/MPLSView Traffic Data object files (created by conviptraf) are created for each day that data is available. Choose particular days to consider for aggregation by checking the box next to that day. Finally, under Specify Aggregate Directory, specify the directory where aggregated results are to be stored. Click on Next, and the aggregation process (run in the background by the rolliptraf program) begins.

Figure 118: Choose Dates for Aggregation
Choose Dates for Aggregation

After traffic data aggregation has completed, you are given the option of changing the directory where the outputs are written to. Click Browse if you wish to change the directory. Click Next to go to the final step to specify further aggregation and demand generation options, as shown in the following figure and table.

Figure 119: Choose Aggregation and Demand Generation Options
Choose Aggregation and Demand Generation Options
  • Output runcode: Specify the runcode for the demand and trafficload files created.     

  • Max BW (bps) per demand: Generated demands will not exceed this value

  • Min BW (bps): Generated demands are at least this value.

  • Range: This option indicates which period to use to generate the demand file. Options include Peak Period, 24 Period, and Period n, where 1<=n<=24

  • ECMP: Used to created ECMP demands as appropriate.

  • Aggregation statistic: This is the computation methods used to calculate traffic. Options include 99th, 95th, 90th, and 80th percentiles, average, and max.

  • Traffic type: Demand and Trafficload files are generated



IP/MPLSView Traffic Data > Aggregate IP/MPLSView Traffic Data



Once third party traffic data has been converted to IP/MPLSView traffic data, this data can be aggregated. This allows multiple traffic sources to be combined into a single trafficload file. If you had previously completed the procedure for importing third party traffic data, they will automatically proceed to the following window. If starting from scratch, simply start the Import Traffic Wizard and select WANDL Traffic Data from the Select Import Type menu.

Figure 120: Selecting IP/MPLSView Traffic Data as the Import Type
Selecting IP/MPLSView Traffic Data as the Import Type

The import item shown in Figure 121 is the Import Directory, which must contain valid IP/MPLSView traffic data. To include subdirectories, check the Include Subdirectories checkbox, then select all the subdirectories to include.

Figure 121: Including Subdirectories With the Import Directory
Including Subdirectories With the
Import Directory

Usually it is wise to include all subdirectories at this step, because data from different days are usually stored in separate subdirectories. If the import directories and subdirectories are configured correctly, clicking Next displays a window containing a list of all the dates associated with the IP/MPLSView traffic data found in the Import directory and subdirectories, as shown in the following figure.

Figure 122: Traffic Import Dates
Traffic Import Dates

Select the dates to be aggregated, and bring them over to the right panel by clicking the right arrow button. Clicking Next at this point will start the aggregation process. Once that is complete, this step is over, and the only step remaining is to convert the aggregate data to a trafficload file.



IP/MPLSView Aggregate Traffic Data > TrafficLoad File



Once IP/MPLSView traffic data has been aggregated, it can be converted to a trafficload file. To do this, select WANDL Aggregate Traffic Data as the Import Type. If continuing from a previous step, this selection will already be made.

Figure 123: Selecting WANDL Aggregate Traffic Data as the Import Type
Selecting WANDL Aggregate Traffic Data as the Import
Type

Clicking Next displays the Aggregation Options window shown here.

Figure 124: Aggregation Options
Aggregation Options

Field

Description

Aggregation Statistic

This is the computation method used to calculate traffic. Options include 99th, 95th, 90th, and 80th percentiles, average, and max.

Traffic Type

This is the traffic type to be converted to the trafficload file. Options include interface traffic, tunnel traffic, and demand traffic.

The following aggregation options are available when the traffic type Demand Traffic Load is selected.

  • Output runcode: Specify the runcode for the demand and trafficload files created.

  • Max BW (bps) per demand: Generated demands will not exceed this value

  • Min BW (bps): Generated demands are at least this value.

  • Range: This option indicates which period to use to generate the demand file. Options include Peak Period, 24 Period, and Period n, where 1<=n<=24

  • ECMP: Used to created ECMP demands as appropriate.