Using an MX Series Router Flow Collector Interface to Process and Export Multiple Flow Records
Basic passive monitoring can sometimes create a large number of flow records. However, you can manage multiple flow records with a flow collector interface. You can create a flow collector interface from a Monitoring Services II PIC. The flow collector interface combines multiple flow records received from a monitoring services interface into a compressed ASCII data file and exports the file to an FTP server.
To convert a Monitoring Services II PIC into a
flow collector interface, include the flow-collector statement
at the [edit chassis fpc fpc-slot pic pic-slot monitoring-services application] hierarchy
level. To restore the monitoring functions of a Monitoring Services
II PIC, include the monitor statement at the [edit
chassis fpc fpc-slot pic pic-slot monitoring-services application] hierarchy level.
After you commit the configuration to convert the PIC between the monitor and
flow-collector service types, you must take the PIC offline and
then bring the PIC back online. Rebooting the router does not enable the new service
type. You can use the Monitoring Services II PIC for either flow collection or
monitoring, but not both types of service simultaneously.
A flow collector interface, designated by the
cp-fpc/pic/port
interface name, requires three logical interfaces for correct operation.
Units 0 and 1 are used respectively as export channels 0 and 1 to send the
compressed ASCII data files to an FTP server. You must include a class-of-service
(CoS) configuration for these two export channels to provide adequate bandwidth for
file transmission. Unit 2 is used as a flow receive channel to receive flow records
from a monitoring services interface.
Unlike conventional interfaces, IP addresses for flow collector logical interfaces set up a
point-to-point connection between the Routing Engine and the flow collector. The
address statement at the [edit interfaces
cp-fpc/pic/port
unit unit-number family inet] hierarchy level
corresponds to the IP address of the Routing Engine. Likewise, the
destination statement at the [edit interfaces
cp-fpc/pic/port
unit unit-number family inet address
ip-address] hierarchy level corresponds to
the IP address of the flow collector interface. As a result, you must configure
the destination statement for Units 0 and 1 (export channels 0
and 1) with local addresses that can reach the FTP server. Similarly,
configure the destination statement for Unit 2 (flow receive
channel) with a local IP address so it can reach the monitoring services
interface that sends flow records.
To activate flow collector services after
the Monitoring Services II PIC is converted into a flow collector,
include the flow-collector statement at the [edit
services] hierarchy level. You also need to configure several
additional components:
Destination of the FTP server—Determines where the compressed ASCII data files are sent after the flow records are collected and processed. To specify the destination FTP server, include the
destinationsstatement at the[edit services flow-collector]hierarchy level. You can specify up to two FTP server destinations and include the password for each configured server. If two FTP servers are configured, the first server in the configuration is the primary server and the second is a backup server.File specifications—Preset data file formats, name formats, and transfer characteristics. Files are sent by FTP to the destination FTP server when the timer expires or when a preset number of records are received, whichever comes first. To set the data file format, include the
data-formatstatement at the[edit services flow-collectorfile-specification file-name] hierarchy level. The default data format isflow-compressed. To set the export timer and file size thresholds, include thetransferstatement at the[edit services flow-collectorfile-specification file-name]hierarchy level and specify values for thetimeoutandrecord-leveloptions. The default values are 600 seconds fortimeoutand 500,000 records forrecord-level.To set the filename format, include the
name-formatstatement at the[edit services flow-collectorfile-specification file-name] hierarchy level. Common name format macros that you can use in your configuration are included in Table 1.Table 1: Name Format Macros Field
Expansion
{am_pm}
AMorPM{date}
Expands to the current date, using the
{month},{day}, and {year}macros.{day}
01to31{day_abbr}
SunthroughSat{day_full}
SundaythroughSaturday{generation_number}
Expands to a unique, sequential number for each new file created.
{hour_12}
01to12{hour_24}
00to23{ifalias}
Expands to a description string for the logical interface.
{minute}
00to59{month}
01to12{month_abbr}
JanthroughDec{month_full}
JanuarythroughDecember{num_zone}
-2359to+2359{second}
00to60{time}
Expands to the time the file is created, using the
{hour_24},{minute}, and{second}macros.{time_zone}
Time zone code name of the locale (
gmt, pst, and so on).{year}
1970, 2008, and so on.{year_abbr}
00to99Input interface-to-flow collector interface mappings—Match an input interface with a flow collector interface and apply the preset file specifications to the input interface. To configure the default flow collector and file specifications for all input interfaces, include the
file-specificationandcollectorstatements at the[edit services flow-collectorinterface-map]hierarchy level. To override the default settings and apply flow collector and file specifications to a specific input interface, include thefile-specificationandcollectorstatements at the[edit services flow-collectorinterface-map interface-name]hierarchy level.Transfer log settings—Allow you to configure the destination FTP server where log files containing the transfer activity history for a flow collector interface are to be archived, the name for the log file, and the amount of time the router waits before sending the log file to the FTP server. To configure, include the
archive-sites,filename-prefix, andmaximum-agestatements at the[edit services flow-collectortransfer-log-archive]hierarchy level. The default value for themaximum-agestatement is 120 minutes, with a range of 1 to 360 minutes. Also, you can configure up to five FTP archive site servers to receive log files.Miscellaneous settings—Allow you to configure values for the IP address of the analyzer, an identifier for the analyzer, the maximum number of times the flow collector interface attempts to send transfer log files to the FTP server, and the amount of time the flow collector interface waits between retry attempts. To configure, include the
analyzer-address,analyzer-id,retry, andretry-delaystatements at the[edit services flow-collector]hierarchy level. The range for theretrystatement is 0 through 10 retry attempts. The default for theretry-delaystatement is 30 seconds and the range is 0 through 60 seconds.
To specify a flow collector interface as the destination
for flow records coming from a Monitoring Services or Monitoring Services
II PIC, include the collector-pic statement at the [edit forwarding-options monitoring group-name family inet output flow-export-destination] hierarchy level.
You can select either the flow collector interface or a flow server
as the destination for flow records, but you cannot select both destination
types simultaneously.
There is also a Juniper Networks enterprise Management Information Base (MIB) for the flow collector interface. The Flow Collector Services MIB allows you to use SNMP to monitor the flow collector interface. The MIB provides statistics on files, records, memory, FTP, and error states of a flow collector interface. It also provides SNMP traps for unavailable destinations, unsuccessful file transfers, flow overloading, and memory overloading. For more information, view the enterprise-specific Juniper Networks MIBs at SNMP MIB Explorer.
In summary, to implement the flow collector service,
include statements at the [edit chassis], [edit interfaces], [edit forwarding-options], and [edit services] hierarchy levels. The excerpt on the following pages shows the flow
collector service configuration hierarchy. For a full configuration
example, see Example: Configuring a Flow
Collector Interface on an M, MX or T Series Router.
[edit]
chassis {
fpc fpc-slot {
pic pic-slot {
monitoring-services {
application flow-collector;
}
}
}
}
interfaces {
cp-fpc/pic/port {
description ”flow_collector_interface”;
unit 0 {
family inet {
address ip-address {
destination ip-address;
}
}
}
unit 1 {
family inet {
address ip-address {
destination ip-address;
}
}
}
unit 2 {
family inet {
address ip-address {
destination ip-address;
}
}
}
}
interface-fpc/pic/port {
description “export_interface”;
unit 0 {
family inet {
address ip-address;
}
}
}
mo-fpc/pic/port {
description “monitoring_services_interface”;
unit 0 {
family inet;
}
}
Ethernet-based-interface-fpc/pic/port {
description “ input_interface”;
encapsulation encapsulation-type;
}
}
forwarding-options {
monitoring group1 {
family inet {
output {
export-format cflowd-version-5;
flow-active-timeout value;
flow-inactive-timeout value;
flow-export-destination collector-pic;
interface mo-fpc/pic/port {
source-address ip-address;
}
}
}
}
}
services {
flow-collector {
analyzer-address ip-address;
analyzer-id name;
retry value;
retry-delay seconds;
destinations {
"ftp://username@ftp-server-address-1//directory/" {
password "encrypted-password";
}
"ftp://username@ftp-server-address-2//directory/" {
password "encrypted-password";
}
}
file-specification {
file-specification-name {
}
data-format flow-compressed;
transfer timeout value record-level size;
}
}
interface-map {
file-specification file-specification-name;
collector cp-fpc/pic/port;
interface-name {
file-specification file-specification-name;
collector cp-fpc/pic/port;
}
}
transfer-log-archive {
filename-prefix filename;
maximum-age timeout-value;
archive-sites {
"ftp://username@ip-address//directory/" {
password "encrypted-password";
}
}
}
}