Configuring Bulkstats Traffic Collection (Advanced)

Note: This is an advanced traffic collection feature which requires a license. Please contact your Juniper representative for more information.

1. Configure the appropriate Bulkstats statements on routers to ftp Bulkstats data to /u/wandl/data/bulkstats/current on the IP/MPLSView server. Provisioning templates for ERX and Junos can be found in /u/wandl/data/templates/configlet/BulkStats. The recommended interval for bulkstats collection 15 minutes, and 15 minutes interval for FTP data transfer.
2. When setting up the router profiles, note that the name in router profile must be exactly the same as the real router name. The vendor type must also be correct for Bulkstats to work. For ERX, only the physical router is needed, not virtual routers. Traffic data for virtual routers will come in one file with the physical router and will be separated automatically. Use the routerName, instead of routerName(default).
3. Start up a traffic data collector from command line:

   /u/wandl/dcollector/dc.sh start <instance #>

4. Open the Traffic Collection Manager GUI.
5. Create a new router group. Uncheck the box Enable Router Group Failover so the traffic data collector will not change when it goes down. Check the box Is this group for Bulkstats Collection? to designate that the router group is for Bulkstats.
6. Assign a traffic data collector and Bulkstats enabled routers into this router group. Change the collection method to Bulkstats.
7. Start traffic collection. Wait for incoming data. You will be able to see traffic on the map after receiving two data sets from a router. Bulkstats should FTP into /u/wandl/data/bulkstats/currentfrom routers. Example of raw data output for ERX is listed below. After the data is processed, /u/wandl/data/.network/interface.traffic.bulk file and object files in /u/wandl/data/bulkstats/wandlobjformat will be created. Raw data will be moved into /u/wandl/data/bulkstats/archive
8. By default, interface.traffic.bulk file has 15min interval, but is updated every 5 min. The reason for this is the passive nature of bulkstats collection in comparison to SNMP/CLI collection. Instead of going into the router to retrieve data, we are waiting for bulkstats data to come in, which could be anytime. So if the data comes in right before the file updates, then we do not want to wait for another 15 minutes to process the data from 30 minutes ago. Therefore, it is better to update interface.traffic.bulk more frequently, 5min by default. So the file updates every 5 minutes, but the data points represent 15 minutes traffic. There will be 8 data points to represent15*8 = 2hours.
9. Historical traffic can be seen from the web: Live Network > View Historical Traffic Reports.

Notes

The Server’s and Routers’ clocks should be configured the same for the collection data to be in sync with the real traffic data. This prerequisite is not required for CLI/SNMP collection, since the CLI/SNMP collection time is decided by the time that collection manager starts talking to the routers and starts collection. Bulkstats, on the other hand, waits for data that could come in anytime. Every 5 seconds Bulkstats checks for new incoming files, whose ftp transfer times are uncertain and are depending on the file size and the connection transfer rate. Therefore the traffic collection manager cannot determine the exact time when the file is generated. It will depend on the time stamps of the files generated by the routers, which means that it will depend on the accurate clock time of the routers and the server.

If the Traffic Collector is running on a separate machine different from the Application Server, then (a) the automatic SSH connection from the traffic collector to the Application Server needs to be setup, and (b) the rsync package needs to be available on the traffic collector.

• To install rsync on the traffic collector, switch to the root user and run the ./inst_rsync.sh package under the installation directory’s replication folder. This directory is not a default package in the installation. Contact Juniper support for assistance if this directory is missing.
• To set up the automatic SSH connection, please refer to the Getting Started Guide for IP/MPLSView, Installing the Rsync Package and Automating SSH login chapter for the detailed procedure of creating a private and public key on the traffic collector, but do it for the wandl user (not root user). Copy over the public key to the remote server’s .ssh/authorized_keys file to enable automatic login. Only one-way SSH from the traffic collector to the IP/MPLSView server is needed in this case.

The processed Bulkstats data is immediately available at the application server for traffic data collectors running on the application server, whereas with remote data collectors the data is rsync'd every 15 minutes.

Note that BulkStats is designed to work with one IP/MPLSView server. For one Bulkstats file in the router, the router can only FTP the data to one remote machine. If you need a second IP/MPLSView server to monitor the traffic, a second copy of the Bulkstats file should be configured in the router to collect the same traffic statistics. This should be done with caution as it may use up more router CPU/memory resources.