sFlow Monitoring Technology

Benefits of sFlow Technology

• sFlow can be used by software tools like a network analyzer to continuously monitor tens of thousands of switch or router ports simultaneously.

• Because sFlow uses network sampling (forwarding one packet from n number of total packets) for analysis, it is not resource intensive (for example processing, memory and more). The sampling is done at the hardware application-specific integrated circuits (ASICs) and, hence, it is simple and more accurate.

sFlow Support on Switches

QFX Series

sFlow technology on the switches samples only raw packet headers. A raw Ethernet packet is the complete Layer 2 network frame.

An sFlow monitoring system consists of an sFlow agent embedded in the device (switch) and up to four external collectors. The sFlow agent's two main activities are random sampling and statistics gathering. The sFlow agent performs packet sampling and gathers interface statistics, and then combines the information into UDP datagrams that are sent to the sFlow collectors. An sFlow collector can be connected to the switch through the management network or data network. The software forwarding infrastructure daemon (SFID) on the switch looks up the next-hop address for the specified collector IP address to determine whether the collector is reachable by way of the management network or data network.

Each datagram contains the following information:

• The IP address of the sFlow agent

• The number of samples

• The interface through which the packets entered the agent

• The interface through which the packets exited the agent

• The source and destination interface for the packets

• The source and destination VLAN for the packets

You can view the Extended router data and Extended switch data headers on collector as part of sFlow records.

The Extended switch data contains information of Flow data length (byte), Incoming 802.1Q VLAN, Incoming 802.1p priority, Outgoing 802.1Q VLAN, and Outgoing 802.1p priority fields

The Extended router data contains information of Flow data length (byte), Next hop, Next hop source mask, and Next hop destination mask fields.

sFlow IP-over-IP

Starting in Junos OS Release 20.4R1, you can use sFlow technology to sample IP-over-IP traffic at a physical port on QFX5100 and QFX5200 devices. This feature is supported for IP-over-IP tunnels with an IPv4 outer header that carry IPv4 or IPv6 traffic. Use sFlow monitoring technology to randomly sample network packets from IP-over-IP tunnels and send the samples to a destination collector for monitoring. Devices that act as a IP-over-IP tunnel entry point, transit device, or tunnel endpoint support sFlow sampling. Table 1 shows the fields that are reported when a packet is sampled at the ingress or egress interface of a device that acts as an IP-over-IP tunnel entry point, transit device, or tunnel endpoint.

Table 1: Supported Metadata

sFlow Field	Tunnel Entry Point	Transit Device	Tunnel Endpoint
Raw packet header	Includes payload only	Includes payload and tunnel header	Egress: Includes payload only Ingress: Includes payload and tunnel header
Input interface	Incoming IFD SNMP index	Incoming IFD SNMP index	Incoming IFD SNMP index
Output interface	Outgoing IFD SNMP index	Outgoing IFD SNMP index	Outgoing IFD SNMP index

QFabric

On a QFabric system, sFlow technology monitors the interfaces on each Node device as a group, and implements the binary backoff algorithm based on the traffic on that group of interfaces.

On the QFabric system, the following default values are used if the optional parameters are not configured:

• Agent ID is the management IP address of the default partition.

• Source IP is the management IP address of the default partition.

In addition, the QFabric system subagent ID (which is included in the sFlow datagrams) is the ID of the Node group from which the datagram is sent to the collector.

On a QFabric system, the sFlow technology architecture is distributed. The global sFlow technology configuration defined on the QFabric system Director device is distributed to Node groups that have sFlow sampling configured on their interfaces. The sFlow agent has a separate sampling entity, known as a subagent, running on each Node device. Each subagent has its own independent state and forwards its own sample information (datagrams) directly to the sFlow collectors.

On the QFabric system, an sFlow collector must be reachable through the data network. Because each Node device has all routes stored in the default routing instance, the collector IP address should be included in the default routing instance to ensure the collector’s reachability from the Node device.

Regardless of the rate of traffic or the configured sampling interval, a datagram is sent whenever its size reaches the maximum Ethernet transmission unit (MTU) of 1500 bytes, or whenever a 250-ms timer expires, whichever occurs first. The timer ensures that a collector receives regularly sampled data.

Packet-based sampling in sFlow is implemented in the hardware. If traffic levels are unusually high, the hardware generates more samples than it can handle, and the extra samples are dropped, producing inaccurate results. Enabling the disable-sw-rate-limiter statement disables the software rate-limiting algorithm and allows the hardware sampling rate to stay within the maximum sampling rate.

Table 2: Supported Metadata

Incoming Interface and Encapsulation	Outgoing Interface and Encapsulation	Required Sampled Content	Forwarding Scenario	Metadata
Access port Layer 2 traffic	Network port	Incoming Layer 2 header + Layer 2 payload	Packets are encapsulated with VXLAN header and forwarded.	Incoming Interface Index or Identifier. Outgoing Interface Index or Identifier
Network port Layer 3 traffic	Access port	Incoming Layer 3 header + VXLAN header + Inner payload	Packets are de-capsulated and forwarded.	Incoming Virtual Tunnel End Point (VTEP) Interface Index or Identifier. Outgoing Interface Index or Identifier
Access port Layer 2 traffic	Network port	Incoming Layer 2 Header + Layer 2 payload	Packets are encapsulated with VXLAN header and forwarded.	Incoming Interface Index or Identifier. Outgoing Interface Index or Identifier
Network port Layer 3 traffic	Access port	Inner payload	Packets are de-capsulated and forwarded.	Incoming VTEP Interface Index or Identifier. Outgoing Interface Index or Identifier

Table 3 provides Metadata information for extended switch data and extended routing data.

Table 3: Supported Metadata for Extended Switch Data and Extended Routing Data

EVPN-VXLAN	Scenario	Traffic Type	sFlow Interface Side	VXLAN Tunnel Type	Extended Switch Data				Extended Routing Data
					IIF VLAN	IIF VLAN Priority	OIF VLAN	OIF VLAN Priority	NH IP	NH SMASK	NH DMASK
CRB	Layer 2 GW Leaf	Layer 2	Ingress	Encap	Yes	Yes	No	No	Yes	Yes	Yes
				Decap	No	No	Yes	No	No	No	No
			Egress	Encap	Yes	No	No	No	Yes	Yes	Yes
				Decap	No	No	Yes	No	No	No	No
	Layer 3 GW Spine	Layer 2	Ingress	No	No	No	No	No	No	No	No
				No	No	No	No	No	No	No	No
				Transit	No	No	No	No	Yes	Yes	Yes
			Egress	No	No	No	No	No	No	No	No
				No	No	No	No	No	No	No	No
				Transit	No	No	No	No	Yes	Yes	Yes
		Layer 3 Traffic (Inter Vlan Case)	Ingress	Encap	No	No	No	No	Yes	Yes	Yes
				Decap	No	No	No	No	Yes	Yes	Yes
				Transit	No	No	No	No	Yes	Yes	Yes
			Egress	Encap	No	No	No	No	Yes	Yes	Yes
				Decap	No	No	No	No	Yes	Yes	Yes
				Transit	No	No	No	No	Yes	Yes	Yes
ERB	Layer 2+Layer 3	Layer 2	Ingress	Encap	Yes	Yes	No	No	Yes	Yes	Yes
				Decap	No	No	Yes	No	No	No	No
			Egress	Encap	Yes	No	No	No	Yes	No	Yes
				Decap	No	No	Yes	No	No	No	No
		Layer 3 Traffic (Inter VLAN Case)	Ingress	Encap	Yes	Yes	No	No	Yes	Yes	Yes
				Decap	No	No	Yes	No	No	No	No
			Egress	Encap	Yes	No	No	No	Yes	Yes	Yes
				Decap	No	No	Yes	No	No	No	No

sFlow Support on Routers

• PTX Series
• GRE Encapsulation
• sFlow Sample Size
• ACX Series

Table 4: Extended Tunnel Egress Structure Fields and Values

Field Name	Value
Protocol reported	0x2f (GRE)
Source IP	IPv4 or IPv6 address of the tunnel source
Destination IP	IPv4 or IPv6 address of the tunnel destination endpoint
length	0
source port	0
destination port	0
tcp flags	0
priority	0

The extended structure for IPv4 and IPv6 GRE tunnels is below:

Sampled IPv4 header structure is below:

Sampled IPv6 header structure is below:

sFlow Limitations on Routers

On routers, limitations of sFlow traffic sampling include the following:

• Trio chipset cannot support different sampling rate for each family. Hence, only one sampling rate can be supported per line card.

• Adaptive load balancing is applied per line card and not for per interface under the line card.

Routers support configuration of only one sampling rate (inclusive of ingress and egress rates) on an line card. To support compatibility with the sFlow configuration of other Juniper Networks products, the routers still accept multiple rate configuration on different interfaces of the same line card. However, the router programs the lowest rate as the sampling rate for all the interfaces of that line card. The (show sflow interfaces) command displays the configured rate and the actual (effective) rate. However, different rates on different line cards is still supported on Juniper Networks routers.

In Junos OS Evolved, you can configure sFlow only on Ethernet interfaces (et-*) for the following PTX Series devices:

• PTX10003-80C and PTX10003-160C

• PTX10008

• PTX10001-36MR

• PTX10004

• PTX10016

You cannot configure sFlow on loopback interfaces (lo0).

sFlow Limitations on Switches

On switches, limitations of sFlow traffic sampling include the following:

• The EX3400, EX4100, EX4300, EX4400, and QFX5K series switches use pseudo-egress sampling for egress sampling. Packets are not true egress samples. They are unmodified copies as they appear in the ingress pipeline of the sflow instance device that is using egress sampling.

• On EX9200 switches, true OIF (outgoing interface) is not supported with sFlow.

EX9200 switches support configuration of only one sampling rate (inclusive of ingress and egress rates) on an FPC (or line card). To support compatibility with the sFlow configuration of other Juniper Networks products, EX9200 switches still accept multiple rate configuration on different interfaces of the same FPC. However, the switch programs the lowest rate as the sampling rate for all the interfaces of that FPC. The (show sflow interfaces) command displays the configured rate and the actual (effective) rate. However, different rates on different FPCs is still supported on EX9200 switches.

Adaptive Sampling on Routers and Switches

Adaptive sampling is the process of monitoring the overall incoming traffic rate on the network device and providing intelligent feedback to interfaces to dynamically adapt the sampling rates on interfaces on the basis of traffic conditions. Adaptive sampling prevents the CPU from overloading and maintains the system at an optimum level, even when traffic patterns change on the interfaces. Whereas the sample rate is the configured number of egress or ingress packets out of which one packet is sampled, the adaptive sample rate is the maximum number of samples that should be generated per line card, that is, it’s the limit given to adaptive sampling. Sample load is the amount of data (or number of packets) moving across a network at a given point of time that is sampled. As you increase the sample rate, you decrease the sample load and vice versa. For example, suppose the configured sample rate is 2 (meaning 1 packet out of 2 packets is sampled), and then that rate is doubled, making it 4, or only 1 packet out of 4 packets is sampled.

You configure the adaptive sample rate, which is the maximum number of samples that should be generated per line card, at the [edit protocols sflow adaptive-sample-rate] hierarchy level.

To ensure sampling accuracy and efficiency, QFX Series devices use adaptive sFlow sampling. Adaptive sampling monitors the overall incoming traffic rate on the device and provides feedback to the interfaces to dynamically adapt their sampling rate to traffic conditions. The sFlow agent reads the statistics on the interfaces every 5 seconds and identifies five interfaces with the highest number of samples. On a standalone switch, when the CPU processing limit is reached, a binary backoff algorithm is implemented to reduce the sampling load of the top five interfaces by half. The adapted sampling rate is then applied to those top five interfaces.

Using adaptive sampling prevents overloading of the CPU and keeps the device operating at its optimum level even when there is a change in traffic patterns on the interfaces. The reduced sampling load is used until:

• You reboot the device.

• You configure a new sampling rate.

• The adaptive sampling fallback feature, if configured, increases the sampling load because the number of samples generated is less than the configured threshold.

If a particular interface is not configured, the IP address of the next interface in the priority list is used as the IP address for the agent. Once an IP address is assigned to the agent, the agent ID is not modified until the sFlow service is restarted. At least one interface has to be configured for an IP address to be assigned to the agent.

Considerations

On the QFX Series, limitations of sFlow traffic sampling include:

• sFlow sampling on ingress interfaces does not capture CPU-bound traffic.

• sFlow sampling on egress interfaces does not support broadcast and multicast packets.

• Egress samples do not contain modifications made to the packet in the egress pipeline.

• If a packet is discarded because of a firewall filter, the reason code for discarding the packet is not sent to the collector.

• The out-priority field for a VLAN is always set to 0 (zero) on ingress and egress samples.

• You cannot configure sFlow monitoring on a link aggregation group (LAG), but you can configure it individually on a LAG member interface.

• On QFX10000 Series switches, for a set of ports in a multicast group, since the actual sampling happens in the ingress pipeline for egress packets, the minimum of the configured sFlow rate or the most aggressive sample rate among those ports is used for sampling across all ports in that group.

• Starting from Junos OS Release 19.4 and later, on QFX10000 Series switches, if the destination port of a sampled UDP packet is 6635 and the packet does not include a valid MPLS header, the flow sampled packet gets corrupted or truncated. The actual packet is forwarded.

• On QFX10000 Series standalone switches and the QFX Series Virtual Chassis (with QFX3500 and QFX3600 switches), egress firewall filters are not applied to sFlow sampling packets. On these platforms, the software architecture is different from that on other QFX Series devices, and sFlow packets are sent by the Routing Engine (not the line card on the host) and are not transiting the switch. Egress firewall filters affect data packets that are transiting a switch but do not affect packets sent by the Routing Engine. As a result, sFlow sampling packets are always sent to the sFlow collector.

• How Adaptive Sampling Works
• Adaptive Sampling Fallback
• Adaptive Sampling Limitations

sFlow Agent Address Assignment

The sFlow collector uses the sFlow agent’s IP address to determine the source of the sFlow data. You can configure the IP address of the sFlow agent to ensure that the agent ID of the sFlow agent remains constant. If you do not specify the IP address to be assigned to the agent, an IP address is automatically assigned to the agent based on the following order of priority of interfaces configured on the device:

Table 5: Interfaces on the Devices

Routers and EX Series Switches	QFX Series Devices
Virtual Management Ethernet (VME) interface Management Ethernet interface	Management Ethernet interface em0 IP address Any Layer 3 interface if the em0 IP address is not available

If neither of the preceding interfaces has been configured, the IP address of any Layer 3 interface or the routed VLAN interface (RVI) is assigned to the agent. At least one interface must be configured on the switch for an IP address to be automatically assigned to the agent. When the agent’s IP address is assigned automatically, the IP address is dynamic and changes when the switch reboots.

sFlow data can be used to provide network traffic visibility information. You can explicitly configure the IP address to be assigned to source data (sFlow datagrams). If you do not explicitly configure that address, the IP address of the configured Gigabit Ethernet interface, 10-Gigabit Ethernet interface, or the RVI is used as the source IP address.