Help us improve your experience.

Let us know what you think.

Do you have time for a two-minute survey?

 
 

Packet Flow Accelerator Diagnostics Software

Understanding Packet Flow Accelerator Diagnostics Software and Other Utilities

You can use Packet Flow Accelerator Diagnostics software to validate the integrity of the QFX-PFA-4Q module and the QFX5100-24Q-AA switch. The Packet Flow Accelerator Diagnostics software contains standard diagnostics, orchestration diagnostics, Precision Time Protocol (PTP) and synchronization diagnostics, and other utilities. The Packet Flow Accelerator Diagnostics software runs in a guest virtual machine (VM) on the QFX5100-24Q-AA switch and requires that you configure guest VM options in the Junos OS CLI.

The QFX-PFA-4Q module contains four 40-Gigabit Ethernet QSFP+ interfaces, an FPGA module, and timing input and output interfaces to support Precision Time Protocol applications. The FPGA module contains logic that you can customize for processing compute-intensive, latency-sensitive, high-volume transactions.

Before you can run the Packet Flow Accelerator Diagnostics software and utilities, make sure you have performed the following tasks:

Understanding External and Internal Ports and Network Interface Card Ports

Packet Flow Accelerator Diagnostics software and utilities validate the data paths between the external and internal ports on the QFX5100-24Q-AA switch and QFX-PFA-4Q module. Figure 1 illustrates the names of the ports on the QFX5100-24Q-AA switch and QFX-PFA-4Q module and how they connect.

Figure 1: Ports on the QFX5100-24Q-AA switch and QFX-PFA-4Q modulePorts on the QFX5100-24Q-AA switch and QFX-PFA-4Q module

Table 1 provides information on the external and internal ports and NIC ports on the QFX5100-24Q-AA switch and QFX-PFA-4Q module.

Table 1: External and Internal Ports on the QFX5100-24Q-AA Switch and the QFX-PFA-4Q Module

A-ports

Interfaces xe-0/0/24 through xe-0/0/39 on the Packet Forwarding Engine (PFE) of the QFX5100-24Q-AA switch connect to the B-ports on the FPGA module on the QFX-PFA-4Q expansion module. A-ports require corresponding B-ports on the FPGA module. You can manage these interfaces through the Junos OS.

B-ports

Internal 10-Gigabit Ethernet ports connect to the FPGA module on the QFX-PFA-4Q module, which then connect to the A-ports on the PFE of the QFX5100-24Q-AA switch. The naming convention for these ports is determined by the guest VM. The guest VM controls the FPGA module.

C-ports

Four, front-facing 40-Gigabit Ethernet ports on the QFX-PFA-4Q module connect to the FPGA module running on the QFX5100-24Q-AA switch and the F-ports on the QFX5100-24Q-AA switch. The guest VM controls the FPGA module.

D-ports

Two 10-Gigabit Ethernet internal ports on the Packet Forwarding Engine of the QFX5100-24Q-AA switch connect to the Ethernet NIC on the QFX5100-24Q-AA switch. The naming convention for these ports is the same one used for the F-ports. You can manage these ports through the Junos OS.

F-ports

Twenty four front-facing 40-Gigabit Ethernet ports on the QFX5100-24Q-AA switch. These ports contain an “et” prefix when in 40-Gigabit Ethernet mode. If you channelize these interfaces, the prefix is “xe.” You can manage these ports through the Junos OS.

NIC ports

Internal interfaces xe-0/0/40 and xe-0/0/41 on the QFX5100-24Q-AA switch connect to the PFE for use on the guest VM. The NIC ports perform the same functions as any other Linux OS NIC port. The NIC ports do not work unless the QFX-PFA-4Q module is installed.

Understanding Packet Flow Accelerator Diagnostics Software Tests and Scripts

You can run Packet Flow Accelerator Diagnostics software to test the following subsystems on the QFX-PFA-4Q module:

  • FPGA

  • QDR SRAM memory

  • DRAM memory

  • DRAM SPDs

  • FPGA-connected PCI Express links

  • FPGA-connected Ethernet data (QSFP interfaces)

  • QSFP I2C I/O

  • PTP I/O

Before you can run any test or script, you need to connect to the console connection of the guest VM. .

The following test sets are available:

  • quick-test—Allows you to perform a basic test of all FPGA-attached functionality. These tests take one or two minutes to complete.

  • burn-in—Allows you to exercise all FPGA-attached functionality. These tests take several hours to complete.

  • individual test mode—Allows you to test a single subsystem with extra configuration options.

Understanding the ikondiag Command

To run any of the tests, issue the ikondiag command with the following arguments:

Note:

Before you can run the tests, you need to connect to the console connection of the guest VM.

  • -t (quick-test | burn-in | <test name>)

    This argument identifies the test.

  • -h

    This argument provides usage details for the test.

  • -V

    This argument provides verbose output for the tests.

For example, to run the PTP test, issue the ikondiag -t PTP command at the guest VM prompt:

ikondiag -t PTP

Understanding Basic Functionality Tests

You can test basic functionality on the PCI Express interface and memory components. Table 2 lists the names of the tests and their functions.

Table 2: Base Tests

Test Name

Description

Details

Optional Arguments

Test Sets

Failure Behavior

FPGABasic

Tests basic FPGA operation.

Configures the FPGA and reads some simple registers over PCI Express.

None.

quick-test and burn-in

Any failures in this test cause the ikondiag command to generate normal test status and error messages, and then to terminate with another error message. You cannot continue testing because all tests rely on the functionality tested by this one.

PCIe

Verifies functionality and stability of bulk transfers of PCIe data.

Repeatedly loops back pseudo random data generated on the CPU to the FPGA and then back to the CPU. Returned data is verified on the CPU.

-i <n> number of repetitions (default = 1 quick-test, 10,000 burn-in)

-j <n> size of individual transfer in Mebibytes (default = 100 MiB).

quick-test and burn-in

This test reports erroneous data values and offsets in data transfer. Any failures in this test will cause the ikondiag command to output normal test status and error messages and then terminate with a further error. You cannot continue further testing because all tests rely on functionality tested by this test.

DIMM

Checks SPD query functionality and verifies that correct DIMMs are installed.

Reads data from SPD device on DIMM modules ,reports contents, and checks for erroneous values and verifies:

  • DIMM part data against expected part data.

  • SPD temperature is in nominal operating range.

None.

quick-test and burn-in

If any values are unexpected, the test reports erroneous values and provides expected values and ranges.

DRAMMemory

Tests data transfer functionality and stability of FPGA-attached DRAM memory devices.

  • Checks that PHYs are initialized correctly.

  • Repeatedly does the following tasks:

    • Writes to memory from the FPGA

      • Each pass switches data between: zeros, ones, counter, random, zeros, random, ones, random.

    • Loops back memory inside the FPGA (simultaneous reads and writes).

    • Verifies memory from the FPGA

-i <n> vary number of iterations) default = 1 for quick-test, 500 for burn-in)

quick-test and burn-in

This test reports the number of errors during verification. Number of errors are specified as an accumulated number of errors per byte-lane and DIMM module.

Understanding and Running Ethernet Tests and Scripts

The Ethernet tests and scripts test C-ports and traffic between A- and B-ports. The traffic between A- and B-ports is tested by passing the data on the F-ports. For the C-ports, you need to loop back the traffic sent on the C-ports. You can use physical copper loopback cables for this purpose. For the F-ports, , you need to loop back the traffic sent on the F-ports. You can use copper loopback cables for this purpose. Include the F-ports in a VLAN. You can use the python PFAD_exec.py -t 1 script as well as the tests below. The python PFAD_exec.py -t 1 script verifies end-to-end L2 traffic on the external QSFP ports and checks the statistics on the interfaces in Junos OS and the statistics on the interfaces in the Packet Flow Diagnostics software VM. This test will fail if traffic loss is seen on any of the interfaces. There is also a provision to test all the combinations of QSFP ports as well.

Table 3 lists the names of the Ethernet tests and their functions. For information on how to install the script, see Installing Packet Flow Accelerator Diagnostics Software.

Table 3: Ethernet Tests and Scripts

Test Name

Description

Details

Optional Arguments

Test Sets

Failure Behavior

QSFPEthernet

Verifies functionality of Ethernet (QSFP) links.

Generates, receives, and verifies Ethernet frames are at line-rate through the FPGA module. The contents and lengths of packets consist of pseudo-random data.

During operation, QSFP connections are all channelized to use 10 Gigabit Ethernet with all 32 Ethernet channels operating in parallel in full-duplex mode.

-i <n> varied number of iterations (default = 1,000 for quick-test, 1e9 for burn-in)

quick-test and burn-in

If the number of packets sent or received correctly are verified as not being equal, this test is considered a failure and the discrepancies between these quantities are reported. This test fails if the external Ethernet connections are not configured for loopback.

QSFPI2C

Checks if there is access to the four QSFP modules located on the front of the QFX-PFA-4Q module.

Performs reads of registers in the I2C modules and verifies that the results are as expected. For this test to pass, QSFP media must be inserted into all four ports on the QFX-PFA-4Q module. Any kind of external media can be used (for example, DAC cables, copper loopback, modules, and optical modules).

None.

quick-test and burn-in

This test fails if it cannot detect the presence of a QSFP module or if the values it reads back are unexpected.

Before you can run the Ethernet tests and script successfully, you need to perform the following tasks:

  • Externally loop back all of the Ethernet connections (QSFP) on the QFX-PFA-4Q module.

    To loop back the QSFP interfaces on the QFX-PFA-4Q module, attach copper loopback modules on the four QSFP+ interfaces installed on the QFX-PFA-4Q module.

    Attach copper loopback modules on the QSFP+ interfaces (ports 10 through port 13) installed on the QFX5100-24Q-AA switch.

  • Channelize ports 10 through 13 on the QFX5100-24Q-AA switch.

  • Pair each of the 16 ikonDiag lanes using the equivalent Junos OS interface names with each of the corresponding Junos OS interfaces that were channelized from ports 10 through 13 on the QFX5100-24Q-AA switch.

    Note:

    Each VLAN must be independent contain exactly two associated ports—one 10-Gigabit Ethernet port that is an F-port, and one 10-Gigabit Ethernet port that is an A-port.

    Table 4 shows the mappings for the 10-Gigabit Ethernet channels on the QFX-PFA-4Q module F-ports.

    Table 4: 10-Gigabit Ethernet Channel Mappings on the QFX-PFA-4Q module F-ports

    ikondiag Names

    Description

    JDFE_XE32_10G

    xe-0/0/32

    JDFE_XE33_10G

    xe-0/0/33

    JDFE_XE34_10G

    xe-0/0/34

    JDFE_XE35_10G

    xe-0/0/35

    JDFE_XE24_10G

    xe-0/0/24

    JDFE_XE25_10G

    xe-0/0/25

    JDFE_XE26_10G

    xe-0/0/26

    JDFE_XE27_10G

    xe-0/0/27

    JDFE_XE28_10G

    xe-0/0/28

    JDFE_XE29_10G

    xe-0/0/29

    JDFE_XE30_10G

    xe-0/0/30

    JDFE_XE31_10G

    xe-0/0/31

    JDFE_XE36_10G

    xe-0/0/36

    JDFE_XE37_10G

    xe-0/0/37

    JDFE_XE38_10G

    xe-0/0/38

    JDFE_XE39_10G

    xe-0/0/39

    Table 5 shows the mappings for the 10-Gigabit Ethernet channels on the QFX-PFA-4Q module C-ports.

    Table 5: 10-Gigabit Ethernet Channel Mappings on the QFX-PFA-4Q module C-ports

    ikondiag Names

    Description

    JDFE_QSFP0_10G_PORT0

    QSFP port #0 10G sub-channel 0

    JDFE_QSFP0_10G_PORT1

    QSFP port #0 10G sub-channel 1

    JDFE_QSFP0_10G_PORT2

    QSFP port #0 10G sub-channel 2

    JDFE_QSFP0_10G_PORT3

    QSFP port #0 10G sub-channel 3

    JDFE_QSFP1_10G_PORT0

    QSFP port #1 10G sub-channel 0

    JDFE_QSFP1_10G_PORT1

    QSFP port #1 10G sub-channel 1

    JDFE_QSFP1_10G_PORT2

    QSFP port #1 10G sub-channel 2

    JDFE_QSFP1_10G_PORT3

    QSFP port #1 10G sub-channel 3

    JDFE_QSFP2_10G_PORT0

    QSFP port #2 10G sub-channel 0

    JDFE_QSFP2_10G_PORT1

    QSFP port #2 10G sub-channel 1

    JDFE_QSFP2_10G_PORT2

    QSFP port #2 10G sub-channel 2

    JDFE_QSFP2_10G_PORT3

    QSFP port #2 10G sub-channel 3

    JDFE_QSFP3_10G_PORT0

    QSFP port #3 10G sub channel 0

    JDFE_QSFP3_10G_PORT1

    QSFP port #3 10G sub-channel 1

    JDFE_QSFP3_10G_PORT2

    QSFP port #3 10G sub-channel 2

    JDFE_QSFP3_10G_PORT3

    QSFP port #3 10G sub-channel 3

    Table 6 provides exact connectivity between the C-ports and A-ports.

    Table 6: Exact Connectivity Between C-Ports and A-Ports

    QSFP Port Number

    Channel Number

    Junos OS Interface

    QSFP port #0

    channel 0

    xe-0/0/32

    QSFP port #0

    channel 1

    xe-0/0/33

    QSFP port #0

    channel 2

    xe-0/0/34

    QSFP port #0

    channel 3

    xe-0/0/35

    QSFP port #1

    channel 0

    xe-0/0/24

    QSFP port #1

    channel 1

    xe-0/0/25

    QSFP port #1

    channel 2

    xe-0/0/26

    QSFP port #1

    channel 3

    xe-0/0/27

    QSFP port #2

    channel 0

    xe-0/0/28

    QSFP port #2

    channel 1

    xe-0/0/29

    QSFP port #2

    channel 2

    xe-0/0/30

    QSFP port #2

    channel 3

    xe-0/0/31

    QSFP port #3

    channel 0

    xe-0/0/36

    QSFP port #3

    channel 1

    xe-0/0/37

    QSFP port #3

    channel 2

    xe-0/0/38

    QSFP port #3

    channel 3

    xe-0/0/39

  • Add these interfaces to a VLAN.

Channelize ports 10 through 13 using the Junos CLI.

  1. Configure ports 10 through 13 on PIC 1 to operate as 10-Gigabit Ethernet ports.

  2. Review your configuration and issue the commit command.

Add the 16 channelized interfaces you just configured to 16 VLANs.

To add the 16 channelized interfaces:

  1. Create 16 VLANs.

  2. Add the channelized interfaces to the VLANs.

  3. Review your configuration and issue the commit command.

Understanding and Using Stress Tests

The stress tests exercise all high-speed I/Os in parallel. The stress tests require the same external media as you used for the Ethernet tests.Table 7 lists the name of the test and its functions.

Table 7: Stress Tests

Test Name

Description

Details

Optional Arguments

Test Sets

Failure Behavior

Stress

Exercises all high-speed I/Os in parallel.

Exercise all of the high-speed I/Os attached to the FPGA in parallel, including:

  • DRAM

  • QDR

  • Ethernet

Each test subsystem is exercised in a similar fashion to the individual tests as previously described.

-i <n> varied number of iterations) default = 1 for quick-test, 1,000 for burn-in)

quick-test and burn-in

If any one sub-system fails, the test is stopped. The first sub-system detected to have failed is reported.

Note:

If multiple subsystems fail, only the first failed subsystem is reported.

Understanding and Running PTP Tests

You can run PTP for hardware used with PTP. These tests are helpful if you are creating timing applications. To run the tests, you need to connect SubMiniature version B (SMB) cables, Ethernet loopback cables, and ToD loopback cables for the clocking I/O, ToD serial port, and 1-Gigabit Ethernet connectors. You must connect the SMB, Ethernet, ToD loopback cables between the 10M and PPS output and input connectors. The ToD loopback cable is a standard RJ45 cable with Pin 3 (Tx Data) connected to Pin 6 (Rx Data). In addition to the PTP tests, you can run scripts included the Packet Flow Accelerator Diagnostics software to test PTP. See Table 9 for information on the PTP scripts. The PTP scripts require you to have a Junos OS image with Enhanced Automation installed on the QFX5100-24Q-AA switch. For information on how to install the scripts, see Installing Packet Flow Accelerator Diagnostics Software.

Table 8 lists the names of the PTP tests and their functions:

Table 8: PTP Tests

Test Name

Description

Details

Optional Arguments

Test Sets

Failure Behavior

PTP

Checks functionality of various FPGA-attached time-synchronizing features of the QFX-PFA-4Q module.

Performs various tests on time-synchronizing functionality of the QFX-PFA-4Q module.

Subtests covered by this test include:

  • Verification of PFE-attached communications.

  • Testing of the PTP PHY

    • Basic configuration.

    • FPGA-attached interrupt line.

    • 1-Gigabit Ethernet loopback (requires external loopback media).

    • QFX-PFA-4Q module time-syncing related clock generators and feedback routing.

    • ToD UART port (requires external loop-back media).

None.

quick-test and burn-in

A failure in any of the subsystems above causes the entire test to fail and generates a report at the end of the test that indicates the pass and fail status of the sub-tests.

Table 9 lists the name of the script and its function. This script is not part of the ikondiag command. You can run this command Junos OS.

Table 9: PTP Script

Script Name

Description

Details

Optional Arguments

Test Sets

Failure Behavior

./run_ptp_test

Checks functionality of various FPGA-attached time-syncing features of the QFX-PFA-4Q module.

Performs various tests on time-syncing functionality of the QFX-PFA-4Q module.

Sub tests covered by this test include:

  • Verification of PFE attached communications.

  • Testing of the PTP PHY

    • Basic configuration.

    • FPGA-attached interrupt line.

    • 1-Gigabith Ethernet loop-back (requires external loop-back media).

    • QFX-PFA-4Q module time-syncing related clock generators and feedback routing.

    • ToD UART port (requires external loop-back media).

None.

None. This test must be run manually.

A failure in any of the subsystems above causes the entire test to fail and generates a report at the end of the test that indicates the pass and fail status of the sub-tests.

Understanding QFX-PFA-4Q Module LED Tests

The LED tests examine the LEDs on the QFX-PFA-4Q module.

To run the LED test, issue the ikon_led_toggle command. The test might take a few seconds to start because the FPGA is being configured. When you see the message Toggling LEDs. Send SIGINT (^C) to exit, the test begins. To terminate the test, type Ctrl-C. Table 10 lists the name of the test and its function.

Table 10: QFX-PFA-4Q Module LED Test

Test Name

Description

Details

Optional Arguments

Test Sets

Failure Behavior

ikon_led_toggle

Flashes the LEDs on the QFX-PFA-4Q module for visual inspection.

The following LEDs on the QFX-PFA-4Q module will repeatedly cycle through the following patterns:

Note:

The AL and ST LEDs are not included in this test.

  • The sixteen bicolor LEDs for QSFP status cycle through green, orange, and off.

  • S0 and S1 LEDs cycle through green and off.

  • Bottom-left RJ-45 PTP status LED cycles through green, orange, and off.

  • Bottom-right RJ-45 PTP status LED cycles through green and off rapidly.

  • Alarm LEDs cycle through orange, red, and off.

None.

None. This test must be run manually.

LEDs might not flash.

Understanding Packet Flow Accelerator Diagnostics Utilities

In addition to the Packet Flow Accelerator Diagnostic software tests, there are utilities included in the Packet Flow Accelerator Diagnostics software that you can use to further diagnose issues on the QFX-PFA-4Q module.

Note:

Before you can run the utilities, you need to connect to the console of the guest VM. For more information on how to access the guest VM, see Installing Packet Flow Accelerator Diagnostics Software.

Table 11 lists the name of the utility and its function.

Table 11: Utilities

Test Name

Descriptions

Details

Expected Output and Behavior

maxtop

Reports FPGA status.

Displays information about the currently configured status of the FPGA module and whether the module is in operation. Checks to make sure very basic driver and FPGA PCI Express link operations are working correctly. If this utility exits with error(s), it is very unlikely that any further FPGA operations will work.

Output should be similar to that shown below. If this output is not shown, there might be a critical failure in the diagnostic software environment, or the PCI Express link to the FPGA is nonfunctional.

MaxTop Tool 2015.1 Found 1 card(s) running MaxelerOS 2015.1 Card 0: QFX-PFA-4Q (P/N: 241124) S/N: 96362301684266423 Mem: 24GB Load average: 0.00, 0.00, 0.00 DFE %BUSY TEMP MAXFILE PID USER TIME COMMAND 0 0.0% - 2fcf249cc7... - - - -

ikon_snake

Enables snake connectivity between all 10-Gigabit Ethernet channels.

Connects the Rx channel of all 32 x 10-Gigabit Ethernet channels on the FPGA module (QSFP interfaces) to the Tx channel of the respective neighboring connection. This allows all 32 channels to be tested using just a 10-Gigabit Ethernet interface external packet generator, copper loopback modules, and a QSFP <-> 4xSFP breakout cable.

After issuing this test, all Ethernet data will be forwarded after the message ‘Snake tool loaded. hit 'enter' to exit.’ is displayed.

Note:

During the time before the operating message is printed, the FPGA module might be in the process of being configured, so no data is forwarded. Pressing ‘enter’ will exit the utility.

After the test has finished, packet data continues to be forwarded until another Ethernet utility or test is run.

ikon_eth_util all --digitalloopback

Enables digital-loopback on all 10-Gigabit Ethernet interfaces on the Enables ‘snake’ connectivity between all QFX-PFA-4Q module 10-Gigabit Ethernet channels.

Connects the Rx side of all 32x 10-Gigabit Ethernet channels on the FPGA module (QSFP) to the Tx side of the same channel.

After issuing this test, all Ethernet data will be forwarded as described after the message ‘running press return key to exit’ is displayed.

Note:

Before the operating message is displayed, the FPGA module might be in the process of being configured, and no data will be forwarded. Pressing Enter exits the utility.

After the test has finished, packet data continues to be forwarded until another Ethernet utility or test is run.

ikon_eth_util

Enables data to pas -through QFX-PFA-4Q module QSFP ports.

Allows data to pass through the QFX-PFA-4Q module QSFP ports on the QFX-PFA-4Q module.

Note:

Because all of the QSFP ports are channelized to 10-Gigabit Ethernet, you must use SFP breakout cables when connecting external media.

After issuing this test, all Ethernet data is forwarded as described after the message ‘ running press return key to exit ’ is displayed.

Note:

Before the operating message is displayed, the the FPGA module might be in the process of being configured, and no data will be forwarded. Pressing ‘enter’ will exit the utility.

After the test has finished, packet data will continue to be forwarded until another Ethernet utility or test is run.

maxnet -v link show

Dumps FPGA packet statistics.

Displays statistics about packets sent and received on all (QSFP) links from the MAC and PHY IP cores in the FPGA. Using the ’v’ option provides verbose output.

Here are some important items to note:

  • Packet statistics are reset whenever the Altera FPGA is reconfigured; that is, when running different applications that make use of the FPGA.

  • The tool only displays data for Ethernet links that are included in the FPGA design. As such, If the FPGA module has not yet been configured, or it is configured with an application that does not use some of the Ethernet links, reduced link details might be displayed.

Sample output for a single 10-Gigabit Ethernet link is as follows:

MaxTop Tool 2015.1
Found 1 card(s) running MaxelerOS 2015.1
Card 0: QFX-PFA-4Q (P/N: 241124) S/N: 96362301684266423 Mem: 24GB

Load average: 0.00, 0.00, 0.00

DFE  %BUSY  TEMP   MAXFILE        PID    USER       TIME      COMMAND         
 0   0.0%   -      2fcf249cc7...  -      -          -         -               
    

host2mem I <filename> -o <filename> -t <DDR | QDR0 | QDRPARITY0 | QDR1 | QDRPARITY1>

Writes and then reads arbitrary data from QDR SRAM or DRAM.

Operates by streaming the contents of a binary file to one of the memory resources on the QFX-PFA-4Q module through the FPGA, and then streams the same data back from the memory to another file.

Note:

You cannot only read back data from RAM because the contents are not preserved between running multiple tests.

Reports PASSED or FAILED depending on whether the returned data matches the input data.

Table 12 lists the command-line arguments for the host2mem utility.

Table 12: Command-Line Arguments

Argument

Description

-- help | -h

Print out usage and exit.

-i <input file>

Input data file.

-o <output file>

Output data file.

-- test | -t <test name>

Test resource. See Table 13 for information regarding resources.

-- verbose | -v

Enable verbose mode.

The file format for input and output files is identical. Data is packed consecutively as words based on the width specified in the test mode table below. The size of an input file might be less but must not exceed the total size of the resource being tested. The size of the output file is the same as the input file and, provided there are no errors, has the same content.

Table 13: File Format Details

Test Mode

Resource

Word Width

Size of Test Data

DDR

DDR SDRAM

192 B

24 GB

QDR0

QDR0 Data

16 B

32 MB

QDRPARITY0

QDR0 Parity bits

2 B

4 MB

QDR1

QDR1 Data

16 B

32 MB

QDRPARITY1

QDR1 Parity bits

2 B

4 MB

The dynamic random-access memory (DRAM) on the QFX-PFA-4Q module contains three dual in-line memory modules (DIMM3, DIMM4, DIMM6), and each data word is split across all three DIMMs.Table 14 lists the allocation of Bytes to DIMMs.

Table 14: Dual In-Line Memory Modules

0

DIMM3

63

64

DIMM4

127

128

DIMM6

191

Sample Output for Packet Accelerator Diagnostics Software

This section provides some sample output for base tests, Ethernet tests, PTP tests, and utilities.

  • ikondiag -t FPGABasic

  • ikondiag -t DIMM

  • ikondiag -t QSFPEthernet

  • ikondiag -t DRAMMemory -i 3

  • ikondiag -t QDRMemory -p -i 3

  • ikondiag -t Stress -p -i 10

  • ikondiag -t PTP

  • ikondiag -t Application -i 2

  • maxtop

  • ikon_eth_util --all-pass-through

Installing Ethernet and PTP Scripts

Installing Ethernet and PTP Scripts

You can use Ethernet and PTP scripts that are included in the Packet Flow Accelerator Diagnostics software to test Ethernet and PTP functionality. Before you can install the scripts, you need to perform the following tasks:

To install the scripts:

  1. Log into the guest VM using the request app-engine virtual-machine-shell guest-VM-name. The maximum length for the guest VM name is 255 characters. Make sure you are logged in as root when you enter this command.
  2. Enter a valid username and password combination for the guest VM.
  3. Enter the guest-util diag-install guest VM IP address command at the shell prompt.

    Use the same IP address you used for configuring the local management address for the guest VM.

  4. Change directories to /var/tmp to edit the PFAD_params.cfg file.
  5. Open the PFAD_params.cfg file using an editor of your choice.

    Here is an example of what is contained in the file:

  6. Configure the management IP address.
  7. Configure the PTP interfaces.

    IF1 is the primary source, and IF2 is the secondary source.

    Configure IF1 as et-0/0/2, and IF2 as et-0/0/3.

  8. Save the changes you made to the PFAD_params.cfg file.
  9. Run the scripts by issuing one of the following commands at the guest VM prompt.
    • To test traffic orchestration:

      python PFAD_exec.py -t 1

    • To test PTP:

      ./run_ptp_test

    • To test Broadsync:

      ./run_broadsync_test

Installing Packet Flow Accelerator Diagnostics Software

Installing Packet Flow Accelerator Diagnostics Software

You can use Packet Flow Accelerator Diagnostics software to test the FPGA module in the QFX-PFA-4Q module installed on the QFX5100-24Q-AA switch as well as the data paths between the FPGA module and the QFX5100-24Q-AA switch. The Packet Flow Accelerator Diagnostics software contains standard diagnostics, orchestration diagnostics, and Precision Time Protocol (PTP) and synchronization diagnostics. In addition to the Packet Flow Accelerator Diagnostics software tests, there are utilities included in the Packet Flow Accelerator Diagnostics software that you can use to further diagnose issues on the QFX-PFA-4Q module. For information on how to install the QFX-PFA-4Q module, see Installing an Expansion Module in a QFX5100 Device.

To run the orchestration diagnostics, PTP and synchronization diagnostics, and utilities contained in the Packet Flow Accelerator Diagnostics software, you need to have a Junos OS Release 14.1X53-D27 software or later with enhanced automation installed on your QFX5100 switch. For information on how to download and install Junos OS software, see Installing Software Packages on QFX Series Devices.

The Packet Flow Accelerator Diagnostics software runs in a guest VM on the switch and requires that you configure guest VM options in the Junos OS CLI.

Verifying That the QFX-PFA-4Q Expansion Module Is Installed

Before you install the Packet Flow Accelerator Diagnostics software, verify that the QFX-PFA-4Q module is installed.

From the CLI prompt, issue the show chassis hardware command.

From the CLI output, you can see that the four QSFP+ interfaces (4x40G QSFP+) contained in the QFX-PFA-4Q module. are installed.

Downloading the Packet Flow Diagnostics Software

Note:

To access the download site, you must have a service contract with Juniper Networks and an access account. If you need help obtaining an account, complete the registration form at the Juniper Networks website

https://www.juniper.net/registration/Register.jsp .

To download the Packet Flow Diagnostics software package from the Juniper Networks Support website, go to https://www.juniper.net/support/ :

  1. Using a Web browser, navigate to https://www.juniper.net/support .
  2. Click Download Software.
  3. In the Switching box, click Junos OS Platforms.
  4. In the QFX Series section, click the name of the platform for which you want to download software.
  5. Click the Software tab and select the release number from the Release drop-down list.
  6. In the Install Package section on the Software tab, select the Install Package for the release.

    A login screen appears.

  7. Enter your name and password and press Enter.
  8. Read the End User License Agreement, click the I agree radio button, and then click Proceed.
  9. Save the pfadiag_vm-rXXXXX.img.gz file on your computer.
  10. Open or save the Packet Flow Diagnostics software package either to the local system in the var/tmp directory or to a remote location. If you are saving the installation package to a remote system, make sure that you can access it using HTTP, TFTP, FTP, or scp.

Copying the Packet Flow Diagnostics Software Package to the Switch

To copy the packet flow diagnostics software package to the switch:

Copy the packet flow diagnostics package to the switch using any file transfer protocol:

For example:

Install the Packet Flow Diagnostics Software on the Switch

To install the packet flow diagnostics software package on the switch:

  1. Install the Packet Flow Diagnostics software on the switch.

    This might take a few minutes.

    If the Packet Flow Diagnostics software resides locally on the switch, issue the following command:

  2. Issue the show version command to verify that the installation was successful.

    The CLI output shows that the Packet Flow Accelerator Diagnostics software was installed.

Configure the Guest VM Options to Launch the Guest VM on the Host

To configure the guest VM options:

  1. Configure the following options for guest VM support in the Junos OS CLI at the [edit] hierarchy.
    • Compute cluster name

    • Compute node name

    • VM instance name

    • Dedicated management interface for guest VM

    • Third-party package name

    • Internal IP address of the guest VM

  2. Configure the name of the compute cluster and compute node.

    The name of the compute cluster must be default-cluster, and the name of the name of the compute node must be default-node; otherwise, launching the guest VM fails.

  3. Configure the name of the VM instance and the name of the third party application.
    Note:

    The package names in the show app-engine virtual-machine-package command and the show version command should match.

  4. Associate the VM instance with the configured compute cluster and compute node.
    Note:

    The name of the compute cluster must be default-cluster, and the name of the compute node must be default-node; otherwise, launching the guest VM fails.

  5. Configure the local management IP address.

    This IP address is used for the internal bridging interface. The host uses this IP address to check the availability of the guest VM.

    Note:

    Do not use 192.168.1.1 and 192.168.1.2 as IP addresses because they are used by the Host-OS and Junos OS respectively.

  6. Configure the management interface for the guest VM.

    This management interface is separate from the one used for Junos OS.

    Note:

    The management interface name must be either em0 or em1. The configuration will fail if you do not configure a management interface and then commit the configuration.

    The new management interface is provisioned for the guest VM.

  7. Commit the configuration.

    Here are the results of the configuration:

Verifying That the Guest VM is Working

To verify that the guest VM is working:

Issue the following show commands to verify that everything is working correctly:
  • root> show app-engine status

    The status should be Online.

  • root> show app-engine virtual-machine instance

    The VM status should be active.

  • root> show app-engine virtual-machine package

Accessing the Guest VM

To access the guest VM:

  1. Log into the guest VM.
    • Specify the guest VM name using the request app-engine virtual-machine-shell guest-VM-name command. The maximum length for the guest VM name is 255 characters. Make sure you are logged in as root when you enter this command.

    • Enter a valid username and password combination for the guest VM.

      Note:

      The first time you log in, the username is root. There is no password. After you log in, you will be prompted to create a password.

      For example:

  2. Issue the ifconfig -a command to see the names of the management interface that is used to access the guest VM from outside of the network, name of the management interface that is used for internal use, and the NIC ports used in the diagnostics VM.

    In this example, the heartbeat address is the IP address that is used for internal use , the management interface is used for external communications, and the xe-0/0/40 and xe-0/0/41 interfaces are the NIC ports used in the diagnostics VM. The heartbeat is configured by default. The IP address of the heartbeat is the same as the IP address you configured for Junos OS.

    You can associate one of the interfaces to the guest VM by issuing the set services app-engine virtual-machines instance name management-interface interface-name. command. Use the same IP address as the one you configured using the set services app-engine virtual-machines instance test local-management family inet address 192.168.1.10. The MAC addresses associated with these interfaces are used for internal bridging.

Verifying That the FPGA Module Is Working

You can use the following utilities to verify that the FPGA module on the QFX-PFA-4Q module is working.

To verify that the FPGA module is working:

  1. Issue the lspci |grep "RAM memory" command at the guest VM login prompt.

    The output shows that Juniper Networks Device 0078 is working.

  2. Issue the lspci |grep Co-processor command at the guest VM login prompt:

    The output shows that Maxeler Technologies Ltd. Device 0006 is working.

  3. Issue the maxtop command at the guest VM login prompt:
    Note:

    If there are errors in the command output, relaunch the guest VM.

Validating Connections Between QFX5100-24Q-AA Switch Network Ports and QFX-PFA-4Q Module Ports

You can use the ikon_eth_util –all-pass-through utility to validate the connections between the QFX5100-24Q-AA switch network ports and the QFX-PFA-4Q module ports.

In this example, the ikon_eth_util --all-pass-through utility will validate the following connections between the F-ports, A-ports, B-ports, and C-ports. Table 15 provides the ports that are validated in this example.

Table 15: Validating Ports

F-Ports

A-Ports

B-Ports

C-Ports

xe-0/0/10:2

This interface is one of the 10-Gigabit Ethernet ports on the QFX5100-24Q-AA switch. You can manage these ports through the Junos OS.

xe-0/0/32

This interface connects the PFE of the QFX5100-24Q-AA switch to the B-ports on the FPGA module on the QFX-PFA-4Q module.

JDFE_XE32_10G

This interface is an Internal 10-Gigabit Ethernet port on the FPGA module on the QFX-PFA-4Q module and connects to the A-ports on the PFE of the QFX5100-24Q-AA switch.

JDFE_QSFP0_10G_PORT0 [External Port 0-0]

This interface is one of the front-facing 40-Gigabit Ethernet ports on the QFX-PFA-4Q module and connects to the guest VM running on the QFX5100-24Q-AA switch and the F-ports on the QFX5100-24Q-AA switch.

To validate the connections between the QFX5100-24Q-AA switch network ports and the QFX-PFA-4Q module ports:

  1. Configure a VLAN and VLAN ID:
  2. Associate the F-port and A-port in this VLAN so that the FPGA and PFE can communicate:
  3. Commit the configuration:
  4. Verify that the VLAN has been created.
  5. Issue the ikon_eth_util --all-pass-through command at the guest VM login prompt:
  6. Send traffic to xe-0/0/10:2 on the QFX5100-24Q-AA switch and receive traffic on the front panel port 0-0 on the QFX-PFA-4Q module.
  7. Send traffic to the front panel port 0-0 on the QFX-PFA-4Q module and receive traffic on xe-0/0/10:2 on the QFX5100-24Q-AA switch.
  8. Verify the statistics for the xe-0/0/10:2 and xe-0/0/32 interfaces by issuing the show interfaces xe-0/0/10:2 extensive and show interfaces xe-0/0/32 extensive commands.
  9. Verify the statistics for the JDFE_XE32_10G and JDFE_QSFP0_10G_PORT0 interfaces by issuing the maxnet link commands at the guest VM prompt for the Packet Flow Accelerator Diagnostics software.

    [root@ikondiag ~]# maxnet link show JDFE_XE32_10G

    [root@ikondiag ~]# maxnet link show JDFE_QSFP0_10G_PORT0

Uninstalling the Guest VM

To remove the guest VM:

  1. Delete the configuration statements and uninstall the Packet Flow Accelerator Diagnostics software package.

    For example, to remove the app-engine statement:

  2. Commit the configuration.
  3. (Optional) Issue the show version command to learn the name of the Packet Flow Accelerator Diagnostics software package.
  4. Issue the request system software delete virtual-machine-package <package-name> command to uninstall the Packet Flow Accelerator Diagnostics software.