Help us improve your experience.

Let us know what you think.

Do you have time for a two-minute survey?

Resolved Issues

 

This section lists the issues fixed in the Junos OS main release and the maintenance releases.

For the most complete and latest information about known Junos OS defects, use the Juniper Networks online Junos Problem Report Search application.

Resolved Issues: 14.1R9

Class of Service (CoS)

  • When the chained-composite-next-hop is enabled for Layer 3 VPN routes, MPLS CoS rewrite rules are attached to the core-facing interface for "protocol mpls-inet-both-non-vpn" and are applied not only to non-VPN traffic (which is the correct behavior) but also to Layer 3 VPN traffic. That is, both MPLS and IP headers in Layer 3 VPN traffic receive a CoS rewrite. PR1062648

  • When customers delete a logical interface from an interface set that has CoS applied to it and activate a CoS profile directly on that logical interface in one single commit, then the commit fails with an error. The commit is successful if it is done one by one. For example, delete the logical interface from the interface set, commit it, and then activate CoS on that logical interface and commit. PR1169272

  • In rare cases, CoS related queue statistics polling with multiple OID packing or multiple SNMP client polling on the same interface simultaneously might cause cosd to generate core files and restart. A cosd process restart does not impact any CoS services. PR1199687

  • On some T Series routers, the LSI statistics are not displayed in aggregated Ethernet interface bundles and also the input statistics counter for aggregated Ethernet does not include MPLS traffic. PR1258003

EVPNs

  • On MX Series routers, with Virtual Ethernet VPN (EVPN) deployed, the routing protocol process (rpd) might crash if the following commands are executed: show evpn database neighbor <neighbor-name> vlan-id <vlan-id> mac-address <address> show evpn database vlan-id <vlan-id> mac-address <address> show evpn database vlan-id <vlan-id> mac-address <address> instance <instance-name>. PR1119301

  • In EVPN active/standby mode, the mark down state of an aggregated Ethernet interface on the Packet Forwarding Engine on the backup designated forwarder (BDF) might get removed, causing traffic loops in rare cases. PR1179026

  • In an EVPN scenario with static MAC configured in the EVPN instance, the remote EVPN instance can see the MAC route information. However, after deactivating and activating the static MAC in the EVPN instance, and then checking the MAC route information in the remote EVPN instance, no such MAC route is found in the EVPN route table. PR1193754

  • In an EVPN-MPLS or EVPN-VXLAN environment, if the subinterface is configured with VLAN-aware (instance-type virtual-switch), in rare cases the FPC/MPC might crash. PR1274976

Forwarding and Sampling

  • In a scenario with a sampled process running, the sampled process is continuously reading the rpd update information and updating the routes in its local storage and at the same time trying to export the updated records to PIC after every periodic rescheduling. If many routes are getting churned, it might cause the sampled process to crash from memory corruption. PR1055686

  • The dfwc might fail to get filter information from the kernel in COMMIT_CHECK (config validation) mode. As a result, the filter index is regenerated starting from index 1. This will create the mismatch of the filter index as compared with the existing filters in the system. PR1107139

  • There is a vulnerability in a specific loopback filter action command that is processed in a specific logical order of operation in a running Junos OS configuration. It allows an attacker with CLI access and the ability to initiate remote sessions to the loopback interface with the defined action to hang the kernel. PR1167423

  • On M7i/M10i with CFEB installed, if you configure "bandwidth-percent" for a firewall policer and use this policer in a firewall filter, and then apply this firewall filter to an interface, then the filter does not work. PR1202181

  • During a chassis reboot and daemon restart, mib2d client tries to connect to the statistics daemon. There are two connections established from mib2d to the statistics daemon. When a few MIB requests are in the queue for processing in both the connections, because of the nature of the two connections, there is a chance of a deadlock. The connection establishment of one connection is blocked by a processing request on another connection, which continues in a loop. These can be listed down as points: 1. Walk on some OIDs (that is, 1.3.6.1.2.1.2 or 1.3.6.1.2.1.31) will not give results. 2. LLDP neighbor information will not be read. PR1221888

  • With Routing Engine-based sampling configured, it might be observed that the chassis stops exporting flow records after every 5-7 days. PR1270723

General Routing

  • This is a timing issue. After deleting and reconfiguring a VRF instance or changing the route-distinguisher in a VRF instance while rpf-check is enabled, the rpd process might crash. The routing protocols are impacted and traffic disruption will be seen due to loss of routing information. PR911547

  • The output of command show interfaces interface-set queue might work incorrectly. PR1014776

  • The following debug logs might be seen frequently in the STFPC system logs. These harmless debugs were removed after the PR was fixed: PKR: GESET optics alarms enable setting value = 0x0 PKR: GESET optics alarms action enable setting value = 0x0. PR1025405

  • There was a timing issue between Junos OS software and the I2C controllers on an MPC5E during a reboot. The software has been corrected to wait for I2C controllers to be ready before monitoring the voltage levels and current levels. PR1051902

  • MX Series Virtual Chassis interchassis link load balancing is broken for MPC5/MPC6 due to a hash mismatch between ingress and egress. As a consequence, when the ingress aggregated Ethernet interface primary link switchover is triggered and then control is switched back, VCP ports carried less traffic in the output direction than the stream ingress interface received. PR1060882

  • On a dual Routing Engine platform with GRES and NSR enabled, after a Routing Engine switchover, the rpd might crash while trying to destroy a composite next hop (CNH). For example, it would be created in scenarios such as a PIM, L3 VPN, and MVPN with a valid reference on it. This issue occurs because during switchover (while backup rpd switches to master), there is a transition period where rpd switches to master mode but the kernel routing table (KRT) is still in backup mode. If KRT (still in backup mode) receives a CNH addition followed by route additions using this CNH during this phase, it would result in CNH in the KRT with valid route references, yet on expiry queue. It is difficult to reproduce. In this case, after a Routing Engine switchover, it occurs twice consecutively. PR1086019

  • MPC connection to Routing Engine closed with the following error logs: [May 19 19:41:23.724 LOG: Err] PQ3_IIC(WR): I/O error (i2c_stat=0xa3, i2c_ctl[1]=0xb0, bus_addr=0x77) [May 19 19:41:23.724 LOG: Err] Failed to enable PCA9548(0x77):grp(0x1)->channel(1) . These logs are generated because the FPC ID EEPROM should not be accessed when the card is online and cty to any FPC results in accessing the ID EEPROM of all the FPCs in the Routing Engine, before establishing the actual cty. Routing Engine (cty) accessing the ID EEPROM and the FPC accessing the Layer 2 controller devices at the same point can result in conflict and might cause a port hang stage and other issues. PR1089266

  • With ECMP-FRR enabled, after rebooting the FPC that is hosting some ECMP links, the ECMP-FRR might not work. Clearing any BGP sessions (that are part of ECMP) might help to clear this issue. PR1101051

  • In rare cases, after a Routing Engine switchover, the MPC PIC might go offline, and some error messages might be seen. Sometimes, chassisd on the Routing Engine might generate core files continuously, making the unit unusable as none of the interface come up. Root cause: After a Routing Engine switchover chassisd fails to get proper status of the FPCs and generates core files due to insufficient ID EEPROM read times. PR1110590

  • The rpd might crash during the deletion of the address family on an interface while rpf check is configured. The fix removes the possible inconsistency chance (that can trigger this type of rpd crash) between rpf-check flags in a kernel routing table and on an interface family data structure for the same interface that has rpf-check enabled or disabled. PR1127856

  • When the MS-MIC or MS-MPC installed in an MX Series router is processing traffic, and the IPsec policy configuration is changed by means of adding or updating a policy, an mspmand process crash might occur. At times IPsec rule configuration changes on the service PIC would not be updated without mspmand generating a core file. PR1166642

  • The cosd, dcd, or rpd might generate core files in subscriber management deployment using dynamic profiles and RADIUS authentication. PR1168327

  • A periodic packet management process (ppmd) is responsible for the periodic transmission of packets on behalf of its various clients and related protocols (for example, LFM, CFM, LACP, BFD, and so on). During a fabric or SIB online process, the client session, which establishes adjacencies with the ppmd process to receive or send periodic packets on those adjacencies, might flap because of a CPU hog issue. PR1174043

  • On dual Routing Engine platforms, if changes occur on an aggregated Ethernet interface that results in marking ARP routes as down (for example, bringing down one of the member links) due to an interface state pending operation issue on the backup Routing Engine, in a race condition, the backup Routing Engine might crash and reboot with the following error message: (panic:rnh_index_alloc: nhindex XXX could not be allocated err=X). PR1179732

  • In an IPv6 sampling environment, when IPv6 routes flap frequently due to a software defect, the Packet Forwarding Engine sometimes fails to insert or retrieve the sampling IPv6 route from the radix node, and the Packet Forwarding Engine might crash. This is a corner case, that is difficult to reproduce. PR1179776

  • In the hsl2 toolkit, there is a process that periodically checks the ASICs that communicate through it. Due to a bug in the toolkit code, the process used invalidates the ASIC, causing a crash. PR1180010

  • If IGMP snooping is configured in a VPLS routing instance and the VPLS instance has no active physical interfaces, multicast traffic arriving from the core might be sent to the Routing Engine. As a result, host queues might get congested, which might cause protocol instability. As a workaround, configure a dummy activate interface in the VPLS routing instance to avoid this issue. PR1183382

  • On MX Series routers, an MS-MIC crash might occur. The exact trigger of the issue is unknown; generally, this issue might happen very rarely without any external triggers. The crash might occur with any services configuration, with core files pointing to a program terminated with signal 4, Illegal instruction. PR1183828

  • On MX Series routers, a vulnerability in IPv6 processing has been discovered that might allow a specially crafted IPv6 Neighbor Discovery Protocol (NDP) packet to be accepted by the router rather than discarded. The crafted packet, destined to the router, will then be processed by the Routing Engine. A malicious network-based packet flood, sourced from beyond the local broadcast domain, can cause the Routing Engine CPU to spike, or can cause the DDoS protection ARP protocol group policer to engage. When this occurs, the DDoS policer might start dropping legitimate IPv6 neighbors as legitimate ND times out. Refer to https://kb.juniper.net/JSA10749 for more information. PR1188939

  • In Junos OS platforms with an SDK service daemon (ssd) running, the ssd process memory leaks slowly because it mistakenly continues to register with the sampled process. PR1192633

  • On MX Series routers with MPC3/MPC4/MPC5/MPC6 (might also affect EX92xx, for example. with EX9200-32XS, SRX5000 platforms), the VSC8248 firmware on the MPC crashes occasionally. This PR enhances the existing VSC8248 PHY firmware crash detection and recovery, helping to recover from a few corner cases where the existing Junos OS workaround does not work. PR1192914

  • Due to a bug in schema with Junos OS Release 14.1Rx and Junos OS Release 15.1Rx, administrators will not be able to push MPLS configurations to devices that include loose strict tags. PR1193599

  • On MX Series platforms with MPC5E installed, in a high-temperature situation, the temperature thresholds for triggering the high temperature alarm and controlling fan speed are based on the FPC level. Any sensor values in the FPC that exceed the temperature threshold of the FPC trigger the actions associated with temperature thresholds. PR1199447

  • Upgrading with unified ISSU might trigger a flap in the interfaces on MX Series routers. The following message might be seen: SFP: pointer Null, sfp_set_present. PR1200045

  • On MX Series routers, the mspmand process might crash on the MS-MPC with XLP B2 chip (for example, REV17). The exact trigger is unknown. It is usually seen with 70 to 90 percent CPU load conditions. PR1200149

  • GUMEM errors for the same address might continually be logged if a parity error occurs in a locked location in GUMEM. Because GUMEM utilizes ECC memory, any error is self-correcting and has no impact on the operation of the router. In a rare case, such a parity error might appear repeatedly at a specific location. As a workaround, the error can be cleared by rebooting the FPC. PR1200503

  • The MSPMAND might crash when an encrypted packet is received out of the range of the replay-window size. The issue might occur in peak loads, where encrypted packets are received out of order due to drops in the network. PR1200739

  • When a dynamic firewall filter is configured to match a packet in prefix or mask format, the firewall filter might not be correctly programmed on the Packet Forwarding Engine. PR1204291

  • In very rare conditions, the FPC might crash when the CLI command request chassis mic offline fpc-slot <fpc-slot> mic-slot <mic-slot> or request chassis pic offline fpc-slot <fpc-slot> pic-slot <pic-slot> is executed. This issue occurs when an SFP diagnostics polling function tries to access an already destroyed SFP data structure by the MIC or PIC offline. PR1204485

  • The Packet Forwarding Engine might install a next hop incorrectly and cause traffic loss if there is a next-hop policy pointing to an IPv6 address. PR1204653

  • False positive message Host 1 failed to mount /var off HDD, emergency /var created is observed after both Routing Engines are upgraded. PR1207864

  • On MX Series devices, when an ARP entry is learned through the aggregated Ethernet interface, and a route is pointing to that ARP next hop, the ARP entry might not be expired even though the ARP IP is no longer reachable. This issue occurs due to the route next hop on the aggregated Ethernet interface getting stuck in a unicast state even if the remote end is not reachable, and the rpd never gets to determine that ARP is invalid. The route next hop on the aggregated Ethernet interface should be displayed in hold state when the remote end is not reachable. PR1211757

  • FPC Type 5 - 3D cards might run into an over-temperature condition in T4000 routers. Under certain circumstances, the chassisd will declare the over-temperature condition and by default the router shuts down in 240 seconds. Over-temperature SNMP traps (jnxOverTemp) are not sent to an external NMS. PR1213591

  • If a zero-length interface name appears in the SDB database, on detection of a zero-length memory allocation in the SDB database, a forced rpd crash might be seen. PR1215438

  • A vulnerability in IPv6 processing has been discovered that might allow a specially crafted IPv6 Neighbor Discovery Protocol (NDP) packet to be accepted by the router rather than discarded. The crafted packet, destined to the router, will then be processed by the Routing Engine. A malicious network-based packet flood, sourced from beyond the local broadcast domain, can cause the Routing Engine CPU to spike, or cause the DDoS protection ARP group policer to engage. When this happens, the DDoS policer might start dropping legitimate IPv6 neighbors as legitimate ND times out. Refer to https://kb.juniper.net/JSA10749 for more information. PR1220213

  • Due to a defect related to autonegotiation in a Packet Forwarding Engine driver, making any configuration change to an interface in MIC "3D 20x 1GE(LAN)-E,SFP" might lead to interface flapping. PR1222658

  • The Routing Engine CPU uses chassis temperature to decide fan speed instead of Routing Engine CPU temperature. This PR has been fixed to use the real Routing Engine CPU temperature to decide the temperature threshold. PR1230109

  • On all platforms, for IPv6 static routes derived from weighted LSPs, unequal load balancing does not work. PR1230186

  • When set port-mirror to an MX Series router, an LSP ping might fail and IP packets with options will not get mirrored due to an unexpected echo reply from downstream unsolicited traffic (DUT): PR1234006

  • When non-Juniper Networks SFP is used in MIC-3D-20GE-SFP-E or MIC-3D-20GE-SFP-EH MIC, the ISR 2 (MIC error interrupt) might be running off for over 2.5 seconds due to unknown reasons, and then the FPC hosting the MIC might restart and crash. The fix adds interrupt throttling for MIC interrupt and restarts the MIC if the interrupts are more than the threshold (> 2500 interrupts every 5 minutes). PR1235475

  • When the interface configured under router-advertisement physically comes up for the first time, the rpd might repeatedly send the router advertisement, which might result in as high as 100% Routing Engine CPU usage. PR1237894

  • In a race condition, ksyncd crash might be seen on the new master Routing Engine after performing a unified ISSU or GRES switchover. This issue is difficult to reproduce. PR1241875

  • When an IPv6 node receives an ICMPv6 PTB (Packet Too Big) message with an MTU less than 1280, the node will emit atomic fragments. This behavior might result in a denial-of-service attack. Refer to https://kb.juniper.net/JSA10780 for more information. PR1250832

  • Malicious LLDP crafted packet leads to privilege escalation, denial of service (CVE-2018-0007). Refer to https://kb.juniper.net/JSA10830 for more information. PR1252823

  • Class of service (CoS) does not correctly classify egress L3 multicast traffic from an ingress VLAN bridge interface after a configuration change. PR1260413

  • On MIC-3D-20GE-SFP-E or MIC-3D-20GE-SFP, when SFP diagnostic information is being read out periodically, due to a malfunctioning SFP or noise on the I2C BUS, the SFP thread might hog CPU resources, and a watchdog check will restart the MPC to recover. Enhancements will prevent the SFP thread hogging and MPC restart. PR1260517

  • On MX Series routers with QSFP optics, receive-loss cleared and set messages will repeat when the laser is down, even when actual flapping does not occur, and overwhelm the messages file. PR1261793

  • When VSTP is enabled on a double-tagged aggregated Ethernet logical interface and there is another single-tagged aggregated Ethernet logical interface configured with the same outer VLAN tag, then the incoming traffic on that VLAN incorrectly hits the AE_RESERVED_IFL_UNIT (AEx.32767) and the traffic gets dropped. PR1267238

  • The CLI configuration command set chassis effective-shaping-rate is enabled for the MX104. PR1267829

  • Changing the mode of interfaces causes the interface to go down or up. For the interface to be down, all the queues (in/out) associated need to be emptied. Due to a certain condition, the queue does not get emptied and the interface pointer does not get freed properly, resulting in FPC crash. PR1273462

  • A routine within an internal Junos OS sockets library is vulnerable to a buffer overflow. Malicious exploitation of this issue might lead to a denial-of-service (kernel panic) or be leveraged as a privilege escalation through local code execution. The routines are only accessible through programs running on the device itself, and veriexec restricts arbitrary programs from running on Junos OS. There are no known exploit vectors utilizing signed binaries shipped with Junos OS itself. Refer to https://kb.juniper.net/JSA10792 for more information. PR1282562

  • GRE Operation, Administration, and Maintenance (OAM) fails to come up when GRE tunnel source and family inet address are the same (as shown in the following configuration statements): set interfaces ge-0/0/0 unit 0 family inet address a.b.c.d/30 set interfaces gr-0/0/1 unit 0 tunnel source a.b.c.d set interfaces gr-0/0/1 unit 0 tunnel destination x.x.x.x set interfaces gr-0/0/1 unit 0 family inet unnumbered-address ge-0/0/0.0 set protocols oam gre-tunnel interface gr-0/0/0.0 keepalive-time x set protocols oam gre-tunnel interface gr-0/0/0.0 hold-time x PR1283646

  • In an equal-cost multipath (ECMP) routing scenario, the next-hop memory partition on the Packet Forwarding Engine might be exposed to a leak in the next-hop application when during link flapping where the link is part of ECMPs or aggregated Ethernet interfaces. PR1285747

  • The rpd might not immediately notify the kernel to reinstate the direct routes associated with an interface coming up. PR1288492

  • As a result of a regression introduced in Junos OS Releases 14.1R5, 14.2R3, 15.1R1, 15.1F2, and later, the G.751-framed E3 interface traffic rate has been limited to 30 Mbps on certain MX Series MICs. This PR has restored the correct E3 rate. PR1304344

  • An FPC degraded fabric condition detected was reported and FPC might be rebooted if fpc-offline-on-blackholing was configured. The trigger in the FPC has only one Packet Forwarding Engine on this slot, but the FPC, which has two Packet Forwarding Engines, was installed on this slot before. PR1320774

  • When XM Chip temperature increases above 67 degrees C, its Packet Forwarding Engine forwarding capacity of 130 Gbps may be reduced by ~3%, which might affect production traffic in certain corner-case scenarios. PR1325271

  • When the XM (center) chip temperature rises above 67o C, its Packet Forwarding Engine capacity of 130 Gbps might be reduced by about 3 percent, which might affect production traffic in certain corner-case scenarios. PR1325271

Infrastructure

  • Incorrect signedness comparison in the ioctl(2) handler allows a malicious local user to overwrite a portion of the kernel memory. Refer to https://kb.juniper.net/JSA10784 for more information. PR1184592

  • On all Junos OS platforms and on the router with PIM enabled that has a local receiver, stale next hops are present because they did not get deleted by processes due to a timing issue. PR1250880

  • Legacy Junos OS kernel might generate a core file on userland_sysctl / sysctl_root / sysctl_kern_proc_env / panic_on_watchdog_timeout. PR1254742

  • On Junos OS devices with legacy FreeBSD (FreeBSD version 6.X) based on Junos OS, the devices might crash and reboot if there is a defect in the Junos OS SDK-based multithreaded application used. PR1259616

Interfaces and Chassis

  • If VRRP is configured to track a route in a routing instance, after graceful Routing Engine switchover (GRES) or Routing Engine switchover, the VRRP group might show an incorrect master/backup state. PR1134189

  • When polling SNMP MIBs for IPv6 traffic (for example, jnxIpv6IfInOctets), the logical interface (IFL) on IQ2 or IQ2E PIC might occasionally report double statistics. PR1138493

  • Customer might see errors when executing show interface interface-set queue <if-set> for a pure numeric interface-set name. router> show interfaces interface-set queue 803 error: can't decode interface name `803': invalid device name. PR1154667

  • On T1600 and T4000 Series routers, when the hold-time for a 100G interface is set or even without a hold-time configured. In an event of 100G interface shutdown, it might cause BFD flapping and transit traffic loss. PR1168536

  • In an MX Series BRAS environment, when you try to remove a demux0 interface, the DCD process might crash and a core file will be generated. PR1175254

  • When there is a configuration change about OAM CFM, cfmd memory leak is observed and might also trigger the following cfmd crash information: /kernel: Process (44128,cfmd) has exceeded 85% of RLIMIT_DATA: used 378212 KB Max 393216 KB PR1186694

  • When you configure VLAN tags for any interface, if the interface configuration is changed continuously, there could be a memory leak during the device control process (dcd). If the memory is exhausted, the dcd process might crash. PR1207233

  • When VRRP is configured on an IRB interface with scaling configuration (300K lines), in a corner case, handles might not be released appropriately after their use is over. As a result, memory leak on a vrrpd might be seen after a configuration commit. PR1208038

  • In a PPP subscriber scenario, if the jpppd process receives a reply message attribute from the RADIUS or the TACACS+ server with a character of %, it might cause the jpppd process to crash and cause the PPP user to be offline. PR1216169

  • The device control process (dcd) cannot start after router reboot because of a nonexisting logical interface referenced in 'demux-options underlying-interface'. PR1216811

  • The configuration change in which a static VLAN demux interface the underlying physical interface is changed to one with a lower bandwidth (for example, from xe to ge) can fail with the following error: "error: Bandwidth on IFL demux0.7000 cannot be greater than that of its IFD". For example: user@router# show | compare [edit interfaces demux0 unit 7000 demux-options] - underlying-interface xe-0/1/0; + underlying-interface ge-0/3/9; user@router# commit re0: error: Bandwidth on IFL demux0.7000 cannot be greater than that of its IFD error: DCD Configuration check FAILED. error: configuration check-out failed PR1232598

  • On an EX Series Packet Forwarding Engine and on MX Series MPC7E/8E/9E line cards, the Packet Forwarding Engine crashes while fetching interface-statistics with extended-statistics enabled (CVE-2017-10611); Refer to https://kb.juniper.net/JSA10814 for more information. PR1247026

  • If more than one logical interface (IFL) is configured under the same physical interface (IFD), and VRRP is configured on one logical interface without VLAN and the lower unit number logical interface has a VLAN configuration present, then VRRPD incorrectly carries the VLAN information from the lower unit number logical interface to this logical interface's configuration. As a result, VRRP might get stuck (state: unknown, VR State: bringup). This might happen if VRRP is configured on the physical interface with flexible-vlan-tagging or the lt interface without flexible-vlan-tagging. PR1247050

  • When an Ethernet OAM LFM session is configured, the line card hosting the LFM session might reboot after the configuration is committed. PR1283280

  • An invalid interface-set configuration might get committed and result in a continuous dcd or chassisd system crash. PR1316976

J-Web

  • An integer signedness error occurs in GD Graphics Library (CVE-2016-3074) that results in a heap overflow when processing compressed data. Refer to https://kb.juniper.net/JSA10798 for more information. PR1218092

  • Junos: Unauthenticated Remote Code Execution through J-Web interface (CVE-2018-0001); Refer to https://kb.juniper.net/JSA10828 for more information. PR1269932

Layer 2 Features

  • If the periodic interval fast is configured for LACP along with the fast-hello-issu command, LACP might time out if there is any interface commit operation on the peer router during unified ISSU, which causes OSPF adjacency flapping. PR1240679

  • The IPv4/IPv6 packets originating from the Routing Engine might be corrupted when the bridge domain has 'vlan-id' set to none, but the outgoing L2 interface for the packet is tagged and CoS is enabled. It only affects packets that originate from the Routing Engine but does not affect transit traffic. It affects both IPv4 and IPv6 packets. PR1263590

MPLS

  • The log messages: /kernel: %KERN-3: tag_nh_iff_record_delete_iff:404 are cosmetic and were switched on in another PR by mistake. PR1171947

  • In LDP-signaled LSPs scenario, if LDP statistic is configured or the command show ldp traffic-statistics is executed, the device processes statistics for every LDP-signaled LSP. If there is an LSP with scaled next hops, it might take too much time to look up all the next hops and overload the rpd. PR1191406

  • Packets will be out-of-order if they are Routing Engine generated and go over unilist/ECMP. PR1193697

  • A denial-of-service vulnerability in the rpd process allows a malformed MPLS ping packet to crash the rpd process if MPLS OAM is configured. Repeated crashes of the rpd process can result in an extended denial-of-service condition for the device (CVE-2017-2347). Refer to https://kb.juniper.net/JSA10795 for more information. PR1204027

  • When nonstop active routing (NSR) is configured with a Label Distribution Protocol (LDP) export policy, or an L2 smart policy, the routing protocol process (rpd) on the backup Routing Engine might crash when LDP tries to delete a filtered label binding. To avoid this issue, remove the LDP export policy or the l2-smart-policy statement at the[edit protocols ldp] hierarchy level or the [edit routing-instances routing-instance-name protocols ldp] hierarchy level. PR1211194

  • In a scaled environment, when there are many unicast next hops related to the same LSP (for example, the same RSVP or LDP label), MPLS traffic statistics collection might take too much CPU time in kernel mode. This can in turn lead to various system impacting events, like scheduler slips of various processes and losing connection toward the backup Routing Engine and FPCs. PR1214961

  • For MX Series devices with Junos OS Release 14.1R9, due to a software defect, the ldp traffic-statistics configuration does not work correctly. The defect not only prevents periodical LDP statistic gathering but also causes kernel memory leaks. Kernel memory leaks may lead to various other issues. PR1258308

  • The routing protocol process (rpd) might crash if egress-policy is configured in LDP and the same route prefixes are in both inet.0 and inet.3. PR1266358

  • At the transit node of a P2MP tunnel, the changes to the reservation state of a sub-LSP might inadvertently cause the reservation state of other sub-LSPs in the same session to skip refresh cycles, which might result in the reservation tears being sent upstream. Flapping of one sub-LSP might cause other sub-LSPs in the same P2MP session to be torn down. PR1272223

  • Junos OS: A crafted MPLS packet may lead to a kernel crash (CVE-2018-0003). Please refer to https://kb.juniper.net/JSA10831 for more information. PR1276786

  • The following log messages might be seen when you have an output firewall filter attached to the loopback interface: >>>>>> kernel: in_dfw_match: invalid IP version. This is caused by incorrect parsing of MPLS l2ckt ping packets. The logs are completely harmless, and it does not mean that any packets have been discarded. PR1288829

  • Traffic engineering database gradual memory leak causes an rpd crash. PR1303239

  • If BGP multipath is configured, in the case where the interface associated with one of the equal-cost paths flaps and eventually comes up within BGP hold-time, the prefixes may be installed in the routing table only with the path corresponding to the flapping interface as the next hop. PR1305228

Multicast

  • The routing protocol process (rpd) creates an indirect next hop when a multicast route (S,G) needs to be installed when listeners show their interest to S,G traffic. The kernel then creates a composite next hop. In this case, it appears to be P2MP MCNH that gets created. When any member interface is not a Packet Forwarding Engine specific interface (for example, Vt, LSI, IRB, or any other pseudointerfaces), the kernel throws a message indicating that FMBB cannot be supported. These messages are harmless and do not have any impact. PR1230465

Network Management and Monitoring

  • The PR will fix the output of CLI command when snmp notify-filter is configured with wildcard characters. Example configuration: set snmp v3 notify-filter nf1 oid .1.*.6 include set snmp v3 notify-filter nf1 oid 1.2.3.4.5 mask 1.0.0.1.1 set snmp v3 notify-filter nf1 oid 1.2.3.4.5 include Before the fix: > show snmp v3 notify filter Filter Subtree Filter Storage Status name type type nf1 1.2.3.4.5 include nonvolatile active <<<< Here, 1.0.0.1.1 mask is not applied nf1 1.42.6 include nonvolatile active <<<< Here 1.*.6 is considered as 1.42.6. (Where 42 is the ASCII equivalent of wildcard "*") After the fix: > show snmp v3 notify filter Filter Subtree Filter Storage Status name type type nf1 1.*.*.4.5 include nonvolatile active <<<< Mask is applied correctly nf1 1.*.6 include nonvolatile active <<<< Wildcard "*" is treated as expected PR1185143

  • On Junos OS devices with SNMP enabled, a network-based attacker with unfiltered access to the Routing Engine can cause the Junos OS snmpd process (daemon) to crash and restart by sending a crafted SNMP packet. Repeated crashes of an snmpd process can result in a partial denial-of-service condition. Additionally, it may be possible to craft a malicious SNMP packet in a way that can result in remote code execution. Refer to https://kb.juniper.net/JSA10793 for more information. PR1282772

  • On MX Series routers, the show arp no-resolve interface command displays the unrelated static ARP entries that are fixed to display proper static ARP entries of the given interface. PR1299619

Platform and Infrastructure

  • If there are large scale routes, the kernel process might crash with a core file generated while issuing the operational command show arp/clear arp. PR1070660

  • With load-configuration remote procedure call (RPC), there is a chance that annotations have been added to no-keyword containers, which is not supported from the CLI. An added comment cannot be removed from a configuration object unless the object itself is deleted. This causes issues when multiple comments are stacked under the first interface configured. The workaround is to delete the orphaned junos:comment entries and to merge both annotations into a single one, which can then be overwritten to only one if needed. PR1102086

  • Inline 6rd and 6to4 support for XL and XL-XM based platforms. PR1116924

  • Configuring a parameter of "broadcast 255.255.255.255" to an interface family inet when executing the commands show arp or clear arp causes a kernel crash. This issue might cause route flap, which impacts traffic. PR1120114

  • A sustained sequence of different types of normal transit traffic can trigger a high CPU consumption denial of service condition in the Junos OS register and schedule software interrupt handler subsystem when a specific command is issued to the device. PR1145306

  • Two vulnerabilities in a Junos OS telnetd service might allow a remote, unauthenticated attacker to cause a denial-of-service through memory, CPU consumption, or both. Refer to JSA10817 for more information. PR1159841

  • Bidirectional Forwarding Detection (BFD) session fails to come up when it is configured over the SONET interface without an IP address. The IFA GET operation performed by BFD fails because there is no address configured on the interface. PR1165720

  • The sequence number in RPM ICMP-PING probes is introduced as a 32-bit variable instead of a 16-bit if it increases and reaches the maximum value of 65,535 but, it does not rollover, causing all the RPM ICMP-PING probes to fail and not succeed any more. PR1168874

  • If you configure micro-BFD on an aggregate interface, when using native-VLAN and if native-VLAN is configured on one of the logical interfaces, then the ARP resolution for that logical interface fails. PR1172229

  • In an MX Series platform, if an aggregated Ethernet interface is configured with policer, which is a shared-bandwidth-policer, and the member link of the aggregated Ethernet interface is removed or added, the policer value might be wrong. PR1173704

  • This PR fixes a file descriptor (FD) leak problem in an mgd process when NETCONF traceoptions are set. If <commit> rpc is executed via a NETCONF session, there is an FD leak in the corresponding mgd PID. PR1174696

  • There is a potential remote code execution vulnerability in PAM (CVE-2017-10615). Refer to JSA10818 for more information. PR1192119

  • On an MX Series Virtual Chassis, an MPC board selects a clock from the next reference after a GRES, which is a line interface. If there is no signal on that line, then the clock is bad and link flaps could occur or the MPC might generate a core file. PR1194651

  • If an IPv6 filter is configured with a next-header condition in one item and a syslog action in a default item, then IPv6 packets with next-header hop-by-hop are evaluated with this IPv6 filter, and it might generate the blank line for a syslog file. PR1201864

  • When a NETCONF get-route-information RPC is executed for all routes through the ssh transport session and the session is terminated before all the route information is retrieved, the mgd and rpd processes cause high CPU utilization for an extended period of time. Example of issues caused by high CPU utilization for an extended period:

    • BGP neighbors hold-down timer expires and becomes ACTIVE.

    • OSPF adjacencies reset during database exchange.

    • OSPF LSA retransmission events on neighboring nodes occur due to missing ACKs.

    • LDP sessions time out.

    • Non-distributed Bidirectional Forwarding Detection (BFD) sessions get reset due to missing keepalives.

    PR1203612

  • With 64-bit routing protocol process (rpd), if BGP is applied an export policy with "from protocol", it might cause an error to filter some routes that do not match the values from "from protocol". PR1206511

  • When inline J-Flow is enabled, the flow sequence number in the flow data template is set to zero on MPC5E/6E/7E/8E/9E and MPC2E-NG/MPC3E-NG while exporting the flow record to collector. Certain collectors, depending on the implementation of the collector, might fail to decode the flow record and missing flows. PR1211520

  • NTP peers fail to synchronize in symmetric active mode when there is significant downtime of one peer (for example, due to power maintenance, such as for hardware or software upgrades). PR1222544

  • In an Advanced Insight Scripts (AI-Scripts) environment, when there is some special combination of jcs:printf(...) and some special characters (such as \n \t \\) at the boundary of the buffer, the scripts process might crash and high routing protocol process (rpd) memory usage is observed. PR1232418

  • The scale-subscriber license count might increase to an invalid license state with L2TP/LTS clients. This is due to the l2tpd daemon not going through a proper state transition on L2TP/LTS clients logout. Hence, the license count does not get updated. The fix ensures the license count is updated on logout regardless of the daemon going through a proper state transition or not. PR1233298

  • NTP.org and FreeBSD have published security advisories for vulnerabilities resolved in ntpd (NTP daemon). Server-side vulnerabilities are only exploitable on systems where NTP server is enabled at the [edit system ntp] hierarchy level. A summary of the vulnerabilities that might impact Junos OS is in JSA10776. Refer to JSA10776 for more information. PR1234119, PR1159544

  • Starting In Junos OS Release 13.3, SRX Series clusters need to run auditd on both nodes. However, on MX-VC Bm and TXP all LCC also add auditd. Because LCC and VC-BM do not have routes for the accounting server, the following message is generated: 813 unreachable infor. user@router> show system processes extensive | match "-re|audit" sfc0-re0: -------------------------------------------------------------------------- 2565 root 1 96 0 3304K 2620K RUN 0:01 0.00% auditd lcc0-re0: -------------------------------------------------------------------------- 2398 root 1 96 0 3240K 2536K select 0:01 0.00% auditd lcc1-re0: -------------------------------------------------------------------------- 2791 root 1 96 0 3244K 2544K select 0:01 0.00% auditd %DAEMON-3: auditd[2398]: sendmsg to 10.233.225.78(10.233.225.78).1813 failed: Network is down %DAEMON-3: auditd[2398]: AUDITD_RADIUS_REQ_SEND_ERROR: auditd_rad_send: sendto/sendmsg: Network is down. PR1238002

  • On rare occasions during the route add/delete/change operation, the kernel might encounter a crash with the error "rn_clone_unwire no ifclone parent". PR1253362

  • On an MX Series Virtual Chassis setup acting as an MVPN bud node and having a downstream local receiver and a PE node, traffic with few multicast groups is reported as not being forwarded to the local receiver. PR1261172

  • In Junos OS, when a new line card or a service card comes online, the real-time performance monitoring (rpm) process might receive the following error message GENCFG: op 9 (RPM Blob) failed; err 1 (Unknown) might be seen. PR1266336

  • In rare cases, the Packet Forwarding Engine might drop the TCP RST (reset) packet from the Routing Engine side while performing GRES or flapping an interface, and traffic might be dropped. PR1269202

  • When the total number of available CoS queues on an MPC Type 1 or Type 2 with an enhanced queuing chip (QX chip) is limited with the 'chassis fpc max-queues' configuration, some interfaces might start dropping all traffic as Tail-/RED-drops. PR1301717

  • If commit script generates invalid configuration and corrects the same by deleting the configuration and after commit with synchronize configured, the patch might generate 0 bytes size, instead of actual diff. Jan 9 10:49:43 re0-abc mgd[3672]: UI_CMDLINE_READ_LINE: User 'netops', command 'commit synchronize force ' Jan 9 10:50:16 re0-abc mgd[3672]: UI_CFG_AUDIT_OTHER: User 'root' delete: [class-of-service interfaces xe-2/1/0] Jan 9 10:50:23 re0-abc mgd[3672]: UI_COMMIT_PROGRESS: Commit operation in progress: filename /var/run/db/juniper.db-patch.sync, size 0 <<<< this indicates no change in configuration, however there is configuration change. PR1329513

Routing Protocols

  • In a BGP scenario where IPv4 and IPv6 neighbors coexist in the same group, if all of the IPv4 peers flap but none of the IPv6 peers flaps, a timing issue might happen where one of the IPv4 peers comes up before inet.0 RIB is cleaned up. As a result, a routing protocol daemon (rpd) crash is seen. PR986272

  • Starting in Junos OS Release 13.3R2 and later, if delegated BFD sessions flap continuously, packet buffer memory may be leaked. The automatic memory leak detection process reports this within the syslog once a certain threshold is reached, like "fpc7 SHEAF: possible leak, ID 8 (packet(clones)) (10242/128/1024)" on MX- MPC or "fpc4 SHEAF: possible leak, ID 9 (packet(clones)) (255/1/5)" on other platforms. Note that BFD sessions operating in centralized mode are not exposed. A complete fix is available from Junos OS Release 14.2R1 and later. Prior to Junos OS Release 14.2R1, there was only a partial fix that did not fix the memory leak completely. PR1003991

  • On the provider PE in a carrier-of-carriers VPN scenario, a route in the vrf.inet.3 table is copied to vrf.inet.0 automatically. It is because the provider carrier's iBGP session has family inet-vpn and only advertises routes from vrf.inet.0. Then the route in vrf.inet.0 is further auto-exported to bgp.l3vpn.0 table. The rpd process might crash when BGP is trying to advertise the route in the bgp.l3vpn.0 table while its original route in the vrf.inet.0 table is in the middle of deletion. This is a timing issue and not easily reproduced. PR1024470

  • When a BGP speaker (router) has multiple peers configured in a BGP group, there is sometimes an inaccurate count of prefixes. This occurs when the BGP speaker receives a route from a peer and re-advertises the route to another peer within the same group. In such instances, the MIB object "jnxBgpM2PrefixOutPrefixes" for peers in the same group reports the total number of advertised prefixes in the group. MIB value "jnxBgpM2PrefixOutPrefixes" is defined as being used on a per-peer basis. However, it is instead being used to report prefixes on a per-group basis. To display an accurate number of advertised prefixes, use the show bgp neighbor command. PR1116382

  • The rpd might crash because of an RPF check when the PIM neighbor goes down. This is a software bug. PR1122530

  • When protocol MSDP is configured and then deleted, the NSR sync status for MSDP might get stuck in "NotStarted" state, and unified ISSU might fail on the master Routing Engine for this reason "CHASSISD_ISSU_ERROR: Daemon ISSU Abort -1(NSR sync not complete: MSDP)". PR1129003

  • When clearing the IS-IS database, the rpd process might crash due to a rare memory de-allocation failure that a task pointer is attempted to be freed twice. In the fix of this issue, the order of referencing the task pointer is being revised to avoid the occurrence of an rpd crash. PR1169903

  • For a specific route received from different eBGP neighbors, if all BGP selection criteria matches, the router ID is used. In an eBGP route, BGP uses the active route as the preferred one. If this specific route flaps with the sequence from the non-preferred to the preferred path, the rpd process runs the path selection. During rpd path selection, a core file might be generated. This issue has no operational impact. A workaround is available to avoid this issue. PR1180307

  • In a dual Routing Engines scenario, if OSPF protocol is configured with MD5 authentication, after Routing Engine switching, the OSPF session might flap for authentication failure. PR1198179

  • With nonstop-routing (NSR) enabled, all running protocols include PIM and NG-MVPN replicated. If NSR is disabled only under PIM using the set protocol pim nonstop-routing disabled statement, this will remove both PIM and next generation MPVN from the replicated list. By adding PIM NSR again with the delete protocol pim nonstop-routing disabled command will not work as expected and PIM will not be added.PR1203943

  • Suppose a BGP route is resolved using a secondary OSPF route that is exported from one routing-instance to another routing-instance. If the BGP route is withdrawn while the OSPF route is deleted, rpd might restart unexpectedly. PR1206640

  • BGP routes are rejected as cluster ID loop prevention check fails due to a misconfiguration. But when the misconfiguration is removed, BGP routes are not refreshed. The fix for this issue sends a soft route refresh dynamically when a cluster ID is deleted. PR1211065

  • In a rare condition after a BGP session flaps, BGP updates might not be sent completely, resulting in BGP routes being shown in the advertising-protocol table on the local end but not shown in the receive-protocol table on the remote end. PR1231707

  • The routing protocol process (rpd) sometimes is interrupted and halted when it tries to free a session reference block. This can occur when the memory red zone check fails and at the same time attempts to free a reference memory block. The failure is caused when the red zone check receives an address that is not the beginning of a memory block. PR1232742

  • In a PIM scenario with BSR configured, after deleting a static rendezvous point (RP) configuration from another router, then checking an RP table on a BSR router, there might be a stale bootstrap RP entry (which is the static RP deleted from another router) in the RP table. PR1241835

  • The rpd might crash after configuring an IP address that does not exist on the device under [routing-options bmp local-address]. PR1244556

  • The OpenSSL project has published a security advisory for vulnerabilities resolved in the OpenSSL library on January 26, 2017. Refer to JSA10775 for more information. PR1249517

  • If Bidirectional Forwarding Detection (BFD) is configured with fast mode (the parameter minimum-interval is configured with microseconds), during the initial phase of the BFD session, because of a network issue or certain filter, the device might drop the BFD packet with the final bit set, then it will cause the BFD session to be stuck at slow timers (e.g. 2 second). This issue might impact the convergence of the network protocol related to drop more packets. PR1254063

  • The rpd process generates a core file due to BGP UPDATE with malformed optional transitive attributes (CVE-2017-10618); Refer to https://kb.juniper.net/JSA10820 for more information. PR1279204

  • On releases before Junos OS Release 16.1 with BGP, Prefix-Independent Convergence (PIC) and the RIB import feature enabled, if Intermediate IS-IS primary route is deleted, the rpd process might crash and a core file might be generated. This could cause routing protocols to restart. PR1303327

Services Applications

  • MS-PIC generates a core file when routing updates (IPv4/MPLS/IPv6) are received in the PIC. PR1170869

  • In an IPsec scenario, the kmd process might crash after configuring a certain IPsec configuration by apply-groups. PR1265404

Subscriber Access Management

  • If RADIUS returns Framed-route="0.0.0.0/0" to a subscriber terminated on a Junos OS platform, the subscriber cannot log in due to an authentication error. PR1208637

  • Customer encountered an issue with hanged out active PPPoE sessions. Even though the session timeout for these sessions expires, the subscribers still show up. These sessions cannot be cleared using network-access or pppoe sessions commands. The only option to clear these sessions is to use a dynamic-configuration session delete method. PR1230315

User Interface and Configuration

  • If a user enters configuration mode with the configure exclusive command after the configuration is automatically rolled back because of an unconfirmed commit, the user can still make configuration changes with the replace pattern command, and the subsequent commit fails with "error: access has been revoked". After exiting configuration mode, the user fails to enter configuration mode with configure exclusive with error: configuration database modified. PR1210942

  • This issue is specific to a router running Junos OS Release 15.1Rx and earlier, which also has authentication-key-chains configured. When the secret for a key is not configured, commit would fail with the message "error: configuration check-out failed: daemon file propagation failed". This issue is not applicable to Junos OS Release 15.1F or Junos OS Release 16.1 and later. PR1213165

VPNs

  • In a BGP VPLS environment, sometimes routes from BGP with invalid next-hop related information. In such scenarios, VPLS should treat them as bad routes and not send them to rpd infrastructure for route resolution. Due to a software defect, the bad routes are passed to the route resolver, which might lead to an rpd process crash. The routing protocols are impacted and traffic disruption will be seen due to loss of routing information. PR1192963

  • The routing protocol process (rpd) may eventually become exhausted and crash when Layer 2 circuit, Layer 2 VPN, or virtual private LAN service (VPLS) configurations are committed. These commit activities may create a small memory leak of 84 bytes in the rpd. If the rpd memory is exhausted, recovery can be accomplished by restarting rpd. If nonstop routing (NSR) is configured, the master Routing Engine can be switched over to the standby Routing Engine, causing the master rpd to exit and restart and free the leaked memory. PR1220363

  • In a next-generation MVPN scenario with the asm-override-ssm configuration statement for source specific multicast (SSM) group, if you issue the clear pim join command on the source PE, downstream interfaces get pruned, causing the multicast flow to stop. If you issue the clear pim join again, then the issue is resolved. PR1232623

  • The Layer 2 circuit does not switch from primary to backup and vice versa based on the Automatic Protection Switching (APS) status change, because when APS switchover happens, the PW switchover does not switch to the new APS active neighbor. PR1239381

  • If Rosen7 (PIM-MVPN) is enabled for IPv4, but does not explicitly set next-generation MVPN to disable for IPv6, then when PIM multicast route is created in IPv4 it will also create the ALT KAT timer. However, when the IPv4 multicast route is removed, PIM checks if next-generation MVPN is enabled for IPv4 only, which is false. So, ALT KAT timer is not deleted. This leads to a memory leak. PR1276041

  • Moving an MC-LAG (Multichassis Link Aggregation Group) interface from LDP based pseudowire to BGP based pseudowire in a single commit might cause rpd crash. PR1325867

Resolved Issues: 14.1R8

General Routing

  • DPD may not work with link-type IPSec tunnels when NAT is present between the IPSec peers. Even when NAT is not present between the IPsec peers, the issue can occur with lesser probability. PR895719

  • On MX Series router, the physical or logical interfaces (ifd/ifl) might be created and marked UP before a resetting FPCs' fabric planes are brought up and ready to forward traffic, as a result, traffic might be black-holed during the time window. This window of traffic black-hole is particular long if the chassis is heavily populated with line-cards, for example, the router has large scale of configuration (routes or subscribers), and coupled with a lot of FPC reset, such as upon a node power up/reset. PR918324

  • In MX Virtual Chassis (MX-VC) environment, the private local next hops and routes pointing to private local next hops are sent to Packet Forwarding Engine from master Routing Engine and not sent to slave Routing Engine, then an Routing Engine switchover happens. Now as the new master Routing Engine does not know about such next hops and routes, they are not cleaned up. When a next hop with same index is added on new master Routing Engine and sent to Packet Forwarding Engine, the Packet Forwarding Engine might crash due to a stale next hop. PR951420

  • The L2ald may crash after interface flap. PR1015297

  • For Junos 13.3R5 14.1R1 and onwards, the MX-VC inter-chassis TCP control flows are changed to VC high priority, so high volume of VC inter-chassis TCP control flow might impact VC stability and responsiveness to external protocol events. Now with the fix, the priority of VC inter-chassis TCP control flow has been reverted. PR1074760

  • Junos OS runs PKId for certificate validation. When a peer device presents a self-signed certificate as its end entity certificate with its issuer name matching one of the valid CA certificates enrolled in Junos, the peer certificate validation is skipped and the peer certificate is treated as valid. This may allow an attacker to generate a specially crafted self-signed certificate and bypass certificate validation. Refer to JSA10755 for more information. PR1096758

  • When DHCP subscribers are terminated at specific routing-instances and the interface stack is IP demux over vlan-subinterface over AE interface, there might be a memory leak in the kernel AE iffamily when subscribers log in/log out. PR1097824

  • If NSR (nonstop routing) is enabled and a TCP session is terminated while there is still data in the socket pending transmission, the MBUF (kernel memory buffer) used to store this data might not get deallocated properly. In order to hit this issue the TCP session must use NSR active socket replication. If the system runs low on MBUF memory, the kernel will automatically throttle down memory allocation on low priority applications and ultimately, if there is no MBUF left, the system could become unresponsive due to its inability to serve I/O requests. PR1098001

  • On MX platform, in rare condition, if Packet Forwarding Engine sends wrong Packet Forwarding Engine id to chassisd as part of capability message, kernel might crash and some FPCs might be stuck in the present state, the traffic forwarding will be affected. This is a corner case, it is not reproduced consistently. PR1108532

  • On MX240/480/960 Series routers with MS-DPC, customer is running BGP over IPSec. This BGP session has a BFD session tied to it. The BGP session is up but the BFD session remains in INIT state. The issue might be seen with any service configured with multi-hop BFD enabled. Traffic forwarding will not be affected. PR1109660

  • In rare condition, after RE switchover, - the MPC PIC might offline, and some error messages might be seen. - at times chassisd on RE goes to continuous coring makes unit unusable as none of interfaces come up. Root cause: After RE switch over chassisd fail to get proper status of the FPCs and cores due to insufficient IDEEPROM read times. PR1110590

  • On FPC-SFF-PTX-P1-A(PTX3000)/FPC-SFF-PTX-T(PTX3000)/FPC-PTX-P1-A(PTX5000)/FPC2 -PTX-P1A(PTX5000), packet loss may be observed in ECMP or AE scenario. That occurs in a race condition: the unilist is created before ARP learned MAC addresses, then the selector table is corrupted. PR1120370

  • On MX Series platforms, the MS-MPC crash might occur. The exact trigger of the issue is unknown; normally, this issue might happen over long hours (e.g, within a week) of traffic run (e.g, running HTTP/HTTPS/DNS/RTSP/TFP/FTP traffic profile). PR1124466

  • With IPv6 access route configured in dynamic profile, when the router receives IPv6 SOLICIT message which request only Prefix Delegation but no IPv6 address, the access route will not be installed successfully. PR1126006

  • If two redundant logical tunnel (rlt) sub-interfaces are configured in the same subnet and in the same routing-instance, a sub-interface will be down (this is expected), but if the sub-interface is removed from the routing-instance later, after disabling and enabling the rlt interface, a sub-interface might remain in the down state unless you remove the configuration of the rlt interface and then do a rollback. PR1127200

  • When software encounters an error configuring the optics type into the VSC8248 PHY retimer component of an MX MIC/PIC (typically done on SFP+ module plugin), this could lead to 100% FPC CPU utilization indefinitely. MPCs and MICs that are potentially affected are: MPC3 + 10x10GE SFPP MIC MPC4 32XGE MPC4 2CGE+8XGE (10G interfaces only) MPC6 + 24x10GE (non-OTN) SFPP MIC PR1130659

  • In a situation where both mirrored interface and mirrored destination are on MPC card and mirror destination interface is a unilist next-hop(e.g. an ae interface), mirrored packets may get dropped. PR1134523

  • On MX platform, the "Max Power Consumption" of MPC Type 1 3D (model number: MX-MPC1-3D) would exceed the default value due to software issue. For example, the value might be shown as 368 Watts instead of 239 Watts when "max ambient temperature" is 55 degree Celsius. PR1137925

  • In the multicast network topology, when making normal changes, such that paths are added or deleted, the rpd leaks 8-bytes memory per operation. The system logs RLIMIT_DATA messages similar to the following when the memory usage reaches 85%: kernel: Process (2634,rpd) has exceeded 85% of RLIMIT_DATA: used 3084524 KB Max 3145728 KB PR1144197

  • Commit error after attempting to delete all guaranteed rates on all traffic-control-profiles associated with demux0 [edit] lab@mx480-J12_09# commit re0: [edit class-of-service interfaces] 'demux0' IFL excess rate not allowed on interface (demux0), please specify guaranteed rate on at least one IFL error: configuration check-out failed PR1150156

  • When using type 5 FPC on T4000 platform, traffic going out of the interface where "source-class-usage output" is configured will be dropped if the Source Class Usage (SCU) or Destination Class Usage (DCU) policy configuration is missing. This issue is caused by incomplete configuration, so to avoid the issue, please make the configuration complete (e.g., with "source-class-usage output" and SCU policy). PR1151503

  • Dynamic-tunnel interface bounces causing memory corruption leading to rpd crash. And the new rpd process once up, sync's up with the kernel, which may have information stored about the GRE tunnel ifl created by previous rpd process. The new rpd process using this information from the kernel leading to subsequent rpd crash being triggered. The following logs might be seen when this issue occurs: root@abc>show log messages| match "Address already in use" %DAEMON-3: Error creating dynamic logical interface from sub-unit 32792: Address already in use %DAEMON-3-RPD_KRT_Q_RETRIES: kqp 0x49df00d0: op add queue low-add attempts 4010 ifd index 284, ifl unit 32792, family 2 instance id 0, state CreateIFL RPD_KRT_Q_RETRIES: IFL IFF Update: Address already in use PR1152912

  • Routers using inline layer 2 services may experience Packet Forwarding Engine wedge leading to fabric degradation and FPC restart. During issue state, the affected FPC will not be able to transmit and traffic will be fully blackholed. This problem is amplified by fragmented and out of order packets. This log entry may be seen during the error state: Host Loopback:HOST LOOPBACK WEDGE DETECTED IN PATH ID 0. PR1153750

  • In sampling feature, certain scenarios force handling of the sampled packet at the interrupt context , which may have chance to corrupt the BMEB packet context , and lead to BMEB FDB corruption. PR1156464

  • Given an active BGP multipath route with 2+ Indirect-Next-Hops and another BGP route which can participate in protocol independent multipath with router-next-hop, rpd might crash if the interface on which first member of Indirect-Next-Hop resolves goes down. PR1156811

  • In the TXP environment, the Line-Card Chassis (LCC) Switch Interface Board (SIB) status is not right when executing command "user@router> show chassis environment". The status may remain on Absent, but with no alarms. This is a minor issue, it does not affect functionality. PR1156841

  • A previous enhancement to strengthen the VC-Heartbeat message exchange resulted rejecting messages at the crucial time of determining the health of the other VC member when all adjacency links fail. Validation of messages has been adjusted to remain strong when the VC is connected, but relaxed during the split conditions to prevent rejecting valid messages. PR1157383

  • Packet Forwarding Engine interfaces on Trio-based line cards might remain down after performing "request system reboot both-routing-engines " or "restart chassisd" several times. Reboot the FPC might restore it. PR1157987

  • On Junos devices with a GRE or IPIP tunnel configured (i.e., devices with a gr- or ip- interface), a specifically crafted ICMP packet can cause a kernel panic resulting in a denial of service condition. Knowledge of network specific information is required to craft such an ICMP packet. Receipt of such a packet on any interface on the device can cause a crash. Refer to JSA10752 for more information. PR1159454

  • On MX Series platform, when MPC experiences a FATAL error, it gets reported to the chassisd daemon. Based on the action that is defined for a FATAL error, the chassisd will take subsequent action for the FATAL error. By default, the action for FATAL error is to reset the MPC. When the MPC reports FATAL error, chassisd will send offline message and will power off the MPC upon the ACK reception. However, if MPC is in busy state for any reason, the ACK doesn't come in time and hence there would be a delay in bringing down the MPC. The fix ensures to bring down the MPC in time upon FATAL error. PR1159742

  • Software OS thread on the line card is doing a busy loop by reading the clock directly from hardware, Sometimes it seems the thread is getting wrong values from HW register and waiting forever in the busy loop. After the busy loop crosses a certain time period, the line card crashes and reboots. This is a rare condition. PR1160452

  • On MX Series routers with enhanced queuing DPCs, there is a memory leak whenever doing SNMP walk to any of COS related OID's or issue the command "show interfaces interface-set queue <interface-set name>". PR1160642

  • The Router Lifetime field is set to 0 in the first Routing Advertisement sent from LNS back to PPPoE subscriber. PR1160821

  • The VCCPD_PROTOCOL_ADJDOWN system log message does not include a 'reason' string to explain why the virtual chassis adjacency was terminated. This information will now be present in the message. PR1161089

  • The default (per-packet load balancing) PPLB export policy created for Ethernet VPN (EVPN) has been removed from JUNOS. It was used to enable per packet load-balance for EVPN routes on certain MX platforms and not all. Now per-packet load balance needs to be configured explicitly. PR1162433

  • The ICMP time exceeded error packet is not generated on an IPsec router on the decap side. The problem is fixed for MS-MPC/MIC and works fine if the session is there. There is no other way to return the time exceeded message over a tunnel. There is no plan to fix this for MS-DPC. PR1163472

  • On MX Series router with MPC3/4/5/6/7E/8E/9E linecard, neither low-light warning nor alarm work on these linecards with 10G or 100G interfaces. When using JAM image, NG-MPC are affected as well. This is optics or fiber issue, no critical service impact. PR1168589

  • Sampled continues logging events in traceoption file after traceoption for sampled deactivated. This can be hit if there is no configuration under 'forwarding-options sampling' but other configuration for sampled is present (e.g. port-mirroring). PR1168666

  • An ungraceful removal of an FPC can trigger fabric healing to kick in. PR1169404

  • Adding keyword 'fast-filter-lookup' to existing filters of an input or output filter list may result in failure to pass traffic. To avoid this issue, the filter list should first be deactivated then the filters updated with a the keyword 'fast-filter-lookup; then the filter list activated. PR1170286

  • If the "no-cell-share" configuration statement under the chassis stanza is activated on MPC3, MPC4, MPC5, or MPC6 cards, the Packet Forwarding Engine will only be able to forward about 62Gbps versus ~130Gbps and causing fabric queue drops. PR1170805

  • The fan speed logic does not operate correctly once PEM on MX104 platforms does automatically shuts down due to over-temperature protection. The fan speed moves back to speed normal. It takes more time for PEM to cool down and come back online automatically with fan at normal speed. PR1174528

  • Storm control feature is not working on MX104 platform. In Packet Forwarding Engine, associated filters and vty commands are not visible as well. It works on other MX series platforms. PR1176575

  • In a multicast scenario where there is PIM configured, if there are PIM assert messages sent or received or there is MVPN configured and NSR enabled, memory leak might happen in rpd. PR1177125

  • This is a display issue and doesn't affect functionality of the power, fixing has been added to commands 'show chassis power' and 'show chassis environment pem', when one of the DC PEM ciruit breaker tripped. PR1177536

  • In EVPN A/S mode, IFL mark down programming at the Packet Forwarding Engine on the BDF gets removed causing traffic loops. PR1179026

  • On 10x10GE(LAN/WAN) SFPP PIC, when the port is configured with WAN PHY mode, the CoS configuration on the port will be incorrectly programmed and it might result in unexpected packet drop. PR1179556

  • In case of point to point interfaces and unnumbered interfaces rpd crash might be seen in corner cases on configuration changes. There is potential fix given through this PR to avoid the crash. PR1181332

  • When "dynamic-tunnels" is configured with knob "gre", performing RE switchover might result in rpd crash. PR1181986

  • In IPv6 environment, adding a link local neigbour entry on subscriber interface then adding a new lo0 address, if delete this neighbour entry and the subscriber interface, due to software defect, the nexthop info is not cleaned properly, the rpd process might crash. The routing protocols are impacted and traffic disruption will be seen due to loss of routing information. PR1185482

  • ksyncd crash might be seen with GRES due to kernel replication error. PR1186317

  • The command "request system reboot both-routing-engines local' on VC-Mm will reboot only one RE on an MX-VC, with this fix, it will reboot both REs of local chassis. In addition, this fix also removes the "set virtual-chassis member <n> role line-card" configuration option on an MX-VC because this option is not supported on MX-VC as designed. PR1188383

  • On MX routers, a vulnerability in IPv6 processing has been discovered that may allow a specially crafted IPv6 Neighbor Discovery (ND) packet to be accepted by the router rather than discarded. The crafted packet, destined to the router, will then be processed by the routing engine (RE). A malicious network-based packet flood, sourced from beyond the local broadcast domain, can cause the RE CPU to spike, or cause the DDoS protection ARP protocol group policer to engage. When this happens, the DDoS policer may start dropping legitimate IPv6 neighbors as legitimate ND times out. Refer to JSA10749 for more information. PR1188939

  • In rare scenario, there may be a transient state when OSPF route leaked from routing-instance to inet.0 is being withdrawn, at the same time the BGP route resolving over that same OSPF route is also being withdrawn, rpd might crash and result in protocols to restart. PR1206640

Class of Service (CoS)

  • This PR does optimization in AE SNMP handling. If all the links in an AE bundle go down, then any COS SNMP query for this AE IFD/IFL will return cached values. PR1140440

  • On MX series routers with Non-EQ DPCs installed, committing the configuration of "rate-limit" under "class-of-service schedulers might fail and the following error reported:Â cosd[2249]: COSD_RATE_LIMIT_INVALID: Unable to apply scheduler map CORE to interface xe-4/0/0. "buffer-size" cannot be configured on rate-limited queues PR1157291

Forwarding and Sampling

  • On MX series routers, a change of policers or counters to an existing firewall filter using physical-interface-filter or interface-specific configuration statements will not be correctly detected by MIB2D. PR1157043

  • After upgrading by using ISSU, as part of bring-up procedure, mib2d will initialize connections to FPC PFEs ( packet forwarding engines ). It might start querying states from PFE while the connection is not ready yet. This failure will cause the connection to reinitialize again. Thus this can form sort of loop which can cause memory and CPU cycle usage to grow. As a result, it causes mib2d to crash. PR1165136

  • Commit gives error as follows when apply-groups is configured under bridge domain. error: Check-out failed for Firewall process (/usr/sbin/dfwd) without details. PR1166537

  • When polling SNMP counters for Trio-Only firewall filters, MIB2D_RTSLIB_READ_FAILURE cosmetic error messages might get reported in syslog. PR1173057

  • statistics-service daemon (pfed) experiences constant memory leak of 10 KB every 2 minutes when MobileNext package is installed: > show version Model: mx480 Junos: 14.1X55-D30.10 JUNOS Base OS boot [14.1X55-D30.10] <...> JUNOS MobileNext Routing Engine Software [14.1X55-D30.10] <<< this package PR1174193

  • Even if packets don't match firewall filter conditions, wildcard mask firewall filter might match any packets. << Sample config >> ------------------------------------------------- set firewall family inet filter TEST-filter term TEST1 from destination-address 0.0.0.255/0.0.0.255 <<<<<< set firewall family inet filter TEST-filter term TEST1 then count TEST1 set firewall family inet filter TEST-filter term TEST1 then discard set firewall family inet filter TEST-filter term TEST2 then accept ------------------------------------------------- This is discard filter for /24 prefix broadcast address. However it might discard other packets. PR1175782

High Availability (HA) and Resiliency

  • Right after all FPC complete their upgrade, the kernel (on the VC-Mm) closes its connection to ksyncd (on the VC-Bm) since it has received a message "invalid IPC type 20". This disconnect causes ksyncd to restart, it then cleans all kernel state in the VC-Bm and starts the replication process. This causes the timer for waiting for the VC to become GRES ready (after FPC upgrade) to expire and abort the ISSU. PR1163807

  • When configure the "nonstop-routing" under one group and apply this group to routing-options configuration hierarchy, sometimes the NSR does not work. As a workaround, please configure the "nonstop-routing" directly under the routing instance hierarchy. PR1168818

  • Backup routing engine (Backup RE) may restart unexpectedly due to memory leak after switchover. PR1198005

Infrastructure

  • In scaling setup (in this case, there are 1000 VLANs, 1000 Bridge Domains, 120 IRB interfaces, 120 VRRP instances, BGP and IGP), if the routing protocols are deactivated and activated, there might be a chance that the pending route stats are not cleaned up, which will cause the stats infra to have stale pointers and lead to memory corruption in socket layers. The system might go to db prompt because of this. All the traffic going through the router will be dropped. PR1146720

  • On M/T/PTX platforms, the SNMP requests may return timeout if SNMP pollings on IF-MIB and COS-MIB for the same ifl/ifd are requested at the same time. This is a generic async stats infra issue in kernel. On MX Series platform, the same issue may not be seen since SNMP pollings for ifl stats go through pfed instead of kernel on MX Series platform. PR1149389

Interfaces and Chassis

  • Demux Subscriber IFLs might show the interface as 'Hardware-Down' even though the underlying ae bundle and its member link(s) shows up. PR971272

  • During failure notification state machine, CFM does not correctly transit from DEFECT CLEARING state to RESET once the error indication has been cleared. As a consequence all the forthcoming errors will be considered post errors and will be reported right away without incurring the fngAlarmTime. This is a cosmetic problem. PR1096346

  • Due to movement of SNMP stats model from synchronous requests to asynchronous requests in Junos OS Release 13.3R1, the IQ2/IQ2E PIC, which has limited memory and CPU power, can not handle scaling SNMP polling at high rate (e.g., a burst of 4800 SNMP requests). This issue comes with high rate SNMP stats polling for IQ2/IQ2E interfaces or Aggregated Ethernet (AE) interface with IQ2/IQ2E as member links. These memory failures can cause IQ2/IQ2E PIC reboot because keep alive messages will also not get memory. PR1136702

  • Starting from 12.3R4, on dual-RE equipped M series routers, due to the mismatch of online status of the missing FRU (e.g. FPC or FEB which is not inserted, but is reported as online on backup Control Board), error messages about the missing FRU might be seen intermittently on the device. PR1148869

  • In affected releases, the following cosmetic alarms are seen after reseating the clocking cables: 2015-11-13 05:22:56 UTC Major CB 0 External-A LOS 2015-11-13 05:22:56 UTC Major CB 0 External-B LOS PR1152035

  • SONET interface on MIC-3D-1OC192-XFP does not count input error correctly. While hardware counts framing error, runts and giants but input error in 'show interface extensive' command reports runts and giants only. PR1154268

  • If an interface configured with VRRP is removed from a routing-instance to global, or from global to a routing-instance, the IFLs of that interface will be deleted and recreated. In ideal cases as the interface gets deleted, VRRP should move to bringup state; when the interface is created again, VRRP goes to previous state. After this, VRRP should get VIP addition notification from kernel and update VRRP state and group id for VIP. However, in race conditions, VRRP might get VIP addition notification from kernel even before the interface creation event happens. If so, VRRP will never be able to update proper VRRP state and group id. So the VIP will reply for the ARP with an incorrect MAC ending with "00", while the correct MAC should end with the groups id configured. PR1169808

  • In previous release, only IEEE classification is supported for CFM OAM packets. In the fix, we will support 802.1AD based filter for CFM OAM packets. when Linktrace and loopback requests are received in MX, 802.1p bits is used to determine the forwarding class and queue for response or linktrace request forwarded to next router, this cause these PDUs are put to wrong queue when input-vlan-map pop is present because received PDU doesn't carry 802.1p bits. In the fix, we will use incoming forwarding class to determine the 802.1p priority and outgoing forwarding class and queue for new generated response or link trace requests. PR1175951

  • On dual RE system, if master RE is running Junos OS 13.3R9/14.1R7/14.2R5/15.1R3/15.2IB or later, backup RE is running Junos OS prior to 13.3R9/14.1R7/14.2R5/15.1R3/15.2IB, a major alarm is raised. This is cosmetic and can be safely ignored. Please upgrade backup RE to the same release with master RE to avoid the issue. user@router> show system alarms 2 alarms currently active Alarm time Class Description 2016-xx-xx xx:xx:xx UTC Major PEM 1 Not OK 2016-yy-yy yy:yy:yy UTC Major Host 1 failed to mount /var off HDD, emergency /var created <<<<<<<<<<<<<<< PR1177571

  • Commit check may exit without providing correct error message and causing dcd exit. The only known scenario to trigger this issue is to configure a IPv6 host address with any other address on the same family. PR1180426

Layer 2 Features

  • In BGP-based VPLS scenarios, changing the configuration of a VPLS mesh group might cause rpd core. FPC reboot might also be seen during the rpd core. PR1123155

  • In a VPLS scenario, when "$junos-underlying-interface-unit" is configured in "dynamic-profiles" hierarchy, which is then implemented in a routing-instance, upgrade/commit will fail with the following error message: Parse of the dynamic profile (<dynamic_profile_name>) for the interface: $junos-interface-ifd-name and unit: $junos-underlying-interface-unit failed. PR1147990

  • The rpd process might crash when adding/deleting Virtual private LAN service (VPLS) neighbors in a single commit. For example, a primary neighbor is changed to become the backup neighbor. PR1151497

Layer 2 Ethernet Services

  • The "Node ID" information is not shown on MX platform when traceoption flag "pdu" is configured to trace Ethernet ring protection switching (ERPS) PDU reception and transmission. PR1157219

  • During l2cpd restart, STP is not receiving restart status. So l2cpd is taking wrong flow during STP initialization and new STP index is allocated for instance "0", and instance "0" is always set to "DISCARDING" status. This might lead to traffic loss. PR1176312

MPLS

  • In MPLS environment, the master Routing Engine might crash due to Mbuffer allocation failure and this crash will trigger an Routing Engine switchover, as a result Backup Routing Engine will become active. The issue is unreproducible, and trigger condition is not clear. PR979448

  • If a RSVP LSP has both primary&&secondary standby path and link-protection enabled, a /32 bypass route is unhidden when the primary link goes down. This /32 route is supposed to be made hidden again when primary link comes back up. But in some cases, due to software defect, this /32 bypass route remains unhidden forever which causes some issues, for example, BFD session down due to better prefix received from Bypass LSP. PR1115895

  • User is allowed to configure both "load-balance-label-capability" and "no-load-balance-label-capability" together. This is incorrect and confusing. PR1126439

  • When a link fails on an RSVP LSP which has link-protection or node-link-protection configured, the PLR (point of local repair) will initiate a bypass LSP and the RSVP LSP will be tunneled on this bypass LSP. However, if now the bypass LSP is brought down because there is a link failure on it, the PLR might only send out a session_preemted PathErr message to the upstream node without sending a ResvTear message. Hence the ingress node does not receive a ResvTear message and the RSVP LSP is not immediately torn down. The RSVP LSP will remain UP for more than 2 minutes until the RSB (Resv sate block) on the ingress's downstream node gets timed out and it sends a ResvTear message to the ingress. PR1140177

  • In LDP P2MP scenario with NSR, after performing multiple iterations of FPC reloads, protocol bounce, interface bounce, GRES, rpd restarts in random, in rare condition, the rpd process might crash, the routing protocols are impacted and traffic disruption will be seen due to loss of routing information. PR1148404

  • With NSR enabled and LDP configured, the rpd process might crash and restart on the new master Routing Engine after a Routing Engine switchover. PR1155002

  • When L2VPN composite next hop configuration statement is enabled along with L2VPN control-word, end-to-end communication fails. Because in this scenario, control-word is not inserted by the ingress PE, but other end expects the control-word. PR1164584

  • In LDP-signaled VPLS environment, other vendor sends an Address Withdraw Message with FEC TLV but without MAC list TLV. The LDP expected that Address Withdraw Message with FEC TLV should always have MAC list TLV. As such, it rejected the message and close the LDP session. The following message can be seen when this issue occurs: A@lab> show log messages |match TLV RPD_LDP_SESSIONDOWN: LDP session xxx.xxx.xxx.xxx is down, reason: received bad TLV PR1168849

  • In MVPN scenario, if active primary path goes down, then PLR(Point of Local Repair) needs to send Label Withdraw for old path and new Label Mapping for new path to the new upstream neighbor. In this case, LDP P2MP path may stay in "Inactive" state for indefinite time if an LSR receives a Label Release, immediately followed by a Label Mapping for the same P2MP LSP from the downstream neighbor. PR1170847

Network Management and Monitoring

  • In customer setup pfe was not able to keep-up with full stats requests from PFED. Because of this delay, PFED runs out of transfer credits to send stats request to PFE and starts returning full stats requests with error response to mib2d with ifl-info flag set to LS STATS and a payload filled with value zero. mib2d was treating the returned 0 filled stats value as correct stats and was returning these 0 values. This results in spike in delta value calculated by the customer side script. PR1010534

  • With Junos OS release 13.3R8/14.1R6/14.1X53-D30/14.2R5/15.1R2/15.1X49-D30 and above, when we configure fxp0 "master-only" address as source address of snmp trap, the snmp trap packets are not sent out after Routing Engine (RE) switchover. To restore this issue, we can use "restart snmp" or "delete/set snmp trap-options". As a workaround, we can use other addresses for snmp trap source. PR1153722

Platform and Infrastructure

  • After configuration of em interface changed (such as configuring family inet or ip address, but MTU is not changed) and system rebooting, the em interface may flap or go down. It could cause RE and FEB connection failure. Under normal circumstances, em interface should not re-initialize when MTU is not changed. So the fix is avoiding reinit of em driver, if MTU is the same. PR983616

  • The management process daemon (mgd) process might be stuck in a loop and cause high CPU usage on RE PR991616

  • When one of the "deny-commands" is incorrectly defined in the profile of TACACS+ server, all "deny-commands" regexes will be ignored, which leads to an over-permissive profile without any warning. PR1078238

  • On MX platform with MPC/MIC or T4000 FPC5, TCP session with MS-Interface/AMS-Interface configuration is not established successfully with the "no-destination-port" or "no-source-port" knobs configured under forwarding-options hierarchy level. PR1088501

  • On MX platform, when offlining the line card (possibly, with any of the line cards listed below), "Major alarm" might be seen due to HSL (link between line card and PFE) faults. This fault is non-fatal and would not cause service impact. The line cards that may hit the issue could be seen as below, MS-MPC/MS-MIC MIC-3D-8DS3-E3 MIC-3D-8CHDS3-E3-B MIC-3D-4OC3OC12-1OC48 MIC-3D-8OC3OC12-4OC48 MIC-3D-4CHOC3-2CHOC12 MIC-3D-8CHOC3-4CHOC12 MIC-3D-1OC192-XFP MIC-3D-1CHOC48 PR1128592

  • Too many duplicate ACK messages are generated from PFE for TCP control connection with RE. This could cause: 1. MX-VC DDoS protection violation for VC-control low queue and makds MXVC split. 2. Cause RE and FPC high CPU utilization. PR1133293

  • On ungraceful exit of telnet (quit/shell logout), perm and env files created by pam were not deleted. PR1142436

  • When ARP is trying to receive a nexthop message whose size (for example 73900 bytes) is bigger than its entire socket receive buffer (65536 bytes), the kernel might crash, and the traffic forwarding might be affected. PR1145920

  • On MX platform with MX Series based line card, inline 6rd with si interface is deployed, if downlink traffic is over ECMP or AE, some traffic might be dropped. PR1149280

  • On MX2000 Series, MPC4 going offline is seen when SFB (Switch Fabric Board) is offlined or removed. This could be caused by the build-up of CDR in ADC which leads to transient packet loss or even getting stuck. The fix prevents line-cards going offline due to transient buildup in ADC. PR1149677

  • When a routing instance is configured with "routing-instances <instance-name> routing-options localized-fib" then VPN localization may fail, causing all routes for the affected routing instance to be installed on all PFEs. PR1149840

  • When the NTP server address is configured in routing instance table and reachable from inet.0 by static configuration (for example, by configuring static/route/next-table/VRF.inet.0), and NTP source-address is configured, the ntpd (the Network Time Protocol daemon running on NTP client) might pick the wrong source-address instead the configured source-address. As a result, NTP server cannot reply the NTP packet back. PR1150005

  • During an ISSU upgrade in MXVC environment, linecards may crash causing service impact. When the linecards come up, there may be a nexthop programming issue as a secondary impact and some IFLs may not pass traffic. Affected linecards need to be rebooted to recover from this condition. PR1152048

  • Fixed an issue with Inline Jflow where the Observation Domain field in exported IPFIX datagrams were always using the value attributed for LU0 in MPCs with multiple LUs per forwarding-engine. PR1152854

  • The logs CHASSISD_READBACK_ERROR are reported on the backup RE for the non-empty FPCs. PR1155823

  • On MX2000 series platform, when MPC goes down ungracefully, other MPCs in the chassis will experience "destination timeout". In this situation, auto fabric-healing will get triggered due to "destination timeout" condition, which may cause Fabric-Plane reset, even all other MPCs to be restarted in some cases. PR1156069

  • Group names handling process enhancement: one of the core functions was optimized by introducing more efficient pointer comparisons instead of CPU-intensive string ones. PR1158652

  • If one logging user is a remote TACACS/RADIUS user, this remote user will be mapped to a local user on device. For permissions authorization of flow-tap operations, when they are set on the local device without setting the permissions on the remote server, they cannot work correctly. The flow-tap operations are as follow: flow-tap -- Can view flow-tap configuration flow-tap-control -- Can modify flow-tap configuration flow-tap-operation -- Can tap flows PR1159832

  • LU(or XL) and XM chip based linecard might go to wedge condition after receiving corrupted packets, and this might cause linecard rebooting. PR1160079

  • Due to software bug on chassisd, backup CB temperature information is missing on cli command 'show chassis environment cb' if it's replaced once. PR1163537

  • For MX Series Virtual Chassis with "default-address-selection" configured, when we have a discard route to a specific subnet ( e.g. 10.0.0.0/8 ) with discard next-hop, and at the same time we have more specific routes through other interfaces ( e.g. 10.1.1.1 through xe-0/0/0 ), if a UDP packet is being sent to 10.1.1.1 through xe-0/0/0 while interface xe-0/0/0 flaps or FPC reboots, it might cause kernel crash on both Master Routing Engine in the Virtual Chassis master router (VC-Mm) and Master Routing Engine in Virtual Chassis backup router (VC-Bm). As a workaround, we can disable "default-address-selection" configuration. PR1163706

  • The following log can be seen on MX2020 after one FPC was pulled out and committing the configuration related interface: CHASSISD_UNSUPPORTED_FPC: FPC with I2C ID of 0x0 is not supported. PR1164512

  • Modifying the configuration of a hierarchical policer when in use by more than 4000 subscribers on an FPC can cause the FPC to core and restart. PR1166123

  • In affected release, if user runs the pfe debug command like "show sample-rr eg-table ipv4 entry ifl-index 1224 gateway 113.197.15.66" will cause the MPC crash. PR1169370

  • Because of an internal timer referring Time in Unix epoch (UNIX epoch January 1, 1970 00:00:00 UTC) value getting wrapped around for every 49 days, flows might get stuck for more than the period of active/inactive time out period. The number of flows that get stuck and how long they get stuck can not be deterministic exactly, which depends on the number of flows at the time of timer wrapping around. PR1173710

  • "show arp" command can't get complete results and reports "error: could not find interface entry for given index". PR1174150

  • On MX2020/2010, chassisd file rotation on commit check will cause the trace file to be stuck and no other operational chassisd events will be logged until chassisd restart. PR1177625

  • If igmp snooping is configured on the system and vpls instances has no active physical interfaces, multicast traffic arriving from the core might be send to the Routing-Engine. Host queues are getting congested and may cause protocol instability. PR1183382

  • VPLS: FPC CPU goes high for several minutes when mac/arp are learnt via lsi interfaces. The FPC CPU goes high during the learning phase and issue can be seen with various triggers that result in mac/arp re-learning e.g. mac flush, FPC reboot or link flap resulting in mac flush etc. For MPC 3D 16x 10GE, the CPU may remain high for upto 30 minutes on learning/re-learning of 10k arp/mac via irb lsi interfaces Problem is only seen if there are ARPs learnt in bulk over irb lsi interfaces. PR1192338

  • A rare VMCORE can occur caused due to process limit being breached by too many RSHD children processes being created PR1193792

  • For certain CLI commands, the log messages might be seen because of a bug where process id for forked process (to service cli command) is not set properly. This should not affect the system behavior PR1199184

  • When a Netconf '<get-route-information>' RPC is executed for all routes via ssh transport session and the session is terminated before all the route information is retrieved, the MGD process and RPD daemon will cause high CPU utilization for an extended period of time. Example of issues caused by this high CPU utilization for an extended period is as follow: BGP neighbors holddown timer expires and become ACTIVE OSPF adjacencies reset during database exchange OSPF LSA retransmissions events on neighboring nodes due to missing ACKs LDP sessions time out non distributed BFD sessions being reset due to missing keepalives PR1203612

Routing Protocols

  • BGP "accepted-prefix-limit" feature might not work as intended when it is configured together with "damping". Root cause of this issue is that when BGP module count the maximum routes accepted from BGP neighbor, it doesn't count the accepted BGP routes which in damping status. So when these damping routes are reused, the total number of received BGP routes exceeds the configured value for "accepted-prefix-limit" . PR897124

  • When route convergence occurred, the new gateway address is not updated correctly in inline-jflow route-record table (route-record table is used by sampling), and the sampling traffic forwarding might be affected, but normal routing would be not affected. PR1097408

  • After executing the CLI commands "show route detail" or "show route extensive," the routing protocol process (RPD) might get stuck in an infinite loop and might stop responding to any events such as CLI commands, protocol keepalives, etc. This would result in a timeout of all protocol adjacencies and a high CPU utilization by RPD might be seen on the device (over 90% used by RPD). In some cases, the memory that is used to store the command output might not be freed during executions, which might lead to an RPD restart because of memory exhaustion (RLIMIT exceeded). PR1104090

  • This issue is a regression defect introduced in Junos OS Release 11.4R11, 12.1R10, 12.2R8, 12.3R6, 13.2R4, 13.3R2, 14.1R1. After upgrading to those releases containing the original fix, when there is no export policy configured for the forwarding table to select a specific LSP, whenever routes are resolved over RSVP (for example, due to aggressive auto-bandwidth), the resolver will spend considerable amount of time on the resolver tree, which contributes to the baseline increase in rpd/Routing Engine CPU. PR1110854

  • IGMPv2 working in v2/v1 compatibility mode does not ignore v2 Leave messages received on a bridge-domain's L2 member interface. Moreover, an IGMP snooping membership entry for the respective group at this L2 member interface will be timed out immediately upon IGMPv2 Leave reception, even when there are some other active IGMP hosts attached to this L2 member interface. It might breaks multicast forwarding for this L2 member interface. PR1112354

  • During many types of configuration changes, especially including import policy, BGP has the need to re-evaluate the routes it has learned from peers impacted by the configuration change. This re-evaluation involves re-running import policy to see if there is any changes to the learned routes after applying the new policy. This work is done in the background as part of an "Import Evaluation" job. When BGP is reconfigured a second time, and the "Import Evaluation job" has not completed, it is necessary to re-run the job from the beginning if there's another change to policy or something with similar impact. This state is noted as "Import Evaluation Pending". However, in this case, there was a bug that caused BGP to always enter the pending state upon reconfiguration, regardless of whether relevant changes were made to import or other similarly impactful configuration. The result is that once it is necessary to start re-evaluation of the routes for a peer, even trivial configuration changes that happen too quickly will cause the "Import Evaluation job" to need to run again as a result of the "Pending" flag being set. To avoid the issue, please ensuring that "ImportEval" is not present in a BGP peer's Flags output from the CLI (show bgp neighbor) prior to doing even trivial commits. PR1120190

  • On Junos-based products, changes in routing-instance, like changing route-distinguisher or routing-option changes in some corner cases might lead to rpd crash. As a workaround always deactivate routing-instance part that is to be changed before committing the changes. PR1134511

  • When Protocol Independent Multicast (PIM) is used, in very rare condition, if the last hop router (LHR) migrates from (Designated Router) DR to non-DR, repeated routing protocol process (rpd) crash may occur due to patricia tree walk issue. PR1140230

  • In MVPN scenario, deleting mvpn configuration from routing instance (e.g. "delete routing-instances <instance-name> protocols mvpn") might cause the routing daemon on master RE to crash. The core files could be seen by executing CLI command "show system core-dumps". PR1141265

  • When multicast-only fast reroute (MoFRR) is enabled in PIM or multipoint LDP domain, memory leak will be observed on generation of the multicast FRR next-hops. The leak rate is 8-byte for IPv4 and 12-byte for IPv6 addresses, per FRR next-hop created. Eventually, the rpd process will run out of memory and crash when it cannot honor some request for a memory allocation. PR1144385

  • With NSR configured, when the BFD sessions are replicated on the backup Routing Engine, the master will not send the source address, instead the backup Routing Engine will query the kernel to get the source address. In rare cases, the query might fail, resulting in the source address as all zeros. Later, if a GRES switchover happens, the new master will have this all-zeros source address. When a BFD packet with this source address is send out, the other end will drop the BFD session due to no matching session (source address). PR1145612

  • With SRLG (Shared Risk Link Group) enabled under corner conditions, after executing command of "clear isis database", the rpd might crash due to the ISIS database tree gets corrupted. PR1152940

  • This core is seen because of incorrect accounting of refcount associated with the memory block which composes the nhid (IRB nh). When the refcount prematurely reaches 0, we released the memory block while it was still referenced from a route. We may see this issue when mcsnoopd becomes a slow consumer of rtsock events generated by rpd (next-hop events in the current case) and messages get delivered in a out-of-order sequence, causing the refcount to be incorrectly decremented. In the testbed where the issue was reported, tracing was enabled for mcsnoopd (for logging all events), causing it to become a slow consumer. However, it may become slow also for other reasons such as processing very high rate of IGMP snooping reports/leaves which could potentially trigger this issue. PR1153932

  • OpenSSH client software supports an undocumented feature called roaming: if the connection to an SSH server breaks unexpectedly, and if the server supports roaming as well, the client is able to reconnect to the server and resume the suspended SSH session. This functionality contains two vulnerabilities that can be exploited by a malicious SSH server (or a trusted but compromised server): an information leak (memory disclosure), and a buffer overflow (heap-based). Refer to https://kb.juniper.net/JSA10734 for more information. PR1154016

  • BGP Monitoring Protocol (BMP) feature is introduced in 13.3R1. When BMP is configured in passive mode and BMP session is closed ungracefully (e.g. No TCP FIN sent), in rare cases, the TCP session might not be cleaned up properly and rpd process crash might be observed during the re-establishment of the previous session. PR1154017

  • In dual REs scenario with NSR and PIM configuration, when backup RE handling mirror updates about PIM received from the master RE, it will delete the PIM session info from its database. But due to a software defect, a leak of 2 memory blocks (8 or 16 byte leaks) will occur for every PIM leave. If the memory is exhausted, the rpd process might crash on backup RE. There is no impact seen on the master RE when the rpd cores on backup. PR1155778

  • In BGP scenario with large scale routing-instances and BGP peers configured, due to a software defect (a long thread issue), BGP slow convergence might be seen. For example, BGP might go down 8-9 seconds after BFD brings down the EBGP session. The rpd slip usually does not hurt anything functionally, but if the slip gets big enough, it could eventually cause tasks to not be done in time. For example, BGP keepalives with lower than 90 seconds hold-time might be impacted. There is no known workaround for this issue, but configuring the knob "protocol bgp precision-timers" can take care of the weak spot like sending BGP keepalives. PR1157655

  • When rib-group copy is done for a route change, the rib-group copy of the secondary route into the destination tables of the copy may not honor maximum-prefixes in some scenarios, such as upon damping changes. The traffic forwarding might be affected. PR1157842

  • In BGP scenario with independent domain enabled in a VRF, when configuring a BGP session in a VRF routing instance with a wrong local-as number, some routes might be declared as hidden because of AS path loop. If later configuring the correct AS number as local-as and committing the configuration, those routes might still remain in hidden state. The hidden routes can be released after performing the commands "commit full" or "clear bgp table <ANY_VRF>.net.0". PR1165301

  • In L3VPN scenario, feature multipath is configured under [set protocols bgp group] with L3VPN chained CNH under routing-options, the feature multipath does not work for L3VPN routes. PR1169289

  • PIM bootstrap export policy is not working as expected when there are no pim neighbors up on the router PR1173607

  • On dual-RE platforms, with NSR enabled for PIM, when change on reverse-path forwarding (RPF) unicast route occurs, routing protocol process (rpd) crash may occur on backup RE. PR1174845

  • In L3VPN scenario, VPN routes with different next-hops were advertised with same label, leading to PE-CE link protection failure and longer than expected traffic loss (as reported 2.6 sec). PR1182777

  • Any configuration change can cause deletion of a firewall filter created for a routing instance if the flowspec routes in that instance are imported using rib-group, and there is no "inet-vpn flow" address family configured and the routing instance does not have any BGP group configured with "inet flow" address family. PR1185954

Services Applications

  • In CGNAT scenario, when we establish simultaneous TCP connects, we need to install timers for each TCP connection/flow. Due to this bug, we ended up creating two timers for the forward and reverse flow separately. Ideally there needs to be only one timer for both the forward and reverse flow. Whenever the session used to get deleted due to timer expiry, the PIC used to crash whenever the code tried to delete the same flow again. PR1116800

  • When making a configuration change to a EXP type rewrite-rule applied to a SONET interface in an MX FPC Type 2 or MX FPC Type 3, if MS-DPC is also installed on the device, a MS-PIC core dump may be generated. PR1137941

  • When NAT for SIP is enabled, in a rare situation where the child SIP flow entries are still present in the parent conversation while they have already been deleted, the service PIC might crash if the SIP parent flow tries to access them. PR1140496

  • When using NAT on the MX, the FTP ALG fails to translate the PORT command when the FTP client using Active Mode requests AUTH(SSL-TLS) and the FTP server does not use AUTH PR1194510

  • When MS-PIC is running on T640/T1600/T4000, the number of maximum service sets if wrongly limited to 4000, instead of 12000. This might impact in scaled service (ipsec, etc) environment. PR1195088

User Interface and Configuration

  • From Junos OS 12.1X44-D50 12.1X46-D35 12.1X47-D25 12.3X48-D15 14.1R3-S1 14.1R4 14.2R1 with large scale configuration configured, due to a software bug --- drastic increase in the number of calls to "action acceptable" function, a performance issue might occur. For example, even thought there is no configuration set for "protocols mpls lsp-external-controller ...", the action acceptable function is called repeatedly when performing a configuration commit. As a result, the configuration load time takes more than before. 15.1 might take more than 10 minutes. The same configuration was able to load in 14.1 in 5 minutes 35 seconds. The fix/optimization has now been provided to decrease processing time during configuration load and rollback. PR1065659

  • Description: Issue --------------- pinned-page found for bucket warning is seen after application (in this case dfwc) is done with the page pool and trying to com eout after ppool_close. Root-cause --------------- This warning is given when the application is done with the page pool and tries to find out if there were any pinned pages in memory. However this warning is basically internal to Junos development team and has been masked in the later releases starting from 15.1 onwards with below: PR https://gnats.juniper.net/web/default/1030715 Fix --------------- We have the taken the relevant changes from PR 1030715 to prevent these flurry of warnings, and to enable these warnings only for Junos development team upon enabling leak check internally. PR1179264

VPNs

  • Upon clearing p2mp lsp in dual-home topology, system is adding the same outgoing interface to the (S,G)OIL multiple times and thus duplicate/multiply the amount outgoing traffic. PR1147947

  • After a GRES with NSR enabled, in NG-MVPN scenario, on the new backup RE RPD is consuming more than 90% CPU. This issue happens rarely and it is not reproducible. PR1189623

  • With MVPN and NSR enabled, high CPU on backup RE might be seen. MVPN on backup RE is re-queuing c-mcast events for flows as it is unable to find phantom routes from master routing-engine . However as routes is not reaching from master routing-engine so backup RE keeps trying causing high CPU triggered by RPD processing. PR1200867

Resolved Issues: 14.1R7

Class of Service (CoS)

  • On MX104 platform, when applying the "rate-limit" and the "buffer-size" on the logical tunnel (lt-) interface on the missing MIC (not inserted on MPC), commit failure with error message would occur. As a workaround, this issue could be avoided by applying the "rate-limit and "buffer-size" on inserted MIC, then commit. PR1142182

Forwarding and Sampling

  • The command "clear firewall all" will now clear the policer stats displayed by "show policer __auto_policer_template_1__", ... "show policer __auto_policer_template_8__". PR1072305

  • On MX Series platform with MX-FPC/DPC, M7/10i with Enhance-FEB, M120, M320 with E3-FPC, when there are large sized IPv6 firewall filters(for example, use prefix lists with 64k prefixes each) enabled, commit/commit check would fail and dfwd process would crash after configuration commit/commit check. There is no operational impact. PR1120633

  • On all Junos OS platforms, when both the filter and the policer are configured for an interface, in rare cases, the policer template may not be received by PFE (from the RE) when it is referenced by the filter term (normally the policer template gets received before the filter term referencing it which is ensured by mechanism in RE kernel). In this situation, the FPC would crash due to this rare timing issue. This issue might be avoided by the recommended steps below: 1. Deactivate the physical interface (IFD) and commit 2. Enable any filter and policer that attached to the interface (e.g. IFL) and commit 3. Activate interface back PR1128518

General Routing

  • In a Layer 3 wholesale configuration, DHCPv6 advertise messages might be sent out with source MAC all zeroes if the subscriber is terminated on the demux interface in a non-default routing instance. For subscribers on default instance there is no such issue observed. PR972603

  • On MX platform, MPC may crash when bringing up the 100-Gigabit Ethernet MIC with CFP2 (model number: MIC6-100G-CFP2) if initialization failure occurs (e.g. when bringing up the MIC6 which has hardware issue). PR1037661

  • On all routing platforms M/MX/T/PTX with BGP configured to carry flow-specification route, in case of deleting a filter term and policer, then add the same term and policer back (it usually happens in race condition when adding/deleting/adding the flow routes), since confirmation from dfwd for the deleting policer might not be received before attempting to add the same policer, the rpd would skip sending an add operation for it to dfwd. As a result, when the filter term is sent to dfwd and tell it to attach to the policer, dfwd had already deleted the policer, and since rpd skipped re-adding it, dfwd will reject the attach filter with policer not found error and rpd will crash correspondingly. PR1052887

  • As a precautionary measure, a periodic sanity check is added to FPC situated on M7i/M10i with enhanced CFEB, M320 with E3-FPC, M120 and MX Series with DPC. It checks FPC error conditions and performs the appropriate actions in case of an error. PR1056161

  • When a labeled BGP route resolves over a route with MPLS label (e.g. LDP/RSVP routes), after clearing the LDP/RSVP routes, in the short window before the LDP/RSVP routes restore, if the BGP routes resolves over a direct route (e.g. a one-hop LSP), the rpd process might crash. PR1063796

  • Upon BFD flapping on aggregate interfaces, the Lookup chip (XL) might send illegal packets to the center chip (XMCHIP) and compromise packet forwarding and an FPC restart is needed to recover from this condition. If Fabric path side is affected, the fabric healing process will initiate this process automatically to recover from such conditions. MPC6E/MPC5E/NG-MPC are exposed to this problem. Corrupted parcels from Lookup chip LU/XL to Center Chip (XM) can also compromise packet forwarding and report DRD parcel timeout errors. An additional parcel verification check is added to prevent sending corrupted parcels to the center chip (XM) PR1067234

  • The license-check process may consume more CPU utilization. This is due to a few features trying to register with the license-check daemon which license-check would not be able to handle properly and result in high CPU on Routing Engine (RE). Optimization is done through this fix to handle the situation gracefully so that high CPU will not occur. PR1077976

  • Scheduler: Protect: Parity error for tick table single messages might appear on MPC3E/MPC4E/MPC5E/MPC6E/T4000-FPC5. PR1083959

  • Wrong diagnostic optics info might be seen for GE-LX10 SFP and SFP+ for SumitomoElectric. The issue only for a specific SFP type - "Xcvr vendor part number : SCP6F44-J3-ANE”, it can be seen with "show chassis pic fpc-slot X pic-slot Y". PR1091063

  • High latency might be observed when continuous IPv6 pings are sent to VMX platform. PR1096403

  • In occasionally , AFEB PCI reads from Cortona MIC with ATM OAM traffic might return garbage values even though the actual content in the MIC has the correct value , this corrupted values would lead to AFEB crash , and also PCI error logs such as : afeb0 PCI ERROR: 0:0:0:0 Timestamp 91614 msec. afeb0 PCI ERROR: 0:0:0:0 (0x0006) Status : 0x00004010 afeb0 PCI ERROR: 0:0:0:0 (0x001e) Secondary bus status : 0x00004000 afeb0 PCI ERROR: 0:0:0:0 (0x005e) Link status : 0x00000011 afeb0 PCI ERROR: 0:0:0:0 (0x0130) Root error status : 0x00000054 afeb0 PCI ERROR: 0:0:0:0 (0x0134) Error source ID : 0x02580258 afeb0 PCI ERROR: 0:2:11:0 Timestamp 91614 msec. afeb0 PCI ERROR: 0:2:11:0 (0x0006) Status : 0x00004010 afeb0 PCI ERROR: 0:2:11:0 (0x004a) Device status : 0x00000004 afeb0 PCI ERROR: 0:2:11:0 (0x0052) Link status : 0x00004001 afeb0 PCI ERROR: 0:2:11:0 (0x0104) Uncorrectable error status : 0x00000020 afeb0 PCI ERROR: 0:2:11:0 (0x0118) Advanced error cap & ctl : 0x000001e5 afeb0 PCI ERROR: 0:2:11:0 (0x011c) Header log 0 : 0x00000000 afeb0 PCI ERROR: 0:2:11:0 (0x0120) Header log 1 : 0x00000000 afeb0 PCI ERROR: 0:2:11:0 (0x0124) Header log 2 : 0x00000000 afeb0 PCI ERROR: 0:2:11:0 (0x0128) Header log 3 : 0x00000000 PR1097424

  • When the clock sync process (clksyncd) is stopped and resumed during link flaps, the clksyncd process might get into an inconsistent state with various symptoms, the clock source might be ineligible due to "Interface unit missing" or "Unsupported interface" with no Ethernet Synchronization Message Channel (ESMC) transmit interfaces. PR1098902

  • Fragmenting a special host outbound IP packet with invalid IP header length (IP header length is greater than actual memory buffer packet header length), can trigger NULL mbuf accessing and dereferencing, which may lead to a kernel panic. PR1102044

  • With Nonstop active routing (NSR) enabled, deleting routing-instance/logical system configuration might cause a soft assert of rpd. If NSR is not enabled, after deleting routing-instance/logical system configuration, executing "restart routing" might trigger this issue too. The core files could be seen by executing CLI command "show system core-dumps". This timing issue has no function impact. PR1102767

  • When using "write coredump" to invoke a live coredump on an FPC in T-series, the contents of R/SR ASIC memory (Jtree SRAM) will get dumped. In the situation that there is a parity error present in the SRAM, then the coredump will abort and the FPC will crash. As a workaround, configuring "set chassis pfe-debug flag disable-asic-sram-dump" before "write coredump" will help to avoid the issue. PR1105721

  • On MX-VC with heartbeat connection, if it is in a scaled subscribers environment, when power down both VCM REs, there might be a delay (minutes) for backup chassis to be master and during which time, traffic blackhole might be seen. PR1115026

  • On a busy MX Series Virtual Chassis platform, for example, with 100k subscribers and 16k subscribers concurrent login/logout, the ksyncd process might crash on Virtual Chassis backup Routing Engines after a local or global graceful Routing Engine switchover (GRES). This issue has no service impact. PR1115922

  • For MPC6E with CFP2, there was a race condition between the Interrupt service routine and the periodic, as a result interface up/down will not happen for laser off/on. PR1115989

  • On TX/TXP platform, when an LCC hit overtemp situation, it might go offline abruptly without notify SFC and other LCCs, which might cause traffic loss or performance degradation. Now with the fix, the overtemp situation on LCC is handled gracefully. PR1116942

  • On MX Series routers containing multiple PFEs (Packet Forwarding Engines) such as MX240/MX480/MX960/MX2010/MX2020, with either MPC3E/MPC4E/MPC5E/MPC6E cards,if the routers have GRE decap, then certain packet sizes coming via these aforementioned line cards, at very high rate can cause these line cards to exhibit a lockup, and one or more of their PFEs corrupt traffic towards the router fabric. PR1117665

  • The commit latency will increase along with the increasing lines under [edit system services static-subscribers group <group-name> interface]. Use ranges to create static demux interfaces is a recommend option. e.g. [edit system services static-subscribers group PROFILE-STATIC_INTERFACE] + interface demux0.10001001 upto demux0.10003000; PR1121876

  • For scaled configuration, it may take too much time for commit and session gets hung because there is an unnecessary check to see if family Ethernet-switching co-exists with family bridge for all interfaces having bridge configuration. PR1122863

  • This is a cosmetic issue that vMX firewall logs may show wrong packet length for dropped packets. PR1124855

  • With BGP configured on CE-faced interfaces (in VRFs), doing 'show route' frequently may cause rpd to slowly leak memory. The leak rate will be one memory block of the size necessary to hold the instance name of the routing instance for a BGP neighbor. If the rpd process memory exhausted, the rpd process might crash, and the routing protocols are impacted and traffic disruption will be seen due to loss of routing information. You can check rpd memory usage with "show task memory brief" command. PR1124923

  • In EVPN scenario, the EVPN route table between the master RE and backup RE would be different (unused garbage routes will appear) once RE switchover (e.g. by rebooting the "old" master RE or performing graceful routing engines switchover) is performed, which may cause kernel crash on the new master RE in some cases. PR1126195

  • When Junos OS devices use Link Layer Discovery (LLDP) Protocol, the command 'show lldp neighbors' displays the contents of PortID Type, Length, and Value (TLV) received from the peer in the field 'Port Info', and it could be the neighbor's port identifier or port description. Junos CLI knob can select which 'interface-name' or 'SNMP ifIndex' to generate for the PortID TLV, so we do not have any problem as long as two Junos OS devices are connected for LLDP, but we might have an interoperability issue if other vender device which can map the configured 'port description' in the PortID TLV is used. In such case, the Junos OS displays the neighbor's PortDescription TLV in the 'Port info' field, and if the peer sets 'port description' whose TLV length is longer than 33 byte(included), Junos OS is not able to accept the LLDP packets then discards packets as errors. The PortID TLV is given as : "the port id tlv length = port description field length + port id subtype(1B)" PR1126680

  • In multi-homing Ethernet VPN (EVPN), if there are two loopback addresses and the router-id and the primary loopback addresses are different on the designated forwarder (DF) PE, when the link between CE and DF PE down, the Type 4 route of old DF are not deleted properly from the backup PE and causing the new DF election failure. The traffic forwarding will be affected. As a workaround, we should configure single primary loopback address and remove "router-id" knob on both multi-homing PEs. PR1126875

  • On M320/T320/T640 with FPC 1/2/3 and their enhanced version (-E2/-E), in multicast scenario and AE interface is within multicast NH (such as, AE interface is the downstream interface for a multicast flow), egress multicast statistics displays incorrectly after flapping of AE member links. PR1126956

  • In current Juniper implementation, the IPv6 multicast Router Advertisement timer is not uniformly distributed value between MinRtrAdvInterval and MaxRtrAdvInterval as described in RFC 4861. PR1130329

  • On MX with MS-MIC (or possibly, MS-MPC is affected as well), changing configuration of sampling input parameters, such as "rate" under forwarding-options is not reflected without restarting the line card. PR1131227

  • On Trio based line card, multiple modifications of firewall filter might cause lookup chip error and traffic blackhole, following jnh_free error messages could help to identify this issue: messages: fpc1 jnh_free(10212): ERROR [FW/3]:1 Paddr 0x006566a9, addr 0x2566a9, part_type 0call_stack 0x40497574 0x418ffa84 0x41900028 0x418ecf94 0x41861690. PR1131828

  • When customers do changes under "protocol router-advertisement interface X" (such as changing timers etc), they expect that commit would trigger an new router-advertisement being sent out to notify hosts about configuration changes. However it does not seem to be a case unfortunately. It makes the router information to expire on hosts and causes obvious loss of connectivity for the hosts. PR1132345

  • On MX platform with non-Q MPC (for example, MPC2-3D) or Q-MPC with enhanced-queueing off, when traffic has to egress on any one of the dynamic PPPoE (pp0), IP-DEMUX (demux0) and VLAN-DEMUX (demux0) IFLs, the queue mapping might get wrong. The traffic forwarding might be affected. PR1135862

  • MXVC-Same subnet VC-heartbeat polling failed to recover PR1136119

  • Upon either FPC boot with 100G CFP2 MIC installed or upon 100G CFP2 MIC installation post FPC boot, if the MIC is unable to initialize correctly, the MPC6E can crash and then restart. This has only been seen when the MIC has suffered a hardware failure. PR1148325

High Availability (HA) and Resiliency

  • On MX Series Virtual Chassis (MX-VC) with scaled configuration, for example, 110000 DHCP and 11600 PPP subscribers, the unified in-service software upgrade (ISSU) might fail due to the management daemon (MGD) timer expiring before Field-replaceable units (FRUs) update finish. PR1121826

  • With NSR enabled on multiple RE system, when dynamic GRE tunnel is configured, performing RE switchover might causing rpd crash repeatedly on backup RE. PR1130203

Infrastructure

  • When "show version detail" cli command has been executed, it will call a separate gstatd process with parameter "-vvX". Because the gstatd could not recognize these parameters, it will run once without any parameter then exit. PR1078702

  • The Remote NFS Server process (nfsd) is not terminated on the new backup Routing Engine (RE) after RE switchover. As a result, it spawns a new one upon RE switchover until running out of memory. PR1129631

Interfaces and Chassis

  • From Junos OS release 13.3R1, any host outbound traffic go across AE interface, will lead to ifstat memory leak, finally result in kernel crash. Please use the command "show system virtual-memory | match "mem|ifstat "" to monitor the memory utilization. PR975781

  • In the bridge domain configuration with IRB interface environment, the IRB interface INET/ISO MTU is set to 1500. When the MTU on IRB interface is deleted, the MTU wouldn't be changed. PR990018

  • On DPC only chassis, after software upgrade or not graceful Routing Engine switchover, Ethernet OAM related LAG bundles might not come up due to the Link Fault Management (LFM) packets arrive on AE interface instead of physical link interface. PR1054922

  • MS-DPC might crash when allocating chain-composite nexthop in enhanced LAG scenario. PR1058699

  • During subscriber login/logout the below error log might occur on the device configured with GRES/NSR. /kernel: if_process_obj_index: Zero length TLV! /kernel: if_pfe: Zero length TLV (pp0.1073751222) PR1058958

  • With "enhanced-ip" mode and AE interface configured, if SCU/DCU accounting is enabled, the MS-DPC might drop all traffic as regular discard. PR1103669

  • The 'optics' option will now display data for VCP ports: show interfaces diagnostics optics vcp-0/0/0 PR1106105

  • On MX240 or MX480 platform with at least two DC modules (PN: 740-027736) equipped, when shutting down one of the PEMs and then turn it on again, even the PEM is functioning, the "PEM Fan Fail" alarm might be observed on the device due to software logic bug. There is no way to clear the ALARM_REASON_PS_FAN_FAIL for I2C_ID_ENH_CALYPSO_DC_PEM once it has been raised. PR1106998

  • On all Junos platform, if the "HDD /var" slice (for example, "/dev/ad1s1f" depending on the type of RE) is not mounted (for example, label missing, file system corrupted beyond repair, HDD/SDD is removed from the boot list, etc), the system may build emergency "/var/", however, no alarm or trap is generated due to the incorrect operation of the ata-controller. Although the boot messages may present the logs, it may not be sufficient enough to identify the issue before encountering other problems (for example, Junos OS upgrade failure and the Routing Engine may hang in a recovery shell). In addition, as a method to check where Routing Engine is running from, a manual check could be done as below, user@re0> show system storage | match " /var$" /dev/ad2s1f 34G 18G 13G 57% /var <<<Indicate that "/var" is mounted from the HDD/SSD

    user@re0>show system storage | match " /var$" <<<<No output here, it means that the RE is running from "emergency /var" PR1112580

  • Junos OS now checks ifl information under the ae interface and prints only if it is part of it. PR1114110

  • On MX Series platform, when using Ethernet OAM Connectivity Fault Management (CFM), the CFM process (CFMD) may crash in either of the following scenarios, - Scenario 1 When CFMD is restarted or GRES. There is no specific defined configuration which could cause this crash, but normally this would be seen with VPLS or Bridge domain with multiple Mesh-groups. The crash happens rarely in this scenario. - Scenario 2 When configuring 2 interfaces in the same bridge-domain (BD) or routing-instance, and both interfaces have maintenance association end point (MEP) configuration along with action-profile enabled. Also there is no maintenance association intermediate point (MIP) configuration on that BD or routing-instance. The crash might be seen with the above configurations and when one of the interfaces is flapped or deleted and then re-created. In addition, in this scenario, this issue may not happen always as this depends on the ordering of kernel event. PR1120387

  • On JUNOS platform, an aggregate-ethernet bundle having more-than one member link can show incorrect speed which wouldn't match to the total aggregate bandwidth of all member links. The issue would be seen when LFM is enabled on the aggregate-ethernet bundle. The issue would be triggered when one of the member link flaps. Although after the flap, the current master Routing Engine would show correct aggregate speed, the backup Routing Engine would report incorrect value. In this state, when Routing Engine mastership is switched, the new master Routing Engine (which was backup) will show incorrect value. One of the side-effect of this issue is that RSVP also reflects incorrect Bandwidth availability for the affected aggregate-ethernet bundle, thus can cause under-utilization of the link with LSP having bandwidth constraints. PR1121631

  • Since a bug which was introduced in 15.1R1, loopback sub-interfaces always have a Flag down in the output of CLI command "show interfaces". PR1123618

Layer 2 Features

  • When AE is core facing ifl in ldp-mesh vpls instance with local-switching in it, the traffic is looped back. PR1138842

  • In a VPLS scenario, when "$junos-underlying-interface-unit" is configured in "dynamic-profiles" hierarchy, which is then implemented in a routing-instance. The upgrade/commit will fail with the following error message, Parse of the dynamic profile (dynamic-profiles) for the interface: $junos-interface-ifd-name and unit: $junos-underlying-interface-unit failed! PR1147990

Layer 2 Ethernet Services

  • On MX series platform with none-stop-routing (NSR) enabled and some L2 protocols configured, performing RE switchover might cause layer 2 control protocol daemon (l2cpd) to crash and FPC to be rebooted. PR1076113

  • On MX platform with Dynamic Host Configuration Protocol (DHCP) maintain subscriber feature enabled, after rebooting the FPC hosts the Demux underlying interfaces, the next-hop for some DHCP subscribers might be marked as dead in the forwarding table. When this issue occurs, we can execute CLI command "clear dhcp server binding <address>" to restore. PR1118421

  • For PVSTP/VSTP protocols, when MX/EX92xx router inter-operate with Cisco device, due to the incompatible BPDU format (there are additional 8 Bytes after the required PVID TLV in the BPDU for Cisco device), the MX might drop these BPDUs. PR1120688

  • In the DHCPv4 or DHCPv6 relay environment with large scaled environment (in this case, 50-60K subscribers), and the system is under stress (many simultaneous operations). The subscribers might get stuck in RELEASE state with large negative lease time. PR1125189

  • In some rare scenarios, the MVRP PDU might unable to be transmitted, which could cause memory leak in layer 2 control plane daemon (l2cpd), and finally results in the l2cpd process crash. PR1127146

  • Input/Output pps/bps statistics might not be zero after a member link of AE interface with distributed ppmd was down in M320/T-Series(GIMLET/STOLI based FPC) PR1132562

Multiprotocol Label Switching (MPLS)

  • With egress protection configured for Layer 3 VPN services to protect the services from egress PE node failure in a scenario where the CE site is multihomed with more than one PE router, when the egress-protection is un-configured, the egress-protection route cleanup is not handled properly and still point to the indirect composite nexthop in kernel, but the composite nexthop can be deleted in rpd even the egress protection route is pointing to the composite nexthop. This is resulting in composite nexthop "File exists" error when the egress protection is re-enabled and reuse the composite nexthop (new CNH addition fails as old CNH is still referenced in kernel). PR954154

  • MPLS auto-bandwidth does not reset MAX Avg Bandwidth when overflow or underflow threshold limit is configured. It may lead to wrong bandwidth reservations occasionally. PR954663

  • In next-generation MVPN extranet scenario, if there is a mix of VT interface and LSI (vrf-table-lable is used) interface on next-generation MVPN egress node, after changing some vrf policies, the routing protocol process (rpd) might crash and reset. PR1045523

  • In MPLS scenarios, removing the "family mpls" configuration from an outgoing interface may cause inet and/or inet6 nexthops associated with that interface to unexpectedly transit to dead state. Even adding back "family mpls" cannot restore it. PR1067915

  • When an LSP is link-protected and has no-local-reversion configured, if the primary link (link1) is down and LSP on bypass (link2), then another link (link3) is brought up, before the LSP switch to link3, if link1 is enabled and link3 is disabled, the LSP will stuck in bypass LSP forever. This is a timing issue. PR1091774

  • From Junos release 13.2R1 and above, in MPLS L3VPN scenario, when "l3vpn-composite-nexthop" knob is enabled on a PE router and an interface style service set is attached to the ingress interface, the L3VPN packets with the MPLS labels will be sent to the service card and dropped. As a workaround, we should disable "l3vpn-composite-nexthop". PR1109948

  • If "optimize-timer" is configured under P2MP branch LSP, this branch LSP will not be re-established if link flap on egress node. If "optimize-timer" is configured at protocols/mpls level, issue could be avoided. PR1113634

  • For an MPLS L3VPN using LDP-signaled LSPs, in a rare racing condition (e.g. large-scale environment or Routing Engine CPU utilization is high), the rpd process might crash after an LDP neighbor down. PR1115004

  • When multipoint LDP (M-LDP) in-band signaling is enabled to carry multicast traffic across an existing IP/MPLS backbone and routing process is enabled to use 64bit mode, the rpd might crash due to accessing an uninitialized local variables. PR1118459

  • When an PLR is a non-Juniper router, Juniper ingress node might stay on the bypass tunnel and ignore the CSPF result. PR1138252

  • There is no entropy label for LDP route in scenario of LDP tunneling across a single hop RSVP LSP with label 0 (explicit-null) used. As workaround, either remove LDP tunneling or RSVP explicit-null will resolve the issue. PR1142357

  • This issue is related to inter-op between multi vendor scenario. This fix will add sub-object RRO which will help change of label during FRR active scenario. PR1145627

Network Management and Monitoring

  • The SNMPv3 message header has a 4 byte msgID filed, which should be in (0....2147483647), when the snmpd process has been running for a long time, the msgID might cross the RFC defined range and causing Net-SNMP errors, "Received bad msgID". PR1123832

Platform and Infrastructure

  • On Trio based line card, when GRE keepalive packets are received on a PFE that is different from the tunnel interface hosted, the keepalive message will apply the firewall filter configured on default instance loopback interface. PR934654

  • Bad udp checksum for incoming DHCPv6 packets as shown in monitor traffic interface output. The UDP packet processing is normal, this is a monitor traffic issue as system decodes checksum=0000. PR948058

  • Under certain conditions the PFE flow export thread and flow update thread might be out of sync resulting in a situation where the update thread might attempt to update a flow record that is being aged-out/deleted by the export thread. As a consequence, PPE traps might be generated during flow processing; the PPE trap signature is very dependent on the operation performed on that particular record: fpc1 PPE Sync XTXN Err Trap: Count 3, PC 637f, 0x637f: flow_export_read_src_address_ipv6 LUCHIP(2) PPE_4 Errors sync xtxn error Under rare conditions, this can ultimately lead to record corruption. Trying to reuse or update such a record would trigger the following error: [LOG: Err] LUCHIP(2) HASH INT Status FPM Error: [LOG: Err] LUCHIP(2) HASH FPM ERROR: Alloc OMI Ram IF Error, TID=1, FP_ID=0x2. - There is no impact to forwarding. - There may or may not be impact on Jflow. - Its a generic problem for any inline-jflow application including IPv4 and IPv6. With 13.2 release, new fields (min, max TTLs/QinQ values) are added to jflow record. These fields need to be updated (if value changes) per packet in the flow. So the probability of hitting the race condition between export thread (deleting the record) and jflow data path code (updating the same record) and is higher in 13.2 release onwards. PR968807

  • In the dual REs scenario with NSR configuration, the knob "groups re0 interfaces fxp0 unit 0" is configured. If disable interface fxp0, backup RE is unable to proceed with commit processing due to SIGHUP not received, the rpd process on backup RE might crash. PR974430

  • When netconf or Junos OS scripts are used to manage the device, the management process gets stuck in a loop, causing high CPU usage. PR991616

  • On MX Series Virtual Chassis (MX-VC) platform, mirroring of OAM packets may not work as expected if the OAM packet traversing through multiple Packet Forwarding Engines (for example, the mirrored port and VCP port are on separate PFEs). PR1012542

  • when one of the "deny-commands" is incorrectly defined on the profile of TACACS+ server, all "deny-commands" regexes will be ignored, which leads to an over-permissive profile without any warning. PR1078238

  • The MIB counter or "show pfe statistics traffic" shows junk PPS and invalid total traffic output counter. PR1084515

  • When BFD or VRRP is running on a multi LU (lookup chip) PFE (such as MPC3 or MPC4), some incoming BFD or VRRP packets might be incorrectly evaluated by a firewall filter configured on a loopback interface of a different logical system or routing instance. Therefore, packets might be unexpectedly discarded leading to session/mastership flaps. PR1099608

  • On an MPC3E or MPC4E or on an EX9200-2C-8XS line card, when the flow-detection feature is enabled under the [edit system ddos-protection] hierarchy, if suspicious control flows are received, two issues might occur on the device: ? The suspicious control flow might not be detected on the MPC or line card. ? After suspicious control flows are detected, they might never time out, even if traffic flows no longer violate control parameters. PR1102997

  • Junos defines SNMP ifXTable (ifJnxInErrors/ifJnxInL3Incompletes) counter as 64-bit width, but it worked as 32-bit width counter. It works as 64-bit width counter after the fix. PR1105266

  • On Trio-based platform, in MX Series Virtual Chassis (MXVC) environment, if the subscriber logical interface (IFL) index 65793 is created (for example, when carrying 15K DHCPv4 subscribers to exceed IFL index creation 65793) and the IEEE 802.1p rewrite rule is configured (for example, using CoS rewrite rules for host outbound traffic), due to usage of incorrect IFL index, the Virtual Chassis Control Protocol Daemon (vccpd) packets (for example, Hello packets) transmission may get lost on all VC interfaces, which may lead to VC decouple (split brain state, where the cluster breaks into separate parts). As a workaround, either delete the rewrite rule (delete class-of-service host-outbound-traffic ieee-802.1 rewrite-rules), or find the IFL in jnh packet trace that is not completing the vccpd send to other chassis and at RE clear that subscriber interface may resolve the issue. PR1105929

  • CVE-2015-5477 A vulnerability in ISC BIND's handling of queries for TKEY records may allow remote attackers to terminate the daemon process on an assertion failure. See https://kb.juniper.net/JSA10718. PR1108761

  • On MX-VC, when traffic with TPID 0x88a8 or 0x9100 is sending over AE interface, the packets which across VCP links might be dropped on egress VCP PFE due to invalid fabric token. PR1112752

  • When inline static NAT translation is used, if two rules defined in two service sets are pointing to the same source-prefix or destination-prefix, changing the prefix of one of the rule and then rolling back the changes is not changing back all the pools correctly. PR1117197

  • On Trio-based line card, the firewall filter may have some issues when matching on Authentication Header (AH) protocol. This can affect VRRP (among others) when authentication is used, and an Routing Engine (RE) firewall filter is matching on protocol AH. As a workaround, we can change the filter to match on other criteria (e.g. source or destination address). PR1118824

  • When the AC Single Phase Power Distribution Module (PDM) is installed on an MX2010 or MX2020 router running Junos release 14.1 or 14.2, the system does not recognize the FRU and alarms are triggered as a result. PR1121068

  • After changing an outer vlan-tags, the ifl is getting programmed with incorrect stp state (discarding), so the traffic is getting dropped. PR1121564

  • With "fast-synchronize" configured, adding a new configuration-group that has configuration relevant to the rpd process and apply it and commit, then any configuration commits might cause the rpd process on the backup Routing Engine (RE) crash. We can reboot the backup RE to restore. PR1122057

  • On Trio-based platform, when fragmented packets go through the inline NAT (including source NAT, destination NAT, and twice NAT), the TCP/UDP checksum would not be correctly updated. In this situation, checksum error would occur on the remote end (inside and outside device). Non-fragmented packets would not be affected by the issue. If possible, this issue could be avoided by either of the following workarounds, * Enable "ignore-TCP/UDP-Checksum errors" at the inside or outside device which processes TCP/UDP data OR * Make sure there will not be any fragments subjected to inline NAT functionality by appropriate MTU adjustment or setting PR1128671

  • Parity error at ucode location which has instruction init_xtxn_fields_drop_or_clip will lead to a LU Wedge. LU is lookup ASIC inside the Trio. The LU wedge will cause the fabric self ping to fail which will lead to a FPC reset. This is a transient HW fault, which will be repaired after the FPC reset. There is no RMA needed unless the same location continues to fail multiple times. PR1129500

  • On Junos device with DHCP Relay config but without accounting config, and the accounting license does not exist, when the first DHCP control traffic is received, the following subscriber-accounting license grace period alarms might be triggered: alarmd[1650]: Alarm set: License color=YELLOW, class=CHASSIS, reason=License grace period for feature subscriber-accounting(30) is about to expire craftd[1592]: Minor alarm set, License grace period for feature subscriber-accounting(30) is about to expire PR1129552

  • For IPv6 packet with "no next header" in Hop-By-Hop header, if the Hop-By-Hop header length field value is large than 112, the router will drop such packet and log the following error: PPE PPE HW Fault Trap: Count 105, PC 60ce, 0x60ce: ipv6_input_finished_parsing LUCHIP(3) PPE_10 Errors lmem addr error PR1130735

  • NTP.org published a security advisory for thirteen vulnerabilities in NTP software on Oct 21st, 2015. These vulnerabilities may allow remote unauthenticated attackers to cause Denial(s) of Service(s), disruption of service(s) by modification of time stamps being issued by the NTP server from malicious NTP crafted packets, including maliciously crafted NTP authentication packets and disclosure of information. This can impact DNS services, as well as certificate chains, such as those used in SSL/https communications and allow attackers to maliciously inject invalid certificates as valid which clients would accept as valid. Refer to JSA10711 for more information. PR1132181

  • Too many duplicate ACK messages are generated from PFE for TCP control connection with RE. This could cause: 1. MX-VC DDoS protection violation for VC-control low queue and makds MXVC split. 2. Cause RE and FPC high CPU utilization. PR1133293

  • With scaled firewall filters attached to interfaces (e.g. 10k+ filters), running "show configuration" command can cause high CPU of the mgd process. As a workaround, we can use "show configuration |display set" command to view the config. PR1134117

  • PPE thread timeout trap may cause XM chip wedge, it will not affect MQ based FPC. PR1136973

  • On MX2020, when we remove whole power of a power zone, and then put the power back to the zone, FANTray LED stays Amber and FANTray LED on craft card stays OFF, and do not revert to green (FANTray LED) or ON (Craft LED) until we reboot the entire chassis system or hot swap that FAN tray. For Zone 0(PSM 0 to 8), FAN 1 shows the above described behavior. For Zone 1(PSM 9 to 17), FAN 3 shows the above described behavior. PR1138209

  • When the cli command "show pfe statistics exceptions | match reject" executed CPROD thread in the PFE may hogg the CPU and result in FPC crash PR1142823

Routing Protocols

  • BGP "accepted-prefix-limit" feature might not work as intended when it is configured together with "damping". Root cause of this issue is that when BGP module count the maximum routes accepted from BGP neighbor, it doesn't count the accepted BGP routes which in damping status. So when these damping routes are reused, the total number of received BGP routes exceeds the configured value for "accepted-prefix-limit" . PR897124

  • Since Junos 13.3R2 and higher if delegated BFD sessions are flapping continuously, packet buffer memory maybe be leaked. The automatic memory leak detection process will report this within the syslog once certain threshold is reached "fpc7 SHEAF: possible leak, ID 8 (packet(clones)) (10242/128/1024)". Please note BFD sessions operating in centralized mode are not exposed. PR1003991

  • From Junos release 14.1R1 or above, the rpd process might crash while executing CLI command "show isis backup spf results". PR1037114

  • EDITED MP 8/31 When a multicast group in protocol independent multicast (PIM) dense mode has a large number of multicast sources, the RPD process can crash after a routing engine switchover. PR1069805

  • On large scale BGP RIB, advertised-prefixes counter might show the wrong value due to a timing issue. PR1084125

  • When a BGP session supports multiple address families, the inactive route of some of the address families might not be flushed correctly, leading to wrong behaviors for some of the features which need to advertise inactive routes(e.g. advertise-inactive, advertise-external, optimal-route-reflection, etc). PR1097297

  • Due to software bug Junos cannot purge so called doppelganger LSP, if such LSP is received over newly formed adjacency shortly after receiving CSNP from the same neighbor. PR1100756

  • When two (or more) route target communities of MP-BGP route match to two (or more) route target communities in VRF import policy of a RI duplicate routing entries might be installed in the RI. In the output of 'show route table <RI-name>.inet.0 detail' two identical routing entries appear with one being marked as 'Inactive reason: Not Best in its group - No difference'. When such duplicate routing information is to be deleted, rpd process process will crash. PR1113319

  • When the Multicast Source Discovery Protocol (MSDP) is used, if the RP itself is the First-Hop Router (FHR) (i.e. source is local), the MSDP source active (SA) messages are not getting advertised by the RP to MSDP peers after reverse-path forwarding (RPF) change (e.g. the RPF interface is changed). PR1115494

  • When a logical unit of an interface is associated with a Bidirectional Forwarding Detection (BFD) session, if changing the unit number of the interface (for example, change the unit number for a running BFD session from ge-1/0/0.2071 to ge-1/0/0.285), the device may fail to change the name due to the missing check for logical interface (IFL) index change. PR1118002

  • On dual Routing Engine platform with Nonstop active routing (NSR) and authentication of the Bidirectional Forwarding Detection (BFD) session enabled, BFD process (bfdd) memory leak may occur on the master RE and the process may crash periodically once it hits the memory limit (RLIMIT_DATA). The problem does not depend on the scale, but the leak will speed up with more BFD sessions (for instance 50 sessions). As a workaround, if possible, disabling BFD authentication will stop the leak. PR1127367

  • In multicast environment, when the RP is FHR (first hop router) and it has MSDP peers, when the rpf interface on RP changed to MSDP facing interface, due to the multicast traffic is still on the old rpf interface, a multicast discard route will be installed and traffic loss will be seen. PR1130238

  • In multicast environment with Protocol Independent Multicast sparse mode (PIM SM) used, if a upstream router of last-hop router receives the (S,G) SPT join while the shortest-path tree (SPT) is not yet established (only because multicast source is not reachable, a reachable route for SPT which is just not established yet will not cause this issue), when the multicast route get deleted on the router (e.g. receives the (S,G) prune from downstream PIM router), the router would incorrectly stop forwarding the multicast traffic even if rendezvous-point tree (RPT) path exists. PR1130279

  • Pending RN from Development PR1135205

  • In rare condition, mt tunnel interface flap cause backup RE core. The exact root cause is not known. While processing updates on the backup RE (received from master RE), accessing free pointer cause the Core. PR1135701

  • On dual Routing Engine (RE) platform with Bidirectional Forwarding Detection (BFD) protocol enabled, after graceful Routing Engine switchover (GRES), the periodic packet management process (ppmd) might crash on backup RE due to a software defect. PR1138582

  • In the BGP labeled unicast environment, the secondary route is configured with both add-path and advertise-external. If the best route and secondary route are changed in a routing table at the same time, add-path might miss to readvertise the changed route. The old route with the old label is still the last route advertised to one router instead of updating the advertisement with the new route and new label. So the traffic forwarding might be affected. PR1147126

Routing Policy and Firewall Filters

  • When a malformed prefix is used to test policy (command "test policy <policy name> <prefix>"), and the malformed prefix has a dot symbol in the mask filed (e.g. x.x.x.x/.24), the rpd process might crash. PR1144161

  • From Junos OS release 13.2R1, an attempt to commit a configuration with a dangling conditional policy referring a non-existent/inactive routing-instance will be permitted. If we have a conditional policy referring an active routing-instance, deleting/deactivating this routing-instance and then committing will cause the rpd process crash. As a workaround, we should always make sure that conditional policies are referring active routing-instances. PR1144766

Services Applications

  • Junos 13.3 and above release, when configuring a /31 subnet address under a nat pool, the adaptive services daemon (SPD) will continuously crash. PR1103237

  • If l2tp is configured under access-group hierarchy, during commit or commit check operation, the pppd process might crash (the configuration could commit successfully). It might result in a minimal impact of system, and it will restore automatically. As a workaround, please configure the same under the access profile client hierarchy. PR1108024

  • SIP one way audio calls when using X-Lite SIP Softphone, in case that SIP media is switched to another media gateway though a SIP RE-Invite message PR1112307

  • In CGNAT environment, when a service PIC is in heavy load continuously, there might be a threads yielding loop in CPUs, which will cause the CPU utilization high, and might cause one the CPUs to be reset. PR1115277

Software Installation and Upgrade

  • In certain conditions, when /var is not mounted from a persistent filesystem, executing a Junos upgrade will have unexpected results. This is caused by an inexact check of whether it is running from an Emergency VAR. PR1112334

Subscriber Access Management

  • For scenarios that are not in a Layer 3 wholesale network environment, we can configure "duplication-vrf" to send duplicate accounting records to a different set of RADIUS servers that reside in either the same or a different routing context. After Routing Engine switchover, the duplicate accounting feature stops work for existing subscribers. PR1121524

  • In subscriber management environment, the authentication process (authd) crash may occur. This issue is not reproduced yet, possibly, it might be seen when generating a CLI Change of Authorization (CoA) request (e.g. via CLI command "request network-access aaa subscriber add service-profile filter-service session-id 10"), then logging out the subscriber (the one with service just activated), if the management CLI session is closed before subscriber entry is reused, the crash may occur. PR1127362

User Interface and Configuration

  • When committing a config with very long as-path, in this case the as-path is almost 12000 characters long, the commitd process might crash. The commitd process restart results in a minimal impact of system. As a workaround, please config as-path less than 4096 characters long. PR1119529

VPNs

  • In a multi-homed source topology in NG-MVPN (applicable to both inter-AS and intra-AS scenario), there are two problems: The first problem is Multicast (S, G) signaling doesn't follow RPF. When the routing table (mvpninstancename.inet0) has two routes, due to the policy configuration, the best route to the source is via the MPLS core, but Multicast (S, G) PIM join and NG-MVPN Type 7 both point to inactive route via local BGP peer. The second problem is when "clear pim join instance NG" is entered, the multicast forwarding entries are wiped out. PR1099720

  • In scenario involving pseudowire redundancy where CE facing interface in the backup neighbor (can be non-standby, standby, hot-standby type), if the virtual circuit (VC) is not present for the CE facing interface, the CE facing interface may go up after committing an unrelated VC interface configuration (e.g. changing description of another VC interface) even though the local pseudowire status is in down state. PR1101886

  • In Global Table Multicast (GTM) scenario (instance-type mpls-internet-multicast), when the GTM instance and master instance are used, if the name of the GTM instance is changed, the routing protocol process (rpd) may crash due to the usage of the incorrect routing table handle. PR1113461

  • In L2circuit environment, if one PE has pseudowire-status-tlv configured but remote hasn't, and at the same time, this PE doesn't support control-word but remote does, then it will not send changed local status code to remote PE, in a rare condition, after enable status-tlv support at remote end, the l2circuit might stuck in "RD" state on remote PE. PR1125438

Resolved Issues: 14.1R6

Class of Service (CoS)

  • After restarting chassisd or doing an in service software updgrade from 13.2R8.2 to 13.3R7.3 results in the following messages seen in syslog: cosd_remove_ae_ifl_from_snmp_db ae40.0 error 2 Messages appear to be harmless with no functionality impact. PR1093090

  • On MX104 platform, when we configure rate-limit for the logical tunnel (lt-) interface, the commit will fail. As a workaround, we can use firewall filter with policer to achieve the same function. PR1097078

  • When performing the Routing Engine switchover without GRES enabled, due to the fact that the Class-of-Service process (cosd) may fail to delete the traffic control profile state attached to logical interface (IFL) index, the traffic-control-profile may not get programmed after the ifl index is reused by another interface. PR1099618

Forwarding and Sampling

  • The issue is seen while moving an interface from one mesh group to another. PR1077432

  • In rare cases, SSH or telnet traffic might hit incorrect filter related to SCU (Source Class Usage) due to the defect in kernel filter match. This issue comes when the filter has match condition on source class ID. PR1089382

  • In rare cases, MX Series routers might crash while committing inline sampling related configuration for INET6 Family only. PR1091435

General Routing

  • Changing the static route configuration from next-hop to qualified-next-hop might result in static route getting missed from the routing table. Restarting routing process can bring back the routes but with the rpd core. PR827727

  • If with accounting/sampling enabled, an unnecessary update from the routing protocol process (rpd) to the route record database might be triggered by certain configuration change. This process causes jump in CPU utilization of all Packet Forwarding Engines. PR1002107

  • SNMP MIB walk of object "jnxSpSvcSet" gives hardcoded value as "EXT-PKG" for SvcType PR1017017

  • In IP security (IPsec) VPN environment, after performing the Routing Engine switchover, the traffic may fail to be forwarded due to the SAs may not be downloaded to the PIC, or due to some security associations (SAs) on the PIC may incorrectly hold references for old Security Policy Database (SPD) handles while SPD has deleted its entries in the Security Association Database (SAD). PR1047827

  • MPC with Channelized OC3/STM1 (Multi-Rate) Circuit Emulation MIC (MIC-3D-4COC3-1COC12-CE) might crash. This problem is very difficult to replicate and a preventive fix will be implemented to avoid the crash. PR1050007

  • When "satop-options" is configured on an E1 with Structure-Agnostic TDM over Packet (SAToP) encapsulation, after Automatic Protection Switching (APS) switchover, some SAToP E1s on the previously protect interface (now working) start showing drops. PR1066100

  • When setting the syslog to debug level (any any), you may note reoccurring messages of the form "ifa for this rt ia is not present, consider ifa as ready". These messages are logged for IPv6 enabled interfaces when receiving forwarded packets and cause no harm. Set a higher debug level to avoid seeing them. PR1067484

  • When VMX is deployed, initially there is no management port configuration, so configuration needs to be applied by serial console. The console for VMX is set to 9600 baud rate, with this rate, only a small number of configuration lines can be pasted at a time. PR1068152

  • For Network Address Translation (NAT), Traffic Detection Function (TDF), or IPsec service configured on MX Series platform with MS-MPC/MS-MIC, the received fragmented IPv4/IPv6 packets will be re-assembled and sent out. Under scaled environment, the mspmand process might crash while MS-MPC/MS-MIC is under process of assembling the fragmented packets. PR1075454

  • Traffic throughput test between MPC1/1E/2/2E card and MPC2E/3E NG card, the flowing from MPC1/1E/2/2E card to MPC2E/3E NG card is lesser then from MPC2E/3E NG card to MPC1/1E/2/2E card. PR1076009

  • When a router with AMS infrastructure has MAC flow control enabled, the continuous fragmented packets might crash the NPU and mspmand process (which manages the Multi-Services PIC). PR1076033

  • In subscriber management environment, the PPP daemon (jpppd) might crash repeatedly due to a memory double-free issue. PR1079511

  • On MX Series platform with MS-MPC/MS-MIC, in some mspmand process crash scenarios, after the mspmand coredump is finished or almost finished, PIC kernel also crashes and dumps vmcore. The mspmand cores in these scenario are readable but vmcores are not. PR1081265

  • The rpd process might crash on both master and backup Routing Engines when a routing instance is deleted from configuration if the routing instance is cleaned up before the interface delete is received from device control daemon (dcd). This is a rare timing issue. PR1083655

  • OTN based SNMP Traps such as jnxFruNotifOperStatus and jnxIfOtnNotificationOperStatus are raised by offline/online MIC although no OTN interface is provisioned PR1084602

  • On MX Series routers with MPCs/MICs, if a rlsq interface is receiving continuous fragmented traffic, doing rlsq switchovers couple of times might cause FPC to crash and reboot. PR1088300

  • Wrong ESH checksum computation with non-zero Ethernet Padding in MX Series router. PR1091396

  • In a fib-localization scenario, IPv4 addresses configured on service PICs (SP) will not appear on FIB-remote FPCs although all local (/32) addresses should, regardless of FIB localization role, install on all Packet Forwarding Engines. There is no workaround for this and it implies that traffic destined to this address will need to transit through FIB-local FPC. PR1092627

  • There are entries for PEM in jnxFruEntry in VMX. It is not necessary and is cosmetic. PR1094888

  • Just after the system reboots, rpd process is determining the Routing Engine mastership mode too early before chassisd is determining the mastership , which would cause overload feature not to work properly. PR1096073

  • For Junos OS Release 13.3R1 and later, the DPC card might experience a performance degradation when it's transferring bidirectional short packets (64B) in inline rate. PR1098357

  • On XL-based cards such as MPC5/MPC6, PPE thread timeout errors (resulting in PPE trap files) can be triggered when the FPC allocates illegal memory space for the forwarding state of router operations. - In certain cases, this can result in packet loss depending on how many packets use this forwarding state. PR1100357

  • When the null pointer of jbuf is accessed (jbuf, that is, a message buffer is allocated only when the packet is ready to process. The buffer is freed after the packet completes ALG handling is accessed), for example, when using the Microsoft Remote Procedure Call (MS RPC) (as observed, issue may also happen on Sun Microsystems RPC) Application-level gateway (ALG) with NAT (stateful firewall is used as a part of the service chain), if the traffic matching configured universal unique identifier (UUID) is arrived on the ALG, the mspmand (which manages the Multiservice PIC) crash occurs. PR1100821

  • After Junos OS Release 13.3R1, IPCMON infra is added to debug IPCs between PFEMAN and the Routing Engine. When convergence occurs, string processing of IPCMOM will take added time. Then the slow convergence will be seen. It is a performance issue, it is visible in scaled scenario (for example, more than 100K routes). As a workaround, please execute command "set pfe ipclog filter clear" to disable IPC logging on all FPCs. PR1100851

  • FFP is a generic process that should be called during commit process, and FFP calls the PDB initialization as part of its process. On the PDB-unsupported platforms (MX Series, M10i, M120, M320 are PDB-supported), when committing configuration, some error messages will be seen. PR1103035

  • If fpc offline knob is configured after the presence of Non-recoverable faults, then offline action will not be performed. PR1103185

  • Non-queuing MPC5E might crash continuously if rate-limit under transmit-rate for scheduler is applied. As a workaround, do not configure rate-limit and use firewall policer for forwarding-class instead. MPC5EQ is not exposed. PR1104495

  • An IPv4 filter configured to use the filter block with term that has both "from precedence" and another non 5-tuple (i.e. not port, protocol, address) will cause an XL based board to reboot. Example: set firewall family inet filter FILTER fast-filter-lookup set firewall family inet filter FILTER term TERM from precedence PRECEDENCE set firewall family inet filter FILTER term TERM from tcp-established PR1112047

  • In the scenario that the power get removed from the MS-MPC, but Routing Engine is still online (for example, on MX960 platform with high capacity power supplies which split into two separate power zones, when the power zone for the MS-MPC line card loses power by switch off the PEM that supports the MS-MPC situated slot), if the power goes back (for example, switch on the PEM), the MS-MPC might be seen as "Unresponsive" (checked via CLI command "show chassis fpc") and not coming up back online due to failure of reading memory. PR1112716

  • Under certain conditions, when the Junos OS Routing Engine tries to send an IP packet over a IPIP tunnel, the lookup might end up in an infinite loop between two IPIP tunnels. This is caused by a routing loop causing the tunnel destination for Tunnel#A to be learned through Tunnel#B and the other way around. PR1112724

  • Under certain conditions, when the Junos OS Routing Engine tries to send an IP packet over a GRE tunnel, the lookup might end up in an infinite loop between two GRE tunnels. This is caused by a routing loop causing the tunnel destination for Tunnel#A to be learned through Tunnel#B and the other way round. PR1113754

Infrastructure

  • On dual Routing Engine platform, if GRES is configured (triggered by "on-disk-failure"), when a disk I/O failure occurs on the master Routing Engine due to hardware issue (for example, SSD failure), the graceful Routing Engine switchover might not be triggered immediately after initial IO failure has been detected. As a result, Routing Engine might enter a state in which it responds to local pings and interfaces remain up, but no other processes are responding. PR1102978

Interfaces and Chassis

  • After changing the speed of fxp0 interface (the management Ethernet interface) to 1G (the maximum speed), the interface process (dcd) configures the interface but reads the speed even before the change takes effect. Although the hardware speed is updated to 1G, from dcd perspective, the speed is still not changed. Then if you change back to the original speed, the change is ignored by dcd. PR976825

  • On MX Series platform, when an aggregated Ethernet bundle participating as L2 interface within bridge-domain goes down, the following syslog messages could be observed. The messages would be associated with FPC0 even if there are no link(s) from this FPC0 participating in the affected aggregate-ethernet bundle. mib2d[2782]: SNMP_TRAP_LINK_DOWN: ifIndex 636, ifAdminStatus up(1), ifOperStatus down(2), ifName xe-3/3/2 mib2d[2782]: SNMP_TRAP_LINK_DOWN: ifIndex 637, ifAdminStatus up(1), ifOperStatus down(2), ifName xe-3/3/3 mib2d[2782]: SNMP_TRAP_LINK_DOWN: ifIndex 740, ifAdminStatus up(1), ifOperStatus down(2), ifName ae102 fpc0 LUCHIP(0) Congestion Detected, Active Zones f:f:f:f:f:f:f:f:f:f:f:f:f:f:f:f fpc0 LUCHIP(0) Congestion Detected, Active Zones 2:0:0:0:0:8:a:0:0:0:0:0:8:4:0:a alarmd[1600]: Alarm set: FPC color=RED, class=CHASSIS, reason=FPC 0 Major Errors craftd[1601]: Major alarm set, FPC 0 Major Errors fpc0 LUCHIP(0) Congestion Detected, Active Zones 2:0:0:0:0:8:a:0:0:0:0:0:8:4:0:a alarmd[1600]: Alarm cleared: FPC color=RED, class=CHASSIS, reason=FPC 0 Major Errors craftd[1601]: Major alarm cleared, FPC 0 Major Errors fpc0 LUCHIP(0): Secondary PPE 0 zone 1 timeout. fpc0 PPE Sync XTXN Err Trap: Count 7095, PC 10, 0x0010: trap_nexthop_return fpc0 PPE Thread Timeout Trap: Count 226, PC 34a, 0x034a: nh_ret_last fpc0 PPE PPE Stack Err Trap: Count 15, PC 366, 0x0366: add_default_layer1_overhead fpc0 PPE PPE HW Fault Trap: Count 10, PC 3c9, 0x03c9: bm_label_save_label fpc0 LUCHIP(0) RMC 0 Uninitialized EDMEM[0x3f38b5] Read (0x6db6db6d6db6db6d) fpc0 LUCHIP(0) RMC 1 Uninitialized EDMEM[0x394cdf] Read (0x6db6db6d6db6db6d) fpc0 LUCHIP(0) RMC 2 Uninitialized EDMEM[0x3d9565] Read (0x6db6db6d6db6db6d) fpc0 LUCHIP(0) RMC 3 Uninitialized EDMEM[0x3d81b6] Read (0x6db6db6d6db6db6d) These message would be transient in nature. The discrepancy of nexthop handling that is addressed in this PR can also manifest itself in form of other issues in the system. Basically when the nexthops go out of sync we are bound to see either Packet Forwarding Engine crashes/traps or Routing Engine crashes. The fix in this PR should take care of this behavior and ensure we handle the nexthops correctly to maintain the synchronization between master Routing Engine, backup Routing Engine and all Packet Forwarding Engine peers. PR990023

  • dcd will crash if targeted-distribution applied to ge ifd via dynamic-profile PR1054145

  • Two redundant logical tunnels (rlt) interfaces are configured with "per-unit-mac-disable" enabled. After configure the second one, the first rlt interface goes down. rlt0 { logical-tunnel-options { per-unit-mac-disable; <<<<<< } } PR1055005

  • It is observed that the syslog messages related to kernel and Packet Forwarding Engine may get generated at an excessive rate, especially in subscriber management environment. Most of these messages may appear repeatedly, for example, more than 1.5 million messages may get recorded in 2 hours, and there are only 140 unique messages. Besides, these messages are worthless during normal operation and due to the excessive rate of log generation, high RE CPU consumption (for example, RE CPU utilization can be stuck at 100% for a long time (minutes or hours), it depends on the activity of subscribers (frequency of logins and logouts) and on the AI scripts used by the customer) by event process (eventd) might be observed on the device. PR1056680

  • For Junos release 13.3R1 or later, after multiple (e.g. 26) iterations of graceful Routing Engine switchover (GRES), the TNP address of management interface might be deleted wrongly during switchover, this leads to all FPCs be offline. PR1060764

  • On MX Series routers, INET MTU (PPP payload MTU, that is IP header plus data excluding any L2 overhead) is being set to lowest MRU of either MX (local device) or peer. This behavior is not inline with ERX behavior, which is set to min(local MTU, peer MRU). This might cause the packet drops in the customer network in the downstream path. PR1061155

  • When the Ethernet Link Fault Management (LFM) action profile is configured, if there are some errors (refer to the configuration, for example, frame errors or symbol errors) happening in the past (even a long past), due to the improper handling of error stats fetching from kernel, the LFM process (lfmd) may generate false event PDUs and send the false alarm to the peer device. PR1077778

  • On MX Series Virtual Chassis (MX-VC) platform with "subscriber-management" enabled, after power up/reboot, the VC backup router (VC-B) experiences a rapid sequence of role transitions from no-role to VC master router (VC-M) to VC-B, the expected local GRES and a reboot of the former master Routing Engine might not happen on the VC-B, some of the FPCs on it might be stuck in "present" state and eventually rebooted. PR1086316

  • When an interface on SFPP module in MIC is set disabled, after pulling out the SFPP and then insert it, the remote direct connected interface might get up unexpectedly. PR1090285

  • After removing a child link from AE bundle, in the output of "show interface AE detail", the packets count on the remaining child link spikes, then if add back the previous child link, the count recover to normal. PR1091425

  • In MX Series Virtual Chassis (MXVC) environment, when rebooting the system or the line cards which contain all the Virtual Chassis port (VCP) links, because line cards may fail to complete the rebooting process within 5 minutes, the timer (that is, the amount of time allowed for the LCC to connect to the SCC) started by the master router may expire which may cause the VCP links establishment failure. In addition, this issue is not specific to the line cards type, based on the observation, the timer (5min) may expire on a MX2020 with all 20 FPCs equipped as well. PR1095563

  • On PB-2OC12-ATM2-SMIR PIC, port 0 and port 1 are configured with clock source as external, if Loss of signal (LOS) is inserted on port 0, the port 0 will down, the expect behavior is clock being used from port 1. But in this case, port 0 down will results in port 1 flapping and reporting SONET phase lock loop (PLL) errors. PR1098540

  • Due to the fact that the error injection rate configured by user on Routing Engine via CLI command "bert-error-rate" may not be programmed in the hardware register, the PE-4CHOC3-CE-SFP, PB-4CHOC3-CE-SFP, MIC-3D-4COC3-1COC12-CE, and MIC-4COC3-1COC12-CE-H may fail to inject bit errors during a Bit Error Ratio Test (BERT). PR1102630

  • On MPC-3D-16XGE-SFPP line card, when an optics (for example, 10G-LR-SFP) is disabled and then enabled administratively, if the SFP is not temperature tolerant (non-NEBS compliant), the TX laser may not be turned on due to the fact that the chassis process (chassisd) may keep sending the "disable-non-nebs-optics" command to the optics if the current temperature of FPC reaches the threshold temperature. PR1107242

  • On MX Series platform, continuous error messages might be seen on the MICs (for 10G/40G/100G MICs) from MIC3 onwards (listed as below) when physical interface (IFD) settings are pushed (e.g. booting the MPC). Based on the current observation, the issue may not have any operational impact and the MICs that may encounter this issue are listed as below, - 10G MICs: MIC3-3D-10XGE-SFPP, MIC6-10G, MIC6-10G-OTN, - 40G MICs: MIC3-3D-2X40GE-QSFPP, - 100G MICs: MIC3-3D-1X100GE-CFP, MIC3-3D-1X100GE-CXP, MIC6-100G-CXP, MIC6-100G-CFP2 PR1108769

Layer 2 Features

  • During interface flaps a high amount of TCN (Topology Change Notification) might get propagated causing other switches to get behind due to high amount of TCN flooding. This problem is visible after the changed done from 11.4R8 onwards which propagates TCN BPDU immediate and not in the pace of the 2 second BPDU Hello interval to speed up topology change propagation. The root cause is the TCNWHILE timer of 4 seconds is always reset upon receiving TCN notifications causing the high churn TCN propagation. PR1089580

  • In MX Series Virtual Chassis (MXVC) environment, when packets come from a interface (for example, xe-16/0/1.542) situated on one member of VC (for example, VC member 1), if the ingress Packet Forwarding Engine (for example,FPC16 PFE0,who runs hash to determine which interface it should send the packet to) decides that it should send the packet via another interface (for example, xe-4/0/1.670) situated on different member (for example, VC member 0), it will send the frame to member 0 via the vcp- intf. In case of xe-4/0/1.670 belongs to an AE bundle which has multiple child links, a hash need to be run on Packet Forwarding Engine carrying the VCP port (receiving side on member 0) to determine which one is the egress Packet Forwarding Engine within member 0 to send the packet out after vcp- intf gets the packet. This hash result should get the same result as the ingress Packet Forwarding Engine. If it is not the case, then the packet would get dropped on Packet Forwarding Engine on member 0. PR1097973

  • With scaled subscribers connected, restarting one of MPCs might cause subscribers unable to log in for about 2 mins. PR1099237

  • In a scenario that BGP based VPLS stitching with L2circuit, with "pseudowire-status-tlv" configured under L2circuit's mesh-group, if L2circuit neighbor doesn't configure "pseudowire-status-tlv", then status of "Negotiated PW status TLV" of VPLS connection is "NO", this will cause BGP based VPLS connection can not up even the L2circuit is up. PR1108208

Layer 2 Ethernet Services

MPLS

  • In Resource Reservation Protocol (RSVP) environment, if CoS-Based Forwarding (CBF) for per LSP (that filter out traffic not related to that LSP) is configured, and either the feature fast-reroute or link-protection is used on the device, when the primary link is down (for example, turning off the laser of the link), due to some next hops of the traffic may be deleted or reassigned to different class of traffic, and the RSVP local repair may fail to process more than 200 LSPs at one time, the traffic may get dropped by the filter on the device before the new next hop is installed. In this situation, the feature (fast reroute or link protection) may take longer time (for example, 1.5 seconds) to function and the traffic loss might be seen at the meantime. In addition, the issue may not be seen if the CBF for per LSP is not configured on the device. PR1048109

  • Junk characters are being displayed in output of show connections extensive command. PR1081678

  • On dual Routing Engine platform with GRES, the kernel synchronization process (ksyncd) may crash on the backup Routing Engine when adding of route pointing to indirect nexthop on system. PR1102724

  • In Junos OS release 13.2R1 and above, in MPLS L3VPN scenario, when "l3vpn-composite-nexthop" statement is enabled on a PE router and an interface style service set is attached to the ingress interface, the L3VPN packets with the MPLS labels will be sent to the service card and dropped. As a workaround, we should disable "l3vpn-composite-nexthop". PR1109948

Network Management and Monitoring

  • Mib2d cores while trying to re-add a lag child into the internal DB. Since the entry is already present in the internal DB. Before adding the child link mib2d does a lookup on the tree, to know if the entry is not already there. However, this lookup returns no results, since the child link is part of snmp filter-interface configuration. PR1039508

Platform and Infrastructure

  • LSI logical interface input packet and byte stats are also added to core logical interface stats, but when the LSI logical interface goes down and the core logical interface stats are polled, there is a dip in stats. The fix is to restore LSI logical interface stats to core logical interface before deleting the LSI logical interface. PR1020175

  • The Priority code point (PCP) and Drop eligible indicator (DEI) bit in 802.1Q header are preserved while packet gets routed within the same Packet Forwarding Engine . The expected behavior is resetting the PCP and DEI bit when the packet is routed. PR1036756

  • Due to a defect in the Junos OS software, when a telnet user experiences some undefined network disconnect, .perm and .env files under /var/run are left behind. This scenario happens only under certain unknown ungraceful network disconnects. When considerable number of .perm/.env files get accumulated under /var/run, issue is seen with telnet users, that they are not able to perform permitted operations on the router, post-login. PR1047609

  • If a Radius server is configured as accounting server, when it is non-reachable, the auditd process might stressed with huge number of audit logs to be sent to the accounting server, which might cause auditd to crash. PR1062016

  • VRRP advertisements might be dropped after enable delegate-processing on the logical tunnel (lt) interface. It would result in VRRP master state observed on both routers. PR1073090

  • Problem: It tries to check allotted power for all the FPCs, here in the CHASSISD_I2CS_READBACK_ERROR logs it shows for the FPCs which are not present in chassis. It just calls i2cs_readback() to read i2c device and fails there as these FPCs? slots are blank and prints those readback errors. Also the errors are harmless: "CHASSISD_I2CS_READBACK_ERROR: Readback error from I2C slave for FPC" Fix: Code to check 'if power has been allotted to this FPC', needs to be executed only if the FPC is present. PR1075643

  • When a MX Series chassis network-services is "enhanced-ip" and an AE is part of a Layer 2 bridge (bridge-domain or VPLS), there is a possibility that an incorrect forwarding path may be installed causing traffic loss. This could happen when first applying the configuration, restarting the system or restarting the line card. PR1081999

  • If with both MPC/MSDPC and other type of DPCs equipped, for local switching at mesh group level, split horizon on PW interfaces won't work and this would cause packets to loop back to same PW interface. PR1084130

  • In Junos OS Releases 13.3R3, 14.1R1, 14.2R1, there is a new feature, an extra TLV term is added to accommodate the default action for the "next-interface" when the corresponding next-interface is down. While doing a unified ISSU from an image without the feature to an image with this feature, all MPCs might crash. PR1085357

  • Issue is specific to 64-Bit RPD and config-groups wildcard config specific as in below case: set groups TEST routing-instances <*> routing-options multicast forwarding-cache family inet threshold suppress 200 set routing-instances vrf1 apply-groups TEST set routing-instances vrf1 routing-options multicast forwarding-cache family inet threshold suppress 600 With this daemon(rpd) reads suppressed value ?200? (i.e. coming from groups) instead of reading value ?600?from foreground and customer sees unexpected behavior with respect to threshold-suppress. Workaround: They can replace wildcard with actual routing-instance name as in below example: set groups TEST routing-instances vrf1 routing-options multicast forwarding-cache family inet threshold suppress 200 set routing-instances vrf1 apply-groups TEST set routing-instances vrf1 routing-options multicast forwarding-cache family inet threshold suppress 600 PR1089994

  • On MX series router, if ifl (logical interface) is configured with VID of 0 and parent ifd (physical interface) with native-vlan-id of 0, when sending L2 traffic received on the ifl to Routing Engine, the VID 0 will not imposed, causing the frames to get dropped at Routing Engine. PR1090718

  • When an interface on MQ-based FPC is going to link down state, in-flight packet on interface transmit path will be stuck on the interface and never drained until the interface comes up again. As a result, small number of such stacked packets will be sent out when the interface is going to UP state. No other major impact should not be seen after those packets are drained. PR1093569

  • On MX2020/2010 router, an SPMB core file will be seen if there are bad XF chips (fabric chip) on SFB, which might trigger RE/CB switchover. PR1096455

  • When a P2MP LSP is added or deleted at ingress LSR, traffic loss is seen to existing sub-LSP(s) at transit LSR which replicates and forwards packet to egress PEs. This issue only affects MX Series routers with MPCs/MICs. PR1097806

  • The "shared-bandwidth-policer" knob is used to enable configuration of interface-specific policers applied on an aggregated Ethernet bundle to match the effective bandwidth and burst-size to user-configured values. But this feature is broken from Junos release 14.1R1 when "enhanced-ip" is configured on MX Series platform with pure trio-based line cards. The bandwidth/burst-size of policers attached to Aggregated Ethernet interfaces are not dynamically updated upon member link adding or deletion. PR1098486

  • On Trio-based platform, when the type of the IPv6 traffic is non-TCP or non-UDP (for example, next header field is GRE or No Next Header for IPv6), if the traffic rate is high (for instance, higher than 3.5Mpps), the packet re-ordering may occur. PR1098776

  • On MX Series-based line cards, when the prefix-length is modified from higher value to lower value for an existing prefix-action, heap gets corrupted. Due to this corruption, the FPC might crash anytime when further configurations are added/deleted. The following operations might be considered as a workaround: Step 1. Delete the existing prefix-action and commit Step 2. Then re-create the prefix-action with newer prefix-length PR1098870

  • In an MPLS L3VPN network with a dual-homed CE router connected to different PE routers, a protection path should be configured between the CE router and an alternate PE router to protect the best path. When BFD is enabled on the BGP session between the CE and the primary PE router, with local traffic flowing from another CE connected with the primary PE to this CE, after bring down the interface on the best path, the local repair will be triggered by BFD session down but it might fail due to a timing issue. This will cause slow converge and unexpected traffic drop. PR1098961

  • On Trio-based platform, before creating a new unilist nexthop, there is a check to see if there is at least 512k DoubleWords (DW) free. So, even the attempting NH requires only a small amount of memory (for example, < 100 DWs), if there is no such enough free DWs (that is, 512k), the check will fail and the end result is that the control plane will quit adding this NH prematurely - stopping at ~80% of capacity. With the fix, it will check for 64k free DWs which is lower reference watermark for available resource, thereby ensuring that can allocate resource. PR1099753

  • From Junos release 14.1 and above, IPv6 mobility packets with Heartbeat option that the length of the mobility header (including the ethernet encapsulation and main IPv6 header) extends beyond 128 Bytes will be discarded as bad IPv6 option packet due to a logic error in packet handling. PR1100442

  • Large scaled inline BFD session (in this case, 6000 inline BFD sessions) are loaded with the minimum-interval value 50ms. If FPC restarts, some BFD sessions might flap. PR1102116

  • On MPC3E/MPC4E line card, when the feature "flow-detection" is enabled (under "ddos-protection" hierarchy), if suspicious control flow is received, two issues may occur on the device: Issue 1: sometimes, the suspicious control flow may not get detected on the line cards Issue 2: once the suspicious control flows are detected, they may never time out even if the corresponding packets stop PR1102997

  • Customer is having the similar issue PR1103771

  • On T4000 platform with FPC Type-5 equipped, after performing unified ISSU, due to the fact that only 6 out of 16 temperature sensors may get initialized, the temperature reading for the line card may be shown as "Absent". PR1104240

  • Any configuration or logical interface (IFL) change will introduce 160 bytes memory leak on MPC heap memory when we have any type of inline sampling configured (ipfix or version 9). Only trigger of issue is the configuration of inline sampling, even without traffic being sampled. The leak is more evident in a subscriber management scenario when we have many IFL addition/deletion. Rebooting MPC in a controlled maintenance window is the only way to restore memory. PR1105644

  • When a common scheduler is shared by multiple scheduler maps which applies to different VLANs of an Aggregated Ethernet (AE) interface, if the knob "member-link-scheduler" is configured as "scale", for some VLANs, the scheduler parameters are wrongly scaled among AE member links. As a workaround, we should explicitly configure different schedulers under the scheduler maps. PR1107013

  • Due to a software defect found in 13.3R7.3 and 14.1R5.4 inclusively, Juniper Networks strongly discourage the use of Junos software version 13.3R7.3 on routers with MQ-based MPC. This includes MX-Series with MPC1, MPC2; all mid-range MX-Series; and some of EX9200 line cards. PR1108826

  • DHCP End options (option 255) is missing by DHCP-relay agent (where 20 bytes DHCP options 82 inserted) for client DHCP discover message with 19bytes padding. PR1110939

  • On Trio-base FPC, when MPLS-labled fragmented IPv6 packets arriving at PE router (usually seen in 6PE and 6VPE scenario), the Packet Forwarding Engine might mistakenly detect such IPv6 header and then drop these packets as "L3 incompletes" in the output of "show interface extensive". PR1117064

Routing Policy and Firewall Filters

  • In Class-of-Service (CoS) environment, there is a possibility (happened twice so far and not reproducible in the lab) that routing protocol process (rpd) may crash because the CoS memory may get incorrectly freed and then allocated again. PR1062616

  • On the platform that M7i/M10i with enhanced CFEB, M320 with E3-FPC, M120, and MX with DPC, when the flood filter is configured in VPLS instance on the Packet Forwarding Engine, if the Packet Forwarding Engine receives a filter change (for example, FPC reboot occur and comes up), the line card may fail to program the filter. PR1099257

Routing Protocols

  • In mutli-topologies IS-IS scenario, there is huge difference between estimated free bytes and actual free bytes when generating LSP with IPv6 Prefix. It might cause LSP fragment exhaustion. PR1074891

  • There are two issues in the PR: (1) In multicast environment, Incoming interface list (IIF) list has only RPF interface, designated forwarder (DF) winners are not added in the list in backup Routing Engine. (2) "Number of downstream interfaces" in show pim join extensive is not accounting Pseudo-VXLAN interface. PR1082362

  • When removing BGP Prefix-Independent Convergence (PIC) from the configuration, the expect behavior is that any protected path would become unprotected. But in this case, the multipath entry that contains the protection path (which is supposed to be removed) remains active, until BGP session flaps or the route itself flaps. As a workaround, we can use "commit full" command to correct or to commit. PR1092049

  • In BGP environment, when configuring RIB copy of routes from primary routing table to secondary routing table (for example, by using the CLI command "import-rib [ inet.0 XX.inet.0]") and if the second route-table's instance is type "forwarding", due to the BGP routes in secondary routing table may get deleted and not correctly re-created, the routes may be gone on every commit (even commit of unrelated changes). As a workaround, for re-creating the BGP routes in secondary route table, use CLI command "commit full" to make configuration changes. PR1093317

  • In Junos OS Release 9.1 and later, RFC 4893 introduces two new optional transitive BGP attributes, AS4_PATH and AS4_AGGREGATOR. These new attributes are used to propagate 4-byte AS path information across BGP speakers that do not support 4-byte AS numbers. In this case, when AS4_AGGREGATOR attribute (18) is received from a 2-byte AS peer (note AS4_AGGREGATOR attribute is only received when the aggregator has 4-byte AS but this peer only supports 2-byte AS), NSR synchronization with standby Routing Engine would fail, causing session constantly bouncing on standby Routing Engine (hogging CPU). PR1093615

  • With this change the default label hold timer was increased for 10 seconds to 60 seconds. PR1093638

  • The rpd process might crash when resolve-vpn and rib inet.3 are configured under separate levels (BGP global, group and peer). The fix is If anybody configures a family at a lower level, reset the state created by either of knobs from higher levels. This behavior conforms with our current behavior of family config - which is that any config at a lower level is honored and the higher level config is reset. PR1094499

  • When BGP routes has multiple protocol nexthops including discard/reject and other IGP nexthops, the discard/reject nexthop will be selected as BGP nexthop, which will cause traffic loss. PR1096363

  • When polling SNMP OID isisPacketCounterTable 1.3.6.1.2.1.138.1.5.3, the rpd process might crash. PR1101080

  • When the ISIS configurations has been removed, the ISIS LSDB contents got flushed. If at the same time of this deletions process, there is an SPF executions (that is, try to access the data structures at same time when/a fraction of secs after freeing its content), routing protocol process (rpd) crash occurs. PR1103631

Services Applications

  • When an MX Series router configured as an LNS sends an Access-Request message to RADIUS for an LNS subscriber, the LNS now includes the Called-Station-ID-Attribute when it receives AVP 21 in the ICRQ message from the LAC. PR790035

  • In IPsec environment, after performing the Routing Engine switchover (for example, performing Graceful Routing Engine Switchover) or chassis reboot (that is, whole device is powered down and powered UP again), due to the key management daemon (kmd) may be launched before the Routing Engine mastership is finalized, it may stop running on the new master Routing Engine. PR863413

  • In the L2TP scenario with dual Routing Engines. After subscriber management infrastructure daemon (smid) being restarted, because the delete notification to backup RE might be lost, the subscriber database (SDB) information does not synchronize between master RE and standby RE. After RE switchover is executed, the Layer 2 Tunneling Protocol daemon (jl2tpd) might crash, and new L2TP subscribers are unable to dial. PR968947

  • On an L2TP access concentrator (LAC) device with more than 8K L2TP sessions up, if execute command "clear services l2tp session all" and then stop the command by using ctrl-C, the Layer 2 Tunneling Protocol process (jl2tpd) might crash. PR1009679

  • On MX-series router that configured as L2TP tunnel switch (LTS), after receiving a Call-Disconnect-Notify (CDN) message on LNS interface from remote LNS, the L2TP daemon (l2tpd) might crash and dump a core file. PR1021881

  • On Trivial File Transfer Protocol (TFTP) Application Layer Gateway (ALG) with NAT translation type "dynamic-nat44" configured, MS-DPC/MS-MPC/MS-MIC might crash when processes the TFTP packets. PR1091179

  • On M series platform, in Layer 2 Tunneling Protocol (L2TP) network server (LNS) environment, not all attributes (Missing NAS-Identifier, NAS-Port-Type, Service-Type, Framed-Protocol attributes) within Accounting-Request packet are sending to the RADIUS server. PR1095315

  • If MS-DPC is used in CG-NAT environment, in a very rare condition, when the MS-DPC tries to delete a NAT mapping entry (e.g. entry timeout), error might occur and the MS-DPC might get rebooted and then dump a core file. PR1095396

  • Some values of MIB object jnxSrcNatStatsEntry might be doubled when AMS (or rsp) interface and NAT are configured together. PR1095713

Software Installation and Upgrade

  • Add "on <host>" argument to to "request system software validate" to allow validation on a remote host/RE running Junos. PR1066150

Subscriber Access Management

  • In scaled DHCP subscribers environment, the authd process might crash and generate a core file after clearing DHCP binding or logout subscribers. PR1094674

VPNs

  • In BGP MVPN scenario, an MSDP timeout on the PE might occur causing the source to be removed even if it is local. This will cause type-5 flaps and traffic loss of 30 to 40 seconds. The issue showed up in a scaling MSDP configuration where the KA timer periodically expires with a CE (not PE) acting as RP. The fix has been provided to add a check for local source (even if not local RP) before withdrawing the type-5 route. PR1011124

  • In Internet multicast over an MPLS network by using next-generation Layer 3 VPN multicast (NG-MVPN) environment, when rib-groups are configured to use inet.2 as RPF rib for Global Table Multicast (GTM, internet multicast) instance, the ingress PE may fail to add P-tunnel as downstream even after receiving BGP type-7 routes. In addition, this issue only affects GTM. PR1104676

Resolved Issues: 14.1R5

Class of Service (CoS)

  • This error message "only per-unit and 2-level hierarchical scheduler are supported on this interface" is a cosmetic regression issue without any functional impact. PR1050512

  • Forwarding class accounting stops working after Routing Engine switchover. This behavior has been corrected in Releases 13.3X2,13.3R7,14.1R5,14.2R3,13.3R6, and 15.1. Issue comes when MPC reboots for any reason with forwarding-class-accounting configured on AE/AS interface. In forwarding-class-accounting feature, counters are allocated based on number of forwarding classes configured in MPC. In error case on MPC reboot, AE interface is getting created before the message for configuring number of forwarding classes in MPC comes. As a result while enabling forwarding-class-accounting feature on AE interface, number of forwarding classes value in MPC is 0 and counters are not allocated causing issue. Cause: Race condition when on MPC reboot AE interface getting created before number of forwarding classes are configured. Fix: When number of forwarding classes are set after MPC reboot, check for any AE interface with forwarding-class-accounting configured and reprogram it. PR1060637

  • Add chassis scheduler map support on gr interface on MS-PIC, which means there will be no commit error if scheduler-map-chassis is applied on gr interface. PR1066735

  • 1. With "hierarchical-scheduler" configured at IFD level 2. Under class-of-service hierarchy "output traffic control profile" configured at "interface-set" as well as IFD level, for the same IFD/IFL. With the above two conditions met, when a Junos OS upgrade is performed on a dual Routing Engine system the configuration validation check would fail on the Routing Engine that is upgraded later with this error message. Error message: "cannot configure a traffic control profile for this ifl when a parent has a traffic control profile that references a scheduler map: ifl xe-11/0/0.5000 refers to traffic-control-profile TCP_PE-CE_30M. It is also a member of interface set xe-11/0/0_OTag=80 which has traffic-control-profile TCP_PE-CE_80M which references scheduler-map SM_PE-CE" conditon-1: lab-re1> show configuration interfaces xe-11/0/0 { hierarchical-scheduler; <<< Condition-2: lab-re1> show configuration interfaces interface-set xe-11/0/0_OTag=80 { interface xe-11/0/0 { <...>; } } lab-re1> show configuration class-of-service interfaces interface-set xe-11/0/0_OTag=80 { output-traffic-control-profile TCP_PE-CE_80M; <<< } <..> xe-11/0/0 { output-traffic-control-profile TCP_Maxbuff; unit 5000 { output-traffic-control-profile TCP_PE-CE_30M <<< } } PR1069477

  • On MX Series platform configured for IP network-services (default) and with MS-DPC/Tunnel-Interface, virtual-tunnel (vt) interfaces are created automatically to support ultimate-hop-popping upon enabling "protocol rsvp". These interfaces are associated with default IP and MPLS classifiers along with MPLS re-write rule. When "protocol rsvp" is disabled/enabled or MS-DPC/FPC (with tunnel-service) restarts, the vt interfaces are deleted and re-added to the system. However during the deletion, these interfaces are not getting released from cosd process and thus leads to memory leak in cosd. PR1071349

  • On MX Series platform, when aggregated Ethernet (AE) interface is in link aggregation group (LAG) Enhanced mode, after deactivating and then activating one child link of the LAG , the feature that runs on AE interface rather than on the child link (for example, IEEE-802.1ad rewrite rule) may fail to be executed. PR1080448

Forwarding and Sampling

  • On MX Series routers with MPCs/MICs, when deleting firewall filter and the routing instance it is attached to, in some race conditions, the filter might not be deleted and remains in resolved state indefinitely. PR937258

  • When a firewall filter, which is used to de-encapsulate the IPv4 packets encapsulated in IPv6 GRE header, is attached to interface hosts on MX Series MPC/MIC, the IPv6 GRE header would be de-encapsulated but the inner IPv4 packet would end up getting dropped and not forwarded. This issue affects the packet with IPv4 over IPv6 GRE header only, and those packets with IPv6 over IPv6 GRE header are not affected. PR1054039

  • If the template of the policer is changed (for example, change the bandwidth-limit value of policer), shared-bandwidth-policer knob may not function properly anymore. PR1056098

  • In some rare cases, SNMP might get Output bytes of Local statistics instead of the Traffic statistics when retrieving Output bytes of Traffic statistics on a logical interface. PR1083246

General Routing

  • On dual Routing Engine platforms, after performing unified graceful Routing Engine switchover (GRES) with 8K subscribers, the ksyncd process may crash due to the replication error on a next hop change operation. The issue is hit when there's memory pressure condition on the Routing Engine and in that case, it may lead to null pointer de-reference and ksyncd crash. Or in some case, the kernel on the new master Routing Engine might crash after Routing Engine switchover if Routing Engine is under memory pressure due to missing null check when trying to add a next hop and the next hop is not found at the time. PR942524

  • When the mirrored interface and mirror destination interface are hosted on different Virtual Chassis (VC) members, the ingress MPLS packets are not getting mirrored to the mirror destination. PR979888

  • In point-to-point (P2P) SONET/SDH interface environment, there is a destination route with this interface as next-hop. When this interface is disabled, the destination route is still kept in the forwarding table and might cause ping fails with "Can't assign requested address" error. PR984623

  • The knob 'gratuitous-arp-on-ifup' should send a gratuitous arp on each unit of a physical interface, but in Release 12.3 and later versions, only the first unit is seeing the configured behavior. PR986262

  • Optics lane#3 and lane#4 TX, RX power alarm data was ignored but the lane#1 and lane#2 data was used for lane#3 and lane#4 respectively. Causing incorrect/false alarm on lane#3 and lane#4 PR1001670

  • When there are no services configured, datapath-traced daemon is not running. In the PIC the plugin continues to try for the connection, and continuous connecion failure logs are seen. PR1003714

  • A raw IP packet with invalid Memory Buffer(mbuf) length may trigger a kernel crash. The invalid mbuf length might be set incorrectly by other daemons. PR1006320

  • During Wan Link flaps , ASIC streams in the Packet Forwarding Engines are disabled/enabled on the fly when traffic is inflight. This is normal and will result in the Cell drops, PKTR ICELL signature errors and SLOUT errors. However under certain rare conditions, Lout IP -Pkt Len Mismatch error is observed which sometimes trigger automatic restart of the FPC. On TXP, TXP-3D in FPC Type 4-ES can experience automatic restart during wan interface flaps. PR1013522

  • If you issue the "show services nat mappings details" command with a large number of service sets configured (such as 1000 service sets) and one or two NAT mappings specified, the command takes a certain amount of time to display the output. During this period, if you deactivate or activate the services, a multiservices PIC management daemon core file is generated. PR1019996

  • Total CPU Utilization and Interrupt CPU Utilization are displayed incorrectly for MPC3E-3D-NG-Q, MPC3E-3D-NG, MPC2E-3D-NG-Q, and MPC2E-3D-NG. This is because the router incorrectly calculates CPU utilization from system startup rather than the CPU utilization at a particular point of time. PR1024150

  • On MPC5E line card, if a firewall filter with large-scale terms (more than 1300) is attached to an interface, traffic drop might be seen. PR1027516

  • In the scenario where router acts as both egress LSP for core network and BRAS for subscribers, RSVP-TE sends PathErr to ingress router due to matching to subscriber interfaces wrongly when checking the explicit route object (ERO), if subscribers are associated with same lo0 address as used by RSVP LSP egress address. PR1031513

  • On the Type 5 PIC, when the "hold-time down" of the interface is configured less than 2 seconds and the loss of signal (LOS) is set and cleared repeatedly in a short period (for example, performing ring path switchover within 50ms), the "hold-time down" may fail to keep the interface in "up" state within the configured time period. PR1032272

  • Problem scenario: Issue is seen only with vMX. It will be seen when the PPPoE session's Keep-alive timer expiry happens. This might be due to non-graceful termination of remote side or due to communication path failure with the remote end. Problem statement: When PPPoE session Keep-alive timer expires the local PPPoE session is NOT closed/logout. PR1034520

  • When the CPU usage is very high (e.g. 100%) on Routing Engine, the MS-MIC might get stuck due to kernel deadlock, which triggers the card to crash and generate a core file. PR1038026

  • If default-address-selection knob is configured on MX-VC, VC-heartbeat connection between member chassis may be unable to come up. PR1041194

  • CLNS ping fails for l3vpn over ethernet scenario. IS-IS routes are being considered as ARP routes which leads to this problem. PR1041251

  • For MLPPP interface on MX Series routers with MPCs/MICs, in some very rare conditions, the received fragmented packets might be dropped. PR1041412

  • This issue is applicable to a case which inline NAT configured on an interface belongs to either an MPCE or an MP3E/MPC4E/T4000-FPC5. Ingress and egress traffic traversing between an MPCE and these cards may cause the router to drop packets. PR1042742

  • On T Series platform running Junos OS Release 12.1 or later, for interface connected via optical system like DWDM, when the interface is admin disabled, there might be a delay (300-400msec) for system to detect the event and during which time, traffic blackhole might be seen. Please note if disable the interface by breaking the Rx or Tx link, issue will not happen. PR1043762

  • Queue stats on LSQ interfaces are not properly cleaned up when queuing is enabled on the IFD and the queues hosted at IFD level. This happens when a subsequent delete and create of the LSQ interface (not always though) - 14.1R4.10 PR1044340

  • On MX Series platform with one of the following protocols configuration, flapping the protocols will trigger the Composite Next-hop change operation. In rare condition, since it is not properly programmed, the FPC might crash. This is a day-1 issue. - LDP - MPLS - Point-to-multipoint LSP - RSVP - Static LSPs PR1045794

  • Once default route 0.0.0.0/0 is added, deleted or changed, the PFEMAN thread running on the MPC/FPC5 needs more than 600msec to program such changes. This is long enough to trigger LFM or BFD flap. Junos OS Release 13.3R2 or later is exposed to this symptom. PR1045828

  • On MX Series platforms, the unilist next-hop member will become 'replaced' status on Packet Forwarding Engine after interface flapping with ARP timeout. While the problem is happening, routing-table will display all right next-hop status but can not forward traffic since forwarding next-hop in Packet Forwarding Engine is in 'replaced' status and no longer active. PR1046778

  • On T Series FPC 1-3 and M320 except E3-FPC with fib-local configuration. If there are multiple FIB local FPCs or the FIB local is a multiple Packet Forwarding Engine FPC, the TCP packets might get out of order, and packets re-ordering would occur. It reduces the application-level throughput for any protocols running over TCP. PR1049613

  • In the PPP dual-stack subscribers environment, in rare condition, if bringing up 1000 dual-stack subscribers quickly, the PPP negotiation might fail. Then PPP retries negotiation, all subscribers fully establish. PR1050415

  • Incorrect flow count is reported in the field 'count' of V9 header in all the packets sent to the collector. PR1050543

  • This problem is because of a race condition, where other FPCs are not able to drain "which is 1 second" Fabric Streams connecting to FPC which is getting offline. With this situation - even when FPC comes online, other FPCs which have observed message "xmchip_dstat_stream_wait_to_drain" will not be able to send traffic to that particular FPC over fabric. There is no workaround. Rebooting FPCs which observed error message "xmchip_dstat_stream_wait_to_drain" is a recovery. PR1052472

  • On all Junos OS based platforms, there are two different types of memory blocks that might be leaked. The first issue is rpd-trace memory block leak. There is one block each for any trace files opened for rpd. They could be leaked for each time a configuration commit is done. Around 40 bytes are leaked per operation. The issue does not occur in Junos OS releases prior to 14.1. The second issue is rt_parse_memory block leak which could happen during the configuration of aggregate routes, configuration information might not be freed. Around 16384 bytes are leaked per operation. This issue is a day-1 issue. PR1052614

  • In subscriber management environment, the Berkeley Database (DB) may get into deadlock state. It is brought on by multiple daemons attempting to simultaneously access or update the same subscriber or service record. In this case, due to the access to DB were blocked by device control daemon (dcd), the subscriber management infrastructure daemon (smid) fails to recover the DB. Consequently, the router may stop responding to all the login/logout request as well as statistics activity. This timing related issue is most likely to occur during login or logout and when the system is busy. PR1054292

  • In subscriber management scenario, when dynamic VLAN (DVLAN) demux interface is configured on MX Series router, the interface may get in stuck state. It could be observed that the statistics of demux0 may stop incrementing. This is because Session Database (SDB) may incorrectly calculate the number of subscribers over DVLANs. When the issue occurs, for example, the router may not able to process any PPPoE Active Discovery Initiation (PADI) packet, and fail to establish the PPPoE session. PR1054914

  • OpenSSL project has published a security advisory for vulnerabilities resolved in the OpenSSL library on January 8th 2015: CVE-2014-3569, CVE-2014-3570, CVE-2014-3572, CVE-2014-8275, CVE-2015-0204, CVE-2015-0205. Refer to JSA10679 for more information. PR1055295

  • In IPv6 environment, after enabling feature "solicit-router-advertisement-unicast", the IPv6 router may fail to reply the Router Advertisement (RA) to the IPv6 host as unicast only. To be exact, the IPv6 router may not only reply to the IPv6 host an RA as unicast to its link local address, but also send the RA as multicast to all nodes group (Multicast Address: ff02::1). The sample configuration could be found as follows: user@router> show configuration protocols router-advertisement interface ge-1/0/3.400 { ... solicit-router-advertisement-unicast; <<<<< "solicit-router-advertisement-unicast" feature is enabled ... } PR1056599

  • IFCM error messages may occur in logs when it is not used. We lowered the severity of the message to avoid confusion. PR1057712

  • In LDP tunneling over single hop RSVP based LSP environment, after enabling "chained-composite-next-hop", the router may fail to create the chained composite next hops if the label value of VPN is equal with the label value of LDP. PR1058146

  • On MX Series routers, the interrupt-driven basis link down detection (an interrupt-driven link-down notification is generated to trigger locally attached systems to declare the interface down within a few milliseconds of failure) may fail after performing unified in-service software upgrade (ISSU). The interrupt might got prevented after performing unified ISSU due to disable the interrupt registers before unified ISSU but never restored after. PR1059098

  • In an IPsec load-balancing environment using MS-MPC cards, the ICMP request and ICMP reply can go through two different IPsec tunnels due to asymmetric routing; that is, ICMP request goes through one PIC, and ICMP reply goes through another PIC. Because of this, the ICMP reply will get dropped and never reach the other side of the IPsec tunnel. PR1059940

  • When enabling pseudowire subscribers the "show subscribers extensive" command does not display CoS policies applied to the subscriber interface. PR1060036

  • Output MTU counter shows incorrect data in the show pfe statistics traffic command output PR1061111

  • Due to incomplete fix, in releases containing PR869773 fix, rate limit drops are seen for Ingress queuing even though rate-limit is not configured or supported for ingress. PR1061256

  • If Bidirectional Forwarding Detection (BFD) protocol is enabled via site-to-site IPsec tunnel, the BFD session may fail to come up. It is because, when the BFD protocol is trying to exchange the packet via IPsec tunnel, the value of the TTL in inner IP header for packet may be decremented, hence the BFD packet gets dropped on the peer side and no BFD session would come up. PR1061342

  • On MX Series router with MPC2E-3D-NG/MPC3E-3D-NG/MPC5/MPC6 linecards, the Ethernet frame loss measurement (ETH-LM) feature does not work. PR1064994

  • If there are application-sets matching conditions in the NAT rule, NAT port might leak after deleting applications under application-set in live network. PR1069642

  • Higher baseline CPU utilization and periodic CPU spikes might be seen on XM-based MPC as compared to MPC-3D-16XGE-SFPP Cards due to below reasons: On XM-based MPC, low priority threads which monitor various things in the background on a periodic basis such as voltage, temperature, stats counters, hardware status etc are existed. When the system is idle these threads are allowed to take more of the load and that is why higher baseline CPU/CPU spikes are seen. This does not prevent other higher priority threads from running when they have to, as these are non-critical activities being done in the background and hence is a non impacting issue. PR1071408

  • overhead-accounting frame-mode command does not work on 100GbE CFP MIC, 100GbE CXP MIC, 2x40GbE QSFP MIC, and 10x10GbE SFPP MIC on MPC3E-3D-NG-Q, MPC3E-3D-NG, MPC2E-3D-NG-Q, and MPC2E-3D-NG PR1072001

  • On MX Series routers, the CLI command set interfaces interface-name speed auto-10m-100m is not supported. PR1077020

  • From Junos OS Release 14.1R1, if the hidden knob "layer-4 validity-check" is configured, the Layer 4 hashing will be disabled for fragmented IP traffic. Due to a defect, the Multicast MAC rewrite is skipped in this case, the fragmented multicast packets will be sent with incorrect destination MAC. PR1079219

  • The "inactivity-timeout" knob under [edit applications application application-name] hierarchy does not take effect for TCP based protocols. PR1080464

  • 'show interfaces queue <ifl>' stats are not correct with RLSQ warm-standby mode. Issue seen on MX Series routers with MPCs or MICs as well in 14.1R4.10 PR1082417

  • On a device with lt and ams interfaces configured, walking ifOutOctets or other similiar OID's may cause a "if_pfe_ams_ifdstat" message to print. This is a cosmetic debug-level entry, which was incorrectly set to critical-level. PR1085926

  • In some rare conditions, depending on the order in which configuration steps were performed or the order in which hardware modules were inserted or activated, if PTP master and PTP slave are configured on different MPCs on MX Series router acting as BC, it might happen that clock is not properly propagated between MPCs. This PR fixes this issue. PR1085994

Infrastructure

  • Fsck flags -C flag would prevent fsck from fixing the filesystem. Removing this flag as per this PR. PR1023164

  • On all Junos OS platforms, when the gstatd triggers false positives, this would result in unnecessary Routing Engine switchover happening. Thus a config option is added to prevent gstatd from initiating Routing Engine unnecessary switchover or Routing Engine relinquishing the mastership. The following error messages are expected to be seen: gstatd: [ad2] average write duration of 1021.34 crossed threshold of 1000.00 /kernel: mastership: routing engine 1 relinquishing as master: voluntarily requested PR1024515

  • Mirroring to next-hop-subgroup stops working, when there is a change in the next-hop-subgroup configuration PR1049631

Interfaces and Chassis

  • PR fix corrected jnxoptIfOTNPMFECIntervalTimeStamp, jnxPMIntTimeStamp, and jnxoptIfOTNPMIntervalTimeStamp reporting incorrect values around sytem-local midnight time as reported in PR 1065110. It also corrected the “SNMP PM Interval - incomplete date & time format without UTC offset”. PR946014

  • On MX Series router, in rare condition, the kernel might crash and the router will go in db prompt when router reboots. PR993978

  • MX-VC multiservices MIC and MPC: Offline is not initiated in ISSU if the MIC is in the VC-B chassis from where VC ISSU is started. PR997255

  • On MX Series routers with MPCs and MICs,in virtual private LAN service (VPLS) environment, the next hop in the kernel allocated by connectivity-fault management process (cfmd) may not be freed even after the CFM session has been removed (for example, deactivating the routing-instance). In this situation, after activating the routing-instance back, the interface within the routing instance would fail to come up because the nexthop is not freed by the cfmd application and hence the VPLS connection is down. PR1000060

  • On standalone T series router or TX platform, during Routing Engine rebooting, a bad (or busy) I2C device on Switch Interface Board (SIB) might cause Switch Processor Mezzanine Board (SPMB) to crash. Please note the TXP platform might also experience same issue due the bad I2C, and it has been addressed in another PR, which has been fixed on Junos OS version 13.1R5 13.2R6 13.3R1 13.3R4 14.1R3 14.2R1 and 15.1R1. PR1010505

  • Some duplicate entries are reported in jnx-chas-defines.mib. This patch removes the duplicate entries to fix the issue. PR1036026

  • On Ethernet PICs with longer hold down timer configured, flapping interface within the hold time might cause traffic loss longer than the hold period. PR1040229

  • At the end of the unified ISSU process, a link flap may be observed on SFP-T (tri-rate) interfaces. Now handling of SFP-T (tri-rate) interfaces has been improved to avoid link flap. PR1040977

  • In case of the IQ2 or IQ2E PIC are working in tunnel-only mode, rebooting the tunnel PIC while the traffic is passing through the tunnel might cause the tunnel PIC to not transfer traffic any more. PR1041811

  • “clear interfaces interface-set statistics all” command fails due to memory limitation. PR1045683

  • On MX Series routers (platforms) with Enhanced Switch Control Board (SCBE), when the fan tray is inserted or pulled out, the chassisd process might crash. PR1048021

  • When configuring the Virtual Router Redundancy Protocol (VRRP) on an interface which is included in a routing-instance via applying groups setting, if changes are made to the interface, the VRRP process (vrrpd) memory leak might be observed on the device. PR1049007

  • dcd is cored by configuring IPv6 address on fxp0.0 with master-only option under interfaces configuration. PR1049450

  • When Inherit is part of lower IFL Unit, vrrpd parses it before Active. In this case, vrrpd attaches a dummy Active to the Inherit, with the assumption that the Active will be available soon and then replication of information from Active to Inherit will take place. However, the replication of the priority might not be done correctly due to which the Inherit group gets stuck with priority of 0. PR1051135

  • The "show chassis network-services" command might not show the correct configured value when executed on the backup Routing Engine. This command should only be executed on the master Routing Engine. PR1054915

  • After performing a unified in-service software upgrade (ISSU) on MX Series Virtual Chassis (MX-VC) platform, all physical interfaces may go down. And the interfaces remain down until a graceful Routing Engine switchover (GRES) is performed. PR1055327

  • There is a mismatch in mac statistics, few frames go unaccounted. This is a day-1 issue with the software fetching of mac statistics. The snap and clear bits were set together on pm3393 chip driver software, so it used to happen that even before the copy of stats to shadow registers happened, clear was happening which used to go unaccounted. Now rollover mechanism has been implemented and tested for two continuous days and everything is fine. PR1056232

  • When a dynamic PPPoE subscriber with targeted-distribution configured on a dynamic vlan demux interface over aggregated Ethernet, the device control daemon (dcd) process might crash during a commit if the vlan demux has mistakenly been removed. The end users can't visit Internet after the crash. This is a rare issue and not easy to be reproduced. PR1056675

  • In subscriber management environment, PPP client process (jpppd) might crash as a result of a memory allocation problem. PR1056893

  • In a Multichassis link aggregation groups (MC-LAGs) environment, the MC-LAG peers have the MAC and port information and can forward the traffic appropriately. If a single VLAN on ICL interface is modified to a different VLAN, and then the administrator rolls back the VLAN configuration to the original one, the remote MAC might be stuck in the "Pending" state and not be installed in the bridge MAC-table, which causes the traffic forwarding to be affected. PR1059453

  • For transit traffic on INLINE LSQ redundancy (rlsq) interface, the input firewall-filter counters are logging zero packet count regardless of traffic flow. Output filter counters are logging correctly. For host-bound traffic, the firewall output counter will get double accounted on Classical rlsq and triple accounted on INLINE rlsq. PR1060659

  • In scaling PPP subscriber environment, when the device is under a high load condition (for example, high CPU utilization with 90% and above), the long delay in session timeout may occur. In this situation, the device may fail to terminate the subscriber session (PPP or PPPoE) immediately after three Link Control Protocol (LCP) keepalive packets are missed. As a result, subscriber fails in reconnect due to old PPP session and corresponding Access-Internal route are still active for some time. In addition to this, it is observed that the server is still sending KA packets after the session timed out. PR1060704

  • For multichassis link aggregation groups (MC-LAGs) running in active-active mode with back-to-back square topology, when the Inter chassis Control Protocol (ICCP) is broken between any MC-LAG devices, the non preferred device reverts to its own local system ID. But its Link Aggregation Control Protocol (LACP) partner on the remote side does not remove the flap link from AE bundle and it remians UP. This might cause a network wide loop resulting in traffic outage until manual intervention. PR1061460

  • In connectivity fault management (CFM) environment, if an AE interface is included in MEP interfaces, and if there is another AE interface configured without any child link (even this AE is not participating in OAM), the CFM sessions might not come up after Routing Engine restart or switchover. PR1063962

  • Error message is continuously logged every second after a particular copper-SFP [P/N:740-013111] is plugged into a disabled port on MIC. ***** error message **** mic_sfp_phy_program_phy: ge-*/*/* - Fail to init PHY link mic_periodic_raw: MIC(*/*) - Error in PHY periodic function PQ3_IIC(WR): no target ack on byte 0 (wait spins 2) PQ3_IIC(WR): I/O error (i2c_stat=0xa3, i2c_ctl[1]=0xb0, bus_addr=0x56) mic_i2c_reg_set - write fails with bus 86 reg 29 mic_sfp_phy_write:MIC(*/*) - Failed to write SFP PHY link 0, loc 29 mic_sfp_phy_mdio_sgmii_lnk_op: Failed to write: ifd = 140 ge-*/*/*, phy_addr: 0, phy_reg: 29 ala88e1111_reg_write: Failed (20) to write register: phy_addr 0x0, reg 0x1d Fails in function ala88e1111_link_init PR1066951

  • In PPPoE over AE subscribers management scenario, if "targeted-distribution" is enabled for subscribers IFL, the dcd process might crash and reboot when try to deactivate the AE interface. PR1067062

  • When adding new VCP port MX-VC, some traffic drops are seen. PR1067111

  • On MX Series Virtual Chassis (MX-VC) platform, due to a timing issue, the physical interface (ifd) on the same Modular Interface Card (MIC) with Virtual Chassis port (VCP) might not be created or take a very long time to be created after reboot the hosted Modular Port Concentrator (MPC). PR1080032

  • In an MX Virtual Chassis (MX-VC) scenario, during a unified ISSU operation, the new master Routing Engine does not have the MX-VC SCC's system MAC address. It just has its local system MAC address. The address is not replicated between local Routing Engines, and the new master Routing Engine has not yet connected to the MX-VC SCC to receive it. Hence, the possibility exists to overwrite the FPC with an address that does not match the previous address. PR1084561

  • The VRRP preempt hold time is not being honored during NTP time sync, and system time is changed. PR1086230

Layer 2 Features

  • In DHCP dynamic subscriber management scenario, when maintain DHCP subscribers during interface delete is configured, some interface indices might be reused by a new interface if system is under stress (such as high connection speed, many clients and individual log files configured to be larger than 100M). In this case, it might result in subscriber being associated with an interface that no longer exists. PR1044002

  • If the ppmd does not send replies to lacpd's periodic request to gather port statistics, the lacpd process may crash and restart due to the process memory consumption being slowly increased and finally reaching RLIMIT_DATA value which is 128MB. PR1045004

  • On multiple Routing Engines system with NSR enabled, if the FEC129 VPLS instance has "no-tunnel-service" configured, the VPLS might show status as "OL" (no outgoing label) after performing Routing Engine switchover. PR1050744

  • The Layer 2 Control Protocol process (l2cpd) leaks memory when interface config is applied to LLDP-enabled interfaces using 'apply-groups'. Size of the leak is ~700 bytes per commit. PR1052846

  • After changng the way of getting the site ID of VPLS from fixed site-id to automatic-site-id on one site while other sites are still using the fixed site-id in the network, the rpd process might crash because the site ID gotten by "automatic-site-id" may conflict with the site ID which was configured as fixed site ID on other sites. PR1054985

  • When MX Series router acts as the Virtual Extensible Local Area Network (VXLAN) Layer 3 gateway, the integrated routing and bridging (IRB) interfaces are configured to connect the VXLANs. The VXLAN packets are dropped when the route to reach a remote virtual tunnel endpoint (VTEP) interface is over an IRB interface. PR1057005

  • BGP peer configured between two routers over lt (logical tunnel) interface, if deactiving and activating scaled configuration a few times, in rare condition, the lt interface might reject all the ARP reply packets, and hence the ARP resolution does not happen over this interface, so the unicast routes are not in the correct states, and pinging to such an lt interface will fail. PR1059662

  • With Dynamic Host Configuration Protocol (DHCP) maintain subscriber feature enabled, when the subscriber's incoming interface index is changed, for example, the interfaces go away and come back after changing the MTU configuration of interface, the existing subscribers may get dropped and new subscribers fail in connection. PR1059999

  • After FPC is rebooted, the filter under Packet Forwarding Engine of the ERPS bridge domain may programe wrong ifl index, it will cause the router to not be able to receive any ERPS packets. PR1070791

  • LACP partner system ID is shown incorrectly when the AE member link is connected to a different device, this might misguide while troubleshooting the LAG issues. PR1075436

  • On MX Series routers, when configuring the dynamic access routes for DHCP subscribers based on the Framed-Route RADIUS attribute, the access route may be created on the device, however, the framed routes may not be installed for subscriber interface (under the "Family Inet Source Prefixes"). PR1083871

  • MTU change is not advised on the Ethernet ring protection (ERP) ring interfaces unless ring is in idle condition. Changing ring interface MTU while ring is not in idle state may result in change in the forwarding state of the interface and which can lead to loop in the ring. PR1083889

MPLS

  • When RSVP label-switched-path (LSP) optimize is enabled, RSVP LSP might stay down after a graceful Routing Engine switchover (GRES). To resolve the problem, the corresponding label-switched-path configuration needs to be deactivated, then, be activated again. PR1025413

  • On the P2MP LSP transit router with link protection enabled, if the LSP is the last subLSP, tearing the last subLSP (for example, a RESV tear message is received from downstream router) might crash the routing protocol process (rpd). PR1036452

  • LDP is not distributing a label for BGP FEC/prefix to downstream on demand (DoD) sessions when Forwarding Equivalence Class (FEC)/prefix learned this from IBGP peer to whom ldp-tunneling is configured. PR1049329

  • In LDP link protection which is protected by dynamic RSVP LSP scenario, when flap the interface having LDP link-protection enabled, the rpd process might crash on backup Routing Engine as soon as the bypass LSP is established. PR1053426

  • On M/MX/T Series routers, dynamic-rsvp-lsp is configured under interface link-protection hierarchy level. After interface flap, the bypass LSP does not come up. PR1054155

  • With BGP prefix-independent convergence (PIC) edge feature enabled, more than one BGP next-hop association will be installed in the Packet Forwarding Engine for MPLS VPN and Internet transit traffic. Deactivating/activating the IGP protocol (IS-IS or OSPF) might cause the backup session to stay down on the Packet Forwarding Engine. PR1058190

  • With graceful-restart configured, an inter-domain point-to-multipoint (P2MP) label switched path (LSP) with ERO defined and CSPF enabled might fail to come up after rpd process restart. PR1058271

  • With BGP labeled-unicast egress protection is enabled in a Layer 3 VPN, the protected node advertises primary BGP labeled unicast routes that need protection. When there is next-hop change for a labeled route, for example, deactivating/activating egress-protection knob or route churn, the memory might be exhausted andleads to an rpd process crash. PR1061840

  • This is a regression issue on all Junos operating systems related to a timing factor. When LDP session flaps, over which entropy label TLV or any unknown TLV is received, the LDP speaker might not send label withdraw for some prefixes to some neighbors. As a result, these neighbors will still use stale labels for the affected prefixes. PR1062727

  • The point-to-multipoint (P2MP) label-switched path (LSP) is unable to re-establish after certain links are down. This issue might be encountered when the links are those that contain the primary and backup LSPs for the P2MP LSP. The P2MP LSP can be restored after the links are up. PR1064710

  • In Junos OS Release 14.1 and later, the "load-balance-label-capability" knob is introduced to enable the router to push and pop the load balancing label, which causes LDP and RSVP to advertise the entropy label TLV to neighboring routers. MX, T4000 and PTX platform have the capability and it is reflected in their default forwarding-options configuration. However, there is a software defect in the way how Entropy Label Capability (ELC) TLV is encoded in the LDP label mapping message. It might cause the LDP session between routers to go down. PR1065338

  • Bypass enabled with optimize-timer will flap during every re-optimization event. PR1066794

  • When CSPF computes the path for node-protected bypass, it considers only the SRLG group configured on next-hop interface along the primary path. However it doesn't consider the SRLG group on next-to-next-hop interface to adequately provide diverse path between primary and node-protected bypass. PR1068197

  • When a primary LSP gets re-routed due to better metric, Link/Node protection for this LSP is expected to come up within 7 seconds provided the bypass-lsp protecting the next-hop link/node is already available. However in some corner cases, the Link/Node protection for re-routed primary LSP will not come up within 7sec even with bypass-lsp availability. The PR fixes this issue and reduces the delay of associating bypass-lsp with primary-lsp from 7 seconds to 2 seconds. PR1072781

  • In scaled l2circuits environment, the rpd process might crash due to a corruption in the LDP binding database. PR1074145

  • In race conditions, the rpd process on backup Routing Engine might crash when BGP routes are exported into LDP by egress-policy and configuration changes during the rpd process synchronizing the state to backup rpd process. PR1077804

Network Management and Monitoring

  • When syslog server is configured using hostname, after Routing Engine switchover router stopped sending the syslogs to external syslog server. Immediately after switchover, DNS was not accessible because it will take some time to learn route to DNS. System stopped retrying DNS resolution and syslogging stopped. System was running GRES (no NSR). PR947869

  • If the size of interface control process (dcd) trace file is configured to large (e.g. 500M), restarting the dcd process when it is doing log rotation (due to size limit reached) might cause dcd process unable to start any more, in the same time, interface ports will be down and "thrashing" error will be seen. PR1047330

  • SNMP mib walk jnxMac does not return value with et- interfaces on MPC3/MPC4/MPC5/MPC6 PR1051960

  • There is no specific counter name in the MIB2D_COUNTER_DECREASING syslog message PR1061225

  • SNMP queries for LAG MIB tables while LAG child interface is flapping, may cause mib2d grow in size and eventually crash with a core file. Mib2d will restart, and recover by itself. PR1062177

Platform and Infrastructure

  • When Network Configuration Protocol (NETCONF) service is used on the device, after the NETCONF session is established, because all the output that contain <error> tag might be incorrectly converted into <rpc error>, the management daemon (mgd) may crash on the device. As the example below, the output that contains <error> tag may lead to the crash. user@re0> show subscribers address 1000 | display xml .. <<<<<< The output contain <error> tag and may trigger the crash. PR975284

  • If the system has service related configurations, Error message generated by mountd might be seen: "can't delete exports for /packages/mnt/jbase: Bad address". PR991814

  • For inline BFD over aggregated Ethernet (AE) interface which member links are hosted on different FPCs, BFD packets coming on ingress line card will be steered to anchor Packet Forwarding Engine through fabric. If FPC reconnect to master Routing Engine (such as Routing Engine switchover operation), the inline BFD session punts the BFD packet to host, the BFD packet should go through loopback interface filter of VRF on which it is received. But in this case, the BFD packet might hit the wrong loopback interface filter from wrong routing-instance since the VRF information is not carried across fabric. PR993882

  • CPQ RLDRAM ECC single and double bit error will generate CM alarm. "show chassis alarms" command can be used to view CM alarm. Details ======= 1> CPQ RLDRAM ECC single bit error in last 10 secs will raise minor CM alarm. 2> No CPQ RLDRAM ECC single bit error in last 10 secs will clear minor CM alarm. 3> CPQ RLDRAM ECC double bit error will raise Major CM alarm (this alarm will not be cleared until the FPC is restarted) PR1023146

  • On MX platform with scaled set-up, after deactivate/activate or renaming a bridge domain (BD) which has irb interface associated, the IGMP snooping configured under the BD might not work any more. Please note it happens only when the router is in "network-services enhanced-ip" mode. PR1024613

  • Recurring local memory (LMEM) data errors may cause LU chip (lookup chip on Trio based FPC) wedge and eventually FPC crash. PR1033660

  • The Priority code point (PCP) and Drop eligible indicator (DEI) bit in 802.1Q header are preserved while packet gets routed within the same Packet Forwarding Engine. The expected behaviour is resetting the PCP and DEI bit when the packet is routed. PR1036756

  • Presence of /8 prefix in two terms results in incorrect filter processing and unexpected behavior. PR1042889

  • When IRB interface is configured with VRRP in Layer 2 VPLS/bridge-domain, in corner cases IRB interface may not respond to ARP request targeting to IRB sub-interface IP address. PR1043571

  • On MX Series platform with Extensible Subscriber Services Management (ESSM) subscribers configured using Junos OS commit script, after performing sequence of operations repeatedly with same set of configuration (subscribers apply-macros'), like adding subscribers, then deleting same subscribers again, then adding, then deleting again and again like so, the memory would leak on mgd process. In a generic scenario where a script just commits transient change and then exits, the issue will not be experienced. PR1048770

  • By default, after MPC 3D 16x 10GE) cards come up, about 75% of queues were allocated to support rich queuing with MQ chip. Such allocation causes MQ driver software module to poll stats. Polling stats cause this rise in CPU usage. PR1048947

  • For a Routing Matrix, if different Routing Engine models are used on switch-card chassis (SCC)/switch-fabric chassis (SFC) and line-card chassis (LCC) (for example, RE-1600 on SCC/SFC and RE-DUO-C1800 on LCC), where the out-of-band (OoB) management interfaces are named differently (for example, fxp0 on SCC/SFC RE and em0 on LCC RE), then the OoB management interface configuration for LCC RE will not be propagated from SCC/SFC RE during commit. PR1050743

  • NTP.org has published a security advisory for multiple vulnerabilities resolved in ntpd (NTP daemon) that have been assigned four CVE IDs. Junos has been confirmed to be vulnerable to one of the buffer overflow vulnerabilities assigned CVE-2014-9295 which may allow remote unauthenticated attackers to execute code with the privileges of ntpd or cause a denial of service condition. Refer to JSA10663 for more information. PR1051815

  • Values for the "input-traffic-control-profile" statement get reset after deactivating/activating the traffic-manager mode. PR1052785

  • Change of MED value fails when configure private is issued PR1055178

  • On Trio-based line cards, in forwarding topology that has unilist next-hop and the unilist next-hop is pointing to indirect next-hop or another unilist next-hop, if there is a change happens on one of the downstream paths, the change may fail to be propagated to the unilist next-hop. As a result, the unilist selector table is not updated. This might lead to traffic blackholing. PR1056150

  • While using certain 14.1 daily builds and the JAM package users might notice erroneous outputs in the fields of Domain ID and FlowSet ID. This issue is fixed in Junos 14.1R4. PR1057450

  • Under very rare situations, Packet Forwarding Engines on the following linecards, as well as the compact MX80/40/10/5 series, may stop forwarding transit traffic: - 16x10GE MPC - MPC1, MPC2 This occurs due to a software defect that slowly leaks the resources necessary for packet forwarding. Interfaces handled by the Packet Forwarding Engine under duress may exhibit incrementing 'Resource errors' in consecutive output of 'show interfaces extensive' output. A Packet Forwarding Engine reboot via the associated linecard or chassis reload is required to correct the condition. PR1058197

  • With the configuration "extend-size", if user loads and commits scaled configuration (in this case, 250K Unique Prefix list policy options), then deletes the knob "extend-size", the dfwd process might crash. PR1058579

  • After committing the Network Time Protocol (NTP) configuration, if the number of routing-instances per source-address exceeds 18, it may cause NTP daemon (ntpd) crash. In this scenario, the NTP feature may not be functional. For example there are 19 routing-instance names per source address statement in the sample configuration below. ntp { server X.X.X.X; source-address X.X.X.X routing-instance [ X1 X2 X3 X4 X5 X6 X7 X8 X9 X10 X11 X12 X13 X14 X15 X16 X17 X18 X19 ]; (19 routing-instance names) } PR1058614

  • On MX Series line cards with MPCs and MICs with Junos OS Release 12.3R3 and later, the system does not push the configured Tag Protocol ID (TPID) value (for instance, 0x88a8) to the packets while sending out the packets; instead it pushes default TPID 0x8100. This might lead to traffic drop on the peer device if it is expecting a particular TPID (for instance, 0x88a8) whereas it receives a different one. PR1059225

  • Modifying IEEE-802.1ad rewrite-rule on the fly might make it unable to change IEEE-802.1p ToS values for inner VLAN in QinQ. PR1062817

  • When MX platform acts as Virtual Extensible Local Area Network (VXLAN) gateway, if there are multiple Packet Forwarding Engines, VXLAN packets will be distributed to available Packet Forwarding Engines in the chassis to perform VXLAN encapsulation/decapsulation, this is not expected (Expect behavior: VXLAN packet processing will be done on the same Packet Forwarding Engine on which it is received). This might result in unexpected packet drop and also overlay ping/traceroute not working. PR1063456

  • Observation domain ID in exported flow records is incorrect in MPC 3E cards. and 200G 40x10G MPC and 200G 4x100G MPC for the MX Series. PR1066319

  • On MX Series routers with MPCs and T4000 routers with Type 5 FPCs, the feature "enhanced-hash-key" is configured to select data used in the hash key for enhanced IP forwarding engines. If "type-of-service" is configured at the [edit forwarding-options enhanced-hash-key family inet] hierarchy level, or "traffic-class" is configured at the [edit forwarding-options enhanced-hash-key family inet6] hierarchy level, the last significant 2 bits of the TOS/TC bytes under the IPv4/IPv6 header are extracted incorrectly as load sharing input parameters, this might cause unexpected load balancing result. PR1066751

  • StartTime and EndTime of the flow in inline-jflow (version 9) has future time-stamp PR1067307

  • Firewall filters that have a prefix-action cannot be configured under [edit logical-system <name> firewall family inet] because the Packet Forwarding Engine won't be programmed for the filter. PR1067482

  • An FPC with interfaces configured as part of an Aggregated Ethernet bundle may crash and reboot when the shared-bandwidth-policer is configured as part of the firewall policer. PR1069763

  • If with about 1M routes on MX series router, there might be more than 1 second (about 1.3s) packets dark window during unified ISSU. PR1070217

  • On MX series routers, when using MX Series based FPC with feature inline sampling activated, memory partition error messages and memory leak might be observed on the FPC. In some cases, this issue only affects sample route-records but not regular Packet Forwarding Engine routes or next-hops. However, in the extreme case, it is also possible to cause the Packet Forwarding Engine failing in installing routes into forwarding next-hops and hence traffic drop. On MX series routers, when using MX Series based FPCs, Junos OS 13.3R5 14.1R4 14.2R1 or higher is exposed. On T4k or TXP-3D routers, when using FPC-3D FPC's, Junos OS 14.2R1 or later is exposed. PR1071289

  • VPLS filter applied under forwarding-options might drop VPLS frame unexpectedly when it's coming from an lt- interface. PR1071340

  • When inline-sampling is enabled, in race conditions, if packet gets corrupted and the corrupted packet length shows 0, which may cause "PPE_x Errors thread timeout error" and eventually cause MPC card to crash. PR1072136

  • After IPv6 RPM(real-time performance monitor) support, snmp server cannot receive some of IPv6 PING-MIB info. For example, snmp server receives "pingCtlRowStatus(23)" and "pingCtlAdminStatus(8)" error and cannot get "pingResultsTable" and "pingProbeHistoryTable" info. << example >> ** The following logs are snmp server logs. "snmpset -v 2c -c xxxxxx" commands are used. ----pingCtlRowStatus(23) error info. Error in packet. Reason: inconsistentValue (The set value is illegal or unsupported in some way) Failed object: SNMPv2-SMI::mib-2.80.1.2.1.23.7.79.87.78.69.82.95.65.6.84.69.83.84.95.65 ---pingCtlAdminStatus(8) error info. Error in packet. Reason: inconsistentValue (The set value is illegal or unsupported in some way) Failed object: SNMPv2-SMI::mib-2.80.1.2.1.8.7.79.87.78.69.82.95.65.6.84.69.83.84.95.65 ** The following logs are snmp server logs. "snmpwalk -v 2c -c xxxxxx" commands are used. pingResultsTable(3) SNMPv2-SMI::mib-2.80.1.3 = No Such Object available on this agent at this OID pingProbeHistoryTable(4) SNMPv2-SMI::mib-2.80.1.4 = No Such Object available on this agent at this OID PR1072320

  • Problem: MAC filter ff:ff:ff:ff:ff:ff is cleared from the Packet Forwarding Engine hardware mac table. So arp requests are not forwarded to irb. Fix: Not all mac entries pointing to invalid l2 token are candidates for being deleted. Static mac entries are managed by control plane only. So Packet Forwarding Engine cannot delete these entries. The logic for skipping mac deletion for static mac entries done earlier is not proper Packet Fowrding Engine. Fixed the same. PR1073536

  • When an MX Series chassis network-services is "enhanced-ip" and an aggregated Ethernet with "family bridge" configuration is first committed, there is a possibility that an incorrect forwarding path may be installed causing traffic loss. PR1081999

  • LMEM is an internal memory in LU/XL ASIC chip. It has private and shared regions for Packet Processing Engines. LMEM data errors are very rare events caused by environmental factors (this is not created by software). Due to a software defect, an error in the shared LMEM region will result in corruption of critical data structures of Packet Processing Engines that causes unpredictable communication of LU/XL ASIC chip with MQ/XM ASIC chip. These events will corrupt the state in MQ/XM and lead to a MQ/XM wedge. The MQ/XM wedge would cause fabric blackhole and finally reboot the line card. PR1082932

  • On MX Series router with MPCs/MICs, the "RPF-loose-mode-discard" feature is not working when configured within a Virtual Router routing instance. The feature is working only when configured in the main instance. PR1084715

  • With MX Series based FPC, load balance hash seed will be changed after ISSU. Since the hash seed value will be reverted to original value by rebooting FPC, there would be hash value inconsistency in the system which might introduce blackholing on multicast flavor traffic (mcast or BUM on vpls/l2-bridge). Affected versions (Other versions do not have the issue) 12.3R7, 13.1R5 and later, 13.2R4 thru 13.2R5, 13.3R2 thru 13.3R3, 14.1R1 and later, 14.2R1 and later. PR1086286

  • The prompt for SSH password changed in Junos OS 13.3, from "user@host's password:" to "Password:". This change breaks the logic in "JUNOS/Access/ssh.pm" which is located in /usr/local/share/perl/5.18.2/ on Ubuntu Linux, for example. PR1088033

Routing Protocols

  • If with both BGP Prefix-Independent Convergence (PIC) edge and "routing-options multipath" configured, when the primary path fails, the protection provided by BGP PIC edge might not work correctly. PR1011596

  • If with BGP PIC edge feature enabled and OSPF protocol as IGP, when the primary route changed, there is a chance that the Packet Forwarding Engine forwarding entry will stay in reroute state which causes session down. PR1015598

  • When BGP add-path feature is enabled on BGP route-reflector (RR) router, and if the RR router has mix of add-path receive-enabled client and add-path receive-disabled (which is default) client, due to a timing issue, the rpd process on RR might crash when routes update/withdraw. PR1024813

  • RIP is applying the RIB import-policy for the primary RIB table, as per the policy configured evaluation fails and routes are removed from primary RIB. But import-policy is applied only for secondary tables. RIP should apply only the protocol import policy and add routes to primary RIB. Routes are leaked to secondary routing table according to import-policy. Fix: As suggested by rpd infrastructure team , removed the import policy filter application to primary routing table by protocol rip. Now import policy application is handled by policy module within RPD. PR1024946

  • When a BGP peer goes down, the route for this peer should be withdrawn. If it happens that an enqueued BGP route update for this peer has not been sent out, issuing the CLI command "show route advertising-protocol bgp <peer-addr>" might crash the routing protocol process (rpd). This is a very corner issue and hardly to be experienced. PR1028390

  • If precision-timers and traceoptions are enabled for BGP then both main-thread and precision-timers pthread try to rotate the same tracefile without taking any locks. As a result all the status commands for rpd and krt may timed-out. PR1044141

  • If labeled BGP routes are leaked from inet.3 table to inet.0, then activation of BGP "add-path" feature might crash the routing process (rpd). PR1044221

  • BFD session might reset on commit if version is configured. The adaptive RX interval gets set to 0 which results in the reset. A sample configuration of BFD version is as following: protocols { bgp { bfd-liveness-detection { version 1; minimum-interval 1000; transmit-interval { minimum-interval 1000; } } } PR1045037

  • When BGP and ICCP are the client of the same multi-hop BFD session, BFD runs in centralized (non-distributed) mode. But if nonstop-routing configuration is added and enabled, runing mode of BFD is changed to distributed mode. This behavior is incorrect but it would not affect to protocols which is client of the BFD session. However, if Routing Engine switchover is performed after enabling NSR, the BFD session will get unstable and all the client protocols also get unstable. PR1046755

  • The Junos OS Multicast Source Discovery Protocol (MSDP) implementation is closing an established MSDP session and underlying TCP session on reception of source-active TLV from the peer when this source-active TLV has an "Entry Count" field of zero. "Entry Count" is a field within SA message which defines how many source/group tuples are present within SA message. PR1052381

  • Either "rib inet.3" or "resolve-vpn" feature is available to be configured in the lower hierarchy for BGP labeled-unicast family routes. These two features are mutually exclusive and only one of them could be used at a single BGP group. If the administrator swaps the two features (for example, using the "resolve-vpn" first, then deactivate it and using "rib inet.3" instead, then use "resolve-vpn" back), the secondary routes (routes in inet.3 which including the ones from this BGP group and from other BGP groups) may got accidentally removed every time on "commit" operation take place. PR1052884

  • After deactivating/deleting BFD configuration, Packet Forwarding Engine receives BFD session down event and it marks corresponding nexthops as down and traffic drops consequently. PR1053016

  • The BGP session sending add-path prefixes can cause an rpd crash when the add-path IDs that it allocates roll over from 65535 to 0. If the routes contributing add-path prefixes are changing, the allocated path-id can eventually reach this value. This fix changes the allocation scheme to always use the lowest available free path-id, so a rollover will never occur. PR1053339

  • The routing protocol process (rpd) might crash when static reverse-path forwarding (RPF) selection is configured and the upstream interface in the VRF routing instance disabled. PR1054913

  • After multicast traffic source incoming interface and source ip RPF (reverse path forwarding) route switching to a different interface, the multicast route cache upstream interface might not be refreshed to be in sync with the pim join upstream interface. This is incorrect and will cause packet blackhole for the affected multicast stream. PR1057023

  • Deletion of a routing-instances may lead to a routing daemon crash. This may happen if routing-instance's Routing Information Bases (RIB) is referenced in an active policy-option configuration. As a workaround, when deactivating the routing-instance, all associated configurations using the route-table names in the routing-instance should also be deactivated. PR1057431

  • When running Simple Network Management Protocol (SNMP) polling to specific ISIS Management Information Base (MIB) with invalid variable, it will cause routing protocol process (rpd) crash. PR1060485

  • In PIM environment, Bootstrap Router (BSR) can be used only between PIMv2 enabled devices. When deactivating all the interfaces which are running PIM bootstrap, the system changes to operate in PIMv1. At this time, all the information learned about/from the current BSR should be cleaned, but actually, BSR state is not cleaned. If the interface which was the previous "elected BSR" is activated, BSR state is PIM_BSR_ELECTED(should be cleaned previously) and the system assumes the BSR timer is still here. When the system tries to access the null BSR timer, the rpd process might crash. PR1062133

  • In Protocol Independent Multicast (PIM) sparse mode environment, in the situation that the router is being used as the rendezvous point (RP) also the last hop router, when the (*,G) entry is present on the RP and a discard multicast route (for example, due to receiving multicast traffic from non-RPF interface) is already existed, if the (S,G) entry is learnt after receiving source-active (SA) of the Multicast Source Discovery Protocol (MSDP), the SPT cutover may fail to be triggered. There is no traffic impact as receivers still can get the traffic due to (*,G) route. PR1073773

  • In an MPLS L3VPN Core network, enable BGP Prefix-Independent Convergence (PIC) Edge feature on a PE router, if the same VPN route is received with different multiple exit discriminator (MED) via two route reflectors (RR), when BGP PIC evaluates those two routes, it disregards the one with higher MED hence fails to build a multipath protection/backup path entry. PR1079949

  • When removing scale BGP configuration, if the BGP session are holding stale routes for the benefit of a restarting peer, the routing protocol process (rpd) may crash. As a workaround, the administrator may use CLI command "show route receive-protocol bgp <peer address> extensive | match STALE" to find the existing stale routes. If there are none, then removing the BGP configuration may not cause the rpd crash. PR1081460

  • If a policy statement referred to a routing-table, but the corresponding routing instance is not fully configured (ie. no instance-type), commit such configuration might cause the rpd process to crash. PR1083257

  • With Multicast Source Discovery Protocol (MSDP) and nonstop active routing (NSR) configured on the Protocol Independent Multicast (PIM) sparse-mode rendezvous point (RP), the rpd process might permanently get stuck when multicast traffic received shortly after Routing Engines switchover. PR1083385

  • When there are a number of secondary BGP routes in inet.0, an SNMP walk of inet.0 by the bgp4 MIB can cause a core if the corresponding primary routes are being deleted. PR1083988

  • When BGP route is leaked to a routing-instance and there is an import policy to overwrite the route preference, if damping is also configured in BGP, the BGP routes which were copied to second table can't be deleted after routes were deleted in master table. This is a day-1 issue. PR1090760

Routing Policy and Firewall Filters

  • When configuring the unsupported IPv6 flow specification feature, that is, when configuring inet6 address as source/destination of inet-flow route, the configuration can pass the commit check and being committed. But it can cause rpd process crash eventually when trying to program this route to firewall process (dfwd, which manages compilation and downloading of Junos firewall filters). If a flow route is received from a BGP neighbor and prefix-length for source/destination is greater than 32, it can lead to rpd process crash too. PR1059542

Services Applications

  • On M/MX/T Series routers (platforms) with Services PIC, the incoming interface is a services interface. If the services interface receives "ICMP MTU Exceeded" message, the message might be dropped. PR977627

  • Added support to bring up Tunnel-switched sessions when tunnel-group is not configured at LTS and tunnel attributes are returned from RADIUS. PR1030799

  • When using both Port Control Protocol (PCP) and traditional NAT (e.g. DSLITE), if you try to setup two pools with overlapping address ranges, this can lead to MS-DPC to crash and generate a core file. PR1036459

  • On M/MX/T Series routers with Multiservices 100, Multiservices 400, or Multiservices 500 PICs with "dump-on-flow-control" configured, if prolonged flow control failure, the coredump file might generate failure. PR1039340

  • Inline IPv6 L2TP on MPC subscriber terminated at a LNS breaks adaptive services SP unicast nexthops on MS-DPC. Even one subscriber causes the issue. PR1054589

  • When the tunnel between L2TP access concentrator (LAC) and L2TP network server (LNS) is destroyed, the tunnel information will be maintained until destruct-timeout expire (if the destruct-timeout is not configured, the default value is 300 seconds). If the same tunnel is restarted within the destruct-timeout expire, the LNS will use the previously negotiated non default UDP port, which might lead to the tunnel negotiation failure. PR1060310

  • A Layer 2 Tunneling Protocol daemon (l2tpd) crash is seen sometimes when the L2TP service interface unit number is configured higher than 8192. A restriction has been added to force unit numbers below 8192. PR1062947

  • On MX Series routers which are acting as LNS to provide tunnel endpoints, it is observed that the service-interfaces are not usable if an MIC corresponding to them is not physically installed on the FPC. If only those service interfaces that belong to the removed PIC are added to service-device-pool, this results in no LNS subscribers able to login. Note that once the MIC is inserted into the FPC, the features could be used. PR1063024

  • When configuring RADIUS authentication for Layer 2 Tunneling Protocol (L2TP), the RADIUS server cannot be recognized because the source address is not being read correctly. As a result, the L2TP session cannot be established. PR1064817

  • L2TP daemon will core in LTS scenario while the subscriber logs out. This happens when the subscriber has "Called Number AVP" attribute. The "Called Number AVP" was not getting relayed correctly across LTS boundary, hence daemon cores. PR1065002

  • Service PIC daemon (spd) might crash with core-dumps due to CGNAT pool's snmp-trap-thresholds configuration. PR1070370

  • In CG-NAT or stateful firewall environment, due to a null pointer check bug, the MS-DPC might crashed every few hours. Note that this is a regression issue. PR1079981

  • The crash happens if in a http flow, the flow structure is allocated at a particular memory region. There is no workaround but the chances of hitting this issue are very low PR1080749

  • On Layer 2 Tunnel Protocol (L2TP) network server (LNS), during L2TP session establishment, when receiving Incoming-Call-Connected (ICCN) messages with Last Sent LCP CONFREQ Attribute Value Pair (AVP) but without Initial Received LCP CONFREQ and Last Received LCP CONFREQ AVPs, the jl2tpd process might crash. PR1082673

  • In a L2TP tunnel-switching scenario, if a tunnel-switched tunnel is cleared with "clear services l2tp tunnel peer-gateway" AND an incoming ICRQ is received simultaneously from the LAC side destined for this tunnel-switched tunnel, this leads to jl2tpd crash. This defect has now been rectified. PR1088355

Software Installation and Upgrade

  • Due to a software defect found in 14.1R5.4, Juniper Networks strongly discourage the use of Junos software version 14.1R5.4 on routers with MQ-based MPC. This includes MX Series with MPC1, MPC2; and all mid-range MX-Series. PR1108826

Subscriber Access Management

  • This issue was introduced as part of another fix. Please contact JTAC for the recommended release for your deployment. PR1049955

VPNs

  • For VPLS over VPLS topology, when the VPLS payload has two labels (Customer-VPLS-label and Customer-MPLS-label), the frame might be dropped by the core facing interface hosted on IQ2 PIC with "L2 mismatch timeout" error. This particular scenario is fixed. But there are some other worse scenarios which might hit this issue again due to the system architecture limitation, which are not fixed but need to avoid: * Addition of VLAN tags on Service provider's or CE's VPLS payload e.g. configuring QinQ. * Addition of MPLS tags on Service provider or CE's VPLS payload. * Enabling VPLS payload load balancing on Service provider's PE router. PR1038103

  • In next-generation MVPN, after the route to C-RP flaps, traffic loss might be seen for a short period of time. PR1049294

  • In next-generation MVPN scenario, when a source is directly connected to a PE that is acting as an RP stops sending the traffic, the PE never withdraws the Type 5 route. This causes the Type 7 routes and forwarding routes to remain on the egress and ingress PEs. PR1051799

  • In L2VPN scenario with local switching enabled, in corner cases, the rpd process might crash after flapping the PE-CE link. For example, if the L2VPN connection type changes from remote to local after link flaps, for a brief period of time, two route entries (for old remote VC connection and for the new local VC connection) might exist for the same egress route (with interface name as destination prefix). In that case, when deleting remote VC connection and route entry associated with that remote connection, the rpd might crash due to trying to reset an internal variable which is already reset during route addition for the new local VC connection. PR1053887

  • In the l2circuit environment, when l2ckt configuration has backup-neighbor, the flow-label operation is blocked at the configuration level. PR1056777

  • With static selective point-to-multipoint LSP configured for an MBGP MVPN, when sending Type 3 S-PMSI A-D BGP route, the Juniper Networks implementation uses the values taken from the selective P-Tunnel configuration, which is not compliant with RFC 6514 section 4.3, which specifies that the source and group length values in Type-3 must be the same as the host prefix length, that is, if the Multicast Source field contains an IPv4 address, then the value of the Multicast Source Length field is 32; if the Multicast Source field contains an IPv6 address, then the value of the Multicast Source Length field is 128. The same is true for group length. PR1058193

  • In a next-generation MVPN scenario, while traffic is not being generated by source for at least 3 and a half minutes and a routing or other multicast issue prevents the multicast traffic from reaching the receiver PE, after the multicast data starts flowing again for about 6 minutes, the Type-7 and Type-5 routes might be withdrawn which causes a discard route to remain present on the RP facing PE and causes the traffic not to be forwarded even if there is state and flowing traffic for that group. PR1058574

  • The rpd process might crash when deactivating a logical system with nonstop active routing (NSR) enabled and BGP multicast virtual private network (MVPN) configured. PR1059057

  • In MVPN RPT-SPT mode, with a mix of local and remote receivers all using (*,g) joins (spt-threshold infinity), the downstream interfaces may not get updated properly and there may be a stuck (s,g) forwarding route. This issue can occur with the following sequence of events: 1. Local receivers are joined 2. Traffic starts, then stops, and the route times out. 3. Remote receiver joins. Both a (*,g) and an (s,g) forwarding route are created. 4. Another local receiver is joined, or an existing one is pruned. 5. In the (*,g) route the downstream interface list reflects the update, but in the (s,g) route the downstream interface list does not. 6. When traffic starts again, the (s,g) route -- which has the wrong interface list -- is used. The traffic flows to the wrong set of receivers. PR1061501

  • Problem, trigger and symptom: On a dual Routing Engine, if mvpn protocol itself is not configured, and nonstop active routing is enabled, the show command "show task replication" on master Routing Engine will list MVPN protocol even though it is not configured. Other than the misleading show output which may be slightly confusing to the user/customer, there is no functional impact due to this issue. There is no workaround available. PR1078305

Resolved Issues: 14.1R4

Class of Service (CoS)

  • For an ATM interface configured with hierarchical scheduling, when a traffic-control-profile attached at ifd (physical interface) level and another output traffic-control-profile at ifl (logical interface) level, flapping the interface might crash the FPC. PR1000952

  • Sometimes MX Series might respond with "no such instance" of the second OID when two CoS OIDs in the single SNMP packet. PR1015342

  • This issue specific to rate-limit on trunk port in DPC due to a software issue that installing rate-limit variables to egress Packet Forwarding Engine does not work normally. PR1022966

  • For ichip based platform, IQ2 pic expects FC index in the cookie from ichip for packet queuing. For Transit traffic, fc index is coming in cookie where are for host outbound traffic, queue number is coming in cookie to IQ2 pic. As IQ2 pic is not aware whether traffic is transit or host outbound, it treats value received in cookie as FC value and looks into fc_to_q table to fetch queue number. This is causing issue in queueing of host outbound traffic in IQ2 PIC in incorrect queue. This is a day one issue and will come if in FC to Queue mapping, fc id and queue number are not same. PR1033572

  • This error message "only per-unit and 2-level hierarchical scheduler are supported on this interface" is a cosmetic regression issue without any functional impact. PR1050512

General Routing

  • show services accounting usage does not populate CPU utilization for XLP-based cards. Use show services service-sets cpu-usage. PR864104

  • On MX Series platform with enhanced DPCs equipped, after router rebooted, the IRB broadcast channel is not enabled, and all the broadcast packets that are received in the IRB interface will get dropped. Also when ping is given the below L2Channel error increases as ping packets are sent: user@router>show interfaces ge-*/*/* extensive | match channel L3 incompletes: 0, L2 channel errors: 10, L2 mismatch timeouts: 0 PR876456

  • When mirror destination interface is a next-hop-subgroup and enhanced-ip chassis knob is enabled, family any mirroring applied on L3 interfaces ( inet/inet6 ) might not work in certain scenarios. PR972138

  • In the dual Routing Engines scenario with large scale nexthops (in this case, more than 1-million nexthops and around 8K VRFs). In rare condition, kernel might crash on backup and/or master Routing Engine due to exhaustion of nexthop index space. PR976117

  • On MX Series routers, delete an interface A from routing-instance VRF1; then create routing-instance VRF2 and interface A is added to VRF2 with qualified-next-hop configured; finally, delete VRF1. Commit the entire above configuration once, in rare condition, rpd might crash. PR985085

  • In the dual Routing Engines scenario, in rare condition, while executing GRES and deleting interfaces at the same time, it is possible that a nexthop delete message is not sent to rpd process, causing rpd to keep a nexthop index (NHID) that kernel has already deleted. Later when kernel allocates this NHID for next new nexthop and sends it to rpd process, rpd process might crash due to duplicate NHID. PR987102

  • An EVPN with support for inter-subnet routing using an irb interface may experience a crash and restart of rpd, leaving a core file for analysis. In this case, EVPN MAC routes contain MAC+IP, and this IP/32 is installed in VRF table on egress router. Core is triggered in the IP/32 route installation flow. There is no special trigger point- it's a timing issue with basic irb configurations. PR992059

  • In Ethernet VPN (EVPN) routing and bridging (IRB) deployment, when all the access interfaces go down under an EVPN bridge domain, the IRB interface in the bridge domain remains up and causing the issue of IRB subnet remaining being advertised in L3 routing which in turn attracts all L3 VPN traffic for the subnet. PR994909

  • MX960/480/240 fantray red alarm temperature changed from 75C to 80C. PR995225

  • In the dual Routing Engines scenario with NSR configuration, backup peer proxy thread is hogging CPU for more than 1 second if there are multiple updates (>5000) going from master Routing Engine to backup Routing Engine. This leads to FPC socket disconnections. The traffic forwarding might be affected. PR996720

  • On MX104 router with SONET/SDH OC3/STM1 (Multi-Rate) MIC. In rare condition, if the MIC is plugged out from MX104, the Packet Forwarding Engine might crash, the traffic forwarding will be affected. These MICs as below belong to SONET/SDH OC3/STM1 (Multi-Rate) MIC: * MIC-3D-8OC3OC12-4OC48 * MIC-3D-4OC3OC12-1OC48 * MIC-3D-8CHOC3-4CHOC12 * MIC-3D-4CHOC3-2CHOC12 * MIC-3D-8DS3-E3 * MIC-3D-8CHDS3-E3-B * MIC-3D-1OC192-XFP PR997821

  • If encapsulation type is "ppp" for the SONET interface on IQE PIC, sometimes the MTU change might not work. PR1001880

  • If the connection with an OpenFlow controller goes down then comes back up repeatedly, an OpenFlow interface on a QFX5100 switch might send an OFPT_ERROR packet with an XID ID 0 but no data to explain why the error packet was sent. PR1003538

  • On TXP with GRES enabled, when performing graceful switchover on all chassis (include line-card chassis (LCC)) from master Routing Engine to backup Routing Engine, minimal IPv4 traffic loss around 0.04% to 0.05% will be observed on aggregated 100GE PIC on FPC type 4. PR1014420

  • If the service option configured on aggregated Multiservices (AMS) interface is different from its member interface, conflict would happen which might cause some serious issue. After this fix, service-options configuration (which includes timeouts/sessios-limit etc.) should only be configured on all members interfaces when configure AMS bundle. PR1014898

  • Under corner cases, if there are multiple back-to-back Virtual Chassis port (VCP) related CLI commands, Network Processing Card (NPC) core may be observed and FPC hosting the VC ports might reboot. PR1017901

  • Enabling sampling on an ms- interface is not supported configuration, if 'forwarding-opions sampling sample-once' is subsequently deactivated the FPC may reboot. PR1021946

  • MQCHIP(0) mqchip_get_q_forwarded_stats() invalid q_sys 0 q_num messages continuously show in logs. It will cause two GE or XGE interfaces to not forward traffic. PR1021951

  • On Offline/Online cycle of a 40GE QSFP card, a 40GE Interface port's Physical Link might remain down. Few events which will result into the Offline/Online cycle of a 40GE QSFP card are router reboot, FPC reboot, or chassis-control restart or 40GE Card offline request followed by a 40GE Card online request. PR1026088

  • The host MPC might continuously crash when trying to online a faulty MS-MIC after discovering the hardware failure. PR1026310

  • Configuring a routing policy with the "no-route-localize" option to ensure that the routes matching a specified filter are installed on the FIB-remote Packet Forwarding Engines, after removing the routing policy and changing the next-hop for the routes, the previously installed routes using "no-route-localize" policy might not get removed from some Packet Forwarding Engines. Then the Packet Forwarding Engines will not forward received packets to the FIB-local Packet Forwarding Engines to perform full IP table lookup but using the staled routes instead. PR1027106

  • On MPC5E line card, if a firewall filter with large-scale terms (more than 1300 etc.) is attached to an interface, traffic drop might be seen. PR1027516

  • In a rare case, rdd core is reported under /usr/sbin/rdd as soon as applying the group and commit is performed. PR1029810

  • On MX Series platform with MS-MPC card, after performing switchover from master RE0 to backup RE1, 2 internal ARP entries for Routing Engine address (128.0.0.1) on MS-MPC PICs pointing to two eth interfaces connect to CB0 and CB1 separately might be wrongly created. Then if pull out RE0/CB0, the MS-PIC would still select the eth interface connects to CB0, which results in loss of connectivity because that path is not available anymore. PR1030119

  • In VMX, the speed of 10GE interface was not being displayed correctly in "show interface" command. This PR fix allows one to configure the speed on the interface. PR1031286

  • With an unrecognized or unsupported Control Board (CB), mismatch link speed might be seen between fabric and FPCs, which results in FPCs CRC/destination errors and fabric planes offline. Second issue is in a race condition, Fabric Manager (FM) might process the stale destination disable event but the error is cleared indeed, it will result in the unnecessary FPC offline and not allowing Fabric Hardening action to trigger and recover. PR1031561

  • This issue only affects OC-48 MICs. If an SFP is inserted into an OC-48 MIC port that has been disabled the SFP will not show up in a >show chassis hardware command. The issue is fixed with a patch. Contact JTAC to find out which version is best for you. PR1031851

  • The Software Development Kit (SDK) Service process (ssd), which runs on the Routing Engine and is responsible for communications between the SDK application and Junos OS, might crash after Routing Engine switchover and following reboot of both Routing Engines. Since the ssd acts as the broker daemon for Applications connecting to Juniper distributed application framework (JDAF) services, the applications will lose JDAF connectivity when ssd restarts. PR1031860

  • With VPLS BGP control word configured, intermittent packet loss might be seen in one direction on VPLS circuit due to the control-word not being programmed at Packet Forwarding Engine after member DPC reboot. The problem can happen on below conditions: 1. LSI interface exists across two or more physical interfaces. 2. Those physical interfaces located in different FPCs. 3. Those physical interfaces consist of equal-cost paths. So, LSI will not be flapped with one member FPC down. 4. Flap the member DPC where one of physical interfaces situated. PR1031863

  • In rare cases, the AUTHD daemon may crash and cause a corruption of subscriber dynamic profiles. In-use profiles may be incorrectly marked as not in use. Any subscribers that reference that profile are forced to remain in Terminating state, until the router is rebooted. Daemon restarts and GRES switches are ineffective in working around this situation. PR1032548

  • On the virtual MX (vMX) platform with high rate data (in this case, 50Mbps). In rare condition, the IPv6 Neighbor Discovery Protocol (NDP) packet might lose, the traffic forwarding will be affected. PR1035852

  • If an IFL is used as the qualified-next-hop (which implies the IFL has unnumbered-address configured), and there are changes in the IFL filter configuration, then the static route might disappear from routing table. To make it reappear, need to delete it from the configuration and add it back. PR1035598

  • Somtimes AE vlan ifl output byte counters are shown as large value and it is a generic issue. PR1036813

  • Using jnxoptIfOTNPMFECIntervalTable and jnxOpticsPMIntervalTable it is not possible to walk these tables from the middle before this fix. PR1039030

  • In a subscriber scenario with auto-sensed VLAN configured, after scaled subscribers (in this case, 16K subscribers) login/logout for several times, the subscriber management process might stuck and not able to restart due to a Session Database (SDB) deadlock issue. PR1041094

High Availability (HA) and Resiliency

  • Configuring the maximum segment size (MSS) for the TCP connection for BGP neighbors, if "mtu-discovery" and "path-mtu-discovery" knobs are removed, the default MSS value of 512 will be used instead. This is not an expect behavior. PR835220

Interfaces and Chassis

  • Refer to the following topology. If we set interface ge-1/0/8 disable, interface xe-2/0/0 and xe-2/1/0 become down status because "asynchronous-notification" feature. However after 3 or 4 seconds, ether OAM detects link-fault status changed to good. And then, interface xe-2/0/0 and xe-2/1/0 change link status from down to up. The condition is the following. 1. Configure MPLS circuit with ether CCC. 2. Configure "asynchronous-notification" on CE facing interface in both PEs. 3. Configure ether OAM to one of PE, CE pair. 4. Use DPC 10 giga-interface on DTU. * This behavior did not occur with MPC and DPC 1 giga-interface. << topology >> ********************************************************************* local link remote link DPC 10ge | xe-2/0/0 V ge-1/0/6 ge-1/0/8 [ CE ]----------[ PE ]---------[ PE ]----------[ CE ] xe-2/1/0 ge-1/0/7 ge-1/0/9 (DTU) <--------> <-------> <--------> ether CCC MPLS ether CCC asynchronous-notification asynchronous-notification <--------> ether OAM *CE:MX240 PE:MX240 ********************************************************************* PR973840

  • IS-IS Adjacency may flap after unified ISSU. This behavior is being further analyzed and is planned to be fixed in further releases. PR1015895

  • On 10GE interface on MIC (e.g. 3D 4x 10GE XFP and 3D 2x 10GE XFP MIC), when "link-down" event under "optics-options alarm low-light-alarm" is configured and the "hold-time down" timer is set greater than 0, the status of the interface will remain up, even when the light power exceed low alarm threshold and traffic being interrupted. PR1018076

  • With vrf-table-label configured on the routing-instances, when a FPC with Enhanced IQ (IQE) PIC is sharing the same Forwarding Engine Board (FEB) with another FPC, and the FEB has two core-facing interfaces configured with the family mpls on aforementioned FPCs separately, the Label-Switched Interface (LSI) interfaces might be removed incorrectly on the working FPC when the other FPC with IQE PIC is set to offline. PR1027034

  • if DPCE 20x 1GE + 2x 10GE X card is present in the chassis, BFD sessions over AE interfaces may not be distributed PR1032604

  • With heartbeat connection for an MX Series Virtual Chassis (MX-VC) enabled, if the heartbeat connection detects that the Virtual Chassis master router (VC-M) is still operating and able to respond during a split caused by a failure of all the Virtual Chassis port (VCP) interfaces, the Virtual Chassis backup router (VC-B) should go offline after the heartbeat timeout period expires. But VC-B retains VC backup role and never go offline although its FPCs went into PRESENT state. In addition to fix the deviation from the expected functionality, the output of CLI command "show virtual-chassis heartbeat [detail]" is enhanced to more clearly indicate the successful detection of the peer MX-VC member chassis over the heartbeat connection when the chassis loses all VCP adjacency links. A unique "detected" state is provided when MX-VC splits and last heartbeat pulse response is successfully received. PR1034096

  • Some duplicate entries are reported in jnx-chas-defines.mib. This patch removes the duplicate entries to fix the issue. PR1036026

  • FRR switching time is much higher than 50ms (e.g. might be 400-900 ms) when protected links are located on MX Series Gigabit Ethernet enhanced and hardened MICs (i.e.MIC model name end with -E or -EH, currently, the supported MICs are MIC-3D-20GE-SFP-E and MIC-3D-20GE-SFP-EH). PR1038999

  • For Ethernet OAM/CFM, if Maintenance-association (MA) ICC format name of length less than 13 characters (13 byte) is used, deactivate/activate of ’protocol oam' may cause CFM operation failures. 'Cross-connect CCM received’ alarm will be seen. There can be other triggers also. ITU CARRIER CODE format uses fix length size for MA NAME (13 octets). Junos OS creates and maintains actual size configured by user. However the length it maintains is 13 octets. For lower size MA name the value accessed is not deterministic. It would work fine if the subsequent memory is initialized to zero. Else would declare cross connect error as the accessed MA name will be different compared to remote end. PR1041482

Layer 2 Features

  • If a customer is using SNMP and performs an snmpwalk on the dhcp binding table, not all of the entries may be displayed. This fix resolves that issue so that bindings for all ip addresses are displayed. PR1033158

MPLS

  • Error "tag_icmp_route:failed to find a chain composite ahead of fwd nh" might be observed when doing traceroute. PR999034

  • When the size of a Routing Engine generated packet going over an MPLS LSP is larger than MTU (i.e. MTU minus its header size) of an underlying interface, and the extra bytes leading to IP-fragmentation is as small as <8 bytes, then that small-fragment will be dropped by kernel and lead to packet drop with kernel message "tag_attach_labels(): m_pullup() failed". For example - If SNMP Response with specific size fall into above mentioned condition then small fragment will be dropped by kernel and eventually the SNMP response will fail. PR1011548

  • TED link information of protocol from highest credibility level is used irrespective of the level at which CSPF is computing. i.e., cspf-metric in "show mpls lsp extensive" would have the sum of te-metric of IGP with highest credibility at each hop in ERO. This has been corrected and the cspf-metric will be sum of te-metric of current credibility at each hop. PR1021593

  • When configuring point-to-multipoint (P2MP) Label Distribution Protocol (LDP) label-switched paths (LSPs), the labels will never be freed even they are no longer needed. This could lead to the MPLS label exhaustion eventually. To clear the state, the rpd process will restart with core dumps. PR1032061

  • When a LDP enabled router receives a LDP label mapping message which includes an unknown TLVs with unknown and forward bit set, the unknown TLV will be re-advertised along with the LDP message to upstream LSR. However, due to merge issue, Junos appends these unknown TLVs multiple times during construction of label mapping message and will has a unknown TLV(0x0000) with length 0 among the appended unknown TLVs, thereby causing the LDP session with the peer that receives this message flapping. PR1037917

Network Management and Monitoring

  • jnxpic 380 and jnxpic 381 definitions has been added in the "mib-jnx-chas-define" file from 14.1R4 release. PR1036706

Platform and Infrastructure

  • With inline jflow enabled, when the flow is exported once and got reinserted, if the low 12 bits of the packet counter are zero (0x000), the packetDeltaCount counter might be incorrect in inline jflow records. There is no traffic impact but may impact billing. PR886222

  • When apply-groups are used in the configuration, the expansion of interfaces <*> apply-groups will be done against all interfaces during the configuration validation process, even if the apply-group is configured only under a specific interface stanza. PR967233

  • BFD session within default routing-instance are not coming up once inline-services pic is configured and fixed class-of-service forwarding-class is assigned. BFD session operating in no-delegate-processing are not affected. PR999647

  • On TX Matrix Plus routers or TX Matrix Plus routers with 3D SIBs, all the incoming interfaces on an FPC are deactivated when none of the fabric planes are functional. By default, the interfaces remain activated. You can enable the deactivation of interfaces by using the fpc-restart configuration statement at the edit chassis fabric degraded hierarchy level. PR1008726

  • On MX Series router with MPCs or MICs, with igmp-snooping enabled and a multicast route with integrated routing and bridging (IRB) as a downstream interface, a multicast composite nexthop is created with a list of L3 and corresponding L2 nexthops. In a rare corner case, the corresponding L2 nexthop to the L3 IRB nexthop is a DISCARD nexthop and will cause the FPC to crash. PR1026124

  • On MX Series router with MPCs or MICs, when the packets are queued for several seconds due to interface congestion and get aged, the ICHIP might not able to detect those aged packets and thus fail to drain the queue out, which results in the FPC showing CRC errors and going into wedge condition. PR1028769

  • MX Series router with MPCs and MICs might crash when trying to install the composite next-hop used for the next-hop-group configuration related to port mirroring of traffic over IRB to an LSI attached to VPLS instance for a remote host. PR1029070

  • For BFD over aggregated Ethernet (AE) interfaces on MX Series routers with MS-MPC that have configured the enhanced-ip option, the BFD distribution to Packet Forwarding Engine for AE interface might not happen. PR1031916

  • This check ( log message) has been added as part an enhancement in the JNH error report. For FC accounting on AE interface, ingress FC accounting is enabled on AE interface nexthops and egress FC accounting is enabled on AE child member next hops. While fetching stats for AE, both member child IFL and AE IFL stats are fetched and added for result. If ingress FC accounting is enabled on AE IFL, while fetching statistics for child member links this error trace is coming because of this newly added JNH error trace. The fix is to put a check to not call for child member FC statistics when egress accounting is not enabled on AE bundle. PR1032952

  • On MX Series router with MPC, when there is a congested Packet Forwarding Engine destination, the non-congested Packet Forwarding Engine destinations might experience an unexpected packet drop. PR1033071

  • When the 'enhanced-hash-key services-loadbalancing' feature is used by MX Series router with MPCs or MICs, load balancing of flows across multiple service PICs via the source-address across does not work when internal BGP (IBGP) is used to steer traffic to the inside service-interface. For example the operator will see on the stateful firewall that the same source-address has flows across multiple service interfaces. PR1034770

  • sa-multicast load sharing method under [chassis <> fpc <> pic <> forwarding-mode] is not working on 100GE interface on TRIO FPC. PR1035180

  • Presence of /8 prefix in two terms results in incorrect filter processing and unexpected behavior. PR1042889

  • In a scaled subscriber management environment, the output of CLI command "show subscribers" and its sub flavors might print more pages and has to be terminated by "Ctrl+c" or "q". But this was not closing the back end Session Database (SDB) connection properly. Over a period of time, this will cause inconsistency and the subscriber management infrastructure daemon (smid) fails to register and no new subscribers could connect. PR1045820

  • On T4000 and FPC Type 5-3D or TXP-3D platforms , BFD sessions operating in 100msec interval with default multiplier of 3 might randomly flap after the enhancements implemented via PR967013. BFD sessions with lower intervals of 100msec or higher intervals are not exposed. The internal FPC thread, monitoring the High Speed Fabric links had a run time of longer then 100msec. PR1047229

  • By default, after 16x 10GE MPC boards come up about 75% of queues were allocated to support rich queuing with MQ chip. Such allocation causes MQ driver software module to poll stats. Polling stats causes this rise in CPU usage. PR1048947

Routing Policy and Firewall Filters

  • In the BGP environment, if operator "!" exists in the regex for as-path, the commit operation fails. PR1040719

Routing Protocols

  • Under following combination of events: * graceful-restart is enabled and * bidirectional PIM is enabled and * rpd is restarted, and * multicast traffic for bidir rp group hits the box. Pim creates the discard route and this traffic is pruned. PR1019560

  • When BGP is doing path selection with default behavior, soft-asserts requests are introduced. If BGP route flap a lot, it need to do path selection frequently, because of which a great deal soft-asserts might be produced which will cause unnecessary high CPU and some service issues, such as SNMP can not respond and even rpd core. PR1030272

  • When policy LFA is being used and backup path selection is first based on the root-metric criteria, the root-metric should be taken from the link metric connecting source to the backup neighbor (the one-hop neighbor or a remote router such as an RSVP backup LSP tail-end router), but it is now taken from the shortest-path-first (SPF) metric from source to backup neighbor if root-metric highest is configured. In some topologies, if the two metrics are different, IS-IS might select incorrect backup next-hop. PR1031408

  • In distributed BFD (which is enabled by default), if the CLIENT session (for example BGP) flaps due to any reason, the multi-hop BFD session that comes Up after the flap would not be delegated to FPC. PR1032617

  • When "clear bfd session" is issued immediately(before the Poll - Final sequence is completed) post config check-in for interval change from higher to lower minimum-interval value, BFD sessions don't revert to lower interval. PR1033231

  • Issue in populating isisRouterTable values. Some entries are not filled correctly. This does not block/affect the functionality of IS-IS or other components. PR1040234

Services Applications

  • The show CLI command "service nat pool detail" always displays the Port range starting from 1024 even when privileged ports are used. PR1019783

  • The session-limit-per-prefix feature for the MX Series DS-Lite server does not take Softwire flow into account when calculating the flow limit. PR1023439

  • In Network Address Translation (NAT) scenario with Endpoint-Independent Mapping (EIM) configured on service PIC, when a new ICMP session is created which matches an existing EIM mapping, the service PIC might crash. PR1028142

  • For T Series or M320 router containing Dynamic Flow Capture (DFC) PIC (either a Monitoring Services III PIC or Multiservices 400 PIC), there are two issues for DFC feature. The first one is the value of "timeout-remaining" for some filters installed on the DFC pic are too huge. The second issue is for some filters, there won't be any flows to which they are attached when forwarding traffic to the content-destination during random DTCP ADDs. PR1029004

  • When NAT has multiple terms that refer to the same NAT Pool, the command 'show snmp mib walk jnxSvcsMibRoot ascii' always print out jnxNatPoolTransHits for the count of jnxNatRuleTransHits in the first term. PR1035635

  • The cause of the KMD crash is not known. This is not due to SA(Security Associations) memory corruption. The code looks that SA is getting freed without clearing the table entry. PR1036023

  • In the context of DS-Lite softwire scenario, the MS-PIC/MS-DPC might crash in rare occasions when the Dual-Stack Lite (DS Lite) softwire concentrator receiving a high volume of outer IPv6 fragmented packets. PR1044143

VPNs

  • Problem Description The problem is that MSDP is periodically polling PIM for S,G's to determine if the S,G is still active. This check helps MSDP determine if the source is active and therefore the SA still be sent. There is a possibility that PIM will return that the S,G is no longer active which causes MSDP to remove the MSDP state and notify MVPN to remove the Type 5. One of the checks PIM makes is to determine if it is the local RP for the S,G. During a re-configuration period where any commit is done, PIM re-evaluates whether it is a local RP. It waits until all the configuration is read and all the interfaces have come up before making this determination. The local rp state is cleared out early in this RP re-evaluation process, however, which allows for a window of time where the local RP state was cleared out but it has not yet been re-evaluated. During this window PIM may believe it is not the local rp and return FALSE to MSDP for the given source. If MSDP makes the call into PIM during this window after a configuration change(commit), then it is possible that the Source Active(Type 5) state will be removed. Fix The fix will be to clear out the local rp state right before it is re-evaluated ie after it reads configuration for all interfaces; to not allow any time gap where it could be inconsistent. PR1015155

  • On MX-VC platform, if with scaled number of MVPN routes, after adding a new interface in the MVPN instance or changing the traceoptions related configuration, the CPU on Routing Engine might experience a high utilization for about 10min. PR1027596

  • Selective provider tunnel might flap few seconds after Routing Engine switchover, type 3 & 4 routes also refreshed, traffic fall to inclusive provider tunnel for a while PR1049352

Resolved Issues: 14.1R3

Class of Service (CoS)

  • SNMP get-request for OID jnxCosIngressQstatTxedBytes (ingress queue) might return the value of jnxCosQstatTxedBytes (egress queue). But SNMP walk works fine since it uses get-next-request. PR1011641

  • Sometimes MX Series routers might respond with "no such instance" of the second OID when two CoS OIDs in the single SNMP packet. PR1015342

Forwarding and Sampling

  • On the 32-bit Junos OS, when a very big burst-size-limit value (2147492676 and above) is configured in the ingress interface policer, the kernel may drop Routing Engine destined traffic. PR1010008

  • Deactivating Inline Jflow configuration doesn't make memory release normally. PR1013320

  • When an ARP policer is applied to an interface, it appears commented out in the configuration with the following message: "invalid path element 'disable_arp_policer”. PR1014598

  • On MX Series routers with MPCs/MICs, if layer 2 hierarchical policer is configured, upon committing it, the firewall daemon (dfwd) might crash. PR1015190

  • Remote vtep interface is not created despite sending traffic from inter segment, after vtep router reboots or chassisd is restarted. It causes dropping packets. PR1016446

  • When a TRIO specific filter is configured on an interface located on a DPC, the filter is not being installed and no warning message is logged on the message log file. PR1022836

  • Adding "fast-lookup-filter" knob to a firewall filter using one or more terms with "next-term" action could cause dfwc crash during commit (commit check phase). Hence because of this bug, this disallows use of "fast-lookup-filter" feature on firewall filters with terms using "next-term". PR1029761

General Routing

  • On TXP/TXP-3D platform, a bad I2C device on SFC Switch Interface Board (SIB) might cause Switch Processor Mezzanine Board (SPMB) to crash and all SIBs unable to online. PR846679

  • A few particular sequence of member failures in an AMS with HA-enabled and with NAPT-44 configured, can cause sessions to reset after a GRES (or SPD restart). PR910802

  • In this scenario the CPCD (captive-portal-content-delivery) is configured for HTTP-REDIRECT for Subscriber Management clients using MS-DPC. When services sessions start to redirect the HTTP traffic, the memory-usage consistently increments for MSPMAND on the multi-service PIC. The memory limit then might cause packets loss. PR954079

  • 1) Due to a previous fix chassisd on the protocol master Routing Engine and the protocol backup Routing Engine connect to the main snmpd on the protocol master using the following methods. a) Chassisd on the protocol master Routing Engine connects using a local socket since snmpd is running locally. b) Chassisd on the protocol backup Routing Engine connects using a TNP socket since snmpd is not local. 2) However this fix changed the way the other daemons connect to snmpd. All important daemons run on the protocol master and should connect to snmpd using a local socket. However the fix changed it so that all daemons that ran on the protocol master (other than chassisd) tried to connect using the TNP socket. SNMPD does not accept these connections. As a fix, in an MX-VC, we made sure that chassisd connects to all processes which run on the protocol master using internal socket while the chassisd process on the protocol backup and protocol linecard connect connect using TNP socket. PR986009

  • In the dual Routing Engines scenario, in rare condition, while executing GRES and deleting interfaces at the same time, it is possible that a nexthop delete message is not sent to rpd process, causing rpd to keep a nexthop index (NHID) that kernel has already deleted. Later when kernel allocates this NHID for next new nexthop and sends it to rpd process, rpd process might crash due to duplicate NHID. PR987102

  • In the VPLS environment with control-word configuration, when the "control-word" is changed to "no-control-word", there are 5 minutes service outage. PR987216

  • Mirroring of CCC traffic would be broken for a very small duration when Routing Engine switchover is happening. Post switch-over, CCC mirroring would work as expected. PR987593

  • In 6PE scenario, when PE router is sending IPv6 TCP traffic to MPLS core, in rare occasions, the kernel might crash and reboot with a vmcore file created. PR988418

  • OpenFlow v1.0 running on an MX Series router does not respond reliably to interface up or down events within a specified time interval. Per a fix implemented in Junos OS Release 13.3R3.6, OpenFlow v1.0 running on an MX Series router responds reliably to interface up or down events if the echo interval timeout is set to 11 seconds or more. PR989308

  • On M7i/M10i with enchanced CFEB, M320 with E3-FPC, M120 and MX Series routers with DPC. If "no-local-switching" is present in the bridge domain, then the IGMP-snooping is not functioning and client can't see the multicast traffic. PR989755

  • Configure the global and interface limit to allow for the maximum configuration of the macs in the overlay network. PR992084

  • Commit error need to be reported when using unsupported NAPT44 nat-options max-sessions-per-subscriber config with MS-MIC/MS-MPC. PR993320

  • MX960/480/240 fantray red alarm temp changed from 75C to 80C. PR995225

  • On T4000 router with type5 FPC. After FPC rebooting, if chassisd process does not get FPC ready/FPC online ACK message from FPC in 360 seconds, the FPC might reset again. PR998075

  • On M Series, MX Series, and T Series routers with Network Address Port Translation (NAPT) configuration. When the router receives the packet whose value of protocol field in the IPv4 header is 61, the router erroneously does NAPT44 translation. In the correct situation, the packet should not be translated and forwarded. PR999265

  • By default, the syslog utility exports 800,000 logs per second to a remote syslog server. You can modify the number of syslogs to be sent by including the message-rate-limit statement at the [edit interfaces interface-name services-options syslog] hierarchy level to suit your deployment needs. The rate at which syslog messages can be sent to the Routing Engine is 10,000 logs per second. PR1001201

  • On MX240/MX480/MX960 routers running as Precision Time Protocol (PTP) master when interconnect with MX104 series routers running as slave, the PTP clocking state might stuck in "INITIALIZING" for the first created PTP port and not be aligned to clocking state. There is another issue that when issue command "show ptp clock", wrong "slot" number might be seen on MX104 slave. PR1001282

  • "Syslog generated for session-open will have nat port information only if it is different from the original source port". PR1001912

  • If issue the command "show services nat mappings endpoint-independent" or "show services nat mappings address-pooling-paired" or "show services sessions" and kill it immediately when using EIM/APP feature with too many EIM/APP entries present in the system, lots of ipc message reply failure messages may be seen in the syslog. PR1002683

  • Multi-Services PIC could crash and restart on receiving a stray SIGQUIT signal due to it not handling the signal. PR1004195

  • During ISSU early stage, when Mm is arming packages on other three Routing Engines, Mm will not copy config/ssh files to Bm, and Bm will not fork mgd to copy the files. This should not be a problem. During ISSU, when backup chassis switchover done, the original Bm (new Bs) will copy the files from the original Bs(new Bm, now it has the latest config files from Mm). So the origianl Bm could always get the latest config/ssh files. PR1004766

  • When several PICs are set up as an aggregated multi-services (AMS) doing load-balancing, if one PIC of the AMS bundle get offline and then get online, a 30 to 40 seconds momentary traffic loss might be seen. PR1005665

  • The l2cpd process might crash if there are multiple unknown type, length, and value (TLV) elements included in received LLDP PDUs. PR1007223

  • MS-DPC memory leak on system service set when HTTP Redirect attempts to process none-HTTP traffic with HTTP ports (80/8080/443). PR1008332

  • Ingress queuing is not supported on MPC5 (With Q-MPC) when Optical Transport Network (OTN) is enabled. Enabling ingress queuing with OTN would lead to line card crash. PR1008569

  • With more than 8 service-sets configured, when using SNMP mibwalk for service-set (object "jnxSpSvcSetTable") info, the mspmand process (which manages the Multi-Services PIC) might crash. PR1009138

  • Add "protocol evpn" configuration under an existing virtual-switch routing-instance might cause EVPN neighborship unable to establish. PR1009339

  • Whenever a FPC goes down suddenly due to hardware failure, the data traffic in transit towards this FPC from the other FPCs could be stuck in the fabric queue thereby triggering fabric drops due to the lack of buffers to transmit the data to active destination FPCs. PR1009777

  • PIC state mgmt is not available in 14.1R2. PR1013480

  • Unknown unicast flood seen with interface flap after router reboot & with static mac,no-mac-learning,interface-mac-limit config for a virtual-switch. PR1014222

  • The routing protocol daemon (rpd) might crash continuously with core-dumps upon adding a sub-interface with "disable" configuration to a MC-LAG interface. PR1014300

  • On TXP with GRES enabled, when performing graceful switchover on all chassis (include line-card chassis (LCC)) from master Routing Engine to backup Routing Engine, minimal IPv4 traffic loss around 0.04% to 0.05% will be observed on aggregated 100GE PIC on FPC type 4. PR1014420

  • A new global knob is added at the top level CLI "set forwarding-options port-mirroring [no-preserve-ingress-tag]" By default the system behavior would remain as it is today where ingress mirrored copy would contain VLAN content exactly as what came in wire over ingress. However, if this knob is configured, if any VLAN modification happens to packet as part of its datapath processing, that would get retained in the ingress mirrored copy that is, we will not restore VLAN to what came in ingress on wire. PR1015149

  • Hash-key command is no longer treated as a hidden command and considered invalid input in 12.3 for small footprint routers (these platforms don't support the hash-key feature), this could cause configuration failure during a software upgrade if hash-key command is configured prior to the upgrade. This PR reverses the above change and allows hash-key command to be ignored on unsupported platforms: show configuration forwarding-options ## ## Warning: configuration block ignored: unsupported platform (mx80) ## hash-key { family inet { layer-3; } } PR1016339

  • MAC accounting support was added for 40G and 100G interfaces on MPC3 and MPC4 cards. PR1017595

  • With Enhanced IP network service mode configured, traffic might fail to be sent out over the inline LSQ bundle interface. PR1018887

  • Traffic destined to the Broadcast or Network address of a NAT pool using the address prefix setting for the MS-MPC card causes a traffic loop that spikes the CPU. PR1019354

  • No performance or functional impact. Can be safely ignored. "Ignore the PTP message (2) as this MPC doesn't support EEC" should be moved from notice to debug level. PR1020161

  • When source address is configured under ms interface, and the service-set has syslog host as local the FPC slot is printed as -ve. PR1020854

  • For M320 or T series FPCs (M320 non-E3 FPC and T Series non-FPC5) with queuing PIC, if the configured total buffer size temporal values exceed the supported maximum scheduler buffer size for the PIC (e.g. For PD-5-10XGE-SFPP PIC, the maximum temporal buffer size that can be configured for a scheduler is 40,000 microseconds), the default scheduler [95,0,0,5] is applied instead of the default chassis scheduler [25,25,25,25], which might result in the packet drops on Q1 and Q2. PR1027547

  • On MX Series routers with MS-MPC card, after performing switchover from master Routing Engine0 to backup Routing Engine1, two internal ARP entries for Routing Engine address (128.0.0.1) on MS-MPC PICs pointing to two eth interfaces connect to CB0 and CB1 separately might be incorrectly created. Then if pull out RE0/CB0, the MS-PIC would still select the eth interface connects to CB0, which results in loss of connectivity because that path is not available anymore. PR1030119

  • PCS statistics counter is now displayed interfaces in this command: monitor interface <intf>. PR1030819

High Availability (HA) and Resiliency

  • This issue occurs in rare condition. In the dual Routing Engines scenario, doing interface flap after Routing Engine switchover. If this action is repeated many times, the stale indirect nexthop entry might be seen in kernel, which leads to traffic blackhole. PR987959

  • In an MX Series Virtual Chassis configuration, a unified in-service software upgrade (ISSU) from Junos OS Release 14.1 or 14.1R2 to Junos OS Release 14.2 fails with traffic loss. PR1014295

Infrastructure

  • SNMP socket sequence error log. PR986613

  • A reboot is needed if "chassis network services enhanced-ip" is configured on MX Series 3D routers or on T4000 Routers with type 5 FPCs. Without the reboot, performing ISSU might cause new master Routing Engine to crash and go to db> prompt. PR1013262

Interfaces and Chassis

  • When the GE port is configured with WAN PHY mode, a "Zero length TLV" message might be reported from the port. PR673937

  • Error message CHASSISD_IPC_DAEMON_WRITE_ERROR is seen in the messages log when there is a Routing Engine mastership change (system reboot, Routing Engine reboot, GRES switchover CLI command), which causes a restart of alarmd,which breaks the IPC connection between alarmd and chassisd. Chassisd does not detect that the IPC connection has been broken, because it is busy processing the mastership change, and then tries to send alarm information to alarmd during this time. So it encounters a write error (broken pipe) and logs the message. PR908822

  • If dynamic VLAN subscriber interface is over a physical interface (IFD), and there are active subscribers over the interface, when deactivate the dynamic VLAN related configuration under the IFD and add the IFD to an aggregated Ethernet (AE) interface which has LACP enabled, the Routing Engine might crash and get rebooted. PR931028

  • In the dynamic-profile environment with preferred-source-address configuration. If subscribers stuck in terminating state, it is impossible to commit changes. PR978156

  • In the PPPoE environment, when the subscriber logs in successfully but profile activate fails, due to code processing error, the address entry is not deleted in the authd's DAP pool. So when the subscriber tries to log in again, it connects fails. PR995543

  • In the demux interfaces over aggregated Ethernet (AE) environment with targeted-distribution configuration. The index of AE interface is confusion when the index is more than 100. It copy only 4 bytes from interface name. (e.g. If bind demux interface to ae110, it will be bound to ae11 at the same time). The traffic forwarding might be affected. PR998906

  • IGMP joins do not work for PPP subscribers that are using MLPPP and LNS. PR1001214

  • In L2 circuit, with async notification configured on a client facing interface goes down, then on the remote PE the corresponding CE interface shows up in show interface terse output while in log snmp reports interface down. PR1001547

  • Fabric Blackholing logic recovery for certain cases will be done with different action (Phase 1/2/3) based on the problem. PR1009502

  • As current Junos OS multichassis link aggregation groups (MC-LAGs) design, the ARP entry will not sync when learning ARP via ARP request but not Gratuitous ARP/ARP reply. In some specific scenarios (e.g. a host changes its MAC address without sending a Gratuitous ARP), traffic loss might occur. PR1009591

  • Here is the expected behavior for CFM CCM: 1. UP MEP CFM session a. If there is a manually configured ieee-802.1 classifier attached to the interface, then forwarding class of the CCM injected should match the respective classifier. b. If there is an interface in which CFM is configured has no ieee-802.1 based 1p classified, then the forwarding class of the CCM will take as configured in "host-outbound-traffic". c. In case if there is no "host-outbound-classifier" present then packets will be treated as network control (Q3). 2. Down MEP CFM session a. forwarding class of the CCM will always depend on the FC classified based on "host-outbound-traffic". If it is not configured, then it will always take Q3. PR1010929

  • IS-IS Adjacency may flap after ISSU. This behavior is being further analyzed and is planned to be fixed in further releases. PR1015895

  • VRRP daemon (vrrpd) memory leak might be observed in "show system processes extensive" when VRRP is set with routing-instance and then change any configuration. PR1022400

  • set forwarding-options enhanced-hash-key symmetric command will not get applied on MX104 Packet Forwarding Engine. PR1028931

  • In addition to fixing the reported deviation from the expected functionality, the show virtual-chassis heartbeat [detail] command output is enhanced to more clearly indicate the successful detection of the peer MX-VC member chassis over the heartbeat connection when the chassis loses all VCP adjacency links. PR1034096

J-Web

  • An insufficient validation vulnerability in J-Web can allow an authenticated user to execute arbitrary commands. This may allow a user with low privilege (such as read only access) to get complete administrative access. This scope of this vulnerability is limited to only those users with valid, authenticated login credentials. Refer to JSA10560 for more information. PR826518

Layer 2 Features

  • In BGP signaled VPLS/VPWS scenario, rpd process memory leak might occur when a groups with wildcard configuration is applied to the routing instance. PR987727

  • After configuration change or convergence events, kernel may report ifl_index_alloc failures for LSI interfaces and cause KRT queue ENOMEM issue, eventually preventing new IFL's from being added to the system. This condition always recovers on its own once convergence is completed. PR997015

  • In a mixed VPLS instance where both ldp and bgp flavors are present, any cli change in that instance will result in RPD crash. PR1025885

Layer 2 Ethernet Services

  • In MX Series Virtual Chassis (MXVC) scenario with LACP configuration. In rare condition, after VC-M chassis power down, the LACP state getting stuck in ATTACHED state, all traffic carried over these affected access LAGs are blackholed. PR959041

  • When "system no-redirect" is configured, l2 descriptor destination MAC address gets overwritten and causes "DA rejects" on next-hop router. PR989323

  • In the Ethernet ring protection switching (ERPS) environment, once graceful Routing Engine switchover happens on the ring protection links (RPLs) owner node, there will be a ~30s Ring automatic protection switching (R-APS) message storm in the ring, which in turn cause some VPLS instance flapping. PR1004066

  • On MX Series routers with DHCP service enabled, issuing CLI command "show dhcp-security binding" might result in jdhcpd process crash. PR1007577

  • With scaled VPLS instances configured, aggressively flapping the interfaces belonging to the VPLS instances might result in l2cpd process memory leak. When l2cpd reaches its max memory limit, l2cpd process crash will be seen. PR1009952

  • Commit is failing on backup Routing Engine with ethernet-ring configuration under "protocol protection-group" hierarchy. user@R1# commit synchronize re0: configuration check succeeds re1: [edit protocols protection-group] 'ethernet-ring vkm1' L2CPD : INVALID node-id configured for pg vkm1 error: configuration check-out failed re0: error: remote commit-configuration failed on re1 Node id value is not available to backup Routing Engine, when it is not configured. As in such case it is derived from chassis mac and on backup Routing Engine chassis-mac remains as 00:00:00:00:00:00. Fix: validation check for node-id value will not be done on backup Routing Engine. PR1011441

  • If "maintain-subscriber" knob is enabled on the router, DHCPv6 server/relay might be unable to process any packet if deactivate and then activate the routing instance, which means the subscribers can not get the IPv6 addresses. Note, even with the fix, the results of this scenario is also expected if with "maintain-subscriber" knob enabled. Consider using the workaround to avoid this issue. PR1018131

  • After FPC restart, bridge domain (BD) implicit filters for Ethernet ring protection switching (ERPS) might get reprogrammed with wrong logical interface (ifl) index, which cause ERPS to not work correctly. PR1021795

MPLS

  • Although NSR does not support MPLSOAMD and it does not run on backup Routing Engine, backup RPD is attempting to do task_connect to MPLSOAMD. This behavior causes periodical message popping up on backup Routing Engine. Feb 21 15:14:13.306 2014 mx480-re1 rpd[2840]: task_connect: task MPLSOAMD I/O./var/run/mplsoamd_control addr /var/run/mplsoamd_control: No such file or directory. PR938284

  • In the MPLS environment with no-cspf & strict ERO configuration. In race condition, if a PATH message with routing loop error is received before standby Routing Engine has resolved the correct PATH message with no loop, some of LSP are not replicated on standby Routing Engine. If Routing Engine switchover occurs, the forwarding traffic might be affected. PR986714

  • BGP may reevaluate all its routes if a protocol mpls stanza is configured, but an egress-protection stanza is not. On a scaled setup, this can keep RPD CPU high for several minutes after each commit. PR1000550

  • Interoperability issue between Junos OS and IOS-XRv, the virtual IOS-XR. It is related to the max_pdu TLV in LDP. IOS-XRv only supports max_pdu 1000 or below. On the other hand, Junos OS only supports max_pdu 1200 or above. So LDP session never comes up successfully. There are fixes in both vendors. On the Junos OS side, max_pdu 1000 is accepted after the fix and the session comes up. PR1007096

  • When the size of a Routing Engine generated packet going over an MPLS LSP is larger than MTU (i.e. MTU minus its header size) of an underlying interface, and the extra bytes leading to IP-fragmentation is as small as <8 bytes, then that small-fragment will be dropped by kernel and lead to packet drop with kernel message "tag_attach_labels(): m_pullup() failed". For example - If SNMP Response with specific size falls into above mentioned condition, then small fragment will be dropped by kernel and eventually the SNMP response will fail. PR1011548

  • The entropy label value allocated at times falls in the reserved mpls label range(0-15). The label value is calculated based on load balancing information and hence only certain mpls flows may encounter this issue. PR1014263

  • In MPLS scenario with TX/TXP router acting as the transit node, performing MPLS LSP ping or traceroute from ingress node might cause kernel crash on the transit node due to improper timer initialization between SCC and LCC chassis. PR1020021

Multicast

  • In multicast environment, if GRES is performed immediately after a routing-instance being deleted, the krt (kernel routing table) queue might get stuck after adding back the routing-instances which were deleted. PR1001122

Network Management and Monitoring

  • Due to a communication error between the master agent (snmpd process) and the subagent (mib2d process), it might cause a failure to register some MIBs. For example: There is no output while running below commands: user@hostname> show snmp mib walk ifTable When user tries polling the device for ifAlias. The following messages might be seen: user@hostname:~$ snmpwalk -v 2c -c snmp@exp X.X.X.X ifAlias IF-MIB::ifAlias = No Such Object available on this agent at this OID This means that there's no OID registered. PR978535

  • The Packet Forwarding Engine local protocol statistics are 32-bit counters. If there is a rollover (typical candidates are ARP/LACP), those counters start from zero. mib2d will add all counters again if one of the Packet Forwarding Engine statistics traffic counter is less than the previous collected counter, causing the multiplication affect. PR986712

  • Alarm management daemon runs on master and backup Routing Engines on dual Routing Engine systems. There is a 80 megabyte alarm.db file that is copied over from master Routing Engine to backup Routing Engine when the alarm-management daemon has come up on both the routing engines. The basic issue is that alarm-management daemon is trying to copy the alarm.db file over and over again in an infinite loop on the system, causing CPU utilization shooting up after every 20 seconds or so. PR988969

  • The snmpd process becomes unresponsive for ~30 minutes after performing GRES when SNMPv3 notify type is configured to be "inform". PR1021943

Platform and Infrastructure

  • MPLS traceroute causes "rttable-mismatch" syslog messages. PR960493

  • When apply-groups are used in the configuration, the expansion of interfaces <*> apply-groups will be done against all interfaces during the configuration validation process, even if the apply-group is configured only under a specific interface stanza. PR967233

  • This is a corner case. On MX Series routers with MPCs/MICs, some stale unilists nexthops are present. If an interface is down for more than ARP timeout interval, the broken selectors in unilist nexthops, and then traffic will be blackholed when the interface is up again. PR980052

  • Have BFD session between one router supporting inline-BFD (Trio and Junos OS 13.3 or higher) and the other which doesn't support inline-BFD (any version and non-Trio, or Trio and Junos OS less than release 13.3). When the "failure detection time" is less than 50ms, the BFD session might flap. PR982258

  • GRES doesn't clear system login to original master-only fxp0 addresses causing stale login sessions. PR991029

  • On MX2020/MX2010 we might see sporadic FO request time-out error reported under heavy system traffic load. This would mean the request returning into a grant took longer then +/-30usec. The packet will still get forwarded through the fabric hence no operational impact. [May 6 18:56:59.174 LOG: Err] MQCHIP(2) FO Request time-out error [May 6 19:33:47.555 LOG: Info] CMTFPC: Fabric request time out pfe 2 plane 6 pg 0, trying recovery. PR991274

  • When we uninstall an SDK package, the config related to that package is still left out in the config file. After this if commit sync is issued, though commit is successful, it leads to a commitd core. Before the un-installation of SDK package, the config statement [set jnx-ifinfo traceoptions flag all] should also get deleted from the config which is relevant to the package being deleted. PR992486

  • On MX Series routers with MPCs/MICs or T4000 router with type5. When the firewall filter under the [forwarding-options] hierarchy within a bridge domain is removed, it might result in lookup error and frame drop might be observed. PR999083

  • In the IRB interface environment with "destination-class-usage" configuration. If the bridge domain ID is the same as Destination Class Usage (DCU) ID (bridge domain ID and DCU ID are generated by system), the firewall filter might match wrong packets, the packet forwarding would be affected. PR999649

  • On M7i, or M10i equipped with Enhanced Compact Forwarding Engine Board (CFEB-E). When a MPLS LSP flaps, the CFEB-E is unable to recover 8 bytes of JTREE memory per event. PR1000385

  • When receiving traffic coming on MPC and going out on DPC, the MAC entry on a Packet Forwarding Engine might not be up-to-date and the frames targeted to a known MAC address will be flooded across the bridge domain. PR1003525

  • With NSR enabled, when activating a BGP session in a routing instance, and the interface route is imported into the main routing instance, the TCP receive window might decrement until it hits 0 after receiving incoming BGP traffic arrives from the main routing instance. PR1003576

  • On MX Series routers with MPCs/MICs, routers in the same VRRP instance might both claim to be VRRP master after performing unified in-service software upgrade (ISSU) upgrading to specific Junos OS versions. PR1004471

  • On MX-VC platform, if there is a dark window larger than 5s (TCP timeout timer) from Packet Forwarding Engine to Routing Engine during unified ISSU for some reasons, some VC members might get unexpectedly rebooted. PR1005309

  • In PPPoE over ATM subscriber management environment with active subscribers is present, when issue the "show arp" command, an ARP core file is generated. PR1006306

  • The non-first IP fragments containing UDP payload may be mistakenly interpreted as PTP packets if the following conditions are met: - the byte at the offset 9 in the IP packet contains 0x11 (decimal 17) - UDP payload - the two bytes at the offset 22 in the IP packet contain the value 0x01 0x3f (decimal 319; byte 22=0x01 and byte 23=0x3f) - PTP protocol The mis-identification of the packet as PTP will trigger the corruption of the fragment payload. PR1006718

  • Micro BFD sessions are used to monitor the status of individual LAG member links. When micro BFD configurations are added after the LAG bundle configuration in separate commit, the micro BFD sessions for all the member links might remain in "Down" state. PR1006809

  • Memory allocated in reference to the BFD session was not getting freed up. This resulted in memory leak and the memory exhaustion triggered crash. PR1007432

  • For MX Series routers with MPCs/MICs or Chassis-based line cards, if there are more than 8K PPPoE subscribers with SRL (ratelimit) on a same FPC, new subscribers might not be able to connect any more due to filter memory threshold. PR1009232

  • If rate-limit has been configured in scheduler for MX-VC VCP ports, ISSU might fail. PR1009590

  • MPLS traffic going through the ingress pre-classifier logic may not determine mpls payload correctly classifying mpls packet into control queue versus non-control queue and expose possible packet re-order. PR1010604

  • Issue: This change addresses missing NULL check in a trace message which was resulting in Packet Forwarding Engine crash. The error path involves scenario where ifbd is not yet created for an IFL. This is possible under certain IPC reordering scenarios. The Packet Forwarding Engine should not crash by differencing a NULL pointer in this case. PR1014090

  • The fix was committed for this PR# but it also needs DDOS configuration additional to this fix and it is as below: 1) check the "show ddos-protection protocols statistics terse" 2) For each of the Control plane protocols on the system like ospf/vrrp/pvstp, it is recommended to configure 2X of the rate as give below example along with increasing DDOS rate for virtual-chassis control. Example, ######## set system ddos-protection protocols virtual-chassis control-high bandwidth 20000 set system ddos-protection protocols virtual-chassis control-high burst 20000 set system ddos-protection protocols ospf aggregate bandwidth 1000 set system ddos-protection protocols ospf aggregate burst 1000 set system ddos-protection protocols vrrp aggregate bandwidth 100 set system ddos-protection protocols vrrp aggregate burst 100 PR1017640

  • On MX Series routers with MPCs/MICs, when there are next-hop changes, the "heap 0" memory of the FPC may experience memory leakage which will eventually causes memory exhaustion. PR1019794

  • For MX Series routers with inline Network Address Translation (NAT) service, when using "source-prefix" or "destination-prefix" in a NAT translation rule, a pool is implicitly created, appending "_jinpool_" with the rule name and term name with a form : _jinpool_{rule_name}_{term_name}. The name might be cropped due to the maximum length limitation (64 characters). If that happens, both pools might get the same name and result in the indeterminate behavior (statistic issue, drop or incorrect translation). PR1020033

  • problem scenario: The error logs "CHASSISD_FCHIP_CONFIG_MD_ERROR" will appear during FPC normal boot up time and also during FPC restart time for each plane and for each LMNR FPC. Problem statement: This Error logs "CHASSISD_FCHIP_CONFIG_MD_ERROR" are observed only in M320 chassis containing FPCs based on LMNR chipsets. Due to this error log, the rate limit for the fabric port connecting the Packet Forwarding Engine 1 will be set to the default values. PR1020551

  • When receiving traffic coming on MPC and going out on DPC, an Ethernet frame with known DMAC will be flooded to the whole bridge domain after flapping the link which the given MAC is learned for more than 32 times. PR1026879

  • When a layer 2 frame entered the VPLS end point on the label-switched interface (LSI) with VLAN tagged, the frame is wrongly interpreted and treated as no VLAN frame. So the VLAN tag will not be popped although the outbound interface has a pop configuration. PR1027513

  • In normal case, network-service enhanced-ip would make BFD over AE distributed to Packet Forwarding Engine (control plane independent). However due to this software issue, it would remain running on Routing Engine. PR1031916

Routing Protocols

  • High CPU utilization is observed by routing process when high number (around 1000) of Rosen based MVRF configuration is committed in one-shot. It will take more than 1 hour for CPU usage by routing process to come to normal condition. PR947732

  • Performing CLI command "clear multicast bandwidth-admission interface <int>" on 64-bit Junos OS results the rpd process crash. The command should be used without the interface qualifier on the impacted releases. PR949680

  • In a scaled setup a restart routing or NSR switchover can result in duplicate msdp entries. PR977841

  • On a platform with an IGMP configuration in which two receivers are joined to the same (S,G) and IGMP immediate-leave is configured, when one of the receivers sends a leave message for the (S,G), the other receiver might not receive traffic for 1-2 minutes. PR979936

  • In the P2MP environment with OSPF adjacency established. One router's time is set to earlier date than another router. OSPF adjacency might not come up when one router goes down and comes up. PR991540

  • Bringing up DFWD based BFD sessions at scale causes a churn in DFW as a result of which the FPC CPU usage remains at 100% for a prolonged timespan. PR992990

  • When all the following conditions are met, if the knob "path-selection always-compare-med" is configured, the rpd process might crash. - routing-instance (VR, VRF) with no BGP configuration - rib-group in default instance with routing-instance.inet.0 as secondary-rib - rib-group applied to BGP in default instance - BGP routes from master tables (inet.0) leaked to the routing-instance table (routing-instance.inet.0) PR995586

  • When IS-IS is configured for traffic engineer (TE), after remove family mpls from the interface and remove the specific interface from [edit protocols rsvp] and [edit protocols mpls] hierarchy level, corresponding link is not removed from the TED as expected. PR1003159

  • When there are more than 65535 "flow-spec" routes existing in the routing table, the rpd process might crash because it exceeds the current maximum supportable scaling numbers (Current scaling numbers are in the range of 10K~16K). PR1004575

  • When having ECMP routes and multiple levels of route/next-hop recursion, a particular sequence of routes churn may result in rpd process crash and traffic outage. PR1006523

  • Abnormal ip6 route-calculation behavior can be seen when ospf3-te-shortcut is configured. PR1006951

  • When the same PIM RP address is learned in multiple VRFs, with NSR configured, RPD on the backup Routing Engine may crash due to memory corruption by the PIM module. PR1008578

  • When deleting a routing-instance or making changes to the routing-instance, the deletion of the routing-instance to kernel might come before the deletion of the IFLs in the routing-instance, resulting in rpd crash. This is a timing issue, hard to reproduce. PR1009426

  • During unified in-service software upgrade (ISSU), when a Bidirectional Forwarding Detection (BFD) session negotiation is happening, if the session is configured with 10 seconds or higher interval, BFD session would flap. PR1010161

  • Misconfiguring BGP route validation session to the router itself might lead to rpd process crash. PR1010216

  • When inet.3/inet6.3 is not enabled, BGP group uses inet6.0 table to advertise the routes for both inet6 unicast and inet6 labeld-unicast families. When BGP family is changed, BGP sessions re-establish. When BGP starts to advertise routes to the peer, BGP expects to see route label however if the old inet6 unicast routes are still present (not completely cleaned), then rpd process crashes. The fix is to separate bgp group for inet6 unicast with inet6 labeled-unicast with same rib. The old peers are cleaned up in the old group and new peers are established in new group. Thus, new peer establishment is not delayed by the cleanup of the old peer. PR1011034

  • Issue: IsisRouterTable MIB issues, when we do "show snmp mib walk isisRouterHostName/isisRouterTable" we were not getting exact hostname as it is in "show isis hostname" so the actual implementation was not as per RFC-4444, because it was showing only the hostnames of the devices which are immideate neighbors of Dut. Fix: added level info to get sysis_entry per each level correctly and filled data(isisRouterTable) correctly. PR1011208

  • In scaled BFD scenarios, BFD ISSU poll negotiation will fail causing the BFD session to flap during ISSU. PR1012859

  • Under certain sequence of events RPD can assert after a RPD_RV_SESSIONDOWN event. PR1013583

  • With multicast discard route present, if a RP router has no pd- interface, it might not generate (S,G) join to upstream when receiving MSDP source active (SA) message. PR1014145

  • For 64-bit Junos OS, the route protocols process (rpd) might crash and dump core during IBGP route churn when using IBGP multipath and multiple levels of IBGP route/next-hop recursion. PR1014827

  • This PR is implementing traceoptions debug enhancements to detect route-record corruption events. The route-record traceoptions debug will be enabled as follows: ---------------------------- user@router> edit Entering configuration mode [edit] user@router# set routing-options traceoptions flag route-record [edit] user@router# commit ---------------------------- PR1015820

  • The OpenSSL project released a security advisory on 2014-08-06 that contained nine security issues. The following four issues affect Junos OS: CVE-2014-5139: Crash with SRP ciphersuite in Server Hello message CVE-2014-3509: Race condition in ssl_parse_serverhello_tlsext CVE-2014-3511: OpenSSL TLS protocol downgrade attack CVE-2014-3512: SRP buffer overrun See JSA10649 for more information. PR1016458

  • When receiving open message with any capability after the "add-path" capability from BGP peer, the session will be bounced. PR1016736

  • With BGP multipath configured, if a BGP route's multiple protocol nexthops are resolved to different types of IGP routes with a same metric, high rpd process utilization might be observed due to the BGP multipath task. PR1017372

  • The snmp trap generated when an ipv6 BFD session goes up/down does not contain the ipv6 bfd session address. PR1018122

  • The Junos OS implementation of RFC3107 uses unspecified label (0x000000) when sending route with label withdrawn message. This means Junos OS sends 0x000000 instead of 0x800000 for label withdrawn, which is inconsistent with RFC 3107. PR1018434

  • Under following combination of events: * graceful-restart is enabled and * bidirectional PIM is enabled and * rpd is restarted, and * multicast traffic for bidir rp group hits the box. Pim creates the discard route and this traffic is pruned. PR1019560

  • Establish two BFD sessions between two routers, one is single-hop BFD for directly connected interface and the other is multi-hop MPLS OAM BFD. If configuring the MPLS OAM on the same interface with single-hop BFD, when bringing down MPLS OAM from the ingress, it might result in the OAM BFD session deleted on ingress but it still receiving OAM BFD down packet from egress. Since there is no session matching this BFD packet, it does a normal look up and brings down the single-hop BFD session which is on the same interface. PR1021287

  • If auto-export feature is enabled together with rib-groups configuration option, the rpd process might crash. PR1028522

Services Applications

  • If a destination-prefix or source-prefix is used like below example, the Network Address Translation (NAT) rule and term names will be used to generate an internal jpool with a form : _jpool_{rule_name}_{term_name}. If the generated jpool name exceeds 64 characters in length, it will get truncated. If the truncated jpool name get overlapped with other generated jpool name it will lead to an inconsistent pool usage. user@router# show services nat rule A_RULE_NAME_WHICH_IS_LONG_12345 { ... term A_TERM_ALSO_WITH_LONG_NAME_1 { from { source-address { 10.20.20.1/32; } } then { translated { source-prefix 10.10.10.1/32; <--- translation-type { source static; } } } } term A_TERM_ALSO_WITH_LONG_NAME_2 { from { source-address { 10.20.20.22/32; } } then { translated { source-prefix 10.10.10.2/32; <--- translation-type { source static; } } } } } First jpool = _jpool_A_RULE_NAME_WHICH_IS_LONG_1234_A_TERM_ALSO_WITH_LONG_NAME_1 > 64 characters. Second jpool = _jpool_A_RULE_NAME_WHICH_IS_LONG_1234_A_TERM_ALSO_WITH_LONG_NAME_2 > 64 characters. The resulted jpool "_jpool_A_RULE_NAME_WHICH_IS_LONG_1234_A_TERM_ALSO_WITH_" will be used wrongly in both terms. PR973465

  • On MX240/MX480/MX960 routers with MS-DPC with "deterministic-port-block-allocation block-size" configuration. In rare condition, when the "block-size" is set to a larger value (in this case, block-size=16128), the Services PIC might crash. PR994107

  • In the NAT environment, a same pool is used in several terms of a nat-rule. If any pool parameter is modified, the configuration change is ignored. PR994200

  • The redundant services PIC (rsp-) interfaces or redundant Multiservices (rms-) interfaces configured with "hot-standby" mode might flap upon committing any configuration change (will happen for even an unrelated interface description change). PR1000591

  • The following messages are being logged at ERR not DEBUG severity: mspd[3618]: mspd: No member config mspd[3618]: mspd: Building package info This PR sets the correct severity. PR1003640

  • When removing a basic-nat44 translation term, there is a chance the prefix that was used for this translation will become wedged. Any attempt to reuse this prefix for dynamic-nat44 or napt-44 will be such that no address/port allocation will succeed. PR1008214

  • Softwire tunnel count management is inconsistent and incorrect, thus the output of "show service softwire statistics" might be incorrect. PR1015365

  • Configured Port Control Protocol (PCP) lifetime is ignored and NAT pool mapping-timeout is used instead for pinholes existence on the CGNAT public interface. PR1017155

  • L2TP LNS dropped all tunnels/sessions after a commit. PR1020420

  • With Real Time Streaming Protocol (RTSP) Application Layer Gateway (ALG) enabled, the PIC might crash in case the Transport header in status reply from the media server is bigger than 240 bytes. PR1027977

Subscriber Access Management

  • MIB entries for jnxUserAAAAccessPoolRoutingInstance may not appear after deleting and re-adding an assignement pool under a routing instance. PR998967

User Interface and Configuration

  • CST: chassis core generated while applying group config on chassis > FPC. PR936150

VPNs

  • In the Rosen MVPN environment, the RP-PE is an assert loser, another PE is sending traffic over the data-mdt. If a new receiver PE with higher rate comes up, because internal workflow processes wrong, the receiver PE might reset data-mdt. This leads to traffic loss. PR999760

  • Serving site B is not receiving all the traffic from serving site A when traffic is reduced from the exceeded cmcast limit. PR1001861

  • In the 12.3 release after issuing a "request pim multicast-tunnel rebalance" command the software may place the default encapsulation and decapsulation devices for a Rosen MVPN on different tunnel devices. PR1011074

Resolved Issues: 14.1R2

Class of Service (CoS)

  • Manually setting max-queues-per-interface to 4 on PB-4OC3-1OC12-SON-SFP doesn't work. The ports will still work with 8 queue while displaying 4 queue from CLI output. PR981253

  • On MX Series routers with MPC and MPCE and other type of linecard, DPCE, when the Default Frame Relay DE Loss Priority Map is configured and commited, all FPCs are getting restarted with core files. PR990911

Forwarding and Sampling

  • Less impact on customer environment, it is just a ease of debugging issue. PR950553

  • DPC crashed after deactivate/activate [routing-instances TPIX bridge-domains IX bridge-options. PR983640

General Routing

  • When nonstop active routing (NSR) is configured and the memory utilization of rpd process on the backup Routing Engine is high (1.4G or above), the rpd crash on the backup Routing Engine may bounce the BGP sessions on the master Routing Engine. PR942981

  • There is a regression issue in Release 14.1 and later for single chassis with NSR and the MXVC environment. RPD might crash during GRES or membership switchover due to asynchronized routing table between Routing Engines. PR950767

  • Under particular scenarios, commit action might lead the Context-Identifier to be ignored when OSPF protocol refreshes its database. Then the PE router will stop advertising this Context-Identifier. PR954033

  • "show interfaces et-x/y/z extensive" will display MRU now. MRU can be configured at "set interfaces et-x/y/z gigether-options mru" If MRU is not configured then it is defaulted to MTU + 8. MRU displayed from the CLI does not include the CRC PR958162

  • On MX Series Virtual Chassis (MX-VC), if multiple VCP ports are configured between MPC5E cards, traffic might not be load balanced over the VCP ports. Besides, packets might get lost due to VC ingress and egress next-hop caches getting out of synchronization. PR960803

  • Although receiving the flow specification (flowspec) routes with packet-length, icmp-code, or icmp-type matching rules from a BGP peer properly, the local firewall filter in the Packet Forwarding Engines might not include these matching rules. PR968125

  • On an MX VC-Mm Routing Engine switch, the last flap time and associated error counters for the VCP interfaces sometimes get reset. The last flap time can be incorrectly reported as 'Never', for those VCP that have previously flapped. PR971995

  • tnping member1-RE0 from member0-RE0 fails because of a replication panic at "rnh_index_alloc: nhindex 624 could not be allocated err=12" PR977445

  • Changing service-set configuration continuously during scaled traffic conditions may result in mspmand process crash and a core file generated. PR978032

  • Juniper Distributed Application Framework (JDAF) serviceability feature enables CLI based inspection of various JDAF service counters. PR978640

  • On T Series router with FIB Localization enabled, if reboot the Routing Engine while scaled traffic running, the FIB-remote FPC might crash. PR979098

  • In rare condition, when PPPoE subscribers log in with large amounts of configuration data, the subscriber management infrastructure daemon (smid) and authentication service process (authd) might crash, and no new subscribers could connect to the router. PR980646

  • In scenario of NG-MVPN with P2MP LSP as provider tunnel, Kernel Routing Table (KRT) might get stuck after making changes for MVPN, then traffic loss will be seen. Besides, rpd process might crash while trying to generate a live core file. PR982959

  • With a firewall policer configured on more than 256 IFFs (interface address family) of a PIC, then offline and online the PIC might cause the FPC to crash. PR983999

  • OpenSSL library in Junos OS was patched to resolve CVE-2010-5298. PR984416

  • On M7i/M10i with enchanced CFEB, M320 with E3-FPC, M120 and MX with DPC. In a race condition, the Dense Port Concentrator (DPC) may crash when ifls get added to an ifl-set while that same ifl-set get deactivated/deleted in class-of-service. For example: # set interfaces interface-set interface_set_JTAC_ge-3/0/0 interface ge-3/0/0 unit 100 # deactivate class-of-service interfaces interface-set interface_set_JTAC_ge-3/0/0 # commit or (quick commit of following changes) # set interfaces interface-set interface_set_JTAC_ge-3/0/0 interface ge-3/0/0 # commit # deactivate class-of-service interfaces interface-set interface_set_JTAC_ge-3/0/0 # commit PR985974

  • When the logical interface's (IFL) MTU is changed (set interfaces et-x/y/z unit 0 family inet mtu xx), the static route goes to dead state and never recovers on its own. PR989021

  • During large scale MVPN routes churn events, some core-facing IGP protocols (like OSPF or LDP) might flap or experience a long convergence time. PR989787

  • When the interface-mac-limit on vtep interfaces is reached, any new OVSDB MACs advertised from the same remote VTEP are never getting added to the bridge mac-table. PR992084

  • Group VPN member registration in MX Series router will not succeed if the same interface is used for both data traffic and server-member communication. This limitation will apply if a group VPN service-set is applied on the interface in which the member is communicating with the Group key server. PR993001

  • The fabric performance of MPC1, MPC2, or 16xXE MPC in 'increased-bandwidth' mode on an MX960 populated with SCBE's will be less compared to redundant mode due to XF1 ASIC scheduling bugs. PR993787

  • On 10X10GE SFPP, when an interface configured for CCC and asynchronous-notification, and it is told to turn off its later, its laser flaps on and off for some period of time. PR996277

  • The PIC memory gauge counters show up as 0 after a GRES switchover in the "show chassis pic fpc-slot X pic-slot Y" output. PR1000111

  • Because of MCNH change from 13.3 to 14.1 and later, which used new FLOOD_MCNH to replace old MCNH_P2MP, while unified ISSU upgrading there would be a RPD crash. PR1000494

  • When using AMS load-balancing if a PIC in the AMS bundled is offline for any reason and the operator on-lines the PIC, there is slight 30 to 40 second momentary traffic loss. PR1005665

Interfaces and Chassis

  • Queue stats counters for AE interface will become invalid after deactivating ifl on the AE interface. PR926617

  • Strange FRU Insertion trap[RE PCMCIA card 0] is generated when Routing Engine master-switching is done on box with RE-1800. PR943767

  • When an ifl containing some vrrp group configuration is deleted, snmp walk on vrrp MIB may loop continuously. PR957975

  • If there is an IRB interface configured for "family inet6" in a bridge-domain on an MX Series router, the Packet Forwarding Engine might not correctly update the next hop for an IPv6 route when the MAC address associated with the next hop moves from an AE interface to a non-AE interface. PR958019

  • Temperature Top and Bottom are swapped in show chassis environments output for Type3/Type4 FPCs of T Series PR975758

  • In the multilink frame relay (mlfr) environment with "disable-tx" configuration, when the differential delay exceeds the red limit, the transmission is disabled on the bundle link. When it is restored, the link should be added back. But in this case, the link stays in the disable state, and it is not rejoined to the bundle. PR978855

  • With nonstop active routing (NSR) enabled, the VRRP tracking routes state on backup Routing Engine might not get synchronized when adding/deleting the tracking routes. PR983608

  • When upgrading to Release 13.3R2, customer may see the following messages: Chassis control process: rtslib: ERROR kernel does not support all messages: expected 104 got 103,a reboot or software upgrade may be required Chassis control process: Chassis control process: rtslib: WARNING version mismatch for msg macsec (103): expected 99 got 191,a reboot or software upgrade may be required Chassis control process: Chassis control process: rtslib: ERROR kernel does not support all messages: expected 104 got 103,a reboot or software upgrade may be required Chassis control process: Chassis control process: rtslib: WARNING version mismatch for msg macsec (103): expected 99 got 191,a reboot or software upgrade may be required These messages are generated during validation of the new chassis management daemon against the old kernel, and are harmless. PR983735

  • 1GbE SFP(EX-SFP-1FE-LX) output optical power is restored after reseating by manual removal/insert of SFP although the IF is disabled. PR984192

  • SNMP OID VRRP-MIB::vrrpAssoIpAddrRowStatus returns only one Ip address when the interface ifl has configured with two virtual-addressees under two vrrp-groups. PR987992

  • Following messages could be seen on the router for the FPC slot which are even empty. These messages are cosmetic and could be ignored. chassisd[1637]: %DAEMON-6: FPC 0 does not support Pic power off config cmd ignoring the config change chassisd[1637]: %DAEMON-6: FPC 2 does not support Pic power off config cmd ignoring the config change. PR988987

  • CFMD may crash after configuration change of an interface in a logical system which is under OAM config for a l2vpn instance. PR991122

  • In Ethernet OAM connectivity-fault-management, Junos OS default encodes MAID(MD name and MA name) in character format. Currently only 43 octets are supported in Junos OS for the MD + MA name. Junos OS needs to support a maximum length of 44 octets for MAID per the standards. PR997834

  • On MX Series router with MPCs or MICs or T4000 router with type5 FPC, when the "Hardware-assisted-timestamping" is enabled, the MPC modules might crash with a core file generated. The core files could be seen by executing CLI command "show system core-dumps". PR999392

Layer 2 Ethernet Services

  • In DHCPv6 subscriber environment, changing the c-tags (inner vlan) without clear the DHCPv6 clients first is not recommended, it might cause the subscriber to use the old inner vlan even after DHCPv6 RENEW process. PR970451

  • When Cisco running in an old version of PVST+, it doesn't carry VLAN ID in the end of BPDU. So Juniper Networks equipment fails to respond Topology Change Notification ACK packet when interoperates with Cisco equipment. After the fix, Juniper Networks equipment will read the VLAN ID information from Ethernet header. PR984563

  • Layer 2 Control Protocol process (l2cpd) is used to enable features such as Layer 2 protocol tunneling or nonstop bridging. If a router receives a Link Layer Discovery Protocol (LLDP) packets with multiple management address TLV, memory leak might occur which resulting in l2cpd process crash. PR986716

  • jnxLacpTimeOut trap may show negative values and incorrect values for jnxLacpifIndex and jnxLacpAggregateifIndex. PR994725

  • In race condition, when FPC gets rebooted or reset, link(s) from this FPC which are part of aggregate-ethernet bundle would remain in LACP "Detached" state indefinitely. user@node> show lacp interfaces ae102 Aggregated interface: ae102 LACP state: Role Exp Def Dist Col Syn Aggr Timeout Activity xe-2/0/0 Actor No Yes No No No Yes Fast Active xe-2/0/0 Partner No Yes No No No Yes Fast Passive xe-2/0/1 Actor No No Yes Yes Yes Yes Fast Active xe-2/0/1 Partner No No Yes Yes Yes Yes Fast Active LACP protocol: Receive State Transmit State Mux State xe-2/0/0 Defaulted Fast periodic Detached xe-2/0/1 Current Fast periodic Collecting distributing user@node> show interfaces xe-2/0/0 terse Interface Admin Link Proto Local Remote xe-2/0/0 up up xe-2/0/0.0 up up aenet --> ae102.0 xe-2/0/0.32767 up up aenet --> ae102.32767 This issue would be seen when associated aggregate-ethernet bundle is configured for vlan-tagging. To clear this condition, the affected interface should be deactivated and activated using cli commands. ============ [edit] user@node# deactivate interfaces xe-2/0/0 [edit] user@node# commit [edit] user@node# activate interfaces xe-2/0/0 [edit] user@node# commit ============ PR998246

MPLS

  • snmpwalk/snmpgetnext or "show snmp mib walk" fail when polling MPLSLSPOCTETS, MPLSLSPPACKETS, MPLSLSPINFOOCTETS or MPLSLSPINFOPACKETS. PR981061

  • LSP metric modification leads to Constrained Shortest Path First(CSPF) computation and resignaling. It should update RSVP routes directly. PR985099

  • In the MPLS environment with "egress-protection" configuration, there is a direct LDP session between primary PE and protector. One context-id is configured as primary PE's loopback address or any LDP enabled interface address. When delete the whole apply-group or delete the ldp policy from apply-group, the routing protocol daemon (rpd) might crash. PR988775

  • In the virtual private LAN service (VPLS) environment with multihoming (FEC 129) configured, when the router receives the label request for the Forwarding Equivalency Class (FEC) 129, if there is no route for the specific FEC 129, the routing protocol daemon might crash. PR992983

Platform and Infrastructure

  • When using OSPF/OSPFv3 with interface type point-to-point, it is possible for the OSPF session (using multicast traffic exclusively) to come up before next-hop resolution is done (ARP, or ND). In this case, transit traffic will be discarded, until resolution is done. When you have multiple links available, then the route will be balanced using a "unilist" next-hop. When one of the links in the "unilist" don't have Layer 2 resolution, these next-hops will actually drop traffic. The fix added by this PR will make unilist not contain forwarding and non-forwarding at the same time. When the next hop resolution will be done, then the link will be added to the unilist. PR832974

  • The error message 'unlink(): failed to delete .perm file: No such file or directory' was logged when disconnecting from a Telnet session to the router. PR876508

  • Starting with Junos OS Release 13.3 and later, the range of CLI screen-width is 40 through 1024 (in earlier Junos OS releases, the range is 0 through 1024). This PR restores the option of setting screen-width to 0 resulting in unlimited screen width. PR936460

  • The Routing Engine and FPCs are connected with an internal Ethernet switch. In some rare case, the FPCs might receive a malformed packet from the Routing Engine (for example, packet gets corrupted somewhere on its way from the Routing Engine to FPC), then the toxic traffic might crash the FPC. PR938578

  • MPC Type 2 3D might crash with CPU hog due to excessive link flaps causing the interrupts to go high. PR938956

  • The issue might come when a non-template filter gets deleted (but does not get completely cleaned up) and the same filter index gets reassigned to a template filter. This could be considered as a timing issue given it comes with a very specific sequence of events only. PR949975

  • On MX Series routers with MPCs or MICs, VPLS traffic might get blocked for about 5 minutes (timer of MAC address aged-out) after re-negotiating control-word. PR973222

  • With NG-MVPN, multicast traffic might get duplicated and/or blackholed if a PE router, with active local receivers, is also a transit node and the P2MP LSP is branched down over an aggregate interface with members on different Packet Forwarding Engines. PR973938

  • On MX Series Virtual Chassis platforms with interface alias configured, this feature might not work as expected and cause interface flapping after commit. PR981249

  • no-propagate-ttl doesn't work for L3VPV when PE is configured with l3vpn-composite-nexthop and its core interfaces are hosted on MPC based FPC. PR985688

  • On MX Series routers with MPCs or MICs, when filter is applied on the interface with the action of "then next-interface", the packets that are forwarded by the firewall filter would be corrupted. PR986555

  • Interface alias was not shown in the show commands when configured. Now interface alias will be shown (IF CONFIGURED) in show commands containing interface names. A |display no-interface-alias command adds the ability to show the actual interface name if it’s needed. PR988245

  • When services packet(interface-style) is diverted to different routing-instance using a firewall filter, route lookup of the services packet was matching a reject route which results in PPE thread timeout. PR988553

  • TXP with Release 13.1R4 might not trigger autoheal after 65535 CRC error event on inter-chassis optical hsl2 link. Customer will need to do manual fabric plane reset to recover the faulty SIBs after the 65535 CRC error event. PR988886

  • NPC core /../src/pfe/ukern/cpu-ppc/ppc603e_panic.c:68 PR989240

  • On logical systems, backup rpd of logical systems is not getting SIGHUP when the "commit fast-synchronize" statement at the [edit system] hierarchy level is enabled. It causes the issue "restarting backup rpd" of logical systems (as part of recovery mechanism). PR990347

  • When two midplane link errors are present between F13 and F2 Sibs then CLOS rerouting logic does not work properly. This can introduce RODR packet drops and result in destination errors in the plane. PR992677

  • "delete" or "deactivate" of apply-group defining the entire TACACS or RADIUS configuration configured under [edit system apply-group <>] does not take effect on commit. This could lead to TACACS or RADIUS based authentication to still continue working despite removal (delete/deactivate) of configuration. PR992837

  • On MX240/MX480/MX960 routers with Multiservices DPC (MS-DPC), the MS-DPC might crash when the MPLS or VPLS with LAG Enhanced is configured. PR993716

  • Packets dropped with IPv6 reject route are currently subjected to loopback IPv6 filter processing on MX Series routers with MPCs or MICs, as a result the packet dropped by a reject route may be seen from the "show firewall log". PR994363

  • On MX Series routers with MPCs/MICs or T4000 router with type5 FPC, if the CoS scheduler is configured without transmit-rate while with buffer-size temporal, the Packet Forwarding Engine might not allocate buffer for the associated queue. The issue might lead to packets loss. PR999029

  • Handle CHASSISD_FRU_UNSUPPORTED event with auto-image-upgrade.slax script. PR1000476

  • MS PIC may reset after GRES in case of excessive resolve traffic. PR1001620

Routing Protocols

  • In PIM-SM network with "bootstrap routing" RP selection mechanism used, it is observed that some bootstrap messages (BSMs) generation and forwarding behavior of Junos OS does not conform to RFC standard, specifically in the section 3.2 (Bootstrap message generation), 3.3 (Sending Candidate-RP-Advertisement Messages) and 3.4 (Creating the RP-Set at the BSR). PR871678

  • In Protocol Independent Multicast (PIM) scenario, if interface get deleted before the (S,G) route is installed in the Routing Information Base (RIB), then this interface index might be re-used by kernel for another interface and thus cause routing protocol process (rpd) core. PR913706

  • In certain rare circumstances, BGP NSR replication to the backup Routing Engine may not make forward progress. This was due to an issue where an internal buffer was not correctly cleared in rare circumstances when the backup Routing Engine was experiencing high CPU. PR975012

  • On EX9200 switches or MX Series platforms with IGMP snooping enabled on an IRB interface, some transit TCP packets may be wrongly considered as IGMP packets, causing packets to be dropped. PR979671

  • Due to some corner cases, certain commits could cause the input and/or output BGP policies to be reexamined causing an increase in rpd CPU utilization PR979971

  • PPMD filter is not programmed properly which is resulting the Routing Engine absorbing BFD packets instead of the Packet Forwarding Engine. PR985035

  • In Junos OS, by default the RIP protocol "send" option is set to Multicast RIPv2. When this "send" option is changed from "multicast"(active) to "none"(passive) or vice-versa, rpd core file might be seen on the router. PR986444

  • in V4 RG, member site receives traffic from both serving sites for few sources upon withdraw/inject routes for 30 seconds. PR988561

Services Applications

  • Clearing the stateful firewall subscriber analysis causes the active subscriber count to display a very huge number. The large number is seen because when a subscriber times out, the number of active subscribers is decremented. If it is set to zero using the clear command, then a decrement would give an incorrect result. There is no impact to the overall functionality. PR939832

  • Jflowd core crashes because of the interface name mismatch between Jflowd config parsing and SRRB. Config parsing treats the interface as ms-*/*/*(without subunit) while SRRB reports ms-*/*/*.*. The fix is to treat interface name without any subunit as interface with subunit .0. PR968922

  • If a PPPoE/PPP user disconnects in the access network without the LAC/LNS noticing it to tear down the connection (also the PPP keepalive hasn't detected yet), and a second PPP request comes from the same subscriber on the L2TP tunnel (same or different LAC/tunnel), then a second route is added to the table having the next hop "service to unknown". PR981488

  • The cflow export would cease due to memory exhaustion when flow-monitoring is enabled using Adaptive Services II PIC due to memory leak condition. While in this condition, user would see increments in "Packet dropped (no memory)" as below: user@node> show services accounting errors Service Accounting interface: sp-3/0/0, Local interface index: 320 Service name: (default sampling) Interface state: Accounting Error information Packets dropped (no memory): 315805425, Packets dropped (not IP): 0 PR982160

  • In H323 ALG with CGNAT scenario, the MS-PIC might crash when the ALG is deleting an H323 conversation due to the deleting port is outside of allocated NAT port-block range. PR982780

  • On M Series, MX Series, and T Series routers (platforms) with Services PIC with dynamic-nat44 translation-type configured, when the flows are cleared the IP addresses in use are never freed. This issue is present in Junos OS Release 11.4R7 and all more recent releases without this fix. PR986974

  • In large scale L2TP LNS environment. When the SNMP MIB JNX-L2TP-MIB is walked continuously, the memory of the L2TP daemon (jl2tpd) increases due to memory leak. PR987678

  • In the Layer-2 Tunneling Protocol (L2TP) environment with "failover-within-preference" configuration. There are two L2TP network servers (LNSs) with different preference, one LNS is primary and another is backup. If the primary LNS is dead, the router doesn't try to create L2TP tunnel to the backup LNS. PR990042

Software Installation and Upgrade

  • By upgrade-with-config, user can specify a configuration to be applied on upgrade, but the configuration file will not be loaded post upgrading. As a result, router will bring up with old configuration. PR983291

User Interface and Configuration

  • When load large scale configuration, due to the ddl object not being freed properly after it's accessed, load configuration failed with error: Out of object identifiers. PR985324

VPNs

  • Upon withdraw/inject bgp routes in the serving PEs for two different route-groups, member/regular sites receive traffic from both serving sites for 60 seconds. PR973623

  • The S-PMSI tunnel might fail to be originated from ingress PE after flapping the routes to customer multicast source. PR983410

  • In MVPN scenario, a multihomed ingress PE might fail to advertise type-4 after losing routes to local sources. PR984946

  • In route-group scenario, source route is flapped on preferred serving site. After that the member site fails to originate type-4 even though it has type-5 and type-3 from non-preferred serving sites. PR994687

  • Make the assert winner send the assert messages in a spaced way just as PIM Hellos and Joins are sent. With fix, the assert winner sends the assert message more often such that helps the other routers on the LAN to maintain state. For now, the robustness count is hard-coded as 3. This will later be enhanced by way of a CLI knob such that the robust count is configurable. PR999019