Help us improve your experience.

Let us know what you think.

Do you have time for a two-minute survey?

 
 

Open Issues

Learn about open issues in this release for MX Series routers.

For the most complete and latest information about known Junos OS defects, use the Juniper Networks online Junos Problem Report Search application.

Class of Service (CoS)

  • The AE interfaces in per-unit-scheduler mode and committing CoS configuration on AE logical interfaces in a single commit leads to race-conditions. PR1666010

EVPN

  • In Provider Backbone Bridging - Ethernet VPN (PBB-EVPN) environment, ARP suppression feature, which is not supported by PBB might be enabled unexpectedly. This might cause MAC addresses of remote CEs not to be learned and hence cause traffic loss. PR1529940

  • This is a case where interface is disabled and comes up as CE after a timeout. A manual intervention of clear ce interface command should restore this. As a workaround, perform the following steps:

    1. Clear auto-evpn ce-interface interface-name.

    2. Configure editactivate interface-name family inet inet6.

    We can fix this by keeping some persistent state on a interface being a core facing interface in some incarnation. PR1630627

  • On all platforms that support Ethernet Virtual Private Networks -Multiprotocol Label Switching (EVPN-MPLS) services, during switchover or l2-learning restart, some EVPN next hops are not correctly associated with routing-instance in Routing Engine (RE), impacting the traffic forwarding. PR1633344

  • When Assisted Replication (AR) feature as Replicator role is used in an Ethernet VPN (EVPN) multi-homed scenario, out-of-bound memory access issue might be observed, which could result in the kernel crash, leading to the service impact. PR1649234

  • On all Junos and Junos Evolved platforms that support Type-2 (T-2) Integrated Routing and Bridging (IRB) symmetric routing, Virtual Extensible LAN (EVPN VXLAN) symmetric type-2 route needs to be imported in Layer 2 mac-vrf instance for IP host route to be added in Layer 3 (L3) vrf. Layer 3 inter-subnet routing will fail if there is no reachability for the remote IP-host route. PR1669585

  • On all Junos platforms running Ethernet Virtual Private Network (EVPN) Multi-Protocol Labeled Switching (MPLS) and a single active Link Aggregation (LAG), the traffic can get forwarded to the Non-Designed Forwarder (NDF) router, which can lead to a traffic drop. This issue is seen in multi-vendor Provider Edge (PE) Customer Edge (CE) setup when there is a DF switchover, that is, Designated Forwarder (DF) role changes and the backup router starts advertising the EVPN type 2 route, now there is another switchover and the old DF again becomes the DF and advertises EVPN type 1 route but the NDF does not withdraw its route consequently the traffic gets forwarded to the NDF. PR1680421

Forwarding and Sampling

  • When GRES is triggered by SSD hardware failure, the syslog error of rpd[2191]: krt_flow_dfwd_open,8073: Failed connecting to DFWD, error checking reply - Operation timed out might be seen. Restart the dfwd daemon to recover the issue. PR1397171

  • When you configure the "fast-lookup-filter" statement with a match that is not supported in the FLT hardware, traffic might be lost. PR1573350

General Routing

  • You might encounter a single event upset (SEU) event that might cause a linked-list corruption of the TQCHIP. The following syslog message gets reported: Fatal error pqt_min_free_cnt is zero Jan 9 08:16:47.295 router fpc0 CMSNG: Fatal ASIC error, chip TQ Jan 9 08:16:47.295 router fpc0 TQ Chip::FATAL ERROR!! from PQT free count is zero Jan 9 08:16:47.380 router alarmd[2427]: Alarm set: FPC color=RED, class=CHASSIS, reason=FPC 0 Fatal Errors - TQ Chip Error code: 0x50002 Jan 9 08:16:47.380 router craftd[2051]: Fatal alarm set, FPC 0 Fatal Errors - TQ Chip Error code: 0x50002 The Junos OS Chassis Management error handling detects such a condition, raises an alarm, and disables the affected Packet Forwarding Engine entity. To recover this Packet Forwarding Engine entity, restart the FPC. Contact your Juniper support representative if the issue persists even after the FPC restarts. PR1254415

  • If you take a vmhost snapshot on an alternate disk and there is no further vmhost software image upgrade, the expectation is that if the current vmhost image gets corrupted, the system boots with the alternate disk so the user can recover the primary disk to restore the state. However, the host root file system and the node boots with the previous vmhost software instead of the alternate disk. PR1281554

  • When you add VLAN as an action for changing the VLAN in both ingress and egress filters, the filter is not installed. PR1362609

  • On MX Series routers with MPC7E, MPC8E, or MPC9E installed, if optics QSFPP-4X10GE-LR from Innolight vendor (subset of modules with part number 740-054050) is used, the link might flap. PR1436275

  • With NAT/Stateful-firewall/TCP tickle (enable by default) configured on MS-MPC/MS-MIC, the vmcore crash might occur along with the mspmand crash if large-scale traffic flows (for example, million flows) are processed by it. PR1482400

  • The issue occurs when there are hardware link errors on all 32 links on an FPC 11. Because of these link errors, all FPCs reported destination errors towards FPC 11 and FPC 11 get taken offline with the reason offlined due to unreachable destinations. PR1483529

  • Runt, fragment, and jabber counters are not incrementing. PR1492605

  • After backup Routing Engine halt, CB1 goes offline and comes back online; this leads to the backup Routing Engine booting up, and it shows the reboot reason as 0x1:power cycle/failure. This issue is only for the Routing Engine reboot reason, and there is no other functional impact of this. PR1497592

  • PR1463859 introduces a software defect that causes a 10GE interface to flap continuously when configuring with the WAN-PHY framing with the default "hold-down" timer (0). Once upgrading a router to an affected software release, the interface might flap continuously. This is not applicable to an interface with the default framing - LAN-PHY. PR1508794

  • The AMS bundle state toggles momentarily as up, down, and up after configuring commit for a scaled scenario. PR1521929

  • In Mac-OS platforms when Juniper Secure Connect client connects successfully, the client is not getting minimized to tray icon and needs to be minimized manually. PR1525889

  • Due to BRCM KBP issue route lookup might fail. Need to upgrade KBP to address this issue. PR1533513

  • If vMX product is configured to run in performance mode by configuring chassis fpc 0 performance-mode (note: performance mode is enabled by default starting from Junos OS Release 15.1F6), flow cache will be used to improve the traffic forwarding performance. With performance mode enabled, if traffic cause a single flow in the flow cache to have a large number of flow actions, which hit the max supported number (i.e. 18) of flow actions. Typically, the addition of lots of firewall counters and policers in a single flow can make it add up, and the riot might crash. It is a rare issue. PR1534145

  • The Flexible PIC Concentrator (FPC) might generate a core file if the flap-trap-monitor feature under set protocols oam ethernet cfm performance-monitoring sla-iterator-profiles is used and performance monitoring flap occurs. PR1536417

  • In scaled MX2020 router, with vrf localisation enabled, 4 million nexthop scale, 800k route scale. FPCs might go offline on GRES. Post GRES, router continues to report many fabric related CM_ALARMs. FPC might continue to reboot and not come online. Rebooting master and backup Routing Engine will help recover and get router back into stable state. PR1539305

  • On MX platforms with MS-MPC/MS-MIC service card installed, the card might run out of memory due to process mspmand memory leak, which might cause traffic interruption if adding and/or deleting of telemetry sensor. This is because these operations will trigger the memory allocation for decoding configuration change messages and will not release the memory at the end of processing. PR1540538

  • During Routing Engine switchover interface flap might be seen along with Scheduler slippage. PR1541772

  • USF-SPC3 : With ipsec PMI/fat-core enabled, show services sessions utilization CLI not displaying correct CPU utilization. PR1557751

  • The Sync-E to PTP transient simulated by Calnex Paragon Test equipment is not real network scenario. In real network deployment model typically there will be two Sync-E sources (Primary and Secondary) and switchover happens from one source to another source. MPCE7 would pass real network SyncE switchover and associated transient mask. PR1557999

  • Support switchover on routing-crash knob during abnormal termination of RPD. PR1561059

  • Due to a race condition, the show multicast route extensive instance instance-name output might display the session status as Invalid. Such an output is a cosmetic defect and not indicative of a functional issue. PR1562387

  • To avoid the additional interface flap , interface hold time needs to be configured. PR1562857

  • When Inline Jflow is configured and high sampling rate (more than 4000 per second) is set, high CPU utilization might be observed and this might result in relevant impacts on traffic analysis and billing. PR1569229

  • Copying files to /tmp/ causes a huge JTASK_SCHED_SLIP. Copy files to /var/tmp/ instead. PR1571214

  • This issue is caused by /8 pool with block size as 1, when you commit the configuration the block creation utilizes more memory causing NAT pool memory shortage, which is currently being notified to customer with syslog tagged RT_NAT_POOL_MEMORY_SHORTAGE. PR1579627

  • In a fully loaded devices, at times, firewall programming was failing due to scaled prefix configuration with more than 64800 entries. However, this issue is not observed in development setup. PR1581767

  • Error message seen on MX10K8 chassis with SyncE/PTP configurations, This does not affect any functionality, The error seen here because the API called is specific to ferrari platform which needs to be vecterized. PR1583496

  • On all devices running Junos 19.1R3-S5-J3, the subscriber IFL(logical interface) might be in a stuck state after the Extensible Subscriber Services Manager (ESSM) deletion. PR1591603

  • Pim Vxlan not working on TD3 chipsets enabling VxLAN flexflow after Junos OS Release 21.3R1. Customers Pim Vxlan or data plane VxLAN can use the 21.3R1 version. PR1597276

  • MX2010, MX2020: MPC11E: Unified ISSU is not supported for software upgrades from Junos OS Release 21.2 to 21.3 and 21.4 releases due to a flag day change. PR1597728

  • During Routing Engine switchover, if there is a burst of ICMP/BFD/SSH/FTP/TELNET/RSVP packets (~18K pps) you might see new backup RE restarting. PR1604299

  • On Virtual Chassis (MX-VC) platforms with MS-MPC or SPC3 service cards and Aggregated Multi-Service (AMS), traffic on the line card in the backup chassis might not be load-balanced properly due to timing conditions. This works well on the line card in the master chassis. There might be traffic loss when interfaces are not properly balanced. PR1605284

  • Leaf difference with regards to memory-usage/heap in the output of Sensor (/junos/system/linecard/firewall) between MPC7E and MPC10E. PR1606791

  • On all MX platforms, in a subscriber management environment, new subscribers might not connect if Class-of-Service (CoS) CR-features (Classifier Rewrite) are used by the Variable Based Flow (VBF) service. The reference count mismatching between Routing Engine (RE) and VBF is caused by VBF flow VAR CHANGE failure. PR1607056

  • If RPD Agent sends INH deletion/additions out of order on a rare occasion to backup RPD, RPD generates core files. PR1607553

  • When user tries to disable AMS ifd using configuration statement, the ipsec tunnels are not deleted. Deactivating the services will provide the desired result. PR1613432

  • In some NAPT44 and NAT64 scenarios, Duplicate SESSION_CLOSE syslog will be seen. PR1614358

  • Memory Zone is not reflecting properly while doing Memory Tests through Vty command test usp service-sets memory-testing start. PR1619499

  • Tunnel statistics displays incorrect values because it was not supposed that tunnel interfaces would cache flow. PR1627713

  • For a topology with VSTP and VRRP configured and IPv6 traffic, if VSTP bridge priority is changed a couple of times (to trigger toggling of root bridge), V6 traffic drop might be seen on some of the streams. PR1629345

  • For ACX5448, MX204, and MX2008 "VM Host-based" platforms, starting with Junos OS Release 21.4R1 or later, ssh and root login is required for copying line card image (chspmb.elf for MX2008) from Junos VM to Linux host during installation. The ssh and root login are required during installation. Use deny-password instead of deny as default root-login option under ssh configuration to allow internal trusted communication. Ref https://kb.juniper.net/TSB18224. PR1629943

  • The fabric statistics counters are not displayed in the output of show snmp mib walk ascii jnxFabricMib. PR1634372

  • On all devices running Junos OS or Junos OS Evolved, where this is a high BGP scale with flapping route and the BGP Monitoring Protocol (BMP) collector/station is very slow, the rpd process might crash due to memory pressure. PR1635143

  • Same vlan cannot be used as data vlan and voip vlan together. PR1637195

  • Script fails while verifying Access Internal Routes after daemon restart during advanced DHCP test. PR1640567

  • WIth PTPoIPv6 on MPC2E 3D EQ, PTP slave stays in acquiring state. PR1642890

  • When CFP2-DCO is used, operator need to configure otn-option- that is the only mode supported. PR1643815

  • Committing configuration changes during the Packet Forwarding Engine reset pause window (when PFE is disabled, yet the PFE reset proper has not started yet) has the potential of causing errors and traffic loss. In particular, configuration changes that result in re-allocating policers (which are HMC-based) might lead to traffic being entirely policed out (i.e. not flowing). Once the PFE reset procedure has started configuration changes ought to be avoided until the procedure is completely done. PR1644661

  • bb device has to be manually enabled in configuration for DHCP and PPP access models for BNG CUPS. Configuration to enable bb device is as follows: set system subscriber-management mode force-broadband-device. PR1645075

  • On Junos platform, PTP does not lock when port speed is not configured under PIC hierarchy or port speed for some additional random ports are configured under the PIC hierarchy or perform PIC deactivate/activate. PR1645562

  • When per-interface egress and per-sid egress SR sensor stats are configured using the CLI commands below, the (pushed) MPLS label length does not get included in the output/Tx octets field that gets exported from the sensor. set protocols isis source-packet-routing sensor-based-stats per-interface-per-member-link egress set protocols isis source-packet-routing sensor-based-stats per-sid egress.PR1646799

  • With overlapping NAT pool configured with different NAT rules under different service sets, when service outside interface is moved between different routing instances (EX: from vr1 to default, and from default to vr1), NAT routes corresponding to the service-set in default routing instance are getting deleted, resulting in reverse path traffic failure for NAT sessions. PR1646822

  • In the IPv6 segment routing deployment, packets are sent out with the incorrect ethernet type. PR1647622

  • V6 default route will not get added after successful dhcpv6 client binding on PTX1000 router during ztp. PR1649576

  • Configuring MPC11 in 4x100G and keeping peer in 400G mode, Link comes up on peer while staying down on local end. This issue is also specific to 400G-ZR optics as it has single media lane. The issue is not seen on other 400G optics supporting 4x100G mode. PR1653946

  • When interop with the following systems, flow control must be enabled when MACsec is configured on the peer system. Because on these systems, flow control is forced to be on regardless of the CLI provisioning. PR1655712

  • On MX304 in few cases if mce-inject is invoked, logs reporting MCE errors might not get registered .PR1656004

  • Core file reported intermittently where random grpc stack crash is observed. The license service will auto restart and recover. PR1656975

  • Interop for 1G interfaces between EX4100 SKUs and ACX5448/ACX5448-M/D or MX480 will not work. PR1657766

  • UDP Telemetry output fields may misalign on AFT-based line cards such as MPC10/11 or Junos Evolved platform. PR1658017

  • ZTP: DHCPACK is not received at ztp-server after zeroize of the device (client). PR1658287

  • SNMP MIB walk on jnxVpnInfo show snmp mib walk jnxVpnInfo for EVPN or EVPN-VPWS routing instance. PR1659466

  • When there is hard failure on the RE-RE link, the system might fail to recognise the fault and report as an alarm. PR1661635

  • The version details for certain daemons will appear in the command output after the device has been rebooted after the completion of the USB installation of Junos. PR1662691

  • MX10k8 with MX10K-LC2101 Linecard(s) supports *PTP* only with JNP10008-SF Switch Fabric Board(s), *PTP* currently doesn't work with JNP10008-SF2 Switch Fabric Board(s). PR1664569

  • RE0 to RE1 interface EM4 MTU is changed to 9192 bytes through PR 1642364. If one of the REs don't have this fix, Routing Engine sync fails. Due to this reason, unified ISSU will not work. In such scenario, cold image upgrade should be done. PR1665690

  • MX240: Verify VRRP stats is failed after Deactivate the Access interface. PR1666943

  • On MX platforms with MIC-MACSEC-20GE, Forwarding Engine Board (FEB) might go down while activating/deactivating Graceful Routing Engine Switchover (GRES) configuration. PR1668983

  • These are expected error logs, and doesn't cause any functional impact. jsr_iha_pri_unrepl_msg_func: Error: Invalid primary handle in msg 0x10006c600000621, error=2 These logs might be seen if the following conditions are met:

    • On all Junos OS platforms

    • Non stop routing is enabled.

    • with scaled setup

    • The possible triggers would be restart chassisd, ksyncd, switchover, re reboot... which causes nsr unreplication/replication.

    PR1675057

  • On the 48 Port SKU, tagged packet greater than MTU size 1518 gets dropped due to BCM EGR_MTU set to 1518 on ports 0-24. Mitigation is to set interface MTU to 1518. PR1677902

  • The IFD remaining stats flag is not set properly in chassid in today's code. It should be set to TRUE only if HCOS is configured on an interface. Else, it should not be SET. Not setting this rightly, results in statistics not being displayed OR the command output not being displayed at all. The impacted command is run show interfaces extensive intf-name and the impact is seen in GNF environment with no explicit CoS configuration on the interfaces. Not using extensive will ensure there is no issue as well. This is specific to MPC11 with sub LC (GNF) setup. PR1678071

  • Invalid PIC configuration inside GNF might add delay to clear interface statistic all command. This issue does not impact any functionality. chassis fpc SLOT pic IS. PR1683312

High Availability (HA) and Resiliency

  • When you perform GRES with the interface em0 (or fxp0) disabled on the primary Routing Engine, then enable the interface on the new backup Routing Engine, it isn't able to access network. PR1372087

Infrastructure

  • You can the following IPC timeouts logs for statistics query to kernel (queried from CLI or daemons querying internally) when there is configuration churn, or large number of IPCs getting exchanged between kernel and pfe in the system.

    if_pfe_msg_handler: pfe_peer_msg_handler error: error for msg type type, msg subtype subtype, opcode op and peer index index

    Default IPC timeout value in kernel for IPC statistics request is 10s. PR1629930

Layer 2 Features

  • In case of the access-side interfaces used as SP-style interfaces, when you add a new logical interface and if there is already a logical interface on the physical interface, there is 20-50 ms traffic drop on the existing logical interface. PR1367488

MPLS

  • In MVPN Case, if the nexthop index of a group is not same between master and backup after a nsr switchover, we might see a packet loss of 250 to 400 ms. PR1561287

  • Ingress will retry after lsp stay down for extended period of time or customer can clear lsp to speed up the retry. PR1631774

Network Management and Monitoring

  • When you configure maximum-password-length and user tries to configure password whose length exceeds configured maximum-password-length, the system displays error, along with error ok/ tag is also emitted. (Ideally 'ok' tag should not be emitted in an error scenario.) The configuration does not get committed.PR1585855

  • The mgd might crash when you configure an invalid value for identityref type leafs/leaf-lists while configuring Openconfig or any other third-party YANG, problem happens with json and xml loads. PR1615773

Platform and Infrastructure

  • On all Junos and Junos OS Evolved platforms, while using source-address NTP configuration parameter and issue the command set ntp date from the CLI, packets will be sent with the source address of the outgoing interface rather than the manually configured IP address. Typically, the manually configured IP address would be a loopback address. The problem does not apply to automatically generated NTP poll packets. PR1545022

  • Don't use the control-type light under platforms where this feature is not supported at present. At present IPv4 and IPv6 twamp-light is supported on the platforms using TRIO and PE chipsets. PR1603128

  • VM generates core files and VC might split with multicast scale scenario. PR1614145

  • With given multi dimensional scale, if configuration is removed and restored continuously for more than 24 times, the FPC might crash and restart. During the reboot, there can be traffic impact if backup paths are not configured. PR1636758

  • The process sshd will crash if Terminal Access Controller Access Control System (TACACS) user authentication is configured and the user is successfully authenticated by the TACACS server. The crash occurs with every successful authentication and an sshd core file will be created. Because of the core ssh, access is denied. PR1672581

Routing Protocols

  • On all platforms, the issue is when the first time when ESIS is coming up sometimes the ESIS route might not get installed. PR1559005

  • On MX platforms, initial multicast register packets might get dropped, this might affect multicast services. PR1621358

  • On all Junos and Junos OS Evolved platforms, when configuring the network instance for openconfig, an error might be observed while executing a commit if the configured network instance type is default_instance but the instance name is not default. PR1644421

  • RFC 8950/RFC 5549, permits the advertisement of a BGP Nexthop of a different family (for example, IPv6) than the NLRI address family (for example, IPv4). The mapping of possible address families that can be used are exchanged using BGP Capabilities. The BGP Capabilities specification, RFC 5492, recommends that a single capability TLV of a given type is advertised when multiple elements within that TLV are present. That RFC also permits multiple capabilities of the same type to be advertised for multiple elements for backward compatibility. Junos BGP handling of the BGP extended nexthop capability did not handle multiple capabilities of the same code point when multiple extended nexthop capabilities were present. It incorrectly kept only the last one sent. This PR addresses that deficiency. PR1649332

  • The show security keychain detail CLI displays algorithm as hmac-* instead of ao. PR1651195

  • When l2cpd (in the context of xSTP) clears the entries that it has programmed on ppmd, that is, when you delete xSTP configurations from the box, there can be a possibility of ppmd core file. If ppmd is in distributed mode then there will be no service impact, else there can be service impact as packet transmission for various protocols will happen via if ppmd is in centralized mode. PR1660299

  • On all Junos and Junos Evolved platforms, by default, inactive routes from inet.3 are advertised in BGP. When add-path is configured, inactive routes from inet.3 are no longer advertised, this behavior is unexpected. PR1665610

  • On all Junos and Junos Evolved platforms, when BGP update-threading is enabled and executed a certain sequence of flap events like flapping IS-IS adjacencies, LDP, iBGP peers, clearing ARP table, interface flap for an hour by turning OFF and ON laser which triggers the rpd crash. It is a rare case. PR1669615

  • The routing protocol daemon (rpd) crash happens when a non Border Gateway Protocol (non-BGP) route is exported through Label Distribution Protocol (LDP) and later on, if the same prefix advertised by BGP is received, we select that as the forwarded route rather than the already advertised route. This is due to the logic in the forwarding route to pick the BGP route for the BGP-owned routes rather than the active route. This is a rare case and the system recovers by itself until the next event. PR1671081

  • Dynamic IFL add request is waiting to be processed in KRT queue during that if chassid down event occurs. To handle the chassid down event RPD Infra sends notification to protocol (producer of Dynamic IFL add request) and it is producer jobs to DELETE the dynamic ifls request which were there in the KRT queue. PIM code is not clearing those Dyanamic IFLs ADD request during chassis fpc down event. Hence getting error ENOENT -- Item not found when chassis comes back up. PR1675212

  • Any platforms with Micro BFD configured on member links of the LAG/ae interface, BFD Session state in RE remains as UP always even though PEER device has ceased. PR1675921

  • On all Junos and Junos Evolved platforms, the routing protocol daemon (rpd) might crash when Protocol Independent Multicast (PIM), Multicast only Fast Reroute (MoFRR) configuration is present and some network churn event such as continuous interface cost changes, resulting in a change of active and backup paths for Equal Cost Multi-Path (ECMP) happens. There will be service impact because of the rpd crash but the system self-recovers until the next crash. PR1676154

  • VRF Rouging table might not get updated immediately upon change of maximum-prefixes. PR1680277

User Interface and Configuration

  • Root cause- In test configuration flow we are calling mustd as /usr/sbin/mustd -q /var/run/db/file.data /var/run/db/file.data+ -F -m where we just copy the existing cog.db (generated out of committed config) as cdg.db+ , use it for testing the configuration passed, and remove it once the testing is done. This is creating the issue because the configuration is tested against the existing cdg.db. Fix would be to create the cdg.db fresh from the configuration to be tested and test the configuration against that. This new cdg.db should not replace the existing one in /var/run/db. PR1671112

VPNs

  • Tunnel debugging configuration is not synchronized to the backup node. It needs to be configured again after RG0 failover. PR1450393

  • Change here is basically reverting to old enum value used for ATM VPN, and using a new value for BGP Multicast address family, and although these is no visible behavior change due to this, there might be impact on unified ISSU for ATMVPN and BGP Multicast address family if enabled. PR1590331

  • When using Group VPN, in certain cases, the PUSH ACK message from the group member to the group key server might be lost. The group member can still send rekey requests for the TEK SAs before the hard lifetime expiry. Only if the key server sends any new PUSH messages to the group members, those updates would not be received by the group member since the key server would have removed the member from registered members list. PR1608290