Open Issues
Learn about open issues in this release for ACX Series routers.
For the most complete and latest information about known Junos OS Evolved defects, use the Juniper Networks online Junos Problem Report Search application.
EVPN
-
On Junos OS Evolved platforms, if MAC-VRF instances with VXLAN encapsulation associated with IRB Logical interfaces in primary instance (and not in a separate VRF) and ospf is enabled on all interfaces, there could be a next-hop loop where remote PE loopback IP first resolves over IP underlay, remote IRB IP is reachable over the VXLAN tunnel to remote PE loopback and then remote PE loopback is resolved due to lower OSPF metric over IRB. Configure
protocols ospf interface irb.x passive
statement remote PE loopback isn't learnt over IRB adjacency. Typically, IRB logical interfaces are added to a VRF, which prevents this loop. PR1626574
Fault Management
-
On FPC log, when the system is up, the Packet Forwarding Engine error message Jexpr: JexprHandleDdos faile to update plct pfe:1f proto:0x8400 will be seen. This error message is just a debugging error message, there is no functional impact with this log error messages. PR1610764
General Routing
-
There is no impact on these traces. Here are some RPC calls which throws error when there is no handler. This does not have any functionality impact. PR1589410
-
4x400G FPC is supported on FPC slot 1 and FPC slot 5 only as per design. 4x400G FPC plugged into any slot might bring FPC online but links come up only in FPC1 or FPC5. When 4x400G FPC is plugged into other slots, it does not allow the FPC to come online and an alarm is raised. PR1582183
-
ACX7509: some of the interfaces from 16x100G and 20XSFP56 will not go down after evo-pfemand restart. PR1592388
-
Error messages are observed while accessing the supercon scratch pad registers on bootup and there is no functional impact. PR1594136
-
On Junos OS Evolved platforms, if a firewall filter has both Source Class Usage (SCU) or Destination Class Usage (DCU) matches and forwarding-class or loss-priority matches used on the same term in the 'from' action, the traffic supposed to be subjected to the firewall filter might not get subjected to it. PR1595788
-
When you restart FPC with scale configuration or routes or MAC entries, picd might generate a core file. PICD backup automatically and user intervention is not required. PR1595861 and PR1602352
-
The timingd application cannot be successfully restarted for ACX7100-32C and ACX7100-48L in the first G.8275.1 software release. PR1597120
-
Currently, support for load balancing is on ECMP and not on ECMP FRR. Behaviour is the same as in ACX7100. PR1599752
-
RPF fail counters reset on family modification is expected due to product limitations. PR1598135
-
ACX7509 will do hardware timestamping, but peer device (in this case MX) does the software timestamping, which takes more time resulting in high delays. This issue will be seen in all the cases where peer setup does software timestamping. PR1599777
-
With IGMPv3 reports received at a higher rate more than 1600 pps, packets are dropped due to control plane rate limit. Therefore, it is not possible to form 2,56,000 IGMP groups. Need to tweak DDOS configuration for reaching 2,56,000 IGMP groups. PR1599998
-
In scaled environment, the interfaces do not come up during FEB offline - online since the Packet Forwarding Engine state is not transitioning to online though FEB has transitioned to online state. PR1601158
-
ACX7509 has a mesh fabric and it is not a real fabric. There is a flow control mechanism from fabric to CGM. In case of fabric congestion there is flow control and back pressure to the VOQs and packets are dropped in VOQs. These VOQs cannot be mapped directly to fabric congestion as VOQ drops can happen due to different reasons. There is no such specific dedicated registers/counters from BCM to provide values for fabric drops. As of now its accounted as queue drops as fabric is back pressuring ingress. PR1601332
-
The output of the
show system processes extensive
command shows high short term CPU utilization. The values ranges from 50 percent or higher for evo-pfemand. This is a single CPU view. As the ACX7509 system is a multi-core CPU, this has no impact on performance. PR1603899 -
When dual Routing Engines are inserted in the system and these logs come from the backup Routing Engine. Supercon fpga has two end point devices (for RE0 and RE1) in which the primary Routing Engine end point device is enabled and the backup Routing Engine is disabled in the hierarchy. During boot up, the backup Routing Engine tries to access [supercon fpga in pci] hierarchy that is disabled. Therefore, supercon scratch pad register read failure logs and pci uncorrected errors are seen. PR1605797 and PR1615157
-
PR 1606585
The error message Failed to attach ACPI GPIO chip seen on console in early bootup is not an issue and it does not have any known functionality issue. It might be ignored. PR1606585
-
The evo-pfemand process that programs the Packet Forwarding Engine ASIC restarts. As the evo-pfemand restarts, it re-programs the system ports. The hardware linkscan thread that monitors the link status records a momentary link down or link up transition. Each time the linkscan thread observes a change, it sends out a notification. Due to state compression in DDS, consumer applications like Picd do not receive intermediate link-down status notification from evo-pfemand and the interface does not toggle. This is because the notifications are produced rapidly. The remote peer interfaces observe an interface down while some of the local interfaces remain up. PR1608215
-
CAUTION:
For 20xSFP FPC, following speed combination are possible
Slot PortGroups Speed 0 0-3, 4-7, 8-11, 12-15 and 16-19. 25g/10g/1g 1 0-7, 8-16, and 16-19 25g/10g 2 0-3, 4-7,8-11,12-15 and 16-19. 25g/10g/1g 3 0-3, 4-7,8-11,12-15 and 16-19. 25g/10g/1g 4 0-3, 4-7,8-11,12-15 and 16-19. 25g/10g/1g 5 0-7, 8-16, and 16-19 25g/10g 6 0-3, 4-7,8-11,12-15 and 16-19 25g/10g/1g 7 0-3, 4-7,8-11,12-15 and 16-19. 25g/10g/1g
Please note that:
- There can be a combination of speed as initial configuration, however, a link flap is observed within the ports of the port groups if the speeds of any port are reconfigured to other speeds.
- To avoid such a situation all the 1614286 ports of the port group can have single-speed or do not reconfigure the speed within the port group.
-
While rebooting the system, the difference between time in which the interface comes up and the time in which RE1 comes up is approximately 3 minutes when compared to RE0 interface up time. PR1608527
-
The syncE to PTP and syncE to 1pps transient response marginally fails. This happens when the servo get the initial 100 nano seconds jump in one measurement window and the next 100 nano seconds in the next measurement window adjusting less initially. PR1608934 and PR1611848
-
The maximum number of ECMP paths supported is 128. PR1609063 and PR1632055
-
A restart of DHCP takes more time because of the internal issues with the SIGTERM event. PR1610229
-
While sending traffic from both the core files or units, all (16) VOQs/connectors need credits from same port. As a result, credits are distributed per the weight ratio and flows that need higher credits get higher bandwidth than the others. PR1611028
-
PTP to PTP noise transfer fails for frequency 0.03125 HZ PR1611838
-
The syncE to PTP and syncE to 1pps noise transfer tests fail for 1. 0.00781 HZ 2. 0.01563 HZ 3. 0.03125 HZ 4. 0.06156 HZ 5. 0.12313 HZ frequencies. PR1611911
-
PR 1612105
it is day1 issue in ACX card. this is not problem only for multicast packet field in mac statistics . it is problem for all fields in mac statistics when queried for AE interface alone. problem is not there for actual ifd interface which is under AE interface. PR1612105
-
On an FTC FRU hot removal on ACX7509, the following error is seen: FAN -Error: value = 0xffff880e ret = 0xfffffffb and there could be a kernel call stack trace core file in journal or dmesg. These are harmless messages and can be ignored. PR1613389
-
"clear mpls lsp" operation is a destructive operation where it wipes off all existing routes and next-hops in the system and does a fresh reinstallation, the 10 seconds delay in traffic restoration for 16000 l3vpn routes might be attributed to programming delay in the hardware units combined with software model and the CPU capacity. PR1614413
-
On ACX7509, 1GE interface does not come up with copper 1G SFP-T optics and this issue is specific to copper 1G cables. PR1614286
-
Transient zl30642 PLL alarms are seen for the Forwarding Engine Board (FEB) or Flexible PIC Concentrators (FPC) seen when FEB or FPC is brought online on ACX7509. These alarms occur during the period of powerup and will clear in a matter of seconds when the PLL locks are achieved. Therefore, it can be safely ignored if they occur and clear during the onlining of these field-replaceable units (FRUs) - Major FEB 0 zl30642 PLL Input Failure Major FEB 0 zl30642 PLL LOCK Failure Major FEB 0 zl30642 PLL OCXO Failure Major FPC 1 zl30642 PLL Input Failure Major FPC 1 zl30642 PLL LOCK Failure Major FPC 1 zl30642 PLL OCXO Failure. PR1615688
-
A race condition between overlay and kernel deregister is seen. Workaround is before doing jackout, run the command
request chassis fpc offline
and then jack-out. PR1618133 -
Following scenarios warrant reboot or power cycle of the box.
- Ungraceful removal of FRU without turning it offline.
- Power failure in FRU.
- PCI link failures in the system.
These issues can not be recovered in a live box and require reboot/power cycle. PR1619368
-
The issue is specific to slot-7, where slot-7, port-13 is not supported. As part of channelization constraints, both adjacent ports have to configure or both to non channelization. Since slot-7 port is unsupported, need to ignore channelization constraints for port the two ports. PR1620425
-
PE1 ---- P ---- PE2
ping mpls l2circuit
does not work in case ofexplicit-null
is configured in PE routers. It does not affect data path traffic. PR1621111 -
In certain Segment Routing topologies having routers with varying CPU load, higher convergence time might be seen for a switchover event because of the micro-loops. This is a known behaviour when one node converges faster than the neighbouring node. PR1621263
-
In case of multicast replication mode as ingress-egress-recycle, load balancing in the aggregate Ethernet member interfaces does not occur. PR1621377
-
Due to major code changes and more regression impact, no details of auto negotiation settings is available on 1GE interfaces. PR1621991
-
The ACX7509 has multiple FPC slots, statistics are maintained with single slot (slot 0 ) since Packet Forwarding Engine is centralized. ACX7509 is the only platform which has multiple FPC slot among ACX Series card. The pfestatsd is querying statistics based upon FPC slot for clear command which is causing issue since Junos OS Evolved infrastructure is not maintaining statistics based upon FPC slot for ACX7509. This needs change in pfestatsd which requires testing in other multi FPC slot platform also . As a workaround, other traffic counters like interface statistics instead of Packet Forwarding Engine statistics can be used. Packet Forwarding Engine statistics is rarely used to debug. PR1622515
-
On upgrading Junos OS Evolved, FPCs transition PowerOff -> PowerOnWait -> PowerOnPermission -> PowerOnStart. But there is a glitch while it was trying to PowerOn and state went to Empty state from PowerOnStart. This leads to Fpc::OnDelete where we set SetFrueDeleted(true); to true. However, it does not go for FpcFrueDeleteCleanup(fpc_name); fruedelete cleanup where we reset the SetFrueDeleted flag back to false as FPC is still in not ready state (FPC did not come online as dmf init not yet completed) and will defer the termination.PR1623455
-
The user can configure a family ethernet-switching filter to match the dmac( 01:80:c2:00:0e) and etype(0x88f7) and drop such packets. Configuration gives us the flexibility to install filter only for the services where the drop is needed and others can forward as required. PR1623756
-
RPC key failures error messages "Invalid RPC request key: 0x00110000" and "Invalid RPC request key: 0x000b0009" are observed while loading multid configuration. This does not have any functionality impact. PR1624635
-
AOC cables go down on multiple restart of FPC. PR1624992
-
The error message is expected while creation of IRB intermittently, this has no functional impact. PR1625782
-
The Routing Engine can panic on back-to-back routing restarts due to timer_list corruption. 22.x images use newer Linux kernel (version 5.2+) where timer subsystem and callers have been updated to prevent this. PR1626361
-
400G-ZR optic transceiver firmware upgrade fails. It is a generic issue seen across all single RU platforms supporting QDD-400G-ZR. PR1626882
-
After picd or rpdagent application restart multpile object-info anomalies for evo-pfemand, below are the types of anomalies seen Type : net::juniper::rtnh::Route Type : net::juniper::rtnh::NHOpaqueTlv Type : net::juniper::rtnh::Nexthop Type : net::juniper::rtnh::Unilist Type : net::juniper::rtnh::BfdSessionId. PR1628843
-
If a system is fully scaled across features and firewall is also scaled, CPU consumption might be more for a small window of around 5 seconds after every 18 seconds or so. Evo-pfemand might be busy collecting the scaled firewall statistics for that 5 second window and any other applications like 'pfe-cli' trying to execute commands might fail during it. PR1629342
-
In the Layer 2 performance test we have observed that for some of the packet sizes we are not able to achieve 100 percent line rate across the mesh fabric between the Q2C s. We use 42, 100G ports and expecting 4.2Tbps. However, we are able to achieve 4.17T instead of 4.2T. This happens for some specific packet sizes after 352 bytes and before 406 bytes. The vendor already clarified that for some of the packet sizes we won't be able to achieve 4.2T because of inefficiency of packing into cells. PR1631948
-
When a fan tray fails an alarm is raised. After rebooting the alarm gets cleared. However, the fan failure condition will be logged in the log file. PR1633353
-
In a working and non-working logs, l2d index is different for vrrp group number 187. This is the same group for which packet is getting dropped out of 400 groups, other groups are working as expected. So there is some fix which went between working and NKWR related to l2dId which has exposed VRRP issue. Both VRRP MAC and interface MAC gets stored in SLU my_mac_hash table. For finding hash index for vrrp mac we use l2dId, protocol type and vrrp group number as a key. In a non-working scenario there is a collision between interface mac and vrrp mac on same hash index. Ideally hash movement should have happened to address collision but somehow it is not properly done. Going further we need to debug why hash movement is not happening and fix that code area. This code is very sensitive and requires a lot of testing before doing any changes. So we should commit it in DCB first before committing to any other release. PR1633986
-
Interfaces in 4x400g linecard are flapping when unrelated FPC is offlined in multiple FPC offline or online test. PR1635115
-
In scaled setup, while FPC restart, PICD is not releasing fruHwStatus status and causing PIC struck in online. PR1635941
-
FEB jackout and jackin might generate a hwd core file while rebooting the system (but not at jackout or jackin time). There will not be any functionality issue as the core file is generated only during reboot sequence. System is expected to come up fine. PR1636243
Interfaces and Chassis
-
On ACX7509, there is a limitation in adding more than 64 member links in 1 AE, whereas from ASIC, there is no limitation. PR1627951
-
On ACX7509, ACX7100-48L, and ACX7100-32C, when a large number of CFM sessions with 10ms or 3.3ms continuity-check interval are configured and evo-pfemand process is restarted or gets crashed, then the CFM sessions might not come up even after evo-pfemand is up. PR1634721
Layer 2 Features
-
On Junos OS Evolved 21.4-based software, with VPLS instances at some scale, if the RPD process is killed, it might lead to inconsistency for some instances. In LSI information between control and forwarding planes, specifically
show vpls connections instance
andshow ethernet-switching table instance
and traffic drops due to incorrect VPLS label imposed at ingress. As a workaround, restart the rpd and/or l2ald gracefully. PR1627593
User Interface and Configuration
-
In a rare event after configuring and deleting the DDL configuration statements such as
set chassis aggregated-devices
andset interface ae1 aggregated-ether-options
, the configuration object gets out of scope with no functional impact. PR1617667 -
File delete with regex might fail, if using filename without regex it works. PR1624562
-
After picd or rpdagent application restarts, multpile object-info anomalies for evo-pfemand are observed. Below are the types of anomalies seen Type : net::juniper::rtnh::Route Type : net::juniper::rtnh::NHOpaqueTlv Type : net::juniper::rtnh::Nexthop Type : net::juniper::rtnh::Unilist Type : net::juniper::rtnh::BfdSessionId. PR1628843