Navigation  Back up to About Overview 
ContentIndex
  
[+] Expand All
[-] Collapse All

No index entries found.

Resolved Issues in Junos OS Release 12.3 for M Series, MX Series, and T Series Routers

The current software release is Release 12.3. For information about obtaining the software packages, see Upgrade and Downgrade Instructions for Junos OS Release 12.3 for M Series, MX Series, and T Series Routers.

Resolved Issues

Forwarding and Sampling

  • On MX Series platform with MX-FPC/DPC, M7/10i with Enhance-FEB, M120, M320 with E3-FPC, when there are large sized IPv6 firewall filters(for example, use prefix lists with 64k prefixes each) enabled, commit/commit check would fail and dfwd process would crash after configuration commit/commit check. There is no operational impact. PR1120633: This issue has been resolved.

General Routing

  • The issue will happen under the following conditions: * The "forwarding-options sampling input maximum-packet-length" configuration statement is configured to a non-zero value; * Packets are sent to be sampled from a Type 5 FPC to an ES-Type FPC housing Multiservices PIC. Trigger: * Receive packets that should be sampled and the IP payload size is 53 bytes or above. Then the respective packets will be dropped and some SONN error logs: SRCHIP(0): 1 Bad packets on p1 SRCHIP(0): 1 SONN errors on p1 PR839696: This issue has been resolved.
  • If the router configuration which contained 2000 service sets are applied and deleted repeatedly, the nexthop indexes are not getting freed, and the nexthop index space might exhausted. So the new nexthop allocation started failing. In this case, it might cause IPSEC tunnel not coming up. PR1008657: This issue has been resolved.
  • In subscriber management environment, a subscriber login and logout might cause a rpd process memory leak of 8-bytes. PR1011825: This issue has been resolved.
  • On all routing platforms M/MX/T/PTX with BGP configured to carry flow-specification route, in case of deleting a filter term and policer, then add the same term and policer back (it usually happens in race condition when adding/deleting/adding the flow routes), since confirmation from dfwd for the deleting policer might not be received before attempting to add the same policer, the rpd would skip sending an add operation for it to dfwd. As a result, when the filter term is sent to dfwd and tell it to attach to the policer, dfwd had already deleted the policer, and since rpd skipped re-adding it, dfwd will reject the attach filter with policer not found error and rpd will crash correspondingly. PR1052887: This issue has been resolved.
  • Wrong diagnostic optics info might be seen for GE-LX10 SFP and SFP+ for SumitomoElectric. The issue only for a specific SFP type - "Xcvr vendor part number : SCP6F44-J3-ANE”, it can be seen with "show chassis pic fpc-slot X pic-slot Y". user@device> show chassis pic fpc-slot 0 pic-slot 0 .. PIC port information: Fiber Xcvr vendor Wave- Xcvr Port Cable type type Xcvr vendor part number length Firmware 0 GIGE 1000LX10 SM OPNEXT INC TRF5736AALB227 1310 nm 0.0 1 GIGE 1000LX10 SM FINISAR CORP. FTLF1318P2BTL-J1 1310 nm 0.0 2 GIGE 1000LX10 SM SumitomoElectric SCP6F44-J3-ANE 1310 nm 0.0 <<<<Error SFP->PR1091063: This issue has been resolved.
  • When using "write coredump" to invoke a live coredump on an FPC in T-series, the contents of R/SR ASIC memory (Jtree SRAM) will get dumped. In the situation that there is a parity error present in the SRAM, then the coredump will abort and the FPC will crash. As a workaround, configuring "set chassis pfe-debug flag disable-asic-sram-dump" before "write coredump" will help to avoid the issue. PR1105721: This issue has been resolved.

Interfaces and Chassis

  • Reconfiguring lt- interface causes dcd memory leak. PR879949: This issue has been resolved.
  • On MPC-3D-16XGE-SFPP line card, when an optics (for example, 10G-LR-SFP) is disabled and then enabled administratively, if the SFP is not temperature tolerant (non-NEBS compliant), the TX laser may not be turned on due to the fact that the chassis process (chassisd) may keep sending the "disable-non-nebs-optics" command to the optics if the current temperature of FPC reaches the threshold temperature. PR1107242: This issue has been resolved.
  • The jpppd process might crash and restart due to a buffer overwrite. The jpppd process restart results in a minimal impact of system and subscribers. All connected subscribers remain connected and only subscribers are attempting to connect at time of process restart would need to retry. PR1132373: This issue has been resolved.

Layer 2 Features

  • It can happen that when changing an interface framing from lan-phy (default) to wan-phy and back a few times, the interface doesn't show up any more in "show interfaces terse". PR836382: This issue has been resolved.
  • DHCPv6 fails for clients using DUID type 2 (Vendor-assigned unique ID), the software was using the DUID to extract MAC address information. This behavior is fixed and tested PR838404: This issue has been resolved.
  • In DHCP relay scenario, some DHCP relay bindings might get stuck in "RELEASE(RELAY_STATE_WAIT_AUTH_REQ_RELEASE" state due to the LOGOUT Request is not processed correctly by authentication manager process (authd) and this is causing clients not to be able to get a lease. PR850187: This issue has been resolved.
  • On MX Series platform with none-stop-routing (NSR) enabled and some L2 protocols configured, performing Routing Engine switchover might cause layer 2 control protocol daemon (l2cpd) to crash and FPC to be rebooted. PR1076113: This issue has been resolved.
  • In the DHCPv4 or DHCPv6 relay environment with large scaled environment (in this case, 50-60K subscribers), and the system is under stress (many simultaneous operations). The subscribers might get stuck in RELEASE state with large negative lease time. PR1125189: This issue has been resolved.

Platform and Infrastructure

  • When a common scheduler is shared by multiple scheduler maps which applies to different VLANs of an Aggregated Ethernet (AE) interface, if the configuration statement "member-link-scheduler" is configured at "scale", for some VLANs, the scheduler parameters are wrongly scaled among AE member links. As a workaround, we should explicitly configure different schedulers under the scheduler maps. PR1107013: This issue has been resolved.
  • On MX Series-based line card, the firewall filter may have some issues when matching on Authentication Header (AH) protocol. This can affect VRRP (among others) when authentication is used, and an Routing Engine firewall filter is matching on protocol AH. As a workaround, we can change the filter to match on other criteria (e.g. source or destination address). PR1118824: This issue has been resolved.
  • After changing an outer vlan-tags, the ifl is getting programmed with incorrect stp state (discarding), so the traffic is getting dropped. PR1121564: This issue has been resolved.
  • NTP.org published a security advisory for thirteen vulnerabilities in NTP software on Oct 21st, 2015. These vulnerabilities may allow remote unauthenticated attackers to cause Denial(s) of Service(s), disruption of service(s) by modification of time stamps being issued by the NTP server from malicious NTP crafted packets, including maliciously crafted NTP authentication packets and disclosure of information. This can impact DNS services, as well as certificate chains, such as those used in SSL/https communications and allow attackers to maliciously inject invalid certificates as valid which clients would accept as valid. PR1132181: This issue has been resolved.

Services Applications

  • In CGNAT scenario, when we establish simultaneous TCP connects, we need to install timers for each TCP connection/flow. Due to this bug, we ended up creating two timers for the forward and reverse flow separately. Ideally there needs to be only one timer for both the forward and reverse flow. Whenever the session used to get deleted due to timer expiry, the PIC used to crash whenever the code tried to delete the same flow again. PR1116800: This issue has been resolved.

Subscriber Access Management

  • In subscriber management environment with AAA authentication, after a few rounds of login/logout, some dynamic PPPoE subscribers might stuck in configured (AuthClntLogoutRespWait) state. PR1127823: This issue has been resolved.

Platform and infrastructure

  • Deny-commands is not working for "show route community-name" PR836624: This issue has been resolved.

Previous Releases

Resolved Issues in Release 12.3R11

Class of Service (CoS)

  • In SNMP environment, when performing multiple walks or parallel snmpget for same interface at the same time (for example, SNMP bulk get/walk, or SNMP polling from multiple devices) on CoS related MIBs (jnxCos table), if the interface state changes or the request gets timeout when FPC is responding the request, memory leak of Class-of-Service process (cosd) about 160 bytes (up to 1500 bytes) may occur, which may cause cosd to crash eventually when limit is exceeded. PR1058915: This issue has been resolved.

Forwarding and Sampling

  • In rare cases, MX Series routers might crash while committing inline sampling related configuration for INET6 Family only. PR1091435: This issue has been resolved.

General Routing

  • Upon a `write core` on an FPC in T-series (Gimlet/Stoli) since Junos 12.3 (PR/843389), we will also dump the contents of the R/SR ASIC memory (Jtree SRAM). If there is a memory parity error present in the SRAM then the coredump will abort and the FPC will crash unless `set chassis pfe-debug flag disable-asic-sram-dump` is configured. PR1105721: This issue has been resolved.

Interfaces and Chassis

  • When the Ethernet Link Fault Management (LFM) action profile is configured, if there are some errors (refer to the configuration, for example, frame errors or symbol errors) happening in the past (even a long past), due to the improper handling of error stats fetching from kernel, the LFM process (lfmd) may generate false event PDUs and send the false alarm to the peer device. PR1077778: This issue has been resolved.
  • After removing a child link from AE bundle, in the output of "show interface <AE> detail", the packets count on the remaining child link spikes, then if you add back the previous child link, the count recovers to normal. PR1091425: This issue has been resolved.

MPLS

  • Junk characters are being displayed in output of “show connections extensive” command. PR1081678: This issue has been resolved.

Platform and Infrastructure

  • When the system parses IPv6 packet, it does not copy the GRE key id from the GRE header to the packet context. Due to this, the IPv6 GRE header hash value is not computed correctly, hence the session lookup based on the hash value fails and duplicate sessions are created. PR916028: This issue has been resolved.
  • VRRP advertisements might be dropped after enabling delegate-processing on the logical tunnel (lt) interface. It would result in VRRP master state observed on both routers. PR1073090: This issue has been resolved.
  • On MX Series router, if ifl (logical interface) is configured with VID of 0 and parent ifd (physical interface) with native-vlan-id of 0, when sending L2 traffic received on the ifl to Routing Engine, the VID 0 will not be imposed, causing the frames to get dropped at Routing Engine. PR1090718: This issue has been resolved.

Routing Protocols

  • RIP is applying the RIB import-policy for the primary RIB table. As per the policy configured, evaluation fails and routes are removed from primary RIB. But import-policy is applied only for secondary tables. RIP should apply only to the protocol import policy and add routes to primary RIB. Routes are leaked to secondary routing table according to import-policy. As suggested by rpd infrastructure team , removed the import policy filter application to primary routing table by protocol rip. Now import policy application is handled by policy module within RPD. PR1024946: This issue has been resolved.

Services Applications

  • In Network address translation (NAT) environment, if the translation type is "dynamic-nat44", when processing bursts of packets (for example, packets coming in one after the other at a delay close to the interpacket gap, based on the replication, issue was seen when processing line rate traffic on a 1GE port), because the device may fail to free up dynamic NAT addresses, the address pool may get exhausted quite fast. In this situation, since no new addresses could be allocated, the incoming traffic drop might be seen on the device, also memory leak (not the main issue here, may become a problem way after the addresses leak) may occur on the device. PR1098583: This issue has been resolved.

Resolved Issues in Release 12.3R10

Class of Service (CoS)

  • Add chassis scheduler map support on gr interface on MS-PIC, which means there will be no commit error if scheduler-map-chassis is applied on gr interface. PR1066735: This issue has been resolved.

Forwarding and Sampling

  • If the template of the policer is changed (for example, change the bandwidth-limit value of policer), shared-bandwidth-policer knob may not function properly anymore. PR1056098: This issue has been resolved.

General Routing

  • On dual Routing Engine platforms, after performing unified graceful Routing Engine switchover (GRES) with 8K subscribers, the ksyncd process may crash due to the replication error on a next hop change operation. The issue is hit when there is memory pressure condition on the Routing Engine and in that case, it may lead to null pointer de-reference and ksyncd crash. Or in some case, the kernel on the new master Routing Engine might crash after Routing Engine switchover if Routing Engine is under memory pressure due to missing null check when trying to add a next hop and the next hop is not found at the time. PR942524: This issue has been resolved.
  • When nonstop active routing (NSR) is configured and the memory utilization of rpd process on the backup Routing Engine is high (1.4G or above), the rpd crash on backup Routing Engine might bounce the BGP sessions on the master Routing Engine. PR942981: This issue has been resolved.
  • In a scaled network (for example, more than 100K unilist next-hop entries) with BGP ECMP configured, when the master Routing Engine crashes and switchover happens, during when the neighbor session of BGP over aggregate Ethernet (AE), interface might get broken. This is because the unilist next-hops of the AE is stuck at standby state and therefore no traffic can be transmitted through. PR953365: This issue has been resolved.
  • The configuration statement 'gratuitous-arp-on-ifup' should send a gratuitous arp on each unit of a physical interface, but in Junos OS Release 12.3 and higher versions, only the first unit is seeing the configured behavior. PR986262: This issue has been resolved.
  • Whenever the logical tunnel (lt-) interface with IPv6 family configured goes down and comes up upon hardware initialization (MPC/FPC replacement/reboot or chassis reboot), due to Duplicate Address Detection (DAD), functionality is not being performed for the logical interface up/down event. The "lt-" interface may get stuck in "tentative" state and thus IPv6 traffic cannot pass over it. PR1006203: This issue has been resolved.
  • On MX Series router with IPv6 subscribers, after performing GRES or reloading one line card that has underlying interfaces for demux, some demux interfaces might get stuck in Tentative state, and some other demux interfaces which have the same link local addresses might be unable to send any IPv6 RA message. PR1026724: This issue has been resolved.
  • Configuring a routing policy with the "no-route-localize" option to ensure that the routes matching a specified filter are installed on the FIB-remote Packet Forwarding Engines , after removing the routing policy and changing the next-hop for the routes, the previously installed routes using "no-route-localize" policy will not get removed from PFE 1 but will from PFE 0 on the same FPC. Then traffic received on PFE 1 will not forward received packets to the FIB-local s to perform full IP table lookup but using the staled routes instead. This situation does also apply if the interface is getting disabled. If traffic destined to the local-address is still received on PFE 1, those stale route lookup entries might have incorrect entries and might lead to one of the following possible symptoms. fpc1 RCHIP(1): 8 Multicast list discard route entries fpc1 PFE: Detected error nexthop: fpc1 RCHIP(1): RKME int_status 0x10000000 RKME and Detected error nexthop will per default will trigger a FPC restart. PR1027106: This issue has been resolved.
  • On the Type 5 PIC, when the "hold-time down" of the interface is configured less than 2 seconds and the loss of signal (LOS) is set and cleared repeatedly in a short period (for example, performing ring path switchover within 50ms), the "hold-time down" may fail to keep the interface in "up" state within the configured time period. PR1032272: This issue has been resolved.
  • This issue is applicable to a case which inline NAT configured on an interface belongs to either an MPCE or an MP3E/MPC4E/T4000-FPC5. Ingress and egress traffic traversing between an MPCE and these cards may cause the router to drop packets. PR1042742: This issue has been resolved.
  • When querying specific entries of the JUNIPER-SUBSCRIBER-MIB, memory leak may occur on the smihelperd process which provides the necessary information over SNMP. PR1048469: This issue has been resolved.
  • This problem is because of a race condition, where other FPCs are not able to drain "which is 1 second" Fabric Streams connecting to FPC which is getting offline. With this situation - even when FPC comes online, other FPCs which have observed message "xmchip_dstat_stream_wait_to_drain" will not able to send traffic to that particular FPC over fabric. There is no workaround. Rebooting FPCs which observed error message "xmchip_dstat_stream_wait_to_drain" is a recovery. PR1052472: This issue has been resolved.
  • In subscriber management environment, the Berkeley Database (DB) may get into deadlock state. It is brought on by multiple daemons attempting to simultaneously access or update the same subscriber or service record. In this case, due to the access to DB was blocked by device control daemon (dcd), the subscriber management infrastructure daemon (smid) fails to recover the DB. Consequently, the router may stop responding to all the login/logout requests as well as statistics activity. This timing related issue is most likely to occur during login or logout and when the system is busy. PR1054292: This issue has been resolved.
  • In subscriber management scenario, when dynamic VLAN (DVLAN) demux interface is configured on MX Series router, the interface may get in stuck state, and it could be observed that the statistics of demux0 may stop incrementing. This is because Session Database (SDB) may incorrectly calculate the number of subscriber over DVLANs. When the issue occurs, for example, the router may not able to process any PPPoE Active Discovery Initiation (PADI) packets, and fail to establish the PPPoE session. PR1054914: This issue has been resolved.
  • OpenSSL project has published a security advisory for vulnerabilities resolved in the OpenSSL library on January 8th 2015: CVE-2014-3569, CVE-2014-3570, CVE-2014-3572, CVE-2014-8275, CVE-2015-0204, CVE-2015-0205. Refer to JSA10679 for more information. PR1055295: This issue has been resolved.
  • Class 4 (32W) Optics are not supported on MPC4E (2CGE+8XGE). Upon insertion and removal of a Class 4 optic, the TX laser will remain powered-off, even when a supported optic is inserted. PR1068269: This issue has been resolved.
  • If a subscriber-facing AE interface has child links which spread over multiple Packet Forwarding Engines on a single FPC, when subscribers attempt to login, "LUCHIP Congestion Detected" error messages will be seen periodically and there might be some potential forwarding issues for subscribers. PR1069292: This issue has been resolved.
  • In subscriber management environment, changing the system time to the past (for example, over one day) may cause the daemons (for example, pppoed, and autoconfd) that are using the time to become unresponsive. PR1070939: This issue has been resolved.
  • In scaled subscriber management environment (for example, 3.2K PPPoE subscribers), after heavy login/logout, the session setup rate keeps decreasing and also PAP-NAK messages are sent with "unknown terminate code". This continues till Broadband Network Gateway (BNG) does not accept PPP sessions and all newly incoming sessions are stuck in PAP Authentication phase (No PAP ACK received). PR1075338: This issue has been resolved.
  • On MX Series routers, the CLI command “set interfaces interface-name speed auto-10m-100m” is not supported. PR1077020: This issue has been resolved.

High Availability (HA) and Resiliency

  • When LACP is configured in 'periodic fast' mode, the traffic loss of more than 30 seconds will be seen during unified ISSU. The workaround is to change LACP to 'periodic slow' mode before unified ISSU. PR1059250: This issue has been resolved.

Interfaces and Chassis

  • Multicast traffic may not be forwarded to the "Downstream Neighbors" as reported by the command "show pim join extensive". There can be occasions where this traffic is blackholed and not forwarded as expected. Alternatively, there may be an occasion where multicast traffic is internally replicated infinitely, causing one or more of the "Downstream Neighbors" to receive multicast traffic at line rate. PR944773: This issue has been resolved.
  • On standalone T Series router or TX platform, during Routing Engine rebooting, a bad (or busy) I2C device on Switch Interface Board (SIB) might cause Switch Processor Mezzanine Board (SPMB) to crash. Please note the TXP platform might also experience same issue due the bad I2C, and it has been addressed in another PR, which has been fixed on Junos OS Releases 13.1R5, 13.2R6, 13.3R1, 13.3R4, 14.1R3, 14.2R1, and 15.1R1. PR1010505: This issue has been resolved.
  • On MX240 or MX480, the SNMP_TRAPs may not be generated when Exhaust temperature for an FRU exceeds threshold. PR1012497: This issue has been resolved.
  • In case the IQ2 or IQ2E PIC is working in tunnel-only mode, rebooting the tunnel PIC while the traffic is passing through the tunnel might cause the tunnel PIC to not transfer traffic any more. PR1041811: This issue has been resolved.
  • jpppd daemon ran out of memory as subscribers login failed due to missing CoS parameters. Following logs will be seen in messages when the subscribers login fail. Nov 16 12:19:21 jtac-host jpppd: Semantic check failed for profile=PPPoE-1-QoS, error=301 Nov 16 12:19:21 jtac-host jpppd: dyn_prof_send_request: add pre_processing failure, error=301 Nov 16 12:19:21 jtac-host jpppd: Profile: PPPoE-1-QoS variable: $junos-cos-shaping-rate value: failed semantic check. PR1042247: This issue has been resolved.
  • It is observed that the syslog messages related to kernel and Packet Forwarding Engine may get generated at an excessive rate, especially in subscriber management environment. Most of these messages may appear repeatedly, for example, more than 1.5 million messages may get recorded in 2 hours, and there are only 140 unique messages. Besides, these messages are worthless during normal operation and due to the excessive rate of log generation, high Routing Engine CPU consumption (for example, Routing Engine CPU utilization can be stuck at 100% for a long time (minutes or hours), it depends on the activity of subscribers (frequency of logins and logouts) and on the AI scripts used by the customer) by event process (eventd) might be observed on the device. PR1056680: This issue has been resolved.
  • In multichassis link aggregation groups (MC-LAGs) environment, the MC-LAG peers have the MAC and port information and can forward the traffic appropriately. If a single VLAN on ICL interface is modified to a different VLAN, and then the administrator rolls back the VLAN configuration to the original one, the remote MAC might be stuck in the "Pending" state and not be installed in the bridge MAC-table, which causes the traffic forwarding to be affected. PR1059453: This issue has been resolved.
  • In scaling PPP subscriber environment, when the device is under a high load condition (for example, high CPU utilization with 90% and above), the long delay in session timeout may occur. In this situation, the device may fail to terminate the subscriber session (PPP or PPPoE) immediately after three Link Control Protocol (LCP) keepalive packets are missed. As a result, subscriber fails in reconnect due to old PPP session and corresponding Access-Internal route are still active for some time. In addition to this, it is observed that the server is still sending KA packets after the session timed out. PR1060704: This issue has been resolved.
  • Error message is continuously logged every second after a particular copper-SFP [P/N:740-013111] is plugged into a disabled port on MIC. ***** error message **** mic_sfp_phy_program_phy: ge-*/*/* - Fail to init PHY link mic_periodic_raw: MIC(*/*) - Error in PHY periodic function PQ3_IIC(WR): no target ack on byte 0 (wait spins 2) PQ3_IIC(WR): I/O error (i2c_stat=0xa3, i2c_ctl[1]=0xb0, bus_addr=0x56) mic_i2c_reg_set - write fails with bus 86 reg 29 mic_sfp_phy_write:MIC(*/*) - Failed to write SFP PHY link 0, loc 29 mic_sfp_phy_mdio_sgmii_lnk_op: Failed to write: ifd = 140 ge-*/*/*, phy_addr: 0, phy_reg: 29 ala88e1111_reg_write: Failed (20) to write register: phy_addr 0x0, reg 0x1d Fails in function ala88e1111_link_init. PR1066951: This issue has been resolved.

Layer 2 Features

  • The Layer 2 Control Protocol process (l2cpd) leaks memory when interface config is applied to LLDP-enabled interfaces using 'apply-groups'. Size of the leak is ~700 bytes per commit. PR1052846: This issue has been resolved.
  • LACP partner system ID is shown wrong when the AE member link is connected to a different device. This might be misleading while troubleshooting the LAG issues. PR1075436: This issue has been resolved.

Network Management and Monitoring

  • SNMP mib walk jnxMac does not return value with et- interfaces on MPC3/MPC4/MPC5/MPC6. PR1051960: This issue has been resolved.
  • SNMP queries for LAG MIB tables while LAG child interface is flapping, may cause mib2d to grow in size and eventually crash with a core file. Mib2d will restart, and recover by itself. PR1062177: This issue has been resolved.

Platform and Infrastructure

  • For inline BFD over aggregated Ethernet (AE) interface in which member links are hosted on different FPCs, BFD packets coming on ingress line card will be steered to anchor Packet Forwarding Engine through fabric. If FPC reconnect to master Routing Engine (such as Routing Engine switchover operation), the inline BFD session punts the BFD packet to the host. The BFD packet should go through loopback interface filter of VRF on which it is received. But in this case, the BFD packet might hit the incorrect loopback interface filter from wrong routing-instance since the VRF information is not carried across fabric. PR993882: This issue has been resolved.
  • In EVPN scenario, MPC may crash with core-file when any interface is deleted and add that interface to an aggregated Ethernet bundle or change the ESI mode from all-active to single-active. PR1018957: This issue has been resolved.
  • If several aggregates are configured with shared-bandwidth-policer and those aggregates share the same Packet Forwarding Engine for child member links and one member links flaps, all traffic might get policed and dropped. The traffic dropped might not be on the bundle whose child member link flapped. PR1035845: This issue has been resolved.
  • MSDPC-HTTP redirect stops working. PR1039849: This issue has been resolved.
  • When IRB interface is configured with VRRP in Layer 2 VPLS/bridge-domain, in corner cases IRB interface may not respond to ARP request targeting to IRB sub-interface IP address. PR1043571: This issue has been resolved.
  • Due to a defect in the Junos OS Software, when a telnet user experiences some undefined network disconnect, .perm and .env files under /var/run are left behind. This scenario happens only under certain unknown ungraceful network disconnects. When considerable number of .perm/.env files get accumulated under /var/run, issue is seen with telnet users, that they are not able to perform permitted operations on the router, post-login. PR1047609: This issue has been resolved.
  • On the MX Series-based line cards, if inline Network Address Translation (NAT) service, Generic Routing Encapsulation (GRE) tunneling and packets fragmentation are performed on the same Packet Forwarding Engine (specifically, after NAT, the packet go to tunnel and then to fragmentation), the fragmented packets may get dropped by FTP client due to the incorrect TCP checksum of the fragmented packet. PR1051144: This issue has been resolved.
  • NTP.org has published a security advisory for multiple vulnerabilities resolved in ntpd (NTP daemon) that have been assigned four CVE IDs. Junos OS has been confirmed to be vulnerable to one of the buffer overflow vulnerabilities assigned CVE-2014-9295 which may allow remote unauthenticated attackers to execute code with the privileges of ntpd or cause a denial of service condition. Refer to JSA10663 for more information. PR1051815: This issue has been resolved.
  • Software upgrade might cause firewall filters to redirect packets to an incorrect routing instance. This issue only affects Junos OS Releases 12.3R7, 12.3R7-S1, 12.3R7-S2, 12.3R7-S3, and 12.3R8. PR1057180: This issue has been resolved.
  • Customers without the fix cannot use 'then decapsulate gre routing instance <routing instance>' without also another action such as 'then [ count | sample ]' or 'then decapsulate gre [ sample <protocol> ] [ forwarding-class <fc> ] [ no-decrement-ttl ]. If they do they could lose all traffic if the board reboots or if the configuration for the routing instance and the filter are both applied at the same time. PR1061227: This issue has been resolved.
  • On MX Series routers with MPCs and T4000 routers with Type 5 FPCs, the feature "enhanced-hash-key" is configured to select data used in the hash key for enhanced IP forwarding engines. If "type-of-service" is configured at the [edit forwarding-options enhanced-hash-key family inet] hierarchy level, or "traffic-class" is configured at the [edit forwarding-options enhanced-hash-key family inet6] hierarchy level, the last significant 2 bits of the TOS/TC bytes under the IPv4/IPv6 header are extracted incorrectly as load sharing input parameters, this might cause unexpected load balancing result. PR1066751: This issue has been resolved.
  • An FPC with interfaces configured as part of an Aggregated Ethernet bundle may crash and reboot when the shared-bandwidth-policer is configured as part of the firewall policer. PR1069763: This issue has been resolved.
  • VPLS filter applied under forwarding-options might drop VPLS frame unexpectedly when it's coming from an lt- interface. PR1071340: This issue has been resolved.
  • When inline-sampling is enabled, in race conditions, if packet gets corrupted and the corrupted packet length shows 0, it may cause "PPE_x Errors thread timeout error" and eventually cause MPC card to crash. PR1072136: This issue has been resolved.
  • MAC filter ff:ff:ff:ff:ff:ff is cleared from the Packet Forwarding Engine hardware mac table. So arp requests are not forwarded to irb. Not all mac entries pointing to invalid l2 token are candidates for being deleted. Static mac entries are managed by Control plane only. So Packet Forwarding Engine cannot delete these entries. The logic for skipping mac deletion for static mac entries done earlier is not proper Packet Forwarding Engine. Fixed the same. PR1073536: This issue has been resolved.
  • On MX Series-based line cards, when the firewall filters with prefixes are configured, the heap memory leak issue might be observed. PR1073911: This issue has been resolved.
  • With MSDPC equipped on BNG, there might be a memory leak in ukernel, which eventually causes MSDPC to crash and restart. PR1085023: This issue has been resolved.

Routing Protocols

  • When a BGP peer goes down, the route for this peer should be withdrawn. If it happens that an enqueued BGP route update for this peer has not been sent out, issuing the CLI command "show route advertising-protocol bgp <peer addr>" might crash the routing protocol process (rpd). This is a very corner issue and hardly to be experienced. PR1028390: This issue has been resolved.
  • Junos OS Multicast Source Discovery Protocol (MSDP) implementation is closing an established MSDP session and underlying TCP session on reception of source-active TLV from the peer when this source-active TLV have an "Entry Count" field of zero. "Entry Count" is a field within SA message which defines how many source/group tuples are present within SA message. PR1052381: This issue has been resolved.
  • When running Simple Network Management Protocol (SNMP) polling to specific IS-IS Management Information Base (MIB) with invalid variable, it will cause routing protocol process (rpd) crash. PR1060485: This issue has been resolved.
  • When there are a number of secondary BGP routes in inet.0, an SNMP walk of inet.0 by the bgp4 MIB can cause a core if the corresponding primary routes are being deleted. PR1083988: This issue has been resolved.

Services Applications

  • On M Series, MX Series, T Series routers with Multiservices 100, Multiservices 400, or Multiservices 500 PICs with "dump-on-flow-control" configured, if prolonged flow control failure, the coredump file might generate failure. PR1039340: This issue has been resolved.
  • When the tunnel between L2TP access concentrator (LAC) and L2TP network server (LNS) is destroyed, the tunnel information will be maintained until destruct-timeout expire (if the destruct-timeout is not configured, the default value is 300 seconds). If the same tunnel is restarted within the destruct-timeout expire, the LNS will use the previously negotiated non default UDP port, which might lead to the tunnel negotiation failure. PR1060310: This issue has been resolved.
  • A Layer 2 Tunneling Protocol daemon (l2tpd) crash is seen sometimes when the L2TP service interface unit number is configured higher than 8192. A restriction has been added to force unit numbers below 8192. PR1062947: This issue has been resolved.
  • Service PIC daemon (spd) might crash with core-files due to CGNAT pool's snmp-trap-thresholds configuration. PR1070370: This issue has been resolved.

Subscriber Access Management

  • The authd process memory leak slowly when subscribers login and logout, which eventually leads the process to crash and core-file. PR1035642: This issue has been resolved.

VPNs

  • In MVPN RPT-SPT mode, with a mix of local and remote receivers all using (*,g) joins (spt-threshold infinity), the downstream interfaces may not get updated properly and there may be a stuck (s,g) forwarding route. This issue can occur with the following sequence of events: 1. Local receivers are joined 2. Traffic starts, then stops, and the route times out. 3. Remote receiver joins. Both a (*,g) and an (s,g) forwarding route are created. 4. Another local receiver is joined, or an existing one is pruned. 5. In the (*,g) route the downstream interface list reflects the update, but in the (s,g) route the downstream interface list does not. 6. When traffic starts again, the (s,g) route -- which has the wrong interface list -- is used. The traffic flows to the wrong set of receivers. PR1061501: This issue has been resolved.

Resolved Issues in Release 12.3R9

Class of Service (CoS)

  • SNMP get-request for OID jnxCosIngressQstatTxedBytes (ingress queue) might return the value of jnxCosQstatTxedBytes (egress queue). But SNMP walk works fine since it uses get-next-request. PR1011641: This issue has been resolved.
  • For ichip based platform, IQ2 pic expects FC index in the cookie from ichip for packet queuing. For Transit traffic, fc index is coming in cookie where are for host outbound traffic, queue number is coming in cookie to IQ2 pic. As IQ2 pic is not aware of whether traffic is transit or host outbound, it treats value received in cookie as FC value and looks into fc_to_q table to fetch queue number. This is causing issue in queueing of host outbound traffic in IQ2 PIC in incorrect queue. This is a day one issue and will come if in FC to Queue mapping, fc id and queue number are not same. PR1033572: This issue has been resolved.

Forwarding and Sampling

  • In rare condition, dfwd process might crash during user logging out and in. PR982477: This issue has been resolved.
  • When a firewall filter has one or more terms which have MX Series routers with MPCs or MICs match condition or actions, such filters will not be listed during SNMP query. This behavior is seen typically after Routing Engine reboot/upgrade/master-ship switch. Restarting mib2d process will cause to learn these MX Series routers with MPCs or MICs filters: cli > restart mib-process After mib2d restart, SNMP mib walk of firewall OIDs will: - list all the OIDs corresponding this MX Series routers with MPCs or MICs filter - count correctly as configured in the filter Now, despite the SNMP mib walk for firewall OIDs lists all OIDs and appropriate values, messages logs will report the following logs for every interface that has this MX Series routers with MPCs or MICs filter applied. > Jul 8 15:52:09 galway-re0 mib2d[4616]: %DAEMON-3-MIB2D_RTSLIB_READ_FAILURE: get_counter_list: failed in reading counter names ae33.1009-i: 288 (No such file or directory) > Jul 8 15:52:09 galway-re0 mib2d[4616]: %DAEMON-3-MIB2D_RTSLIB_READ_FAILURE: get_counter_list: failed in reading counter names ae31.1004-i: 257 (No such file or directory) > Jul 8 15:52:09 galway-re0 mib2d[4616]: %DAEMON-3-MIB2D_RTSLIB_READ_FAILURE: get_counter_list: failed in reading counter names ae33.1010-i: 289 (No such file or directory) > Jul 8 15:52:09 galway-re0 mib2d[4616]: %DAEMON-3-MIB2D_RTSLIB_READ_FAILURE: get_counter_list: failed in reading counter names ae31.1004-i: 257 (No such file or directory) The above two issues are addressed in this PR fix. PR988566: This issue has been resolved.
  • On the 32-bit Junos OS, when a very big burst-size-limit value (2147492676 and above) is configured in the ingress interface policer, the kernel may drop Routing Engine destined traffic. PR1010008: This issue has been resolved.
  • When an ARP policer is applied to an interface, it appears commented out in the configuration with the following message: "invalid path element 'disable_arp_policer'". PR1014598: This issue has been resolved.

General Routing

  • Changing the static route configuration from next-hop to qualified-next-hop might result in static route getting missed from routing table. Restarting routing process can bring back the routes but with rpd core. PR827727: This issue has been resolved.
  • In this scenario the CPCD (captive-portal-content-delivery) is configured for HTTP-REDIRECT for Subscriber Management clients using MS-DPC. When services sessions start to redirect the HTTP traffic, the memory-usage consistently increments for MSPMAND on the multi-service PIC. The memory limit then might cause packets loss. PR954079: This issue has been resolved.
  • In large scale L3VPN environment (in this case, there are 80K L3VPN routes) with Non-Stop Routing (NSR) enabled. When the L3VPN routes are added and deleted frequently, in rare condition, the Composite Next Hop (cnh) deletion from kernel after backup rpd process learns cnhs with duplicated key but with different nhids, this might lead to rpd process crash on backup Routing Engine. This issue is not reproducible and only happened once. PR959331: This issue has been resolved.
  • Although receiving the flow specification (flowspec) routes with packet-length, icmp-code or icmp-type matching rules from a BGP peer properly, the local firewall filter in the Packet Forwarding Engines might not include these matching rules. PR968125: This issue has been resolved.
  • In the dual Routing Engines scenario with 8K PPP dual stack subscribers. In rare condition, after Routing Engine switchover, some subscribers are stuck in terminating state forever. PR974300: This issue has been resolved.
  • In the dual Routing Engines scenario with large scale nexthops (in this case, more than 1-million nexthops and around 8K VRFs). In rare condition, kernel might crash on backup and/or master Routing Engine due to exhaustion of nexthop index space. PR976117: This issue has been resolved.
  • On MX Series router with MX Series linecard. When a faulty Non-Ethernet clear channel OCx MIC is inserted into MPC, the MPC goes offline. After removing the MIC, the MPC starts going online and offline continuously. These MICs as below belong to SONET/SDH OC3/STM1 (Multi-Rate) MIC: * MIC-3D-8OC3OC12-4OC48 * MIC-3D-4OC3OC12-1OC48 * MIC-3D-8CHOC3-4CHOC12 * MIC-3D-4CHOC3-2CHOC12 * MIC-3D-8DS3-E3 * MIC-3D-8CHDS3-E3-B * MIC-3D-1OC192-XFP. PR976675: This issue has been resolved.
  • With nonstop active routing (NSR) enabled, deleting non-forwarding routing instance might result in the rpd process crash on backup Routing Engine. The core files could be seen by executing CLI command "show system core-dumps". PR983019: This issue has been resolved.
  • On MX Series, delete an interface A from routing-instance VRF1; then create routing-instance VRF2 and interface A is added to VRF2 with qualified-next-hop configured; finally, delete VRF1. Commit the entire above configuration once, in rare condition, rpd might crash. PR985085: This issue has been resolved.
  • 1) Due to a previous fix chassisd on the protocol master Routing Engine and the protocol backup Routing Engine connect to the main snmpd on the protocol master using the following methods. a) Chassisd on the protocol master Routing Engine connects using a local socket since snmpd is running locally. b) Chassisd on the protocol backup Routing Engine connects using a TNP socket since snmpd is not local. 2) However this fix changed the way the other daemons connect to snmpd. All important daemons run on the protocol master and should connect to snmpd using a local socket. However the fix changed it so that all daemons that ran on the protocol master (other than chassisd) tried to connect using the TNP socket. SNMPD does not accept these connections. As a fix, in an MX-VC, we made sure that chassisd connects to all processes which run on the protocol master using internal socket while the chassisd process on the protocol backup and protocol lincecard connect using TNP socket. PR986009: This issue has been resolved.
  • In 6PE scenario, when PE router is sending IPv6 TCP traffic to MPLS core, in rare occasions, the kernel might crash and reboot with a vmcore file generated. PR988418: This issue has been resolved.
  • In the dual Routing Engines scenario with NSR configuration, backup peer proxy thread is hogging CPU for more than 1 second if there are multiple updates (>5000) going from master Routing Engine to backup Routing Engine. This leads to FPC socket disconnections. The traffic forwarding might be affected. PR996720: This issue has been resolved.
  • When having ECMP routes and multiple levels of route/next-hop recursion, a particular sequence of routes churn may result in rpd process crash and traffic outage. PR1006523: This issue has been resolved.
  • MS-DPC memory leak on system service set when HTTP Redirect attempts to process none-HTTP traffic with HTTP ports (80/8080/443). PR1008332: This issue has been resolved.
  • When deleting a routing-instance or making changes to the routing-instance, the deletion of the routing-instance to kernel might come before the deletion of the logical interfaces in the routing-instance, resulting in rpd crash. This is a timing issue, hard to reproduce. PR1009426: This issue has been resolved.
  • Whenever a FPC goes down suddenly due to hardware failure on T Series router, the data traffic in transit towards this FPC from other Enhanced Scaling (ES) FPCs could be stuck in the fabric queue thereby triggering fabric drops due to the lack of buffers to transmit the data to active destination FPCs. PR1009777: This issue has been resolved.
  • On MX Series platforms with ADPC FPCs, M120 or M7i/M10i with Enhanced CFEB each VPLS LSI interface flapping triggers a memory leak in jtree segment 0. There is no memory leak in FPC heap 0 memory. PR1009985: This issue has been resolved.
  • Unknown unicast flood is seen with interface flap after router reboot and with static MAC, no-mac-learning, interface-mac-limit configured for a virtual-switch. PR1014222: This issue has been resolved.
  • The routing protocol daemon (rpd) might crash continuously with core-dumps upon adding a sub-interface with "disable" configuration to a MC-LAG interface. PR1014300: This issue has been resolved.
  • The OpenSSL project released a security advisory on 2014-08-06 that contained nine security issues. The following four issues affect Junos: CVE-2014-5139: Crash with SRP ciphersuite in Server Hello message CVE-2014-3509: Race condition in ssl_parse_serverhello_tlsext CVE-2014-3511: OpenSSL TLS protocol downgrade attack CVE-2014-3512: SRP buffer overrun See JSA10649 for more information. PR1016458: This issue has been resolved.
  • The existing PTSP_SUBS_TIMEOUT_NONE is within the max of 86400 (PTSP_SUBS_PKT_TIMEOUT_INTVL) range. Hence subscriber may not be cleaned from pkt_timeout table when the time interval of 0xFFFF is hit. Defined a separate value to clean the packet idle timeout subscribers. PR1016896: This issue has been resolved.
  • The jptspd_debug print JPTSPD_DEBUG_TRACE(JPTSPD_TRACE_SRC) << __JPTSPD_HERE__ << ": No session found " + sid << endl; will cause problems with "+" before sid instead of << with. PR1016959: This issue has been resolved.
  • In dynamic subscribers management environment with "maintain-subscriber" feature enabled, when scaling up the logged in subscribers, the demux interface might not be associated with the subscriber and "show auto-configuration extensive" CLI command only print partial output. PR1017544: This issue has been resolved.
  • MAC accounting support was added for 40G and 100G interfaces on MPC3 and MPC4 cards. PR1017595: This issue has been resolved.
  • Under corner cases, if there are multiple back-to-back Virtual Chassis port (VCP) related CLI commands, Network Processing Card (NPC) core may be observed and FPC hosting the VC ports might reboot. PR1017901: This issue has been resolved.
  • In the scenario where router acts as both egress LSP for core network and BRAS for subscribers, RSVP-TE sends PathErr to ingress router due to matching to subscriber interfaces wrongly when checking the explicit route object (ERO), if subscribers are associated with same lo0 address as used by RSVP LSP egress address. PR1031513: This issue has been resolved.
  • With an unrecognized or unsupported Control Board (CB), mismatch link speed might be seen between fabric and FPCs, which results in FPCs CRC/destination errors and fabric planes offline. Second issue is in a race condition, Fabric Manager (FM) might process the stale destination disable event but the error is cleared indeed, it will result in the unnecessary FPC offline and not allowing Fabric Hardening action to trigger and recover. PR1031561: This issue has been resolved.
  • In rare cases, the AUTHD daemon may crash and cause a corruption of subscriber dynamic profiles. In-use profiles may be incorrectly marked as not in use. Any subscribers that reference that profile are forced to remain in Terminating state, until the router is rebooted. Daemon restarts and GRES switches are ineffective in working around this situation. PR1032548: This issue has been resolved.
  • If a logical interface is used as the qualified-next-hop (which implies the IFL has unnumbered-address configured), and there are changes in the logical interface filter configuration, then the static route might disappear from routing table. To make it reappear, need to delete it from the configuration and add it back. PR1035598: This issue has been resolved.
  • When recovering from a split master Virtual Chassis (VC), the line-cards in the new VC-Bm chassis may contain provisioning data out of synchronization versus the master Routing Engine. PR1036795: This issue has been resolved.
  • In a subscriber scenario with auto-sensed VLAN configured, after scaled subscribers (in this case, 16K subscribers) login/logout for several times, the subscriber management process might get stuck and not able to restart due to a Session Database (SDB) deadlock issue. PR1041094: This issue has been resolved.
  • If the flow routes (flow route is an aggregation of match conditions for IP packets) are active in the kernel, the rpd process might crash after executing command "show route table <x>.inetflow.0 extensive". PR1047271: This issue has been resolved.
  • On T Series FPC 1-3 and M320 except E3-FPC with fib-local configuration. If there are multiple FIB local FPCs or the FIB local is a multiple Packet Forwarding Engine FPC, the TCP packets might be out of order, packets re-ordering would occur. It reduces the application level throughput for any protocols running over TCP. PR1049613: This issue has been resolved.

High Availability (HA) and Resiliency

  • This issue occurs in rare condition. In the dual Routing Engines scenario, doing interface flap after Routing Engine switchover. If this action is repeated many times, the stale indirect nexthop entry might be seen in kernel, this leads to traffic blackhole. PR987959: This issue has been resolved.

Interfaces and Chassis

  • Error message CHASSISD_IPC_DAEMON_WRITE_ERROR is seen in the messages log when there is a Routing Engine mastership change (system reboot, Routing Engine reboot, GRES switchover CLI command), which causes a restart of alarmd,which breaks the IPC connection between alarmd and chassisd. Chassisd does not detect that the IPC connection has been broken, because it is busy processing the mastership change, and then tries to send alarm information to alarmd during this time. So it encounters a write error (broken pipe) and logs the message. PR908822: This issue has been resolved.
  • If dynamic VLAN subscriber interface is over a physical interface (IFD), and there are active subscribers over the interface, when deactivate the dynamic VLAN related configuration under the IFD and add the IFD to an aggregated Ethernet (AE) interface which has LACP enabled, the Routing Engine might crash and get rebooted. PR931028: This issue has been resolved.
  • Link speed of a LAG bundle may not properly reflect the total bandwidth, when microBFD is enabled on the LAG interface. PR967046: This issue has been resolved.
  • In the dynamic-profile environment with preferred-source-address configuration. If subscribers stuck in terminating state, it is impossible to commit changes. PR978156: This issue has been resolved.
  • On MX Series router, in rare condition, the kernel might crash and the router will go in db prompt when router reboots. PR993978: This issue has been resolved.
  • In L2 circuit, with async notification configured on a client facing interface goes down, then on the remote PE the corresponding CE interface shows up in show interface terse output while in log snmp reports interface down. PR1001547: This issue has been resolved.
  • As current Junos OS Multichassis link aggregation groups (MC-LAGs) design, the ARP entry will not sync when learning ARP via ARP request but not Gratuitous ARP/ARP reply, in some specific scenarios (e.g. a host changes its MAC address without sending a Gratuitous ARP), traffic loss might occur. PR1009591: This issue has been resolved.
  • VRRP daemon (vrrpd) memory leak might be observed in "show system processes extensive" when VRRP is set with routing-instance and then change any configuration. PR1022400: This issue has been resolved.
  • if DPCE 20x 1GE + 2x 10GE X card is present in the chassis, BFD sessions over AE interfaces may not be distributed. PR1032604: This issue has been resolved.

Layer 2 Features

  • After configuration change or convergence events, kernel may report ifl_index_alloc failures for LSI interfaces and causing KRT queue ENOMEM issue, eventually preventing new IFLs being added to the system. This condition always recovers on its own once convergence is completed. PR997015: This issue has been resolved.
  • In the Ethernet ring protection switching (ERPS) environment, once Graceful Routing Engine Switchover (GRES) happens on the ring protection links (RPLs) owner node, there will be a ~30s Ring automatic protection switching (R-APS) message storm in the ring, which in turn cause some VPLS instance flapping. PR1004066: This issue has been resolved.
  • If "maintain-subscriber" knob is enabled on the router, DHCPv6 server/relay might be unable to process any packet if deactivate and then activate the routing instance, which means the subscribers can not get the IPv6 addresses. Please note, even with the fix, the results of this scenario are also expected if with "maintain-subscriber" knob enabled. Consider using the workaround to avoid this issue. PR1018131: This issue has been resolved.
  • After FPC restart, bridge domain (BD) implicit filters for Ethernet ring protection switching (ERPS) might get reprogrammed with wrong logical interface (ifl) index, which cause ERPS cannot work correctly. PR1021795: This issue has been resolved.
  • In a mixed VPLS instance where both LDP and BGP flavors are present with "best-site" knob configured under "site" block, any cli change in that instance will result in rpd crash. PR1025885: This issue has been resolved.
  • If a customer is using SNMP and performs an snmpwalk on the dhcp binding table, all of the entries might not be displayed. This fix resolves that issue so that bindings for all ip addresses are displayed. PR1033158: This issue has been resolved.

MPLS

  • Error "tag_icmp_route:failed to find a chain composite ahead of fwd nh" might be observed when doing traceroute. PR999034: This issue has been resolved.
  • When the size of a Routing Engine generated packet going over an MPLS LSP is larger than MTU (i.e. MTU minus its header size) of an underlying interface, and the extra bytes leading to IP-fragmentation are as small as <8 bytes, then that small-fragment will be dropped by kernel and lead to packet drop with kernel message "tag_attach_labels(): m_pullup() failed". For example - If SNMP Response with specific size falls into above mentioned condition, then small fragment will be dropped by kernel and eventually the SNMP response will fail. PR1011548: This issue has been resolved.
  • TED link information of protocol from highest credibility level is used irrespective of the level at which CSPF is computing. i.e., cspf-metric in "show mpls lsp extensive" would have the sum of te-metric of IGP with highest credibility at each hop in ERO. This has been corrected and the cspf-metric will be sum of te-metric of current credibility at each hop. PR1021593: This issue has been resolved.
  • When RSVP label-switched-path (LSP) optimize is enabled, RSVP LSP might stay down after a graceful Routing Engine mastership switchover (GRES). To resolve the problem, the corresponding label-switched-path configuration needs to be deactivated, then, be activated again. PR1025413: This issue has been resolved.
  • When configuring point-to-multipoint (P2MP) Label Distribution Protocol (LDP) label-switched paths (LSPs), the labels will never be freed even they are no longer needed. This could lead to the MPLS label exhaustion eventually. To clear the state, the rpd process will restart with core files. PR1032061: This issue has been resolved.

Network Management and Monitoring

  • Due to a communication error between the master agent (snmpd process) and the subagent (mib2d process), the device fails to register some MIBs. For example, the following commands do not display any output when you run the command: user@hostname>show snmp mib walk ifTable user@hostname:~$ snmpwalk -v 2c -c snmp@exp X.X.X.X ifAlias The following message is displayed: IF-MIB::ifAlias= No Such Object available on this agent at this OID. This means the OID is not registered. PR978535: This issue has been resolved.

Platform and Infrastructure

  • If the "Host Loopback" error is triggered after configuration change (e.g. it was seen after just changing the config from XE to AE with scaled IGMP receivers), it will in turn triggers error-reporting infra (CMERROR). Since host-loopback falls under Major error class, default action for Major class is "log". Hence it triggers a TFTP for transferring debug info to Routing Engine. If this tftp write fails due to lower layer issue (i.e. RE-PFE link issue), TFTP would try to send again (max 5 times). Every time lower protocols append headers but when it fails these headers will not get removed. This causes packet corruption and Packet Forwarding Engine would crash while freeing packet. PR935764: This issue has been resolved.
  • In a highly congested system (e.g. high multicast traffic rate), traffic/subscribers loss might occur while performing unified In-Service Software Upgrade (ISSU). PR945516: This issue has been resolved.
  • When apply-groups are used in the configuration, the expansion of interfaces <*> apply-groups will be done against all interfaces during the configuration validation process, even if the apply-group is configured only under a specific interface stanza. PR967233: This issue has been resolved.
  • On MX Series routers with MX Series linecards in a setup involving Packet Forwarding Engine fast reroute (FRR) applications, if an interface is down for more than ARP timeout interval or if ARP entries are cleared by cli commands, then after the interface is up again packet forwarding issues may be seen for traffic being forwarded over that interface. PR980052: This issue has been resolved.
  • On MX2020/MX2010 we might see sporadic FO request time-out error reported under heavy system traffic load. This would mean the request returning into a grant took longer then +/-30usec. The packet will still get forwarded through the fabric hence no operational impact. [May 6 18:56:59.174 LOG: Err] MQCHIP(2) FO Request time-out error [May 6 19:33:47.555 LOG: Info] CMTFPC: Fabric request time out pfe 2 plane 6 pg 0, trying recovery. PR991274: This issue has been resolved.
  • BFD sessions within default routing-instance are not coming up once inline-services pic is configured and fixed class-of-service forwarding-class is assigned. BFD sessions operating in no-delegate-processing are not affected. PR999647: This issue has been resolved.
  • MPLS traffic going through the ingress pre-classifier logic may not determine mpls payload correctly classifying mpls packet into control queue versus non-control queue and expose possible packet re-order. PR1010604: This issue has been resolved.
  • When a MIC is inserted into a freshly booted modular MPC3E on MX2020 router, it does not get detected and when an attempt is done to bring it online using CLI command "request chassis mic fpc-slot <> mic-slot <>" it shows the slot is empty. PR1012004: This issue has been resolved.
  • The fix was committed for this PR# but it also needs DDOS configuration additional to this fix and it is as below: 1) check the "show ddos-protection protocols statistics terse" 2) For each of the Control plane protocols on the system like ospf/vrrp/pvstp, it is recommended to configure 2X of the rate as give below example along with increasing DDOS rate for virtual-chassis control. Example, ######## set system ddos-protection protocols virtual-chassis control-high bandwidth 20000 set system ddos-protection protocols virtual-chassis control-high burst 20000 set system ddos-protection protocols ospf aggregate bandwidth 1000 set system ddos-protection protocols ospf aggregate burst 1000 set system ddos-protection protocols vrrp aggregate bandwidth 100 set system ddos-protection protocols vrrp aggregate burst 100. PR1017640: This issue has been resolved.
  • On the MX2020 platform, the system might fail to replicate multicast packets to the downstream interface located on the FPC slot 12 or above. There is no workaround. PR1019414: This issue has been resolved.
  • On MX Series platform with scaled set-up, after deactivate/activate or renaming a bridge domain (BD) which has irb interface associated, the IGMP snooping configured under the BD might not work any more. Please note it happens only when the router is in "network-services enhanced-ip" mode. PR1024613: This issue has been resolved.
  • On MX Series based platform, with igmp-snooping enabled and a multicast route with integrated routing and bridging (IRB) as a downstream interface, a multicast composite nexthop is created with a list of L3 and corresponding L2 nexthops. In a rare corner case, the corresponding L2 nexthop to the L3 IRB nexthop is a DISCARD nexthop and will cause the FPC to crash. PR1026124: This issue has been resolved.
  • MX Series-based line card might crash when trying to install the composite next-hop used for the next-hop-group configuration related to port mirroring of traffic over IRB to an LSI attached to VPLS instance for a remote host. PR1029070: This issue has been resolved.
  • The MX960 Rx Seq # should start at "0", but instead is random and does not reset to zero. Sequence number starts off at what appears to be a "random" value but then it does progress sequentially. The starting value is most likely where the last session ended and the counter was not reset by the control plane. It only occurs in 12.3. PR1031201: This issue has been resolved.
  • On MX Series 3D MPC, when there is a congested Packet Forwarding Engine destination, the non-congested Packet Forwarding Engine destinations might experience an unexpected packet drop. PR1033071: This issue has been resolved.
  • When the 'enhanced-hash-key services-loadbalancing' feature is used by MX Series based line cards, load balancing of flows across multiple service PICs via the source-address across does not work when internal BGP (IBGP) is used to steer traffic to the inside service-interface. For example the operator will see on the stateful firewall that the same source-address has flows across multiple service interfaces. PR1034770: This issue has been resolved.

Routing Policy and Firewall Filters

  • Executing CLI command "show route resolution" and stopping the command output before reaching the end of the database, the rpd process might crash when executing the same command again. PR1023682: This issue has been resolved.

Routing Protocols

  • The case occurs in the BGP multipath scenario, in this case, 3 EBGP CEs inject same route to VRF (the origin attribute of these routes are "Incomplete"). One of the routes is selected, based on the order they come in, an older one is selected (not the one with lowest router id). After that, two remote PEs also inject same route which is imported as secondary (the origin attribute of these routes are "IGP"), then the remote PE (secondary) routes become active due to origin attribute ("IGP" is preferred over "Incomplete"). When the remote PE (secondary) routes become inactive, a different EBGP CE path becomes active (lowest router-id). The original EBGP CE path that have the multipath nexthop is never turn down and it still retains the multipath nexthop, whereas the new EBGP CE path does not have any multipath nexthop, that leads to no multipath in dataplane. PR835436: This issue has been resolved.
  • When rpf-selection is configured with next-hop specified, if FPC is restarted, in rare condition, the rpd process might crash. PR915622: This issue has been resolved.
  • In scaled BGP routes environment, the BGP router has dual Routing Engines, Graceful Routing Engine Switchover (GRES) and Nonstop Active Routing (NSR) is configured, after performing the operation of deactivate/activate BGP groups and commit the configuration, the BGP router might be stuck in "not-advertising" state. PR961459: This issue has been resolved.
  • In the multicast environment, when the name of a VRF is changed, if an IGMP interface is trying to be associated with a new multicast instance before the new multicast instance is created, the rpd process might crash. PR962885: This issue has been resolved.
  • In the dual Routing Engines scenario with "commit synchronize" configuration. In a corner case, when commit operation is executed, if BGP session flaps before master Routing Engine receives the sync done message from standby Routing Engine, the routing protocol daemon might crash. PR976184: This issue has been resolved.
  • In scaled BGP environment, if an NSR enabled router does not have any routing-instance configured, after flapping BGP groups with multiple peers, some BGP neighbors might get stuck in 'not advertising' state. PR978183: This issue has been resolved.
  • When all the following conditions are met, if the knob "path-selection always-compare-med" is configured, the rpd process might crash. - routing-instance (VR, VRF) with no BGP configuration - rib-group in default instance with routing-instance.inet.0 as secondary-rib - rib-group applied to BGP in default instance - BGP routes from master tables (inet.0) leaked to the routing-instance table (routing-instance.inet.0) PR995586: This issue has been resolved.
  • Abnormal ip6 route-calculation behavior can be seen when ospf3-te-shortcut is configured. PR1006951: This issue has been resolved.
  • When the same PIM RP address is learned in multiple VRFs, with NSR configured, rpd on the backup Routing Engine may crash due to memory corruption by the PIM module. PR1008578: This issue has been resolved.
  • Under certain circumstances, route validation session might result in rpd process crash. PR1010216: This issue has been resolved.
  • When inet.3/inet6.3 is not enabled, BGP group uses inet6.0 table to advertise the routes for both inet6 unicast and inet6 labeled-unicast families. When BGP family is changed, BGP sessions re-establish. When BGP starts to advertise routes to the peer, BGP expects to see route label however if the old inet6 unicast routes are still present (not completely cleaned), then rpd process crashes. The fix is to separate BGP group for inet6 unicast with inet6 labeled-unicast with same rib. The old peers are cleaned up in the old group and new peers are established in new group. Thus, new peer establishment is not delayed by the cleanup of the old peer. PR1011034: This issue has been resolved.
  • IsisRouterTable MIB issues, when we do "show snmp mib walk isisRouterHostName/isisRouterTable" we were not getting exact hostname as it is in "show isis hostname" so the actual implementation was not as per RFC-4444, because it was showing only the hostnames of the devices which are immediate neighbors of Dut. Fix: added level info to get sysis_entry per each level correctly and filled data(isisRouterTable) correctly. PR1011208: This issue has been resolved.
  • Under certain sequence of events RPD can assert after a RPD_RV_SESSIONDOWN event. PR1013583: This issue has been resolved.
  • When receiving open message with any capability after the "add-path" capability from BGP peer, the session will be bounced. PR1016736: This issue has been resolved.
  • The snmp trap generated when an ipv6 BFD session goes up/down does not contain the ipv6 bfd session address. PR1018122: This issue has been resolved.
  • Junos OS implementation of RFC 3107 uses unspecified label (0x000000) when sending route with label withdrawn message. This means Junos OS sends 0x000000 instead of 0x800000 for label withdrawn, which is inconsistent with RFP 3107. PR1018434: This issue has been resolved.
  • When BGP add-path feature is enabled on BGP route-reflector (RR) router, and if the RR router has mix of add-path receive-enabled client and add-path receive-disabled (which is default) client, due to a timing issue, the rpd process on RR might crash when routes update/withdraw. PR1024813: This issue has been resolved.
  • When BGP is doing path selection with default behavior, soft-asserts requests are introduced. If BGP route flaps a lot, it needs to do path selection frequently, because of which a great deal soft-asserts might be produced which will cause unnecessary high CPU and some service issues, such as SNMP not responding and even rpd core. PR1030272: This issue has been resolved.

Services Applications

  • Softwire tunnel count management is inconsistent and incorrect, thus the output of "show service softwire statistics" might be incorrect. PR1015365: This issue has been resolved.
  • With Real Time Streaming Protocol (RTSP) Application Layer Gateway (ALG) enabled, the PIC might crash in case the transport header in status reply from the media server is bigger than 240 bytes. PR1027977: This issue has been resolved.
  • In Network Address Translation (NAT) scenario with Endpoint-Independent Mapping (EIM) configured on service PIC, when a new ICMP session is created which matches an existing EIM mapping, the service PIC might crash. PR1028142: This issue has been resolved.
  • For T Series or M320 router containing Dynamic Flow Capture (DFC) PIC (either a Monitoring Services III PIC or Multiservices 400 PIC), there are two issues for DFC feature. The first one is the value of "timeout-remaining" for some filters installed on the DFC pic are too huge. The second issue is for some filters, there won't be any flows to which they are attached when forwarding traffic to the content-destination during random DTCP ADDs. PR1029004: This issue has been resolved.
  • The cause of the KMD crash is not known. This is not due to SA(Security Associations) memory corruption. The code looks that SA is getting freed without clearing the table entry. PR1036023: This issue has been resolved.

User Interface and Configuration

  • CST: chassis core generated while applying group configuration on chassis > FPC PR936150: This issue has been resolved.

VPNs

  • For VPLS over VPLS topology, when the VPLS payload has two labels (Customer-VPLS-label and Customer-MPLS-label), the frame might be dropped by the core facing interface hosted on IQ2 PIC with "L2 mismatch timeout" error. This particular scenario is fixed. But there are some other worse scenarios which might hit this issue again due to the system architecture limitation, which are not fixed but need to avoid: * Addition of VLAN tags on Service provider's or CE's VPLS payload e.g. configuring QinQ. * Addition of MPLS tags on Service provider or CE's VPLS payload. * Enabling VPLS payload load balancing on Service provider's PE router. PR1038103: This issue has been resolved.
  • In NG MVPN, after ipv6 VRF RP config change, we may hit ipv6 data loss for a short period of time. PR1049294: This issue has been resolved.

Resolved Issues in Release 12.3R8

Class of Service (CoS)

  • On MX Series routers with both MX Series linecards (in this case, MPC and MPCE on the box) and other type linecard (DPCE on the box), when the Default Frame Relay DE Loss Priority Map is configured and committed, all FPCs are getting restarted with core files. PR990911: This issue has been resolved.

Forwarding and Sampling

  • DPC crashed after deactivate/activate [routing-instances TPIX bridge-domains IX bridge-options]. PR983640: This issue has been resolved.

General Routing

  • RPD crashed on backup Routing Engine when trying to compare gateways of two different types of nexthops, like table next hop which is installed in kernel for one route, router next hop which is selected in backup RPD. PR828797: This issue has been resolved.
  • Changing the redundancy mode of rlsq interface from "hot-standby" to "warm-standby" on the fly might lead kernel crash and the router will go in db> prompt. PR880451: This issue has been resolved.
  • Under particular scenarios, commit action might lead the Context-Identifier to be ignored when OSPF protocol refreshes its database. Then the PE router will stop advertising this Context-Identifier out. PR954033: This issue has been resolved.
  • Although receiving the flow specification (flowspec) routes with packet-length, icmp-code or icmp-type matching rules from a BGP peer properly, the local firewall filter in the Packet Forwarding Engines might not include these matching rules. PR968125: This issue has been resolved.
  • With a firewall policer configured on more than 256 IFFs (interface address family) of a PIC, then offline and online the PIC might cause the FPC to crash. PR983999: This issue has been resolved.
  • On M7i/M10i with enchanced CFEB, M320 with E3-FPC, M120 and MX with DPC, If "no-local-switching" is present in the bridge domain, then the IGMP-snooping is not functioning and client can't see the multicast traffic. PR989755: This issue has been resolved.
  • During large-scale MVPN routes churn events, some core-facing IGP protocols (like OSPF or LDP) might flap or experience a long convergence time. PR989787: This issue has been resolved.
  • Issuing the CLI command "restart packet-triggered-subscribers" might cause sessions to be out-of-sync between the MX SAE and SRC (external policy manager), which results in new subscribers not being able to be created. PR990788: This issue has been resolved.
  • The fabric performance of MPC1, MPC2, or 16xXE MPC in 'increased-bandwidth' mode on an MX960 populated with SCBE's will be less compared to redundant mode due to XF1 ASIC scheduling bugs. PR993787: This issue has been resolved.
  • The routing protocol daemon (rpd) might crash continuously with core-dumps upon adding a sub-interface with "disable" configuration to a MC-LAG interface. PR1014300: This issue has been resolved.

Interfaces and Chassis

  • When the GE port is configured with WAN PHY mode, a "Zero length TLV" message might be reported from the port. This is a cosmetic issue. PR673937: This issue has been resolved.
  • Due to a bug in Packet Forwarding Engine microkernel driver for MX Series MICs, slight variations of the readings from a built-in DC-DC converter may cause ports of the MIC to go down, or even result in MPC crash, with the following message logged: Ixchip(0): pio_handle(0x4b1ea7d8); pio_read_u32() failed: 1(generic failure)! ix_inq-addr=00200a30 Only MIC-3D-2XGE-XFP (750-028380) cards are affected by this software defect. This issue is resolved in 11.4R10, 12.1R9, 12.2R7, 12.3R5. PR919618: This issue has been resolved.
  • Queue stats counters for AE interface will become invalid after deactivating ifl on the AE interface. PR926617: This issue has been resolved.
  • If there is an IRB interface configured for "family inet6" in a bridge-domain on an MX Series router, the Packet Forwarding Engine may not correctly update the next-hop for an IPv6 route when the MAC address associated with the next-hop moves from an AE interface to a non-AE interface. PR958019: This issue has been resolved.
  • Demux Subscriber IFLs might show the interface as 'Hardware-Down' even though the underlying ae bundle and its member link show up. PR971272: This issue has been resolved.
  • Temperature Top and Bottom are swapped in show chassis environments output for Type3/Type4 FPCs of T Series. PR975758: This issue has been resolved.
  • 1GbE SFP(EX-SFP-1FE-LX) output optical power is restored after reseating by manually removal/insert of SFP although the IF is disabled. PR984192: This issue has been resolved.
  • SNMP OID VRRP-MIB::vrrpAssoIpAddrRowStatus returns only one Ip address when the interface ifl has been configured with two virtual-addresses under two vrrp-groups. PR987992: This issue has been resolved.
  • CFMD may crash after configuration change of an interface in a logical system which is under OAM configuration for a l2vpn instance. PR991122: This issue has been resolved.
  • On MX Series router with MX Series linecard or T4000 router with type5 FPC, when the "Hardware-assisted-timestamping" is enabled, the MPC modules might crash with a core file generated. The core files could be seen by executing CLI command "show system core-dumps". PR999392: This issue has been resolved.
  • As current Junos OS Multichassis link aggregation groups (MC-LAGs) design, the ARP entry will not sync when learning ARP via ARP request but not Gratuitous ARP/ARP reply. In some specific scenarios (e.g. a host changes its MAC address without sending a Gratuitous ARP), traffic loss might occur. PR1009591: This issue has been resolved.

Layer 2 Features

  • In BGP signaled VPLS/VPWS scenario, rpd process memory leak might occur when a group with wildcard configuration is applied to the routing instance. PR987727: This issue has been resolved.
  • In the large-scaled VPLS scenario with LSI (label-switched interface) or VT (virtual tunnel) interface, in race condition, when the LSI or VT interface is deleted, the kernel might crash. PR990269: This issue has been resolved.

Layer 2 Ethernet Services

  • Layer 2 Control Protocol process (l2cpd) is used to enable features such as Layer 2 protocol tunneling or nonstop bridging. If a router receives a Link Layer Discovery Protocol (LLDP) packet with multiple management address TLV, memory leak might occur resulting in l2cpd process crash. PR986716: This issue has been resolved.
  • When "system no-redirect" is configured, l2 descriptor destination MAC address gets overwritten and causes "DA rejects" on next-hop router. PR989323: This issue has been resolved.
  • jnxLacpTimeOut trap may show negative values and incorrect values for jnxLacpifIndex and jnxLacpAggregateifIndex. PR994725: This issue has been resolved.
  • In race condition, when FPC gets rebooted or reset, link(s) from this FPC which are part of aggregated Ethernet bundle would remain in LACP "Detached" state indefinitely. user@node> show lacp interfaces ae102 Aggregated interface: ae102 LACP state: Role Exp Def Dist Col Syn Aggr Timeout Activity xe-2/0/0 Actor No Yes No No No Yes Fast Active xe-2/0/0 Partner No Yes No No No Yes Fast Passive xe-2/0/1 Actor No No Yes Yes Yes Yes Fast Active xe-2/0/1 Partner No No Yes Yes Yes Yes Fast Active LACP protocol: Receive State Transmit State Mux State xe-2/0/0 Defaulted Fast periodic Detached xe-2/0/1 Current Fast periodic Collecting distributing user@node> show interfaces xe-2/0/0 terse Interface Admin Link Proto Local Remote xe-2/0/0 up up xe-2/0/0.0 up up aenet --> ae102.0 xe-2/0/0.32767 up up aenet --> ae102.32767 This issue would be seen when associated aggregated Ethernet bundle is configured for vlan-tagging. To clear this condition, the affected interface should be deactivated and activated using CLI commands. ============ [edit] user@node# deactivate interfaces xe-2/0/0 [edit] user@node# commit [edit] user@node# activate interfaces xe-2/0/0 [edit] user@node# commit. PR998246: This issue has been resolved.

MPLS

  • In LDP signaling L2circuit or VPLS scenario, if L2-smart-policy is configured under LDP protocol, and the LDP interface flaps, RPD might crash. PR899810: This issue has been resolved.
  • When a First hop LSR is sending Resv Message with non-directly connected IP as nexthop (in Resv HOP object), Junos OS on head end will try to install this in forwarding table. As the nexthop to be used is a non-directly connected address, forwarding table update will fail with following KRT_Q_STUCK message: RPD_KRT_Q_RETRIES: Route Update: Invalid argument. PR920427: This issue has been resolved.
  • In the MPLS environment with "egress-protection" configuration, there is a direct LDP session between primary PE and protector. One context-id is configured as primary PE's loopback address or any LDP enabled interface address. When delete the whole apply-group or delete the ldp policy from apply-group, the routing protocol daemon (rpd) might crash. PR988775: This issue has been resolved.

Platform and Infrastructure

  • With Junos OS 12.3R7, high CPU utilization (100%) will be seen on MPC if inline sampling/Jflow is enabled. The high CPU will affect the MPC's normal tasks. PR671136: This issue has been resolved.
  • The Routing Engine and FPCs are connected with an internal Ethernet switch. In some rare case, the FPCs might receive a malformed packet from the Routing Engine (e.g. packet gets corrupted somewhere on its way from Routing Engine to FPC), then the toxic traffic might crash the FPC. PR938578: This issue has been resolved.
  • The issue might come when a non-template filter gets deleted (but does not get completely cleaned up) and the same filter index gets reassigned to a template filter. This could be considered as a timing issue given it comes with a very specific sequence of events only. PR949975: This issue has been resolved.
  • On MX Series based line card, VPLS traffic might get blocked for about five minutes (timer of MAC address aged-out) after re-negotiating control-word. PR973222: This issue has been resolved.
  • If a router has DDoS protection enabled, when DDoS attack happens, the router might not process a DDoS event properly, which results in kernel crash. PR987193: This issue has been resolved.
  • When services packet(interface-style) is diverted to different routing-instance using a firewall filter, route lookup of the services packet was matching a reject route which results in PPE thread timeout. PR988553: This issue has been resolved.
  • Issues in shared bandwidth policer handling may manifest itself as NPC core, very low call setup rate and various error messages, especially when configuration includes VLAN demux over AE interface. PR989240: This issue has been resolved.
  • "delete" or "deactivate" of apply-group defining the entire TACACS or RADIUS configuration configured under [edit system apply-group <>] does not take effect on commit. This could lead to TACACS or RADIUS based authentication to still continue working despite removal (delete/deactivate) of configuration. PR992837: This issue has been resolved.
  • Packet dropped with ipv6 reject route is currently subjected to loopback ipv6 filter processing on MX Series-based line cards. As a result, the packet dropped by a reject route might be seen from the "show firewall log". PR994363: This issue has been resolved.
  • On MX Series router with MX Series linecard or T4000 router with type5 FPC, if the CoS scheduler is configured without transmit-rate while with buffer-size temporal, the Packet Forwarding Engine might not allocate buffer for the associated queue. The issue might lead to packet loss. PR999029: This issue has been resolved.
  • In the IRB interface environment with "destination-class-usage" configuration, if the bridge domain ID is the same as Destination Class Usage (DCU) ID (bridge domain ID and DCU ID are generated by system), the firewall filter might match wrong packets, and the packet forwarding would be affected. PR999649: This issue has been resolved.
  • On M7i, or M10i equipped with Enhanced Compact Forwarding Engine Board (CFEB-E). When a MPLS LSP flaps, the CFEB-E is unable to recover 8 bytes of JTREE memory per event. PR100385: This issue has been resolved.
  • MS PIC may reset after GRES in case of excessive resolve traffic. PR1001620: This issue has been resolved.
  • The non-first IP fragments containing UDP payload may be mistakenly interpreted as PTP packets if the following conditions are met: - the byte at the offset 9 in the IP packet contains 0x11 (decimal 17) - UDP payload - the two bytes at the offset 22 in the IP packet contain the value 0x01 0x3f (decimal 319; byte 22=0x01 and byte 23=0x3f) - PTP protocol The mis-identification of the packet as PTP will trigger the corruption of the fragment payload. PR1006718: This issue has been resolved.
  • MPLS traffic going through the ingress pre-classifier logic might not determine mpls payload correctly classifying mpls packet into control queue versus non-control queue, and expose possible packet re-order.PR1010604: This issue has been resolved.

Routing Protocols

  • The case occurs in the BGP multipath scenario, in this case, three EBGP CEs inject same route to VRF (the origin attribute of these routes are "Incomplete"). One of the routes is selected, based on the order the router come in, and an older one is selected (not the one with lowest router id). After that, two remote PEs also inject same route which is imported as secondary (the origin attribute of these routes are "IGP"), then the remote PE (secondary) routes become active due to origin attribute ("IGP" is preferred over "Incomplete"). When the remote PE (secondary) routes become inactive, a different EBGP CE path becomes active (lowest router-id). The original EBGP CE path that has the multipath next hop is never turned down, and it still retains the multipath next hop, whereas the new EBGP CE path does not have any multipath next hop, which leads to no multipath in dataplane. PR835436: This issue has been resolved.
  • In inter-AS option-C Layer 3 MPLS VPN scenarios, routing protocol process (rpd) might crash when multi hop EBGP is configured between the two autonomous system boundary routers (ASBRs) and static LSPs are configured between the ASBRs to resolve the indirect next hops. PR869488: This issue has been resolved.
  • In PIM-SM network with "bootstrap routing" RP selection mechanism used, it is observed that some bootstrap messages (BSMs) generation and forwarding behavior of Junos OS does not conform to RFC standard, specifically in the sections 3.2 (Bootstrap message generation), 3.3 (Sending Candidate-RP-Advertisement Messages) and 3.4 (Creating the RP-Set at the BSR). PR871678: This issue has been resolved.
  • In Protocol Independent Multicast (PIM) scenario, if interface gets deleted before the (S,G) route is installed in the Routing Information Base (RIB), then this interface index might be re-used by kernel for another interface and thus cause routing protocol process (rpd) core. PR913706: This issue has been resolved.
  • In scaled BGP routes environment (global table ~1.5 million routes). First flapping one BGP session (e.g. change the BGP authentication method can get it), after that deleting another BGP session that holds the active routes, might lead to routing protocol daemon (rpd) scheduler slips. PR928223: This issue has been resolved.
  • Performing CLI command "clear multicast bandwidth-admission interface <int>" on 64-bit Junos OS results in the rpd to crash. The command should be used without the interface qualifier on the impacted releases. PR949680: This issue has been resolved.
  • In certain rare circumstances, BGP NSR replication to the backup Routing Engine may not make forward progress. This was due to an issue where an internal buffer was not correctly cleared in rare circumstances when the backup Routing Engine was experiencing high CPU. PR975012 : This issue has been resolved.
  • On MX Series platforms with IGMP snooping enabled on an IRB interface, some transit TCP packets might be incorrectly considered as IGMP packets, causing packets to be dropped. PR979671: This issue has been resolved.
  • There are two receivers joined to same (S,G) and IGMP immediate-leave is configured. When one of the receivers sends the leave message for (S,G), another receiver is not receiving the traffic for 1-2 minutes. PR979936: This issue has been resolved.
  • In Inter-AS FEC-129 VPLS scenario with route-reflector (RR) in each AS, if the RR router is also a VPLS PE, it stops reflect VPLS routes received from MP-EBGP peer to its clients. PR980834: This issue has been resolved.
  • Ppmd filter is not programmed properly which is resulting Routing Engine to absorb BFD packets instead of Packet Forwarding Engine. PR985035: This issue has been resolved.
  • In Junos OS, by default the RIP protocol "send" option is set to Multicast RIPv2. When this "send" option is changed from "multicast"(active) to "none"(passive) or vice-versa, rpd core might be seen on the router. PR986444: This issue has been resolved.
  • In V4 RG, member site receives traffic from both serving sites for few sources upon withdraw/inject routes for 30 seconds. PR988561: This issue has been resolved.
  • In the P2MP environment with OSPF adjacency established, one router's time is set to earlier date than another router. OSPF adjacency might not come up when one router goes down and comes up. PR991540: This issue has been resolved.
  • When IS-IS is configuring for traffic engineer (TE), after removing family mpls from the interface and removing the specific interface from [edit protocols rsvp] and [edit protocols mpls] hierarchy levels, corresponding link is not removed from the TED as expected. PR1003159: This issue has been resolved.
  • When there are more than 65535 "flow-spec" routes existing in the routing table, the rpd process might crash because it exceeds the current maximum supportable scaling numbers (Current scaling numbers are in the range of 10K~16K). PR1004575: This issue has been resolved.
  • Under certain sequence of events, RPD can assert after an RPD_RV_SESSIONDOWN event. PR1013583: This issue has been resolved.

Services Applications

  • If a PPPoE/PPP user disconnects in the access network without the LAC/LNS noticing it to tear down the connection (also the PPP keepalive hasn't detected yet), and a second PPP request comes from the same subscriber on the L2TP tunnel (same or different LAC/tunnel), then a second route is added to the table having the next hop "service to unknown". PR981488: This issue has been resolved.
  • In H323 ALG with CGNAT scenario, the MS-PIC might crash when the ALG is deleting an H323 conversation due to the deleting port is outside of allocated NAT port-block range. PR982780: This issue has been resolved.
  • On M/MX/T Series routers (platforms) with Services PIC with dynamic-nat44 translation-type configured, when the flows are cleared the IP addresses in use are never freed. This issue is present in Junos OS 11.4R7 and all more recent releases without this fix. PR986974: This issue has been resolved.
  • In large scale L2TP LNS environment, when the SNMP MIB JNX-L2TP-MIB is walked continuously, the memory of the L2TP daemon (jl2tpd) increases due to memory leak. PR987678: This issue has been resolved.
  • The redundant services interfaces (rsp) configured with "hot-standby" mode might flap upon commit. PR1000591: This issue has been resolved.
  • The following messages are being logged at ERR not DEBUG severity: mspd[3618]: mspd: No member config mspd[3618]: mspd: Building package info This PR sets the correct severity. PR1003640: This issue has been resolved.

Subscriber Access Management

  • MIB entries for jnxUserAAAAccessPoolRoutingInstance may not appear after deleting and re-adding an assignement pool unter a routing instance. PR998967: This issue has been resolved.

User Interface and Configuration

  • When load large scale configuration, due to the ddl object not being freed properly after it's accessed, load configuration failed with error: Out of object identifiers. PR985324: This issue has been resolved.

VPNs

  • In NG-MVPN scenario with S-PMSI tunnel has been triggered, performing Routing Engine switchover right after deleting the MVPN routing instance might cause rpd process on master Routing Engine to crash. PR941160: This issue has been resolved.
  • The S-PMSI tunnel might fail to be originated from ingress PE after flapping the routes to customer multicast source. PR983410: This issue has been resolved.
  • In MVPN scenario, a multihomed ingress PE might fail to advertise type-4 after losing routes to local sources. PR984946: This issue has been resolved.
  • Make the assert winner send the assert messages in a spaced way just as PIM Hellos and Joins are sent. With fix, the assert winner sends the assert message more often such that helps the other routers on the LAN to maintain state. For now, the robustness count is hard-coded as 3. This will later be enhanced by way of a CLI knob such that the robust count is configurable. PR999019: This issue has been resolved.

Resolved Issues in Release 12.3R7

Class of Service (CoS)

  • If any of the schedulers have an ID of zero, cosd process might crash following a commit. PR953523: This issue has been resolved.
  • Sometimes the cosd process generates the corefile when add/delete child interface on the LAG bundle. PR961119: This issue has been resolved.
  • Applying a scheduler with transmit rate below 65,535 bps and rate-limit option fails the commit if the associated interface is a non-existing interface or a virtual interface. PR964647: This issue has been resolved.
  • On MX Series router with non-Q DPC (in this case, DPCE 40x 1GE R), when the "interface-set" is configured on a non-Q DPC, then execute the command "show interfaces interface-set queue <interface-set-name>", the DPC might crash. PR979668: This issue has been resolved.

Forwarding and Sampling

  • VPLS mac-table doesn't get populated with mac of previous lt interface after replacing the lt interface in the configuration, which might cause CE connected to the lt interface to get isolated. PR955314: This issue has been resolved.
  • When port-mirroring or sampling is configured, if a lot of route updates are happening in the system, the routing protocol convergence time might be long and packet loss might be observed. PR963060: This issue has been resolved.
  • Under rare circumstances, when a forwarding table filter (FTF) is configured and applied as a routing-instance (i.e. VPLS) in one same configuration change (i.e. in a same commit operation), if the Packet Forwarding Engine receives the forwarding-table binding message before the firewall filter content from the Routing Engine, then the MPC may restart. PR963584: This issue has been resolved.
  • In the large scaled DHCP subscribers setup (e.g. 54,000 dual-stack DHCP subscribers), dynamic firewall daemon (dfwd) memory leak occurs during DHCP subscribers login/logout. PR967328: This issue has been resolved.
  • DPC crashed after deactivate/activate [routing-instances TPIX bridge-domains IX bridge-options]. PR983640: This issue has been resolved.

General Routing

  • MPC might crash with corefiles due to watchdog timer expired. PR593444: This issue has been resolved.
  • When gr- interface is disabled, the DECAP-NH also needs to be deleted / set to discard. PR791277: This issue has been resolved.
  • The ingress family feature unicast Reverse Path Forwarding (uRPF) check execution order was invalidated when Filter Based Forwarding (FBF)was enabled on MX Series routers with MPCs or MICs. This solution repositions uRPF just prior to FBF, so that both actions are compatible and applicable. This applies to both IPv4 and IPv6. PR805599: This issue has been resolved.
  • RPD crashed on backup Routing Engine when trying to compare gateways of two different types of nexthops, like table next hop which is installed in kernel for one route, router next hop which is selected in backup RPD. PR828797: This issue has been resolved.
  • When dynamic profile is used in a Dual-Stack network, the IPv4 routes of the subscriber might unexpectedly get deleted after deleting IPv6 routes. Same thing happened when delete IPv4 subscriber routes. PR864696: This issue has been resolved.
  • MXVC /kernel: rts_ifstate_client_open:Number of ifstate clients have reached threshold,current = 63 maximum = 63. PR894974: This issue has been resolved.
  • The chassisd process is getting stalled and later on restarts with core. The stalling might be caused by debug code which is used to log the errors in the counters when i2c driver or kernel bug shows. PR912990: This issue has been resolved.
  • Leak in /mfs/var/sdb/iflstatsDB.db. PR924761: This issue has been resolved.
  • On MX Series routers containing multiple Packet Forwarding Engines such as MX240/MX480/MX960/MX2010/MX2020, with either MPC3E or MPC4E cards (MPC3 Type 3 3D/MPC4E 3D 2CGE+8XGE/MPC4E 3D 32XGE), if multicast traffic or Layer 2 flood traffic enters the router via these MPC3E or MPC4E line cards, these line cards may exhibit a lockup, and one or more of their Packet Forwarding Engines corrupt traffic towards the router fabric. PR931755: This issue has been resolved.
  • In the MX-VC scenario, have chassis fabric redundancy mode set to increased bandwidth (root@user# set chassis fabric redundancy-mode increased-bandwidth). Then configure the "offline-on-fabric-bandwidth-reduction" for any slot (root@user# set chassis fpc <slot> offline-on-fabric-bandwidth-reduction). After that execute commit, the commit check failed and chassisd crashed with core-dumps. PR932356: This issue has been resolved.
  • In Junos OS versions later than 11.2 where IFL localization is enabled, Routing Engine mastership switchover could lead to IFL indexes inconsistency in Ichip FPCs when graceful Routing Engine switchover (GRES) is configured. This inconsistency could gradually lead to IFL index overlaps and traffic blackholing. PR940122: This issue has been resolved.
  • When configuring "no-readvertise" flag to existing static route, then this static route will not be exported to other VPN routing and forwarding (VRF) tables from onwards which is expected, however, because the static route has already been exported to other VRF tables before "no-readvertise" configuration, no deletion event occurs. Also, the "rt-export" bit still set for the static route which is exported to other routing tables after "no-readvertise" configuration. PR950994: This issue has been resolved.
  • On MX Series router with MX Series linecard or T4000 router with type5 FPC, and NAT is configured when random size packets are sent, some packets don't translate correctly. PR951232: This issue has been resolved.
  • If rpd ACK feature is enabled through command "indirect-next-hop-change-acknowledgements", when a route being added and a quick route change happens on the same route, high routing protocol process (rpd) CPU utilization might be seen and stays high (above 90%) until rpd is restarted. PR953712: This issue has been resolved.
  • Under particular scenarios, commit action might lead the Context-Identifier to be ignored when OSPF protocol refreshes its database. Then the PE router will stop advertising this Context-Identifier out. PR954033: This issue has been resolved.
  • FPC might lose the socket connection to the Routing Engine during the time kernel live-core dump is active. IGP session might get dropped after the socket connection got closed. The FPC will get restarted by the kernel once the live-core dump has finished. PR954045: This issue has been resolved.
  • In subscriber management environment, upgrade Junos OS to specific version (include 12.3R6 13.2R4 13.3R2) via ISSU might make subsequence subscribers fail to connect with following error: "jdhcpd_profile_request: Add Profile dhcp request failed for client in state LOCAL_SERVER_STATE_WAIT_AUTH_REQ: error = 301". PR959828: This issue has been resolved.
  • On MX Series router with dynamic vlan scenario, when improper sort order data is sent to dynamic vlan on the Packet Forwarding Engine, the Modular Port Concentrator (MPC) might crash and generate core files. PR961645: This issue has been resolved.
  • On all Junos OS platforms, if there are some events that cause Packet Forwarding Engine to restart (e.g. changing system mode on a QFX5100 switch), service might be interrupted because the stale interface index is not deleted successfully. PR962558: This issue has been resolved.
  • In the initial router configuration, if static routes are configured over GRE interface and OAM is enabled, then the static routes may remain active while the GRE tunnel is down. PR966353: This issue has been resolved.
  • The symptom of the issue: When the PIC reboots, new subscribers are not coming up. PR967070: This issue has been resolved.
  • Support for Layer 3 VPN localization has been deprecated in the Junos OS releases and platforms listed below. This affects the following CLI command: "set routing-instances [instance-name] routing-options localize" Junos OS releases: - 12.3R7 (CLI command is hidden) - 13.1R5 (CLI command is hidden) - 13.2R5 (CLI command is hidden) - 13.3R3 (CLI command is removed) - 14.1 (CLI command is removed) - 14.2 (CLI command is removed) Platforms: - M320 Series router - MX Series routers (all) - T Series routers (all). PR967584: This issue has been resolved.
  • PPP over ATM transit traffic was not being fragmented correctly by ATM MIC. The changes allow the fragmentation of the transit traffic to work properly. PR976508: This issue has been resolved.
  • On T Series router with FIB Localization enabled, if reboot the Routing Engine while scaled traffic running, the FIB-remote FPC might crash. PR979098: This issue has been resolved.
  • In the high scale P2MP LSP environment, heap memory leak might occur when the LSP flaps. Then some P2MP LSPs might be not installed, so the traffic will lose. PR979211: This issue has been resolved.
  • In rare condition, when PPPoE subscribers log in with large amounts of configuration data, the subscriber management infrastructure daemon (smid) and authentication service process (authd) might crash, and no new subscribers could connect to the router. PR980646: This issue has been resolved.
  • In the BFD environment with static route, the BFD session is established between two routers. When disable the subinterface on one router, the BFD AdminDown packet will be sent out from the router (this is not expected). But according to RFC 5882, another router receives the AdminDown packet, and the static route will never be deleted on it. That might cause traffic packets to be dropped. PR982588: This issue has been resolved.
  • On M7i/M10i with enhanced CFEB, M320 with E3-FPC, M120 and MX with DPC. In a race condition, the Dense Port Concentrator (DPC) may crash when ifls get added to an ifl-set while that same ifl-set gets deactivated/deleted in class-of-service. For example: # set interfaces interface-set interface_set_JTAC_ge-3/0/0 interface ge-3/0/0 unit 100 # deactivate class-of-service interfaces interface-set interface_set_JTAC_ge-3/0/0 # commit or (quick commit of following changes) # set interfaces interface-set interface_set_JTAC_ge-3/0/0 interface ge-3/0/0 # commit # deactivate class-of-service interfaces interface-set interface_set_JTAC_ge-3/0/0 # commit. PR985974: This issue has been resolved.
  • MX Series creates ptsp sessions with large delay. PR986905: This issue has been resolved.
  • The fabric performance of MPC1, MPC2, or 16xXE MPC in 'increased-bandwidth' mode on an MX960 populated with SCBE's will be less compared to redundant mode due to XF1 ASIC scheduling bugs. PR993787: This issue has been resolved.

High Availability (HA) and Resiliency

  • In a scenario where backup Routing Engine is resyncing, there is time window between the backup Routing Engine is marked as graceful Routing Engine switchover (GRES) ready and it factually ready. Then graceful Routing Engine switchover (GRES) might fail if it is performed during that time window. PR958453: This issue has been resolved.

Infrastructure

  • On RE-S-1800 family of Routing Engines, after an intensive writing to SSD, the immediate rebooting might cause SSD to be corrupted. PR937774: This issue has been resolved.

Interfaces and Chassis

  • When using vrrp inheritance, the vrrp stat could stick in bring-up state if it uses wrong vrrp parent, which will affect all new added vrrp even with a correct parent. As immediate recovery, it is suggested to correct or remove these two interfaces: xe-1/0/0.67 and xe-1/0/0.1066 and restart the vrrp process with the command : restart vrrp. PR820298: This issue has been resolved.
  • If the "tunnel-destination" address of a Generic Routing Encapsulation (GRE) interface is placed in one instance and the GRE interface is placed in another routing-instance, the lookup for the GRE tunnel destination is done on inet.0 instead of the appropriate routing instance's inet.0 table. The similar issue could happen on IP-over-IP or Automatic Multicast Tunneling (AMT) tunnels too. PR851165: This issue has been resolved.
  • In scenario when CCM has been running for a while and user issues the following CLI command: "show oam ethernet connectivity-fault-management interfaces interface-name extensive", the initial value reported for CCMs sent is wrong and then the command is executed immediately again and the value is correct. PR880615: This issue has been resolved.
  • Traffic which uses MPLS next-hops enters bridge-domain via IRB interface and if forwarding next-hop moves from non-aggregate interface to aggregate interface (MAC move), the MPLS next-hops are not correctly programmed in the Packet Forwarding Engine and dropped. The child next-hop of the aggregate interfaces are missing. Once IRB MPLS next-hop moves from aggregate interface to non-aggregate interfaces are not affected. IPV4 traffic will not trigger traffic drop upon mac move. The second symptom is a possible kernel core-dump on the new backup Routing-Engine after mastership switch. This applies to an IRB mac move for ipv4,ipv6 and mpls next-hops. PR924015: This issue has been resolved.
  • Queue stats counters for AE interface will become invalid after deactivating ifl on the AE interface. PR926617: This issue has been resolved.
  • "Too many I2C Failures" alarm happens when a FRU (in this case: PWR-MX960-4100-AC-S) experienced six consecutive i2c read/write failure. While the PEM still providing power to the chassis, chassisd daemon cannot read/write information from the PEM until it is reseated. In recent investigation, engineering team has come up some enhancements for this MX960 HC-AC and HC-DC PEM: 1. PEM i2c bus hang avoidance 2. Junos OS recovery from a hung i2c bus 3. noise reduction This Junos OS eliminates the need for the PEM FW upgrade, and at the same time is 100% compatible with those PEMs which have been upgraded. PR928861: This issue has been resolved.
  • Digital Optical Monitoring MIB jnxDomCurrentRxLaserPower gives wrong value in 12.3R3-S6. PR946758: This issue has been resolved.
  • When Connectivity Fault Management (CFM) is configured, if maintenance domain intermediate point (MIP) session associated to default maintenance domain (MD) is inactive, a deletion of interface can not delete MIP session structure, hence might causing memory leak. This crash could also be seen if delete more than one Virtual private LAN service (VPLS) routing instance with no neighbor configuration. PR947499: This issue has been resolved.
  • On MX Series routers (platforms) with Small Form-factor Pluggable (SFP). There are two connected ports. Both of them are configured with auto-negotiation, and one port is configured speed with 10M. So both the ports auto negotiate to 10m. After removing the SFP from one port and reinserting it, then the port is not coming up. PR953518: This issue has been resolved.
  • Kernel crash may happen when a router running a junos install with the fix to PR 937774 is rebooted. This problem will not be observed during the upgrade to this junos install. It occurs late enough in the shutdown procedure that it shouldn't interfere with normal operation. PR956691: This issue has been resolved.
  • If there is an IRB interface configured for "family inet6" in a bridge-domain on an MXSeries router, the Packet Forwarding Engine may not correctly update the next-hop for an IPv6 route when the MAC address associated with the next-hop moves from an AE interface to a non-AE interface. PR958019: This issue has been resolved.
  • In the large scaled VPLS environment , during delete routing-instance of type VPLS, the memory is not getting freed. The connectivity-fault management daemon (cfmd) might crash with a core file generated.,The core files could be seen by executing CLI command "show system core-dumps". PR975858: This issue has been resolved.
  • In the multilink frame relay (mlfr) environment with "disable-tx" configuration, when the differential delay exceeds the red limit, the transmission is disabled on the bundle link. When it is restored, the link should be added back. But in this case, the link stays in the disable state, and it is not rejoined to the bundle. PR978855: This issue has been resolved.
  • After the following process, we can find MCAE becomes standby/standby status. Even if we set "set interfaces aeX aggregated-ether-options mc-ae events iccp-peer-down prefer-status-control-active" for both routers, we can find this issue. << topology example >> iccp ge-1/0/1 ge-1/0/1 [ MX80(router A)]-----------------[MX240(router B)] \ ae0 ae0 / --active-- \ / --standby-- \ MC-LAG / \ / \ / ae0(ge-0/0/0)\ /ae0(ge-0/0/1) [ EX4200(switch C) ] << process >> initial status router A : active router B : standby 1. disable ae0 of router A. 2. disable iccp link of router A. 3. disable ae0 of switch C 4. enable iccp link of router A. (Please wait until iccp status up.) 5. enable ae0 of switch C 6. enable ae0 of router A. PR982713: This issue has been resolved.

Layer 2 Ethernet Services

  • When DHCP local server and DHCP relay are both configured on same router, the DHCP relay binding might get lost if a graceful Routing Engine switchover (GRES) is performed. PR940111: This issue has been resolved.
  • IP address change of a DHCP relay interface does not get reflected in gateway IP address (giaddr) when maintain-subscribers knob is enabled, which needs to restart DHCP daemon to make it work again. PR951909: This issue has been resolved.
  • In L3 Wholesale environment, the DHCP clients might fail to renew their address in DHCP relay scenario. PR956675: This issue has been resolved.
  • In the MPLS environment, LDP session protection is configured and LDP session establishes via IRB interface. When disable IRB interface, LDP session protection does not work. The LDP session goes down even if there is an alternate route. PR959396: This issue has been resolved.
  • Configuring Ethernet Ring Protection Switching (ERPS), after changing interface's MTU on Ring Protection Link (RPL) owner, all the interfaces on RPL owner change into forwarding state, hence cause a layer 2 loop. PR964727: This issue has been resolved.
  • On MX Series platform with Ethernet Ring Protection Switching (ERPS) configuration, after disabling Ring Protection Link (RPL) interface and then moving RPL from west interface to east interface, as a result, the ERPS east and west interface might go into discard state at same time. PR970121: This issue has been resolved.
  • When Cisco running in an old version of PVST+, it doesn't carry VLAN ID in the end of BPDU. So Juniper Networks equipment fails to respond Topology Change Notification ACK packet when interoperates with Cisco equipment. After the fix, Juniper Networks equipment will read the VLAN ID information from Ethernet header. PR984563: This issue has been resolved.

MPLS

  • When a First hop LSR is sending Resv Message with non-directly connected IP as nexthop (in Resv HOP object), Junos OS on head end will try to install this in forwarding table. As the nexthop to be used is a non-directly connected address, forwarding table update will fail with following KRT_Q_STUCK message: RPD_KRT_Q_RETRIES: Route Update: Invalid argument. PR920427: This issue has been resolved.
  • When Packet Forwarding Engine fast reroute (FRR) applications are in use (such as MPLS facility backup, fast-reroute or loop free alternates), a primary path interface flap could be triggered due to Operation, Administration, and Maintenance (OAM) link failure detection or by Bidirectional forwarding detection (BFD). However, this interface flap might lead to a permanent use of the backup path, which means the original primary path could not be active again. PR955231: This issue has been resolved.
  • We add timer for all aggregate LDP prefixes but are not deleting it when the timer expires because of a bug. Since the timer is not expiring, we never update the route for any change. This will be sitting in the routing table as a stale entry. PR956661: This issue has been resolved.
  • When the Label Distribution Protocol (LDP) feature is enabled and the background job "LDP sync send filtered label job" is running, when shut down the LDP, due to LDP failing to delete a job that didn't exist while shutting down, routing protocol process (rpd) might crash. PR968825: This issue has been resolved.
  • In the large scaled MPLS setup with NSR enabled, when restart routing protocol process (rpd) on standby Routing Engine, or reload standby Routing Engine, or reload router, some filtered output label bindings might be missed on the backup Routing Engine, which leads to Label Distribution Protocol (LDP) database between the master and backup Routing Engines to be inconsistent. PR970816: This issue has been resolved.
  • In a scaled MPLS environment, whenever fast reroute (FRR) or Link Protection (LP) or Node Protection (NP) is configured, the switchover from the primary LSP to the secondary LSP might cause traffic loss for a few seconds. PR973070: This issue has been resolved.
  • In the MPLS environment, when execute the command "show snmp mib walk mplsXCTable" to walk the MPLS cross connect table, the routing protocol daemon (rpd) CPU utilization might reach over 90%, and the rpd process doesn't respond to any CLI show commands. PR978381: This issue has been resolved.
  • snmpwalk/snmpgetnext or "show snmp mib walk" fail when polling MPLSLSPOCTETS, MPLSLSPPACKETS, MPLSLSPINFOOCTETS or MPLSLSPINFOPACKETS. PR981061: This issue has been resolved.
  • In the MPLS environment with "egress-protection" configuration, there is a direct LDP session between primary PE and protector. One context-id is configured as primary PE's loopback address or any LDP enabled interface address. When delete the whole apply-group or delete the ldp policy from apply-group, the routing protocol daemon (rpd) might crash. PR988775: This issue has been resolved.

Platform and Infrastructure

  • NPC core observed @nh_dfw_get_correct_next_intf_prefix. PR801607: This issue has been resolved.
  • Unwanted continuous vc_ifd_is_remote(): is_remote_ifd ptr NULL! syslog messages being printed by the MS-DPC. PR868273: This issue has been resolved.
  • When the instance have vlan-id all and adding interface unit with "vlan-tags outer X inner Y" to this instance, traffic from ALL instance VLANs is leaking over that unit tagged with outer tag X and each VLANs own inner tag A,B.C. PR883760: This issue has been resolved.
  • On MX Series based line card, for interfaces tagged with VLAN ID same as the native-vlan-id configured on the interface, FPC adds Native VLAN ID to the packets received on the interface and destined to the host. This is irrespective of the packet content. This results in the packets getting doubly tagged when receiving packets which are already tagged with VLAN ID matching the Native VLAN ID, and thus cause ARP resolution failure on Native VLAN. For example, the ARP packets to IRB (on VLAN 101) are tagged with VLAN ID 101 (which is also the native VLAN ID) and are getting additionally tagged. Hence they are dropped by the IRB and this can cause the ARP request packet to not get resolved on Native VLAN. PR917576: This issue has been resolved.
  • When the transit traffic is hitting the router and the destination is a local segment IP which requires ARP resolution, it's mis-classified by the DDOS filter and an incorrect policer is applied. This leads to host queue congestion. PR924807: This issue has been resolved.
  • MPC Type 2 3D may crash with CPU hog due to excessive link flaps causing the interrupts to go high . PR938956: This issue has been resolved.
  • On I-chip platforms, when forwarding table filter (FTF) is configured for a virtual private LAN service (VPLS) routing instance, the jtree memory corruption might occur if the routing table attached by FTF is destroyed. The route table that is attached by FTF can get destroyed with different events such as interface which is part of the VPLS routing instance flaps or route-distinguisher is changed, etc. PR945669: This issue has been resolved.
  • If a PE router is both egress and transit node for a p2mp lsp, the Packet Forwarding Engine may report errors and install a discard state for the fib entry representing the p2mp lsp label with bottom of stack bit set to 0 . This problem does not have any impact since there is no application using the s=0 entry of a p2mp lsp. PR950575: This issue has been resolved.
  • MIC-3D-40GE-TX (3D 40x 1GE(LAN) RJ45) restarts with core-dumps repeatedly after configuring "VRRP interface" and "traffic-manager mode ingress-and-egress" on PIC2 or PIC3. PR950806: This issue has been resolved.
  • Current display of "cli> request chassis routing-engine hard-disk-test show-status" command for Unigen SSD identified by "UGB94BPHxxxxxx-KCI" is incorrect and can be misleading when use for troubleshooting. For example, attribute 199 is display as "UDMA CRC Error Count" is actually "Total Count of Write Sector". PR951277: This issue has been resolved.
  • For Trio-based platforms,traffic unbalance (60:40 or 70:30) can be seen in the output interface of 2nd node in the cascaded topology. Current Junos OS hash-seed implementation on Trio-chipset can be used to protect the hash-cascade problem (unbalance at 2nd node output, 0:100 for example) but it doesn't work very well(60:40 or 70:30 can be seen). This fix made enhancement, it can deliver nearly 50:50 LB performance. PR953243: This issue has been resolved.
  • On Trio based platforms, the counter for Network Time Protocol (NTP) protocol of the output of "show ddos-protection protocols ntp" would be always null, even though it is confirmed that there are NTP unicast packets received. The reason for this is that only multicast NTP packets are treated as NTP packet type by DDoS protection policer in current implementation, whereas the unicast NTP packets are not. PR954862: This issue has been resolved.
  • * MX2020 FanTray power specification. - zone#1:FT#3 - gets power from zone#1 only - zone#1:FT#2 - gets power from zone#0 in case of no-power in zone#1 - zone#0:FT#1 - gets power from zone#0 only - zone#0:FT#0 - gets power from zone#1 in case of no-power in zone#0 - Critical(Minimum) number for MX2020 operation is 3 If one of zone has no PSM, then it means FAN single-fault in the chassis's point of view. For example, if zone#1 has no PSM, then the FT#3 doesn't get power as it is local-powered FT. Hence, in this case, the FT#3-LED should show ORANGE to notify the single-fault to user, while FT#2 can shows GREEN if it gets enough power from zone#0. In addition, CRAFT-LED for FT#3 should be turned off. * Due to HW-limit(bicolor), it could not show ORANGE color. In current implementation, both CRAFT-LED, FT#3-LED show GREEN. That's problem. * NOTE: Junos OS doesn't support FT double-fault scenario. (MX2020 needs minimum 3 FTs.) If FT#2 gets in trouble in above case(i.e.,FT double-fault), the user should see serious cooling-trouble on SFMs within 1 minute. PR957395: This issue has been resolved.
  • When the special character "%" in comment field, execute the command "root@user# show | display xml", management daemon (mgd) might crash. PR957651: This issue has been resolved.
  • Unable to modify dynamic configuration database after first commit. PR959450: This issue has been resolved.
  • When we set "traffic-manager mode ingress-and-egress" on "MIC-3D-40GE-TX (3D 40x 1GE(LAN) RJ45)", we cannot use ingress queue correctly on PIC2 and PIC3. *Note: We cannot see this issue if we set the above configuration to PIC0 or PIC1. PR959915: This issue has been resolved.
  • In current Junos OS, a PSM shows dc output value even though it is turned off by switch. This cosmetic bug causes miscalculation of actual usage in 'show chassis power'. PR960865: This issue has been resolved.
  • Upon the deletion of a routing-instance and subsequent commit, error logs are generated from each Type 1 - 3(non E3) based FPC. These logs are cosmetic and can be ignored. PR964326: This issue has been resolved.
  • A defect in L3VPN Make Before Break code was resulting in freeing memory corresponding to old nexthops which is being used by egress Packet Forwarding Engine. This was resulting in memory corruption. PR971821: This issue has been resolved.
  • With NG-MVPN, multicast traffic might get duplicated and/or blackholed if a PE router, with active local receivers, is also a transit node and the p2mp lsp is branched down over an aggregate interface with members on different Packet Forwarding Engines. PR973938: This issue has been resolved.
  • SNMP alarms/traps could be generated for unpowered fan trays when only one zone is powered. PR982970: This issue has been resolved.

Routing Protocols

  • On MX Series routers containing multiple Packet Forwarding Engines such as MX240/MX480/MX960/MX2010/MX2020 routers, with DPC (Dense Port Concentrator) or FPC (Flexible Port Concentrator) or with line cards designated with "3D", RPD may restart when attempting to send a PIM assert message on an interface (whose interface index exceeds 65536). It is likely that RPD restarts repeatedly, since after RPD has restarted and protocols have converged, the same PIM assert will trigger further RPD restarts. PR879981: This issue has been resolved.
  • The rpd process might crash when executing the command "show route advertising-protocol bgp <nbr>" without a table option, or with a table that is not advertised by BGP. PR959535: This issue has been resolved.
  • With BGP import policy as next-hop peer-address, if the local router receives inet (or inet-vpn) flow network-layer reachability information (NLRI), routing protocol process (rpd) might crash. Junos OS is designed to create a fictitious nexthop for inet flow and inet-vpn flow families as they don't send/expect-to-receive nexthops. So in this case when the import-policy set a non-null next-hop for the received inet (or inet-vpn) flow route, it could not handle properly which might result in rpd crash. PR966130: This issue has been resolved.
  • In the dual Routing Engine scenario, after a Routing Engine switchover, the periodic packet management daemon (ppmd) might exit. PR979541: This issue has been resolved.
  • Due to some corner cases, certain commits could cause the input and/or output BGP policies to be reexamined causing an increase in rpd CPU utilization. PR979971: This issue has been resolved.
  • Forwarding cache limit not properly meeting threshold after configuration change when configured per address family. PR980578: This issue has been resolved.

Services Applications

  • SIP call forwarding may fail when NAT is used between parties even though the SIP ALG is in use. PR839629: This issue has been resolved.
  • 11.4 Junos OS releases introduce the IKEv2 support and a stricter check on IKE/IPsec SAs proposal parameters. PR843893: This issue has been resolved.
  • Any SIP MESSAGE request will be dropped by the SIP ALG. This type of request is unsupported from day one. This is rare type of request that will not prevent more usual SIP operations such as voice calls, but it may affect some instant messaging applications based on SIP. PR881813: This issue has been resolved.
  • Clearing the stateful firewall subscriber analysis causes the active subscriber count to display a very huge number. The large number is seen because when a subscriber times out, the number of active subscribers is decremented. If it is set to zero using the clear command, then a decrement would give an incorrect result. PR939832: This issue has been resolved.
  • DNS multiple queries A and AAAA might cause the Service-PIC to restart. PR943425: This issue has been resolved.
  • When sending traffic from Internet end, the softwire count is incorrect. PR948583: This issue has been resolved.
  • The dynamic flow control process (dfcd) might core dump when Dynamic Tasking Control Protocol (DTCP) trigger request is same for both the VLAN and DHCP subscriber. PR962810: This issue has been resolved.
  • In the context of DS-Lite softwire scenario, where the Address Family Transition Router (AFTR) node performs NAT with Endpoint Independent Filtering (EIF) and Endpoint Independent Mapping (EIM) enabled, the simultaneous arrival of two packets from opposite sides of the NAT will trigger the creation of the same flow, which in a race condition results in the Service-PIC restart. PR966255: This issue has been resolved.
  • Jl2tpd could crash because of loss of private sdb table delete operations when the mirroring connection bounces on the standby. PR968947: This issue has been resolved.
  • When transferring large FTP file, the server might send packets with incorrect Layer 4 checksum. If inline NAT service is enabled on the router, it might transit the packets to client instead of dropping it, which eventually causes the client FTP time out. PR972402: This issue has been resolved.
  • Due to a race condition, the Service PIC handling the MX Series DS-LITE tunnels may core during a softwire lookup. PR978598: This issue has been resolved.

Software Installation and Upgrade

  • Routing Engine could be brought to DB mode when rebooting after interrupted downgrade. PR966462: This issue has been resolved.

User Interface and Configuration

  • When load large scale configuration, due to the ddl object not being freed properly after it's accessed, load configuration failed with error: Out of object identifiers. PR985324: This issue has been resolved.

VPNs

  • In Multicast VPN scenario, when multiple instances exist and tunnel-source is configured in more than one instance, the routing protocol process (rpd) might reinitialize with core file if one of the instances is deleted or Route Distinguisher (RD) ID is changed. PR877682: This issue has been resolved.
  • With these high amount of streams, we have higher number of data-mdt-tlvs to process which is becoming a bottleneck. PR957280: This issue has been resolved.
  • Upon withdraw/inject, bgp routes in the serving PEs for two different route-groups,member/regular sites receive traffic from both serving sites for 60 seconds. PR973623: This issue has been resolved.
  • Route group member site and regular site may receive data from two serving sites of two groups for the same (S,G). This only happens when in one RG there are no receivers. PR974245: This issue has been resolved.
  • In Rosen MVPN environment, if there a two multihomed ingress PEs, when the route to multicast source flaps, the receiver router might keep switching between sender Data MDTs, which result in traffic loss. PR974914: This issue has been resolved.
  • In the Rosen MVPN environment, setting the TOS IP control packet bit can avoid the possibility of data-mdt TLV messages being dropped in the core during congestion. But in this case, the TOS field to indicate its IP control packet (0xc0) is not set. This might lead to traffic loss. PR981523: This issue has been resolved.

Resolved Issues in Release 12.3R6

Class of Service (CoS)

  • After swapping MPC2E-3D-Q card with MPC2E-3D-EQ card, an interface is still running out of queues with only 32k queues in use. PR940099: This issue has been resolved.

Forwarding and Sampling

  • Filter state failed to be present in the kernel and was not created on Packet Forwarding Engine. Added check to retry creating filter state before pushing to Packet Forwarding Engine. PR937607: This issue has been resolved.

General Routing

  • When graceful Routing Engine switchover (GRES) and ARP purging is enabled, frequent route flapping, route entry and nexthop fail to sync up between the master Routing Engine and the backup Routing Engine. So when the master Routing Engine would like to add a new nexthop but see the backup Routing Engine has already found a nexthop with same destination. It makes the backup Routing Engine reboot and crash on both the Routing Engines. PR899468: This issue has been resolved.
  • RPD on backup Routing Engine might hit out of memory condition and crash if BGP protocol experiences many flaps. PR904721: This issue has been resolved.
  • After FPC/MPC is reset or while PPPoA customer login, in rare case, the ppp daemon (jpppd) might get an incorrect value from device control daemon (dcd) which might cause all the new Link Control Protocol (LCP) messages to be ignored and results in static PPPoA sessions can not come up. This problem is seen on MX Series platform products so far, but the problem is mostly common and if other products are using the same version of Junos OS software it might apply to them. PR912496: This issue has been resolved.
  • Leak in /mfs/var/sdb/iflstatsDB.db PR924761: This issue has been resolved.
  • MX80 routers now support CLI command "show system resource-monitor summary". PR925794: This issue has been resolved.
  • High routing protocol process (rpd) CPU utilization is seen and it stays high (above 90%) until the rpd is restarted. PR925813: This issue has been resolved.
  • If MX Series router is in increased-bandwidth fabric mode, pulling out one SCB might cause packets loss. PR934544: This issue has been resolved.
  • tcp_inpcb buffer leak in ADC and TLB service pics. PR934768: This issue has been resolved.
  • When an SNMP walk is performed to query the native VLAN (mib-2.17.7.1.4.5.1...: dot1qPvid) or the logical type (trunk or access) of the interface (mib-2.17.7.1.4.3.1.5...: dot1qPortVlan), the SNMP walk might cause a memory leak on the Layer 2 address learning process (l2ald), and the process might crash with a core file generated. PR935981: This issue has been resolved.
  • If IPv6 duplicate address is detected, interface can't recover to normal state after flapping interface. Reconfigure IPv6 address will resolve this issue. PR936455: This issue has been resolved.
  • Master Routing Engine reboot due to "panic: pfe_free_peer: not in peer proxy process context" Trigger: replacement of backup Routing Engine. PR936978: This issue has been resolved.
  • LNS drops the LCP Compression Control Protocol (CCP) packet silently comes from L2TP tunnel. PR940784: This issue has been resolved.
  • In subscriber management environment, profile database files at backup Routing Engine get corrupted when the dynamic profile versioning and commit fast-synchronize are enabled in configuration. After graceful Routing Engine switchover (GRES) when the backup Routing Engine become master, all the existing DHCP subscribers stuck in RELEASE State and new DHCP subscribers can't bind at this point. PR941780: This issue has been resolved.
  • MP-BGP route withdraw update might not been sent after deletion of a routing-instance configured with resolve import policy. PR942395: This issue has been resolved.
  • Egress multicast statistics displays incorrectly after flapping of ae member links on M320 or T Series FPC (M320 non-E3 FPC and T Series non-ES FPC). PR946760: This issue has been resolved.
  • With scaled configuration of ATM VCs (~4000 VCs) on a single MIC-3D-8OC3-2OC12-ATM ATM MIC, the MIC might crash. The crash is not seen with lower scale (i.e. less than 3500 VCs per MIC). PR947434: This issue has been resolved.
  • CLI command "show interfaces queue" does not account for interface queue drops due to Head drops. This resulted in the "Queued" packets/bytes counter to be lesser than that was actually received and dropped on that interface queue. This PR fixes this issue. Head-drops, being a type of RED mechanism, is now accounted under the "RED-dropped" section of the CLI command "show interfaces queue". PR951235: This issue has been resolved.

High Availability (HA) and Resiliency

  • With minimal flow configuration, if graceful Routing Engine switchover is not enabled, routing protocol process (rpd) crashes during shutting down the rpd process due to missing safety checks. The core files could be seen by executing CLI command "show system core-dumps". PR852766: This issue has been resolved.

Interfaces and Chassis

  • If there are several logical systems in one router, basically one logical tunnel (lt-) interface needs to work with another lt- interface, which is peer lt- interface. If one of them allocates a MAC address first and the other attempts to allocate a MAC address, then panic happens since it is a reallocation which finally results in the kernel crash. The problem might be seen when deactivating and then activating logical systems or renaming the lt- interface. PR837898: This issue has been resolved.
  • The eeprom SFP-Type descriptor has been updated to display different unique values for fixed-rate or tri-rate copper SFPs. Going forward, the model SFP-1GE-T shows as "1000BASE-T Copper SFP" while model SFP-1GE-FE-E-T shows as "Tri Rate Copper SFP". PR877152: This issue has been resolved.
  • In scenario when CCM has been running for a while and user issues the following CLI command: "show oam ethernet connectivity-fault-management interfaces interface-name extensive", the initial value reported for CCMs sent is wrong and then the command is executed immediately again the value is correct. PR880615: This issue has been resolved.
  • Problem scenario: CFM UP MEP for Bridge/VPLS is configured on MPC with action profile as 'interface down' Problem statement: When the CFM sessions go down due to network outage at the core, action profile is triggered and configured interface is brought down. When the Core network failure is corrected, CFM will not automatically recover because interface will continue to remain down. PR884323: This issue has been resolved.
  • When MX Series routers are running with MC-LAG in active-active mode, the layer 2 address learning daemon (l2ald) might crash if a MAC address is being deleted from one port while the same entry is locally learned on a different port. PR888636: This issue has been resolved.
  • In Point-to-Point Protocol over Ethernet (PPPoE) scenario, if some PPPoE session was added and deleted, after performing Routing Engine switchover operation, the Broadband Remote Access Server (BRAS) might fail to allocate PPPoE session IDs on interFace Descriptor (ifd). PR896946: This issue has been resolved.
  • In Multichasis Link Aggregation (MC-LAG) scenario, when MC-LAG works on Active-Active mode, if the link of MC-LAG flaps repeatedly, the layer 2 address learning daemon (l2ald) might crash with a core file generated. PR913222: This issue has been resolved.
  • Problem Statement: OAM Packets do not gets forwarded with UP and Down MEP configured in access and core interfaces of the bridge down respectively along with MIP configured on the BD. The above configuration was resulting in not honor split horizon forming a loop in core network. This results in packet drop in core network. PR925288: This issue has been resolved.
  • "Too many I2C Failures" alarm happens when a FRU (in this case: PWR-MX960-4100-AC-S) experienced 6 consecutive i2c read/write failures. While the PEM still providing power to the chassis, chassisd daemon cannot read/write information from the PEM until it is reseated. In recent investigation, engineering team has come up some enhancements for this MX960 HC AC PEM: 1. PEM i2c bus hang avoidance 2. Junos OS recovery from a hung i2c bus 3. noise reduction This Junos OS eliminates the need for the PEM FW upgrade, and at the same time is 100% compatible with those PEMs which have been upgraded. PR928861: This issue has been resolved.
  • In PPPoE subscriber management environment, when PPP daemon is receiving an LCP packet with an invalid code ID and without any option, jpppd process crashes with a core file dumped. PR929270: This issue has been resolved.
  • After APS switchover, duplicate packets might be received from the backup circuit under SONET APS configuration with channelized enhanced intelligent queuing (IQE) interface. PR930535: This issue has been resolved.
  • This is a day-1 issue. When a member link was added to or removed from an aggregate bundle like AE on a dual Routing Engine sytem without graceful Routing Engine switchover (GRES), Kernel in the backup Routing Engine would crash due to assertion failure in the function rt_pfe_nh_cont_nh_decrement_ack_count. PR935729: This issue has been resolved.
  • Traffic is not flowing over Demux input interface. PR937035: This issue has been resolved.
  • PCS statistics counter(Bit errors/Errored blocks) not working on Mammoth PIC(xge). PR942719: This issue has been resolved.

Layer 2 Features

  • ========== BACKGROUND ========== A global graceful Routing Engine switchover (GRES), which will cause a master Routing Engine to transition to backup, WILL require all Kernel state to be cleaned so that it can start a fresh resync from the new master. Ksyncd is tasked with cleaning up Kernel state. On cleaning routing tables, if any table has a non-zero reference count, it will return "Device Busy" to the ksyncd. Ksyncd will try 5 successive cleanup attempts after which it will trigger a live Kernel core. ======= PROBLEM ======= In ksyncd's kernel cleanup, the Bridge Domain mapped to a VPLS routing table is deleted AFTER an attempt is made to delete the route table. This is a catch-22 since BDs hold reference counts to the routing table. ===== FIX ==== Cleanup of VPLS routing tables should proceed bottom up in the following order: NextHop Deletes, User Route Deletes, Interface Deletes(ifd,ifl,iff), STP Deletes, Bridge Domain Deletes, Mesh Group Deletes and finally Routing Table delete. This ensures that when we get to routing table delete, all dependencies, that could hold a ref cnt to the routing table, are now gone. PR927214: This issue has been resolved.

Layer 2 Ethernet Services

  • In multilink scenario, while polling the multilink statistics, the Packet Forwarding Engine statistics thread might be yielded. This might happen where there are large number of bundles and links. When the statistics thread is yielded, the context switches to the Packet Forwarding Engine manager thread to handle link and bundle delete operations, some pointers used by the statistics thread are freed up and so when the statistics thread regains control it crashes because of the dangling pointers. PR827326: This issue has been resolved.
  • In MX Virtual Chassis (MXVC) scenario, under high scale system environment (many Aggregated Ethernet interfaces, many logical interfaces), after performing global graceful Routing Engine switchover by CLI command "request virtual-chassis routing-engine master switch", the Link Aggregation Control Protocol (LACP) state of access Link Aggregation Group (LAG) interface might change and therefore resulting in traffic loss. PR885013: This issue has been resolved.
  • In Ethernet ring protection scenario, upon FPC reboots the STP index will get mis-aligned causing traffic drop. when this issue occurs following message can be seen. Before FPC restarts: user@router> show protection-group ethernet-ring vlan Ethernet ring IFBD parameters for protection group Ring1 Interface Vlan STP Index Bridge Domain xe-5/3/0 302 222 default-switch/v302 xe-0/2/0 302 223 default-switch/v302 xe-5/3/0 308 222 default-switch/v308 xe-0/2/0 308 223 default-switch/v308 After FPC restarts: user@router> show protection-group ethernet-ring vlan Ethernet ring IFBD parameters for protection group Ring1 Interface Vlan STP Index Bridge Domain xe-5/3/0 302 245 <<<< default-switch/v302 xe-0/2/0 302 223 default-switch/v302 xe-5/3/0 308 222 <<<< default-switch/v308 xe-0/2/0 308 223 default-switch/v308 PR937318: This issue has been resolved.
  • Service accounting interim updates not being sent. PR940179: This issue has been resolved.

MPLS

  • When static LSPs are configured on a node, RPD could assert upon committing a MPLS-related configuration change. Example: router> show system rollback compare 9 8 [edit protocols mpls] interface ae11.0 { ... } + interface as3.0 { + admin-group red; +} [edit protocols ISIS interface as3.0 level 2] ! inactive: metric 2610; The following error is seen in /var/log/messages in-relation to a static lsp, immediately following the above-mentioned configuration change: rpd[1583]: UI_CONFIGURATION_ERROR: Process: rpd, path: [edit groups STATELESS_ARIADNE protocols mpls static-label-switched-path static-lsp], statement: transit 1033465, static-lsp: incoming-label 1033465 has already been configured by this or other static applications PR930058: This issue has been resolved.
  • In certain circumstance, the Junos OS rpd route flash job and LDP connection job are always running starving other work such as stale route deletion. These jobs are running as LDP is continuously sending label map and label withdraw messages for some of the prefixes under ldp egress policy. This is due to LDP processing a BGP route from inet.3 for which it has a ingress tunnel (the same prefix is also learned via IGP) creating a circular dependency as BGP routes can themselves be resolved over a LDP route. PR945234: This issue has been resolved.
  • In a highly scaled configuration the reroute of transit RSVP LSPs can result in BGP flap due to lack of keepalive messages being generated by the Routing Engine. PR946030: This issue has been resolved.
  • On IS-IS interfaces configured with point-to-point and ldp-synchronization, after a change of IP address on the interface from the remote router, and if the old LDP adjacency times-out after the new LDP adjacency is up, the ISIS protocol will be notified about old LDP adjacency down event and the LDP sync state will remain in hold-down even if the new LDP adjacency is up. PR955219: This issue has been resolved.
  • We add timer for all aggregate LDP prefixes but are not deleting it when the timer expires because of a bug. Since the timer is not expiring, we never update the route for any change. This will be sitting in the routing table as a stale entry. Issue is being fixed in later versions. PR956661: This issue has been resolved.

Platform and Infrastructure

  • In the Network Time Protocol (NTP) configuration, if the specified source ip address is not in current routing-instance, the router will use primary address of interface (which will be used to send packet) as source address, Client routers will treat the NTP packets as incorrect packets, and then NTP synchronization failed. PR872609: This issue has been resolved.
  • When tagged frames with larger than MTU size are received, some frames are not counted as oversized frames on 20x1GE MIC. PR879519: This issue has been resolved.
  • After interface reset, CoS information may not be applied correctly to the Packet Forwarding Engine, leading to inconsistency in scheduling/shaping in Qx Chip. PR908807: This issue has been resolved.
  • In a MX-VC environment, in certain situations the inter-chassis traffic may not be equally balanced across all available vcp links after adding extra links. PR915383: This issue has been resolved.
  • The system MAC address is not getting saved in a unified in-service software upgrade (ISSU) blob and it is not getting programmed again by the Routing Engine when the Packet Forwarding Engine re-connects. The hash seed is generated by using the system MAC address and since it is not saved in a unified ISSU blob, after an ISSU it is 0 and the hash seed is generated using that. If a FPC reboot, then it will get the correct system MAC address and generate the hash seed based on that. This will cause different FPCs in the system to have different hash seeds and could cause AE multicast traffic loss if the ingress and egress FPCs have different hash seeds. PR915933: This issue has been resolved.
  • In subscriber management scenario, memory leak might occur when the firewall fast-update-filter feature is configured, and it will impact any new subscriber login. Such memory leak can be seen with following command, root@router> show chassis fpc

    Temp CPU Utilization (%) Memory Utilization (%)

    Slot State (C) Total Interrupt DRAM (MB) Heap Buffer

    0 Online Absent 8 0 1024 70 << 13

    1 Online Absent 8 0 1024 29 13 PR926808: This issue has been resolved.

  • Under certain timing conditions the MPC/TFEB can receive the firewall filter configuration before it is fully booted/UP/ONLINE. Because the firewall filters can depend on certain default values which are not yet programmed the MPC/TFEB will crash/core-dump and reboot/restart/reload. PR928713: This issue has been resolved.
  • When replacing ichip FPC with MX Series FPC, "traceroute" packets going through a MX Series FPC may experience higher drop probability than when using an Ichip FPC. PR935682: This issue has been resolved.
  • On MX Series routers with DPC type FPCs running a 11.4 (or later) Junos OS release disabling family inet with uRPF enabled on a logical interface might result in another logical interface on the router to drop all incoming IPv4 packets. The lookup index is calculated by taking the lower 16 bits of the logical interface index (also called the IFL index). In other words lookup index = IFL index MOD 65536. It is normal, valid and expected to have logical interfaces which share the same lookup index. The problem described in this PR is not the fact that the lookup indexes are the same. Here is an example of two different logical interfaces on two different FPCs which share the same lookup index: Interface ge-1/1/0.0 has an IFL index of 141073 and a lookup index 10001: > show interfaces ge-1/1/0.0 Logical interface ge-1/1/0.0 (Index 141073) (SNMP ifIndex 2318) ^^^^^^ Flags: SNMP-Traps 0x4004000 Encapsulation: ENET2 Input packets : 0 Output packets: 0 Protocol inet, MTU: 993 ^^^^ Flags: Sendbcast-pkt-to-re, uRPF ^^^^ Addresses, Flags: Is-Preferred Is-Primary Destination: 1.1.1.0/30, Local: 1.1.1.1, Broadcast: 1.1.1.3 Protocol multiservice, MTU: Unlimited Flags: Is-Primary And interface ge-2/0/7.1647 has an IFL index of 10001 and a lookup index of 10001: > show interfaces ge-2/0/7.1647 Logical interface ge-2/0/7.1647 (Index 10001) (SNMP ifIndex 20551) Flags: SNMP-Traps 0x4000 VLAN-Tag [ 0x8100.1647 ] Encapsulation: ENET2 Input packets : 0 Output packets: 0 Protocol inet, MTU: 8978 Flags: Sendbcast-pkt-to-re, uRPF, uRPF-loose Protocol multiservice, MTU: Unlimited In the example above if family inet is disabled on ge-2/0/7.1647 then ge-1/1/0.0 will start dropping all incoming packets silently. PR936249: This issue has been resolved.
  • On TXP system false "SIB Cell Drop Error" alarm might be raised for LCC-SIB after autohealing CRC errors on corresponding HSL2 channel.

    This alarm should be treated as a false one as there are no drops of valid data cells.

    This alarm considered false only when it was raised during fabric autoheal. Fabric autoheal log can be checked as:

    > show chassis fabric errors autoheal 2013-12-04 18:40:52 CET Req: LCC0 plane 4 ln 15 LCC-to-SFC

    2013-12-04 18:40:52 CET Succeeded: LCC0 plane 4 ln 15 LCC-to-SFC PR937330: This issue has been resolved.

  • On front panel display LED status for PSM is incorrect after manually Remove/Insert of PSM. PR937400: This issue has been resolved.
  • "Total errors" counter of MAC statistics on MX DPC(ge/xge) is always 0. PR942183: This issue has been resolved.
  • TWAMP connection/session will come up only if the session padding length is greater than or equal to 27 bytes on the TWAMP Client, the valid range of padding length supported by the TWAMP Server is 27 bytes to 1400 bytes. If IXIA is used as the TWAMP Client, packet length range from 41 bytes to 1024 bytes is supported. PR943320: This issue has been resolved.
  • In PPPoE subscriber management environment, if the BRAS router is MX Series router with MS-DPC equipped and traffic from the subscribers is NATed on MS-DPC card, when PPPoE subscribers flap, heap memory leak might occur on the MS-DPC. PR948031: This issue has been resolved.

Routing Policy and Firewall Filters

  • Policy with Install-nexthop lsp may not work as expected when there is a LSP path change triggering route resolution. PR931741: This issue has been resolved.

Routing Protocols

  • When the IPv6 address on fxp0 is active during boot up, the joining of the all-router group causes the kernel to create a ff02::2 route with a private nexthop, which is not pushed to the Packet Forwarding Engine. When a non-fxp0 interface is active later, the private nexthop will be shared by the non-fxp0 interface as well, resulting in packet drops destined to ff02::2 on the non-management interface. - After this PR, the advertising interface should be configured via the following CLI. [edit protocols] + router-advertisement { + interface. <interface_name>; + } PR824998: This issue has been resolved.
  • When inter operate with Cisco router, OSPF adjacency might be brought down by Cisco end, if Junos OS CPU is high and LSA ACK is delayed for over 2 minutes. PR846182: This issue has been resolved.
  • "show route advertising-protocol bgp <nbr> table foo.mvpn.0" stops working after PR-908199 fix. PR929626: This issue has been resolved.
  • On the first hop router if the traffic is received from a remote source and the accept-remote-source knob is configured, the RPF info for the remote source is not created. PR932405: This issue has been resolved.
  • If you have fix for PR-929626, Avoid the following show command in a VPN setup "show route advertising-protocol bgp <nbr_addr> table foo.inet.0" Where <nbr_addr> is peer within routing-instance "foo" PR936434: This issue has been resolved.
  • In MVPN scenario, while performing CLI command "show route advertising-protocol bgp <neighbor>", the rpd might crash due to a timing issue that BGP rib for bgp.mvpn-inet6.0 table is NULL. PR940491: This issue has been resolved.

Services Applications

  • Max number of supported IPSec tunnels might depend on networking activity as well. Under heavy networking activities, while DPD (Dead Peer Detection) is enabled, the maximum number of supported IPSec tunnels can drop to about 1800. PR780813: This issue has been resolved.
  • In Carrier Grade NAT scenario, MS-PIC might crash and core dump when Port Block Allocation (PBA) block size is relatively big (8192 ports per block), this issue usually happens when a new block need to be allocated because the block currently is exhausted. PR874500: This issue has been resolved.
  • In the Session Initiation Protocol(SIP) Application Layer Gateway(ALG) with port block allocation enabled scenario("user@root# set services nat pool <pool-name> secured-port-block-allocation block-size <block-size>"), a SIP call to be set up and the ports block are allocated for the media flows. When the SIP media flows time out, the APP mapping starts using another port block. But if no enough port block to be allocated, the services Physical Interface Card(PIC) might crash. PR915750: This issue has been resolved.
  • In the IPsec scenario, when all available SAs are expired and the sequence number is wrapping for the IPsec packets, the Physical Interface Card(PIC) will delete the Security Association(SA), however this is not reported back to key management process(kmd). This would cause kmd and the PIC being out of sync regarding the known IPsec SAs, then the traffic blackhole might occur. PR933026: This issue has been resolved.
  • No SNMP trap generated when NAT or Flow sessions reach the threshold. PR933513: This issue has been resolved.
  • Interim-logging is now supported with NAT64 on microkernel (MS-DPC) platforms. The same pba-interim-logging-interval knob under 'service-options' under the service interface will enable the feature for NAT64 as well. PR935606: This issue has been resolved.
  • FW is trying to create a new pair of flows while a drop flow with the same selector is being installed for traffic initiated from the outside by a different CPU. There is a race condition while accessing the flow type field: - CPU1 (installing the drop flow) - creates the flow and adds it to the flow table while holding the corresponding bucket lock. However, the flow type field is filled in later. - CPU2 (installing another flow with the same selector as the drop flow above) CPU2 will find the entry added by CPU1 but will fail to notice that it corresponds to a drop flow because the type field hasn't been set yet by CPU1. This will lead to checking if there is any softwire info available for the existing flow. The drop flow is installed for outside traffic so no softwire information is available causing the assertion to fail. PR940014: This issue has been resolved.
  • Snmp traps are not generating when port utilization threshold is crossed. PR941931: This issue has been resolved.
  • During a rare scenario, switchover on another sp interface can crash a service PIC when running a traffic in hairpinning scenario. PR945114: This issue has been resolved.

Software Installation and Upgrade

  • In this case, since the high level package (i.e. jinstall) is signed, the underlying component packages are not required to be signed explicitly. However the infra was written such a way to display warning message if the component package is not signed (i.e. jpfe). PR932974: This issue has been resolved.

Subscriber Access Management

  • Radius attribute ignore logical-system-routing-instance not ignoring VSA26-1. PR953802: This issue has been resolved.
  • Configuration change of the IPv4 address range in address-assignment pool does not always take effect. PR954793: This issue has been resolved.

User Interface and Configuration

  • If a configuration file which contains groups related configuration is loaded by command "load replace", a "commit confirmed" operation might fail. When this issue occurs, the new configuration is committed even if you do not confirm it within the specified time limit. PR925512: This issue has been resolved.

VPNs

  • Configuration version (child rpd) of rpd generates a core file when doing a commit or commit check. PR930080: This issue has been resolved.
  • The issue happens when the virtual routing forwarding (vrf) is configured "no-vrf-propagate-ttl" and the vrf import policy changes the local preference of the vrf route. With "no-vrf-propagate-ttl", BGP will resolve the primary l3vpn route and the vrf secondary route separately. The root cause is overwriting the route parameters of the second vrf route with the route parameters of the primary route. So when changes the local preference of the vrf route might not work. PR935574: This issue has been resolved.
  • 'show route table VRF.mvpn.0 extensive|detail' for mvpn VRF routing tables will not show BGP TSI info (which previously contained the MVPN PMSI attribute) for outgoing MVPN route advertisements. Since PR 908199, TSI info for these routes is shown on the copy of the route advertised from the main bgp.mvpn.0 table. 'show route table VRF.mvpn.0 extensive|detail' now shows the MVPN PMSI attribute in the main body of the route output. PR939684: This issue has been resolved.

Resolved Issues in Release 12.3R5

Class of Service (CoS)

  • The names "best-effort", "assured-forwarding", "expedited-forwarding", "network-control" are reserved and cannot be currently used in Forwarding Class alias configuration, with several classes mapped to the same queue: user@router# show class-of-service user@router# set class-of-service forwarding-classes class best-effort queue-num 0 user@router# set class-of-service classifiers inet-precedence test forwarding-class best-effort loss-priority low code-points 000 user@router# commit check configuration check succeeds user@router# set class-of-service forwarding-classes class myBE queue-num 0 user@router# commit check [edit class-of-service classifiers inet-precedence test forwarding-class] 'best-effort' forwarding class undefined: best-effort error: configuration check-out failed PR827496: This issue has been resolved.

Forwarding and Sampling

  • After committing some configuration changes (e.g. deactivate an interface), while the Packet Forwarding Engine daemon (PFEd) tries to get statistics of some nodes, it may encounter a NULL nkode, causing PFEd to crash and generate a core file. PR897857: This issue has been resolved.
  • When pfed get restarts during a period when pfed is communicating with mib2d, because the communication sockets have been terminated and failed to be re-opened after pfed back up again, mib2d might crash and generate a core file. The core files could be seen by executing CLI command show system core-dumps. PR919773: This issue has been resolved.

General Routing

  • Only 94 GRE(plain) sessions are in Established state after chassisd restart. PR801931: This issue has been resolved.
  • BFD packets sent from FPC (distributed mode) over normal physical interfaces are set with ttl 0 so that it gets decremented by 1 and becomes 255 once it is sent out on the wire. This behavior is not the case when the BFD packets are sent over IPsec routed tunnels where the packets are sent from the corresponding service PIC. In this case, the ttl should be set to 255 as no such decrement action takes place when it is sent from a service PIC. But in the current scenario, the ttl is set to 0 as a result of which the service pic drops the outgoing packet. This was an untested scenario till date. PR808545: This issue has been resolved.
  • When the 10x10GE PIC (PD-5-10XGE-SFPP) is configured to run in linerate-mode under [set chassis fpc fpc-number pic pic-number] hierarchy, and an input-scheduler-map with Class of Service (CoS) queues including any of queue 4 to queue 7 is applied to an interface on the 10x10GE PIC, the ingress queues may not map correctly to the internal hardware ingress queues, hence as a result, packet drops may be seen in a higher priority queue than that which is expected. PR818605: This issue has been resolved.
  • IPv6 address syntax on rpd log is violation of RFC 5952. For example, 2002:db8:0:0:1:0:0:1 must be logged as 2002:db8::1:0:0:1 in the logs, but it's logged as 2002:db8:0:0:1::1. 2001:0:0:0:db8:0:0:1 must be logged as 2002::db8:0:0:1 in the logs, but it's logged as 2001:0:0:0:db8::1. PR840012: This issue has been resolved.
  • If a router receives the BGP keepalive at time t, the next keepalive is expected at time t+30 secs (+/- 20% jitter). However, right around the time when the next keepalive is expected to be received, the BGP keepalive packet is dropped due to some network issue (e.g. uplink towards peer flaps). During this scenario, retransmission of BGP keepalive message on BGP peer would take long time and the BGP session will be terminated due to hold timer expiry. PR865880: This issue has been resolved.
  • SNMP trap is not generated upon Fabric chip failure/offline/online state on MX Series routers. PR877653: This issue has been resolved.
  • When syslog feature is configured in firewall filter, one of the Junos OS message creating function has a bug, where the whole string is copied directly with no check for overflow. This could easily overflow and results in no null-termination which causes memory corruption and linecard crash. The core files could be seen by executing CLI command "show system core-dumps".PR888116: This issue has been resolved.
  • Backup Routing Engine failed to commit with error "pdb_update_ddl_id: cannot get new id for " dynamic-profiles dynamic-profiles profile-name"",commit full is a workaround. PR888454: This issue has been resolved.
  • TLB: Observed a traffic-drd daemon hang once after logging into service PIC and restarting the net-monitord process. This is not an operational procedure, not always reproducible, and the work-around is to restart the traffic-drd daemon using the restart traffic-dird command. PR889982: This issue has been resolved.
  • When a bgp route is resolved using a next-hop that is also learned in bgp (i.e. there are multiple levels of next-hop resolution) and bgp multipath is also used, during a route churn next-hop for such a bgp route could be incorrectly programmed. PR893543: This issue has been resolved.
  • An MX-VC NSR master switch might put kernel control socket in stale state, and in the subsequent NSR master switch, the kernel will refuse the connection from FPC. As a result, the FPC would be rebooted during the switchover process. PR896015: This issue has been resolved.
  • Some ATM interfaces may stay down after flapping the Circuit Emulation MIC. PR900926: This issue has been resolved.
  • 100G Ethernet interface (Finisar FTLC1181RDNS-J3) on T4000 type-5 FPC might flap once after bringup . The solution is to change the register bandwidth. PR901348: This issue has been resolved.
  • RPD on backup Routing Engine might hit out of memory condition and crash if BGP protocol experiences many flaps. PR904721: This issue has been resolved.
  • What was seen was that on certain occasions that the 10GE PHY does not recover from the transition from down->up. This can happen in cases where a link flaps or an SFP+ in inserted. What was also seen that the same set of events did not yield the same set of outcomes i.e. JTAC was not even to replicate this issue in the labs even though this was seen in the field occasionally. Upon further analysis it was found that the PHY was stuck in Freeze state as an explicit command to take it out of that state was able to recover the link. There was no other issue found except this while all the other parameters of optical power the pluggable optics were all taken into account. PR905589: This issue has been resolved.
  • "set system ddos-protection protocol sample aggregate bandwidth" command is not taking effect. This can cause packet loss in ukernel for Routing Engine based sampling if sampling rate exceeds 1000pps. PR905807: This issue has been resolved.
  • bootp configuration on TXP platform referencing routing-instance fails to commit. PR906713: This issue has been resolved.
  • MX-VC: VC port conversion not working for second set of added VC ports for VCB. PR906922: This issue has been resolved.
  • VCMm-power down creates stale vlan demux0 entries at the Packet Forwarding Engine level. PR908027: This issue has been resolved.
  • When adding the "no-tunnel-services" knob under vpls protocols of routing-instances, during the processing gap of the new knob, if routing protocol process (rpd) restarts (i.e rpd crashes), logical interfaces with VPLS family do not show up, and there are no logical interfaces available for the corresponding VPLS routing instances. Hence VPLS connections might be down (stuck in LD state) and can not be recovered automatically. PR912258: This issue has been resolved.
  • After changing interface description, it doesn't get updated in "show lldp neighbors" output. PR913792: This issue has been resolved.
  • 10GbE interface on MIC3-3D-10XGE-SFPP stays up even if far end is disabled and goes down. Since the interface on MIC3-3D-10XGE-SFPP cannot react to remote failure, CCC circuit cannot change the state correctly, if port of MIC3-3D-10XGE-SFPP is configured as CCC end point. PR914126: This issue has been resolved.
  • The following note applies to 16x10GE MPC: With respect to this feature, when ISSU is performed from feature non-supporting version (ex. 12.2, 13.1) to feature supporting version (12.3R5, 13.2R3, or 13.3), then16x10GE FPC needs to reboot in order to use this feature. PR914772: This issue has been resolved.
  • A log message "%DAEMON-3: Cannot perform nh operation ADDANDGET nhop 0.0.0.0 type unicast nhindex 0x0 ifindex 0xd3e <interface name> fwd nhidx 0x0 type unicast errno 45 suppressed <number of suppressed> logs" is generated if access-internal route is created during the dynamic interface configuration process. The log message can be permanent or not. Besides this message there were no side effects. PR917459: This issue has been resolved.
  • FPC crash can be triggered by a SBE event after accessing a protected memory region, as indicated in the following log: "System Exception: Illegal data access to protected memory!" The DDR memory monitors SBEs and reports the errors as they are encountered. After the syslog indicates a corrupted address, the scrubbing logic tries to scrub that location by reading and flushing out 32-byte cache line containing that location in an attempt to update that memory location with correct data. If that memory location is read-only, it causes illegal access to protected memory exception as reported and resets the FPC. The above-mentioned scrubbing logic is not needed because even if SBE is detected, the data is already corrected by the DDR and CPU has a good copy of the data to continue its execution path. PR/919681 can be triggered on both PTX and T4000 platforms and can be seen in Junos OS releases 12.1 and 12.3. Fix is available in 12.3R5, 12.3R3-S6, 13.3R1, 13.2R2. Crash signature in the FPC shell shows the following: SNGFPC4(router-re0 vty)# sh nvram System NVRAM : 32751 available bytes, 2477 used, 30274 free Contents: [LOG] Set the IP IRI for table #1 to 0x80000014 [LOG] IPV4 Init: Set the IP IRI to 0x80000014 [LOG] GN2405: JSPEC V 1.0 Module Init. <..> Reset reason (0x84): Software initiated reset, LEVEL2 WATCHDOG [Sep 6 17:16:07.231 LOG: Warning] <164>DDR: detected 3 SDRAM single-bit errors [Sep 6 17:16:07.231 LOG: Warning] <164>DDR: last error at addr 0x108d2378, bad data/mask0x00240401fffffff7/0x0000000000000008 bad ecc/mask=0xbe/0x00 System Exception: Illegal data access to protected memory! <<< Event occurred at: Sep 6 17:16:07.231087 Juniper Embedded Microkernel Version 12.1X48-D30.1 Built by builder on 2012-08-23 04:28:12 UTC Copyright (C) 1998-2012, Juniper Networks, Inc. All rights reserved. Context: Thread (Periodic) Registers: R00: 0x10354900 R04: 0xffffffe0 R08: 0x00000001 R12: 0x24002084 R16: 0x7040c48c R20: 0xff700000 R24: 0x10de0000 R28: 0x00000003 MSR: 0x00029200 CR: 0x44002048 ESR: 0x00000000 R01: 0x13fcb8c8 R02: 0x13fc99f8 R03: R05: 0x00000001 R06: 0x00000000 R07: R09: 0x00000000 R10: 0x00000000 R11: R13: 0x53564110 R14: 0x210191df R15: R17: 0x05240140 R18: 0x3e023840 R19: R21: 0x447ac8dc R22: 0x10deb408 R23: R25: 0x00000000 R26: 0x10df0000 R27: R29: 0x13fcb8d8 R30: 0x00000000 R31: CTR: 0x00000000 Link: 0x10354908 SP: XER: 0x20000000 DEAR: 0x00000000 PC: K_MSR: 0x00001000 0x108d2360 0x00000001 0x00000001 0x2e105223 0xd855014f 0x10de0000 0x13fcb930 0x108d2378 0x13fcb8c8 0x100425ec Stack Traceback: Frame 01: sp = 0x13fcb8c8, Frame 02: sp = 0x13fcb928, Frame 03: sp = 0x13fcb958, Frame 04: sp = 0x13fcb988, Frame 05: sp = 0x13fcb998, Frame 06: sp = 0x13fcb9c8, Frame 07: sp = 0x13fcb9f8, [LOG] syslog called with interrupts off (caller pc:0x10549988) [LOG] Dumping core-SNGFPC4 to 1 [LOG] syslog called with interrupts off (caller pc:0x105489c0) [LOG] Coredump finished! [LOG] Set the IP IRI for table #1 to 0x80000014 [LOG] IPV4 Init: Set the IP IRI to 0x80000014 [LOG] GN2405: JSPEC V 1.0 Module Init. pc pc pc pc pc pc pc = = = = = = = 0x10354900 0x100461a0 0x1003c5fc 0x1003c894 0x1003c5fc 0x1003c7b4 0x10030b8c Reset reason: Software reset <..> PR919681: This issue has been resolved.
  • Following chassisd messages might be observed after executing the "show chassis fabric summary" command, FM: Plane Sate: 1 1 1 1 2 2 0 0; staggered_pmask: 15 2a 00 00 00 00 00 00 FM: Mux active/trained: 0/0 0/0 0/0 0/0 0/0 0/0 0/0 0/0; Mode:1 act_mask:3f These are non-impacting debug messages. PR927453: This issue has been resolved.
  • MS-PIC might crash in IPsec environment after deleting "tcp-mss" knob under IPsec "service-sets" hierarchy. PR930741: This issue has been resolved.
  • Polling the OID mib-2.17.7.1.4.3.1.5...: dot1qPortVlan or mib-2.17.7.1.4.5.1...: dot1qPvid might cause a memory leak on the l2ald process, and the process might create core files. PR935981: This issue has been resolved.

High Availability (HA) and Resiliency

  • On TX or TXP Line Card Chassis (LCC) with graceful Routing Engine switchover enabled, if a mastership switch is being requested on a LCC whose backup Routing Engine's em0 interface is physically failed (due to hardware failure or driver stops working), this will cause all FPCs on the LCC disconnect from the old master Routing Engine, but cannot reconnect to the new master one either. PR799628: This issue has been resolved.
  • During every failover of redundancy-group 0, the /etc/ssh and /var/db/certs directories are copied from primary node to secondary node. However, the directories are not copied correctly and nested directories such as /etc/ssh/ssh, /etc/ssh/ssh/ssh are created. PR878436: This issue has been resolved.
  • In certain systems configured with graceful Routing Engine switchover (GRES), there is the possibility for the master and the backup Routing Engine to reach an inconsistent view of installed state. This fault may be exposed if the master Routing Engine experiences a mastership watchdog timeout at a time when it is not in sync with the backup Routing Engine for a particular piece of state. In practice, this possibility exists only for a short time period after a Routing Engine mastership change. Under such conditions, a replication failure may cause the backup Routing Engine to panic. If the failure is seen, the backup Routing Engine will recover on restart. In 11.4 and 12.1 releases without this fix, the fault may be experienced on any GRES-enabled, non-multichassis configuration on a T Series router. For 12.2 and later releases without this fix, the fault may be experienced on any GRES-enabled, non-multichassis configuration on a T Series or MX Series router. PR910259: This issue has been resolved.

Infrastructure

  • If a router receives the BGP keepalive at time t, the next keepalive is expected at time t+30 secs (+/- 20% jitter). However, right around the time when the next keepalive is expected to be received, the BGP keepalive packet is dropped due to some network issue (e.g. uplink towards peer flaps). During this scenario, retransmission of BGP keepalive message on BGP peer would take long time and the BGP session will be terminated due to hold timer expiry. PR865880: This issue has been resolved.
  • When multicast is running on a multi-chassis environment, during flapping of 224/4 or ff00/8 pointing to mResolve(NH), the LCC master might get replication error which causes all FPCs going offline. This flapping of resolve route for multicast can occur because of any of the following reasons: enabling or disabling multicast, deletion of resolve route, or routing restart. PR897428: This issue has been resolved.

Interfaces and Chassis

  • DCD reports error when configuring hierarchical-scheduler on MX80 with QX chipset. This is cosmetic error and it should not have functional impact. PR807345: This issue has been resolved.
  • An MX Series router may cosmetically log "Bottom Fan Tray Unable to Synch". PR833047: This issue has been resolved.
  • Tx and Rx Spanning-tree BPDU stopped intermittently during ISSU. PR849201: This issue has been resolved.
  • The particularity of logic that DCD daemon crashes when "aggregated-ether-options load-balance" is committed. PR854207: This issue has been resolved.
  • M7i Routing Engine crashed with last reboot reason panic:page fault and kernel core, after commit. PR868212: This issue has been resolved.
  • "Link down" alarms should never exist on the VC Protocol Backup Routing Engine. They should only be on Protocol Master, if any. The bug is that the "Link down" alarms are not cleared from the Protocol Backup after/during a graceful Routing Engine switchover (GRES) event. Restarting alarmd removes these alarms from the Protocol Backup. PR886080: This issue has been resolved.
  • If an AE interface is brought down by protocol CFM/LFM/STP, the interface will go down permanently and can not recover automatically. PR888728: This issue has been resolved.
  • While a duplicate interface address (IFA) is configured for two interfaces, software will accept that and generate an error message like this: %CONFLICT-4-DCD_PARSE_WARN_INCOMPATIBLE_CFG: [edit interfaces ge-0/0/0 unit 0 family inet address x.x.x.x/xx] : Incompatible configuration detected : identical local address is found on different interfaces But at kernel side cannot accept duplicate IFA, and needs to delete the next-hop created for this operation. Due to code problem, the cleanup doesn't remove the duplicated IFA under heavy kernel workload. And it will crash while trying to update this duplicated IFA to Packet Forwarding Engine side. PR891672: This issue has been resolved.
  • Following is the document change proposed :- traceroute-ethernet-command :- Source MAC address : MAC address of 802.1ag node responding to the LTM Next-hop MAC address: MAC address of egress interface of the node where LTM would be forwarded show-oam-ethernet-connectivity-fault-management-linktrace-path-database-com mand :- Source MAC address : MAC address of 802.1ag node responding to the LTM Next-hop MAC address: MAC address of egress interface of the node where LTM would be forwarded The display of next-hop MAC address is incorrect for linktrace path database command. PR895710: This issue has been resolved.
  • On MX Series platforms which are running Junos OS Release 12.3R3/R4 and 13.2R1 and Operation Administration and Maintenance (OAM) is activated, periodic packet management daemon (ppmd) might crash after changing the ppm distribution state from distributed to centralized and then restarting connectivity fault management deamon (cfmd) or any action to create ppm interface. It is suggested to deactivate and activate OAM configuration during this configuration change to avoid the problem. PR905812: This issue has been resolved.
  • The MX Series router does not always process the first LCP request for a static PPPoE subscriber. PR908457: This issue has been resolved.
  • Issue is because of vrrpd not configuring vrrp group id, and state when it’s in transition state. In normal scenario when vrrp moves to master it signals dcd to add the VIP. When VIP gets added vrrpd gets a notification and updates state and groupd id corresponding to that VIP. While updating state vrrpd checks the current state. If state is master it updates state as master and if its backup it updates it as backup. But if vrrp state is in transition it does not do anything. It may not be seen every time and is a timing issue. One can confirm the incorrect mac address by capturing monitor traffic on the affected irb interface specifically one who is master VRPP instance. PR908795: This issue has been resolved.
  • When an interface is configured with VRRP protocol, IP address associated with this interface might disappear after deactivating then activating the interface. When this issue happens, KRT may be getting stuck and never clean up. If the interface belongs to a routing-instance, then deactivate/activate the routing-instance can also trigger the same issue. Issue command 'show krt queue' to verify: root@ABC-re0> show krt queue Routing table add queue: 1 queued ADD table index 37, gf 1 (1377) error 'File exists' PR912295: This issue has been resolved.
  • In multicast over AE scenario, if there is a different order of child logical interfaces (logical interface) under parent AE at the master Routing Engine and the backup Routing Engine, then after Routing Engine switchover, multicast traffic might get lost. PR915440: This issue has been resolved.
  • For IQ2 PIC, when the setting shaping rate is too high, when configured it with "set chassis fpc 0 pic 1 traffic-manager logical-interface-base-shaping-rate 16" and this will reset the shaping rate to 1Gbps. The corresponding messages are logged in debug level. In the fix, it is corrected into info level. PR920690: This issue has been resolved.
  • In MX-VC environment, if LT interface's encapsulation type is ethernet-ccc, after rebooting FPC with LT interfaces or rebooting system, the LT interface might not come up again. PR922673: This issue has been resolved.
  • Unified ISSU fails on upgrade to 11.4R5.7. with the following message Logged messages: MIC 4/0 will be offlined (In-Service-Upgrade not supported) MIC 4/1 will be offlined (In-Service-Upgrade not supported) Do you want to continue with these actions being taken ? [yes,no] (no) yes error: /usr/sbin/indb failed, status 0x200 error: ISSU Aborted! Chassis ISSU Aborted ISSU: IDLE Issue happens when a MIC-3D-4OC3OC12-1OC48 card is offline via CLI and removed from the chassis prior to the ISSU. PR923569: This issue has been resolved.
  • Traffic which uses MPLS next-hops enters bridge-domain via IRB interface and if forwarding next-hop moves from non-aggregate interface to aggregate interface (MAC move), the MPLS next-hops are not correctly programmed in the Packet Forwarding Engine. The child next-hop of the aggregate interfaces are missing. Once IRB MPLS next-hop moves from aggregate interface to non-aggregate interfaces are not affected. IPV4 traffic is not affected. PR924015: This issue has been resolved.
  • The MX960 works as LNS can't accept l2tp session packet from Huawei GGSN. PR926919: This issue has been resolved.

Layer 2 Features

  • If STP is configured on AE interface, the l2cpd might be under high utilization and VRRP repeatedly flaps after the VRRP active router reboots. The root cause here is when STP is configured on AE interface, the corresponding Bridge Protocol Data Unit (BPDU) will messages go to Routing Engine instead of processed in Packet Forwarding Engine. PR882281: This issue has been resolved.
  • In VPLS environment, while deactivating/activating VPLS routing-instances, in rare conditions, routing protocol process (rpd) tries to free an already used route, then rpd process crashes with core files. PR908856: This issue has been resolved.
  • "show snmp mib walk ascii jnxVpnIfStatus" doesn't work for BGP vpls when there is incompleted BGP VPLS instance configuration or LDP VPLS instance. PR918174: This issue has been resolved.
  • In BGP autodiscovery for LDP VPLS scenario, as FEC129 VPLS does not support nonstop active routing (NSR), VPLS fails to come up after Routing Engine switchover and traffic will never resume. PR919483: This issue has been resolved.

MPLS

  • This message was used to record error condition from nexthop installer. Over time, it becomes common function and same message will be printed in many valid conditions, leading to confusion on these message-logs. PR895854: This issue has been resolved.
  • IPv6 traceroute may not show some hops for scenarios where 1) Two LSPs are involved. 2) INET6 Shortcuts are enabled. In such scenarios, hops that are egress for one LSP and ingress for the next LSP in the traceroute do not show up. This was a software issue with icmp error handling for packets with ipv6 payload having a ttl of 1. PR899283: This issue has been resolved.
  • RPD might crash under specific conditions and after executing "ping mpls l2vpn interface <interface>" command. PR899949: This issue has been resolved.
  • If the maximum-ecmp next-hops under [edit chassis] hierarchy is configured as 32 or 64 (more than the default value of 16), the routing protocol process (rpd) might crash on the new master Routing Engine after performing graceful Routing Engine switchover. The root cause here is while merging nexthops, the Junos OS is iterating over only 16 gateways instead of configured maximum-ecmp number and finally results in an assert. The core files could be seen by executing CLI command show system core-dumps. PR906653: This issue has been resolved.
  • The output of "show ldp overview" command regarding graceful restart is based on per protocol LDP graceful restart settings. Where graceful restart is enabled by default. So when graceful restart is disabled this command shows it's enabled for LDP. However graceful restart should be enabled globally for LDP graceful restart to operate. PR933171: This issue has been resolved.

Network Management and Monitoring

  • Mib2d may get ATM VPI updates before the ATM IFDs are learned. In such cases instead of discarding the updates mib2d has started caching them untill the IFD is learned. PR857363: This issue has been resolved.
  • In an IS-IS scenario, with traceoptions enabled under protocol ISIS and syslog level set to debug under routing-options options for a router, if the router has two IS-IS neighbors which have the same router-id configured, after configuring the same ISO system-id on these two IS-IS neighbors, routing protocol process (rpd) on the router will crash with core files dumped. PR912812: This issue has been resolved.

Platform and Infrastructure

  • Commit may fail, when a config object is deleted and re-added as transient change from a commit script. PR814796: This issue has been resolved.
  • When the egress Next Hop is IRB interface, the Routing Engine to Packet Forwarding Engine output function for the IRB overwrites some info on the packet mbuf to indicate the underlying egress Layer2 interface, with the expectation that the UDP packet needs to be forwarded. However, in the case of this being a traceroute UDP packet with a limited TTL, the TTL limit check decides to drop the packet and issue an icmp response to the sender. Because of the mentioned IRB overwrite, the icmp code could no longer determine how to send a packet to the originator. The fix is to re-arrange the sequence of events so that the overwriting of the mbuf info is done later in the sequence. After all the possible layer3 (IP) TTL checks are done. PR816202: This issue has been resolved.
  • Since the AC Power System on the MX2020 is a N+N feed redundant and N+1 PSM redundant, there are two separate input stages per PSM, each connected to one of the two different/redundant feeds. However, only one stage is active at a time. This means, the other input stage (unused input stage) may be bad and system will not know about it till it tries to switch to it in case of a feed failure. This is a pretty bad corner case and needs to be addressed. The way to work around this problem is by testing both stages when the power supply is first powered on. This test is done by the system software and an alarm is raised if any feed failure is detected. PR832434: This issue has been resolved.
  • There are two symptoms covered this issue: If there is a mix of high and low priority fabric traffic as can be seen by checking "show class-of-service fabric statistics", the following error messages can be seen when there are bursts of high priority fabric traffic, while low priority fabric traffic is present :- May 6 14:58:41 routername-re0 fpc1 MQCHIP(0) FI Reorder cell timeout May 6 14:58:41 routername-re0 fpc1 MQCHIP(0) FI Cell underflow at the state stage A second symptom with this mix of low and high priority fabric traffic present; if an FPC that is the recipient of this high and low priority fabric traffic restarts, it is possible for the ingress FPC forwarding ASIC to lockup. In this case the following log message might be simultaneously logged :- Jun 5 13:46:50 router fpc4 MQCHIP(0) CPQ Queue underrun error, Qsys1 Queue 42 Jun 5 13:46:50 router fpc4 MQCHIP(0) CPQ Freecnt nearing empty error, Qsys mask 0x2 PR877123: This issue has been resolved.
  • High rate of traffic to the Routing Engine may cause control traffic stoppage to the Routing Engine. The indication is the following type of messages: "WEDGE DETECTED IN PFE ... TOE host packet transfer: reason code 0x1 PR896592: This issue has been resolved.
  • On MX Series router with MPC, firewall filter counter doesn't count packets when firewall is configured on discard interface. PR900203: This issue has been resolved.
  • If there are private sessions in place, it should not abort the effective/revoke of conditional groups. In affected releases, it is not working. PR901976: This issue has been resolved.
  • In MX-VC setup using virtual-switch instance type, there can be scenarios where the outer vlan-tag of PPPoE/PADI packets on egress can be stripped off when ingress interface is a LAG with 2 member links spread across the 2 Chassis members. PR905667: This issue has been resolved.
  • Junos OS 12.3R3, 12.3R3S1 and 12.3R3S2, interfaces with interface-mode trunk connected on top PFE[0] and with IRB interfaces, might corrupt forwarding-state on lowest Packet Forwarding Engine of the FPC. This is applicable to system operating with network-services enhanced-ip mode and systems operating in virtual-chassis mode. PR907291: This issue has been resolved.
  • Command "show ddos-protection protocols" doesn't report correct Arrival and Max arrival pps rates. One bit of rate value at Packet Forwarding Engine is incorrectly set which results in a wrong ddos rate value. PR908803: This issue has been resolved.
  • In MX virtual-chassis (MX-VC) scenario, when the VC-M (master member of VC) reboots and then comes up, the MPC with virtual-chassis port (vcp) configured might crash due to the memory overflowed. PR910316: This issue has been resolved.
  • The DDOS classification for Dynamic Host Configuration Protocol (DHCP) "leasequery" message is not working. This message is treated as "unclassified". PR910976: This issue has been resolved.
  • IPv6 UDP checksum is implemented, but computed UDP checksum for IPv6 IPFIX export packets gets invalid occasionally. When this issue is seen the following capture would be seen in the collector. 14:05:06.810436 In Juniper PCAP Flags [Ext, no-L2, In], PCAP Extension(s) total length 16 Device Media Type Extension TLV #3, length 1, value: Ethernet (1) Logical Interface Encapsulation Extension TLV #6, length 1, value: Ethernet (14) Device Interface Index Extension TLV #1, length 2, value: 139 Logical Interface Index Extension TLV #4, length 4, value: 71 -----original packet----- PFE proto 6 (ipv6): (hlim 64, next-header: UDP (17), length: 144) xxxx:xxx:ffff:ffff::yy.33068 > xxxx:xxx:0:yyy::yyy.2055: [bad udp cksum 72ff!] UDP, length 136 (IPv6 address masked). PR911972: This issue has been resolved.
  • When enhance-route-memory is enabled along with SCU, configuration might cause Jtree Memory corruption on MX Series DPCs. PR914753: This issue has been resolved.
  • Description of T4000 midplane changes after "show snmp mib walk jnxContentsDescr" PR915393: This issue has been resolved.
  • On MX2020, SNMP traps are generated only for SFB slot 6 and 7 upon graceful Routing Engine switchover (GRES) enabled Routing Engine switchover. PR915423: This issue has been resolved.
  • Changing the domain-name doesn't reflect in DNS query unless a Commit full is done. This bug in management daemon (mgd) has been resolved by ensuring mgd propagates the new domain-name to file /var/etc/resolv.conf, so that this can be used for future DNS queries. PR918552: This issue has been resolved.
  • Issue observed in inline Jflow during route-record collection. For route-record function in inline-Jflow it is expected that for any aggregated type next hops a child next-hop must be present. This child next-hop info is updated as gateway info for aggregated next-hop. In scenario, where we have valid aggregated next hop id but no child next-hop, system is crashing in inline-jflow during route-record collection. PR919415: This issue has been resolved.
  • Without this PR fix, commit script applied configuration may emit the XNM RPC errors when there is XML tag mismatch detected: error: [filename: xnm:rpc results] [line: 771] [column: 7] [input: routing-engine] Opening and ending tag mismatch: routing-engine line 7 and rpc-reply error: [filename: xnm:rpc results] [line: 773] [column: 6] [input: rpc-reply] Opening and ending tag mismatch: rpc-reply line 6 and junoscript error: [filename: xnm:rpc results] [line: 774] [column: 2] [input: junoscript] Premature end of data in tag junoscript line 2 PR922915: This issue has been resolved.
  • DDOS_PROTOCOL_VIOLATION alarm shows incorrect timestamps <time-first-detected> and <time-last-detected> on messages. Both fields indicate the same timestamps. Timestamps <time-first-detected> and <time-last-detected> are overwritten. PR927330: This issue has been resolved.
  • If port-mirror is used on member-0 interfaces and mirrored on member-0 interfaces, mirrored traffic is incorrectly sent to member-1 and dropped. Fabric drop counters are counted. PR928315: This issue has been resolved.
  • Under certain timing conditions the MPC/TFEB can receive the firewall filter configuration before it is fully booted/UP/ONLINE. Because the firewall filters can depend on certain default values which are not yet programmed the MPC/TFEB will crash/core-dump and reboot/restart/reload. PR928713: This issue has been resolved.
  • When replacing ichip FPC with MX Series FPC, "traceroute" packets going through an MX Series FPC may experience higher drop probability than when using an ichip FPC. PR935682: This issue has been resolved.

Routing Policy and Firewall Filters

  • Install-nexthop lsp-regex does not work as expected when multiple recursive routes share same protocol next hop having different export policy with regular expression option. Route is not updated with correct export forwarding nexthop as same nexthop select handle is calculated for any set of configured export policy with "install-nexthop lspregx" option. PR863341: This issue has been resolved.
  • Junos OS releases with a fix for PR/706064 have a regression where the vrf-import policy sanitation logic is faulty. A "# commit check" will fail when the first term references a 'target' community and the second term references an 'origin' community. This should pass the check. PR911350: This issue has been resolved.

Routing Protocols

  • With this fix, "jnxBgpM2PrefixesInPrefixesRejected" counter will return the number of prefixes from a BGP peer, that are not eligible to become active. This change makes the variable conform to definition in the specification http://tools.ietf.org/html/draft-ietf-idr-bgp4-mibv2-03. There is a new variable "jnxBgpM2PrefixInPrefixesActive" introduced, to return the number of active prefixes from a BGP peer. So the new sequence of variables for the table is as follows: root@root> show snmp mib walk jnxBgpM2PrefixCountersTable jnxBgpM2PrefixCountersAfi.0.1.1 = 1 jnxBgpM2PrefixCountersSafi.0.1.1 = 1 jnxBgpM2PrefixInPrefixes.0.1.1 = 0 jnxBgpM2PrefixInPrefixesAccepted.0.1.1 = 0 jnxBgpM2PrefixInPrefixesRejected.0.1.1 = 0 jnxBgpM2PrefixOutPrefixes.0.1.1 = 3 jnxBgpM2PrefixInPrefixesActive.0.1.1 = 0 PR778189: This issue has been resolved.
  • Junos OS label block allocation can only return block size as power of 2 (e.g. 2, 4, 8, 16,...). In inter-as option-b L2VPN scenario, routing protocol process (rpd) core is seen when the ASBR receives a non-power-of-2 label block size from other vendor's device. The root cause here is when rpd requests the non-power-of-2 label block size, an assert occurred. The core files could be seen by executing CLI command show system core-dumps. PR848848: This issue has been resolved.
  • When configuring CAC for a physical interface, the software might enable CAC for unit 0 on that interface, but might not be able to delete it when the configuration is removed. PR850578: This issue has been resolved.
  • There are improper </route-family> tags added to all multicast route summary commands when we perform command such as show multicast route summary | display xml. PR859104: This issue has been resolved.
  • On T640/T1600 routers with Enhanced Scaled (ES) FPCs equipped and all MX Series routers with MPC, the Bidirectional Forwarding Detection (BFD) sessions over aggregated Ethernet (AE) interfaces might be down after performing unified In-Service Software Upgrade (ISSU). Note, the problem is only seen on FPC (Packet Forwarding Engine) based BFD (contrasts with Routing Engine based BFD), and the problem is mostly seen on T640/T1600 routers even thought the problem affects MX Series routers in principle. PR859324: This issue has been resolved.
  • The remote discriminator is not reinitialized after bfd session state moves to down (with diagnostic code: control detection time expired) as per RFC 5880 requirement. PR889970: This issue has been resolved.
  • In a scenario with graceful restart (GR) enabled for BGP between Cisco platform and Juniper Networks platform, Junos OS is helper (default) and Cisco being restarting router, when Cisco restarts BGP process, Juniper deletes all BGP routes due to doesn't receive End Of RIB (EOR) markers for all configured NLRIs from Cisco. PR890737: This issue has been resolved.
  • BGP "accepted-prefix-limit" feature might not work as intended when it is configured together with "damping". Root cause of this issue is that when BGP module counts the maximum routes accepted from BGP neighbor, it doesn't count the accepted BGP routes which are in damping status. So when these damping routes are reused, the total number of received BGP routes exceeds the configured value for "accepted-prefix-limit" . PR897124: This issue has been resolved.
  • In PIM dense mode, if the Assert loser router receives a join/prune (S,G) message with upstream neighbor is the loser router, it should send an Assert(S,G) on the receiving interface to initiate a new Assert negotiation to correct the downstream router's RPF neighbor, but our device will not. PR898158: This issue has been resolved.
  • Sometimes "Advertised prefixes" counter for some RIBs may be incorrect for some BGP neighbors. This is a cosmetic issue. Use "show route advertising-protocol bgp <nbr> table <tblname> | match Nexthop | count" to know the right advertised prefixes count. PR899180: This issue has been resolved.
  • In multicast scenario with PIM enabled, when you configure both static RP mapping with override knob and dynamic RP mapping (such as auto-RP) in a single routing instance, allow the static mapping to take precedence for a given group range, and allow dynamic RP mapping for all other groups, but a software defect cause that RP is selected based on dynamic RP mapping address, instead of accounting for this static override knob. PR912920: This issue has been resolved.
  • DR sends a delayed ACK to the LSA on the interface on which the LSA is flooded. This leads to BDR sending only directed ACK to DR, DR-Other is therefore not receiving this ACK and is hence retransmitting the LSA to BDR. PR914803: This issue has been resolved.
  • When the interface goes down, the direct route for that peer address is removed from the routing table before BGP processes interface down event and brings down the session. When BGP calculates multipath routes, since the knob "accept-remote-nexthop knob" is configured, BGP needs to determine whether we can reach the nexthop address (ebgp peer address) directly. BGP did not find direct route for this nexthop address and so asks for route nexthop resolution. In this case, the first BGP path from the peer with up interface has direct router nexthop, the second path is set to have indirect nexthop due to the down interface, BGP passed a wrong mixed multipath nexthop, which caused RPD crash. PR917428: This issue has been resolved.
  • When NSR is configured and path-selection is changed, there might be a non-functional impacting softcore generated during the commit process. PR928753: This issue has been resolved.
  • "show route advertising-protocol bgp <nbr> table foo.mvpn.0" stops working after PR-908199 fix. PR929626: This issue has been resolved.
  • If you have fix for PR-929626, avoid the following show command in a VPN setup "show route advertising-protocol bgp <nbr_addr> table foo.inet.0" Where <nbr_addr> is peer within routing-instance "foo" PR936434: This issue has been resolved.

Services Applications

  • NAPT: Packet Forwarding Engine side report port range starts from 512 because napt mib counter wrong. This fix make the port range in Packet Forwarding Engine start from 1024. PR828450: This issue has been resolved.
  • Any port or IP address value set in SIP VIA header for 'rport' and 'received' attributes will not be checked or translated by the SIP ALG. There is usually no impact from this to a voice call. The contact address inserted by the client in future requests will be the external one but this will not disrupt the SIP ALG. Some rare clients however may have some unexpected reaction that causes problem such as trying to register two IP addresses, the internal one AND the public one, in the same register message which is unsupported by the ALG and causes the message to be dropped. PR869725: This issue has been resolved.
  • When a snmp query is running that accesses information from service PIC, and during that an MS-DPC or service PIC restarts, then the adaptive services process (spd) may hang. As a result the thread will never complete, and we will never be able to update and delete the routes through RPD (e.g. routes that point to NAT pool ranges are marked as dead routes because they still point to old logical interfaces). This can result in routes in incorrect state and black-holing of traffic. PR874347: This issue has been resolved.
  • MIB-NJX-L2TP syntax errors, commas missing on line 401 and 930. PR881423: This issue has been resolved.
  • The jpppd crash on LNS happened because the size of the udp based l2tp packet exceeded the buffer length available. The modification was done to discard the packet instead of creating core. PR888691: This issue has been resolved.
  • In rare conditions with large number of traffic flows ( like NAT and IPsec flows ), the Service PIC may get stuck or crash as a result of prolonged flow-control assertions towards the Packet Forwarding Engine. In order to trigger this issue, many Compute CPUs inside the Service PIC should be overloaded. This will never happen under normal operation, where CPUs can handle large amount of traffic without any issues. PR900227: This issue has been resolved.
  • The SIP ALG is unfit for EIM due to standing limitations, hence, SIP and EIM is currently unsupported configuration. PR900412: This issue has been resolved.
  • In Carrier Grade Network Address Translation (CGNAT) environment, if memory utilization of MS-DPC/service PICs are in the yellow zone and they are configured with cgn-pic knob, there can be a race condition where there are two timers created for the same flow and during the timer processing, the MS-DPC/service PIC may experience a crash and generate a core file. PR901795: This issue has been resolved.
  • In an L2TP scenario, after performing an SNMP walk of "jnxL2tpTunnel" or "jnxL2tpSession" MIBs, the SNMP reply message fails to be written because write buffer is exceeding MTU, causing Routing Engine CPU spikes to 100%. PR905218: This issue has been resolved.
  • In some cases rtsp data flows will be left without cleanup when rtsp master flow closes. This will cause some conversation data flows left on router with very huge timeout values. PR909091: This issue has been resolved.
  • IKE UDP 500 packet is not processed in correct routing-instance. PR909909: This issue has been resolved.
  • In a CGNAT environment, active FTP operations fail when there is latency issue in network. When TCP retransmission, FTP ALG is not translating any fields in the Request: PORT command. As a result server tries to establish the data flow to the private IP address and to a wrong TCP port and it fails as expected. PR916376: This issue has been resolved.
  • In Carrier Grade Network Address Translation (CGNAT) with high memory utilization environment (Memory is in yellow zone and use CLI "show services service-sets memory-usage" to check), this crash might be seen in hairpinning scenario where Endpoint Independent Filtering (EIF) is enabled and the initial packet of a specific flow that hits the MS-DPC is dropped by an ALG due to various reasons (malformed or non complying packet/headers). PR918663: This issue has been resolved.
  • In Carrier Grade NAT (CGNAT) environment, during heavy setup rate of CGNAT flows, High Availability (HA) sync flaps and then keepalive messages are lost, as there is no control flow prioritization configured. HA sync connection keeps disconnecting. After a long period of time PIC silently reboots. Following syslog message might be seen when issue occurs: Sep 7 16:39:29 ROUTER-RE0 (FPC Slot 2, PIC Slot 0) PFEMAN: Lost contact with master routing engine PFEMAN: Forwarding will cease in 4 minutes, 59 seconds Sep 7 16:40:23 ROUTER-RE0 (FPC Slot 3, PIC Slot 1) PFEMAN: Lost contact with master routing engine PFEMAN: Forwarding will cease in 4 minutes, 59 seconds. PR920723: This issue has been resolved.
  • In Carrier Grade Network Address Translation (CGNAT) environment whenever an inbound UDP packet did not hit any rule, a check is performed whether the destination ip and port match any SIP registration. If this check is successful and 'learn-sip-register' is enabled (which is the default in the junos-sip application), if packets are counted as SIP ALG parsing errors, no flow is created and the packet will be forwarded without any transformation. In the case of NAT, the destination address will remain within the nat pool and the packet will keep coming back to the service PIC, causing a routing loop and high CPU utilization. PR923630: This issue has been resolved.
  • If multiple service sets with different number of NAT rules/pools are configured, Services PIC might crash when SNMP walk is performed on jnxSrcNatStatsTable. PR928169: This issue has been resolved.
  • When tcp session is initiated from inside client and three-way handshake is not completed because that client did not ack the syn-ack send from the server, the service pic will send a tcp reset to the server after the timer expires. In this case tcp reset is send in the wrong direction. Instead of sending in the outbound direction to the server, the service pic will send it in the inbound direction. This PR fixes this issue. No service impact is seen because of this. PR931433: This issue has been resolved.

Subscriber Access Management

  • Due to some timing issues, MX Series was generating incorrect LLPDF logs "LLPDF: llpdf_client_connection: Unknown session" every 10 seconds. PR894013: This issue has been resolved.
  • This netstat core can be generated during certain upgrade/downgrade scenario. The fix must be present in the image being upgraded/downgraded to. The trigger for having this netstat core generated is having secureId configuration present on the chassis. When the validate phase of "request system software add" runs, the netstat core may be generated. PR911232: This issue has been resolved.
  • Authd - UserAccess log events not sent to syslog host when destination-override is used. PR931975: This issue has been resolved.

VPNs

  • In an FEC129 VPLS scenario, VPLS pseudowire (PW) processing might hit an assert, causing rpd process to crash with a core file. PR843482: This issue has been resolved.
  • In affected releases, the C-PIM Assert mechanism is not working correctly in a Multicast VPN environment. A typical scenario includes an access VLAN with four routers (CE1, CE2, PE1 and PE2) which are C-PIM neighbors of each other. If CE1 sends a PIM Join to PE1, and CE2 sends a C-PIM Join to PE2, both PEs start to inject the C-Multicast flow in the access VLAN. This triggers the PIM Assert mechanism, which should result in either PE1 or PE2 (one of them, not both) injecting the traffic. However the following two situations might occur during 1 minute or more: - Both PE1 and PE2 keep injecting traffic in the VLAN. - Both PE1 nor PE2 stop injecting traffic in the VLAN. Releases with the fix work fine regarding the PIM Assert mechanism and do not show this abnormal behavior. PR880575: This issue has been resolved.
  • In L2circuit scenario, after L2circuit established, if pseudowire flaps (e.g. interface flapping) while routing protocol process (rpd) processing this change, memory corruption might occur, causing rpd process to crash with core files. PR900257: This issue has been resolved.
  • In l2circuit scenario, if there is no MPLS route to neighbor and there is a static route with discard nexthop in inet.3 table as follows:
    user@router# show routing-options rib inet.3 { static { route 0.0.0.0/0 discard; } }

    Then the l2circuit connection will use the above static route in inet.3 table to connect its neighbor as follows:

    user@router# run show route table mpls mpls.0: 7 destinations, 7 routes (7 active, 0 holddown, 0 hidden) + = Active Route, - = Last Active, * = Both 0 *[MPLS/0] 00:01:42, metric 1 Receive 1 *[MPLS/0] 00:01:42, metric 1 Receive 2 *[MPLS/0] 00:01:42, metric 1 Receive 13 *[MPLS/0] 00:01:42, metric 1 Receive ge-0/0/1.601 *[L2CKT/7] 00:01:38, metric2 0

    Discard In this situation, routing protocol process (rpd) will core dump while walking Simple Network Management Protocol (SNMP) MIB"jnxVpnPwEntry". PR906519: This issue has been resolved.

  • This PR enables default advertisement of MVPN from the main BGP routing tables bgp.mvpn.0 and bgp.mvpn-inet6.0 instead of VRF routing table foo.mvpn.0 or foo.mvpn-inet6.0. It also removes withdraw suppression for extranets. If extranets are used, advertise-from-main-vpn-table is enabled by default for an MVPN NLRI. PR908199: This issue has been resolved.
  • In Rosen and NG-MVPN running in rpt-spt mode, valid (*,G) forwarding state can be created (it can not be created in spt-only mode). If there is rpf-check-policy added to MVPN instance and the rpd check is associated on the (*,g) forwarding route installation, the rpd might crash. PR915672: This issue has been resolved.

Resolved Issues in Release 12.3R4

Class of Service (CoS)

  • When 'scheduler-map-chassis derived' configuration is used under class-of-service, interface related configuration changes can lead to cosd process crash. PR863734: This issue has been resolved.
  • During addition/deletion or just deletion of interfaces with configuration for shared scheduler, some portion of memory is not reclaimed back normally. So continuous addition/deletion of these interfaces results in memory depletion, packet loss and other issues. PR890986: This issue has been resolved.

Forwarding and Sampling

  • Possibility of duplicate packets when sampling and interface-style nat are configured. PR861984: This issue has been resolved.
  • On M7i/M10i with enchanced CFEB, M320 with E3-FPC, M120 or MX DPC, if there is distributed Bidirectional Forwarding Detection (BFD) running on Aggregated interface and a firewall filter is configured on loopback interface (lo0), the lo0 will bind an implicit filter, after FPC restarts or the Routing Engine switchover, the next hop of the implicit filter is not updated with the corresponding link word to point to CLI filter, causing the CLI filter to be not executed. To resolve the issue, deactivate the firewall filter under loopback interface and then activate it again. Note:The default operational mode of bfd for all protocols is distributed mode (runs on pfe), one excpetion being ospf v3 which runs on the Routing Engine by default (centralized mode).+ +So ospf v3 is not affected by this issue. PR864665: This issue has been resolved.
  • Outbound control traffic is not counted by accounting-profile which applied to logical interfaces of AE (Aggregated Ethernet). PR866181: This issue has been resolved.
  • In T4000 platforms with ES-FPC, for IPv6 firewall filters with match conditions on address prefixes longer than 64 bits, in some corner cases, the filter may not be correctly evaluated and packet loss may occur. PR879829: This issue has been resolved.
  • user@host> show services accounting flow-detail destination-prefix 20.1.1.2/32

    Service Accounting interface: sp-2/0/0, Local interface index: 147

    Service name: (default sampling) Interface state: Accounting

    Protocol Input Source Source Output Destination Destination Packet Byte Time since last Packet count for Byte count for interface address port interface address port count count active timeout last active timeout last active timeout udp(17) xe-0/0/3.0 10.1.1.2 whois++(63) xe-0/0/2.0 20.1.1.2 whois++(63) 1075917 49492182 00:17:55 1780922 81922412 tcp(6) xe-0/0/3.0 10.1.1.2 0 xe-0/0/2.0 20.1.1.2 0 106479 4898034 00:01:46 1835070 84413220 PR881629: This issue has been resolved.

  • In scaled MPLS scenario, when LSP path switchover happens, sample process deletes sampling parameters from the Packet Forwarding Engine and as a result of that Packet Forwarding Engine stops exporting flows to the collector. PR891899: This issue has been resolved.
  • When router goes into Amnesiac mode with 'commit failed' due to statements constraint check failed while upgrading Junos OS to 11.4 or later, ARP Replies will be dropped due to incorrect default arp policer on interface even after fixing the commit errors. PR895315: This issue has been resolved.
  • After committing some configuration changes (e.g. deactivate an interface), while the Packet Forwarding Engine daemon (PFEd) tries to get statistics of some nodes, it may encounter a NULL node, causing PFEd to crash and generate a core file. PR897857: This issue has been resolved.

General Routing

  • When an MPC fails in a specific manner, while failing it continues to send traffic into the switching fabric for a time, the fabric ASICs report errors such as these with large counts: chassisd[82936]: %DAEMON-3: New CRC errors found on xfchip 0 plane 0 subport 16 xfport 4 new_count 17651 aggr_count 17651 chassisd[82936]: %DAEMON-3: New CRC errors found on xfchip 0 plane 0 subport 17 xfport 4 new_count 17249 aggr_count 17249 chassisd[82936]: %DAEMON-3: New CRC errors found on xfchip 0 plane 0 subport 18 xfport 4 new_count 65535 aggr_count 65535

    This can cause DPC(s) to stall and not send traffic into the switching fabric to other DPCs or MPCs. Messages such as these may be reported by the affected DPC(s) :

    [Err] ICHIP(1)_REG_ERR:packet checksum error in output fab_stream 4 pfe_id 64 [Err] ICHIP(1)_REG_ERR:packet checksum error in output fab_stream 6 pfe_id 64 [Err] ICHIP(1)_REG_ERR:packet checksum error in output fab_stream 8 pfe_id 64

    This failure on the affected DPCs persists, and will likely affect all traffic destined to the fabric from affected DPCs. The only temporary resolution is to restart the affected DPCs, which will resume fabric traffic from the affected DPCs. PR856560: This issue has been resolved.

  • ATM MIC back-to-back, to many logical interfaces(more than 8k) may cause certain logical interfaces down. PR859165: This issue has been resolved.
  • When the fxp0 interface on a k2re is administratively disabled, the local end shows the link as down while the far end device displays the status as up. PR862952: This issue has been resolved.
  • During a reference clock switch T4 will be switched off. PR868161: This issue has been resolved.
  • The 1588v2 BMCA procedure causes a frequency hold-over event in the system under test. PR868422: This issue has been resolved.
  • Configuration of Container Interfaces for APS on MX FPCs is not allowed since Junos OS 12.1. If this feature is needed on MX Series legacy FPCs use a release with this PR fixed. PR869192: This issue has been resolved.
  • Under high scale, expiry of a Kernel side reconnect timer would cause it to send a non-servicable message to the Packet Forwarding Engine(asking the line cards to restart and resync since reconnect failed). Since there is no ack- to this kernel message, kernel thought it sent the message and untoggles the GRES flag. The Packet Forwarding Engine wasn't expecting anything so it continued along. The EFFECT: The system is permanently not ready for GRES... CLI GRES check will always report: [cmd] request chassis routing-engine master switch check Apr 14 19:03:13 [INFO ] warning: Standby Routing Engine is not ready for graceful switchover. PR873679: This issue has been resolved.
  • On systems containing XM-based linecards(for example, MPC3, type 5 FPCs), if a member link of an aggregate Ethernet (AE) bundle is repeatedly flapped, the flapped member link may stop transmitting traffic. Traffic isn't getting dropped, as the remaining member-links will pick up the slack. But in some cases (the traffic is large or some members encounter the problem together), traffic loss will happen. PR875502: This issue has been resolved.
  • On MX-VC platform, when the master Routing Engine declares GRES ready by CLI command, there is a time window before some FPCs to be actually ready. After performing GRES, these GRES unready FPCs might get rebooted, resulting in traffic loss. PR877248: This issue has been resolved.
  • authd reports sytax error, although the syntax is correct, when trying to activate service profile for subscriber and fails to activate the service. PR883065: This issue has been resolved.
  • lrmuxd core seen when commiting changes related to BD or routing-instance. Below messages appears and commit fails user@host# commit error: Check-out pass for logical system multiplexer process (/usr/sbin/lrmuxd) dumped core (0x86) error: configuration check-out failed. This Issue is resolved now. PR883090: This issue has been resolved.
  • The Routing Engine might become non-responsive due to the exhaustion of kernel mbufs with following messages. /kernel: Mbuf: High Utililization Level: (Low) Throttling low priority requests (10 ms) /kernel: Mbuf: High Utililization Level: (Medium) Throttle low priority requests (150 ms) /kernel: Mbuf: High Utililization Level: (High) Block low priority requests PR886083: This issue has been resolved.
  • RPD might core dump if HFRR (Host Fast Reroute) is enabled on two logical interfaces in the same routing instance for IPv6 and if link-local address is configured on those logical interfaces. The core files could be seen by executing CLI command show system core-dumps. PR886424: This issue has been resolved.
  • The backup Routing Engine failed to commit with error "pdb_update_ddl_id: cannot get new id for " dynamic-profiles dynamic-profiles profile-name"",commit full is a workaround. PR888454: This issue has been resolved.
  • When multiple framed-route(type-22) AVPs are present in Radius access accept message, the router will install only the first route into the routing table. PR891036: This issue has been resolved.
  • Following a global GRES event, the new Master(VC-Mm) will expect relayd to reconnect to it in less than 40 seconds. However under high scale, such as with 54k dual-stack(v4v6) or 110k+ single-stack DHCP subscribers, owing either to a slow relayd(relay daemon) control connection to the Kernel, or due to slow Packet Forwarding Engine reconnects to relayd, we are not able to meet the 40 seconds timer requirement causing subsequent FPC reboots and traffic loss. PR891814: This issue has been resolved.
  • When performing DSCP rewrite on LMNR or T640-FPC4-ES and PC-OC-768 cards, the ECN bits will get reset to 0. PR896847: This issue has been resolved.
  • Interfaces on FPC slot 5 will disapper after offlining of any fpc(not fpc-slot 5) on M40e/M160. PR898415: This issue has been resolved.
  • In subscriber management environment, in a rare case, VLAN auto-sensing daemon (autoconfd) might crash and create a core file due to Session Database (SDB) is inaccessible. PR899747: This issue has been resolved.
  • Some ATM interfaces may stay down after flapping the Circuit Emulation MIC. PR900926: This issue has been resolved.

Infrastructure

  • Unsolicited Neighbor Advertisement is not sent from backup when vrrp switchover is initiated. PR824465: This issue has been resolved.
  • Kernel may crash when delete routing instance under the donor and unnumbered address borrower scenario. When the deleting for the donor is before the deleting of the corresponding unnumbered borrower, in this window, the donor interface does not have an address, and arp processing over the borrower interface during this window may trigger the crash. The core files could be seen by executing CLI command show system core-dumps. PR880179: This issue has been resolved.
  • Every 10 minutes kernel reports "%KERN-6: MTU for 2001:4c0:1:1301:0:1:0:250 reduced to 1500" after reducing MTU once. There is no impact to the system due to this additional log message. PR888842: This issue has been resolved.
  • In a multihop IPv6 BGP session scenario, after configuring single-hop BFD session on the multihop IPv6 BGP neighbor, kernel might try to access a NULL pointer, causing kernel to crash and generate a core file. PR898153: This issue has been resolved.
  • Checksum error seen on ICMP reply when 'sequence, data' field in request set to '0'. PR898487: This issue has been resolved.

Interfaces and Chassis

  • On E1 interface, when interface flaps on CE side of connection, interface will flap a second time on the PE side. PR690403: This issue has been resolved.
  • Traffic loss is seen. Multiple inbound and outbound IPSEC tunnels are created for a single SA during tunnel renegotiation after the lifetime expiry. PR827647: This issue has been resolved.
  • Planes might go into faulty state during the SCB initialization when the SERDES on the SF chip failed to come up. PR839509: This issue has been resolved.
  • IQ2 core is seen after unified ISSU and traffic will be lost for awhile(about 40s). The crash happens during processing of scheduler free message which comes just after unified ISSU complete on IQ2. Then the heap structure is invalid causing panic. The fix is moving the process to unified ISSU sync stage. PR845257: This issue has been resolved.
  • On the following MIC-3D-20GE-SFP only, if the 1GE interface is put into loopback mode, all packets larger then 306 Bytes are truncated on the wire. The solution is to bring the interface down once loopback is configured, to prevent truncated packets to be sent out. PR856892: This issue has been resolved.
  • Dump-on-flow-control knob might not work correctly for RSP interfaces configured in “warm-standby” mode. After an RSP switchover, either manual or following a crash, the dump-on-flow-control flag might get cleared from the MS-PIC. PR867394: This issue has been resolved.
  • snmpwalk of "jnxPPPoEIfLockoutTable" didn’t capture pppoe locked out clients. PR869024: This issue has been resolved.
  • MC-LAG will no longer change just the LACP System Identifiers directly, but will also remove the "Synchronization, Collecting, Distributing" bits from the Actor State bits advertised in the PDU. PR871933: This issue has been resolved.
  • Injecting Enhanced RDI-P(G1 bit5-7:0x2 Payload defect) alarm to a MPC 10GbE WAN-PHY interface causes RDI_P and LCD-PAIS-V alarm on messages. This is due to string typo. RDI_P and LCD-P should be printed on messages. PR872133: This issue has been resolved.
  • On MX Series router with MPC with 20port GE MIC, interface stores packets when disabled and transmits stored packets after enabled. PR874027: This issue has been resolved.
  • In subscriber management environment, with dynamic-profiles configured for subscribers, if the routing instance returned from radius is not configured on BRAS, dynamic-profile add fails and there are some places the memory not freed, causing device control daemon (dcd) memory leak. The memory usage of dcd process can be observed by following command:

    user@router> show system processes extensive | match dcd

    PID USERNAME THR PRI NICE SIZE RES STATE TIME WCPU COMMAND 7076 root 1 97 0 1047M 996M select 6:05 2.88% dcd PR880235: This issue has been resolved.

  • MX Series router is not passing transit IPv6 traffic received on a RLSQ interface with fib-localization enabled. PR880245: This issue has been resolved.
  • VC-Boot loop when installing new local backup Routing Engine. PR881906: This issue has been resolved.
  • Problem scenario: CFM UP MEP for Bridge/VPLS is configured on MPC with action profile as 'interface down' Problem statement: When the CFM sessions go down due to network outage at the core, action profile is triggered and the configured interface is brought down. When the Core network failure is corrected, CFM will not automatically recover because the interface will continue to remain down. PR884323: This issue has been resolved.
  • On LAG interface gratuitous ARP is neither generated nor sent out upon link up even when gratuitous-arp-on-ifup is configured. PR889851: This issue has been resolved.
  • In dynamic PPPoE subscriber management environment, when MS-DPC card is added and "adaptive-services service-package laryer-2" is configured, while PPPoE subscribers log in, kernel might encounter a memory corruption, causing kernel to crash and generate a core file. PR894440: This issue has been resolved.
  • The C-LMI (Consortium LMI) is supported on all I-chip based FPC. Support for the MX-FPC 2 and 3 was missing and now added. PR895004: This issue has been resolved.
  • On MX Series based platforms, when PIC is configured with traffic-manager mode ingress-and-egress, after PIC offline, PIC detach does not clean up the corresponding entries completely. Subsequent PIC online results in corresponding entries add failure since previous entries are still intact, resulting in interface attach failure at the Packet Forwarding Engine level. Due to interface add failure, protocols on the interface never come up. PR895305: This issue has been resolved.
  • IPv6 IIF-index load-balance works unwantedly when IIF-V4 is enabled alone and vice versa. PR898676: This issue has been resolved.
  • On front panel display LED status for PSM is incorrect after manually Remove/Insert of PSM. PR937400: This issue has been resolved.

Layer 2 Features

  • When VPLS is configured with GRES, the backup Routing Engine responds to certain route replication requests by simulating address learning. If the route being replicated is associated with an LSI or VT interface, the address learning code references a special LSI or VT nexthop. Thus, there is a dependency between that route and that nexthop. This fix is to explicitly enforce this ifstate dependency, ensuring that the special nexthop is seen by the peer before the route. PR867929: This issue has been resolved.
  • In Releases 12.1R3, 12.2R3, 12.3R4, 13.1R1, and 13.2R1, for a configuration with bridge domains containing aggregate interfaces, traffic whose destination address is broadcast, multicast, or unknown will not be load-balanced across the member links of such interfaces. Instead, all such traffic will be sent out a single link of the aggregate interface. With this PR change, load-balancing will always be applied to such configurations for traffic whose destination address is broadcast, multicast, or unknown. This change restores the functionality of older releases. PR888232: This issue has been resolved.

Layer 2 Ethernet Services

  • jdhcpd interface traceoptions are not saved to the default log file jdhcpd and require an explicit file name. PR823129: This issue has been resolved.
  • New knob is provided to set the prefix to compare requested ip and server address. Knob is configured as - [edit system services dhcp-local-server] #set requested-ip-network-match <0-31> For V6 [edit system services dhcp-local-server] #set dhcpv6 requested-ip-network-match <0-127> Default will be 8 for v4 and 16 for v6 (first terms). PR872145: This issue has been resolved.
  • When IPv6 is configured on integrated routing and bridging(IRB) interfaces that have AE interfaces as child links, after GRES was enabled and one child link failed or was removed, the kernel crashed. PR878470: This issue has been resolved.
  • DHCPv6 Local Server implementation deletes the client on a reconfigure, so that client can reconfigure. DHCPv6 relay is not forwarding the Reply to the client and simply tearing the client down (generating a release to the server). PR879904: This issue has been resolved.
  • When executing show dhcp relay binding command with high scales of bound subscribers and with several hundred renewing at a given time, DHCP drops the renew packets. PR882834: This issue has been resolved.
  • In an IP demux/vlan demux configuration, where the primary address for the loopback is different from the preferred in the dynamic profile, the ACK to the first RENEW will have the primary address in loopback as server ID since RENEW arrives on ip demux interface. The client will send the next RENEW to that server ID, and the router will drop it. The fix is to always use the server ID from the underlying interface. PR890562: This issue has been resolved.

MPLS

  • The LDP protocol might use the lowest IP address configured on an interface even if there is another (higher) address that is explicitly configured as primary. This can lead to unexpected LDP session flap if the lowest but non-primary address is being removed from the configuration. PR858838: This issue has been resolved.
  • In an RSVP environment with AutoBw, the Bandwidth Adjustment timer for new LSPs added simultaneously is not smeared along with the rest of the existent LSPs when the smearing algorithm is triggered. PR874272: This issue has been resolved.
  • When BGP labeled-unicast route has BGP label as null and its indirect next-hop requires adding 2 or more labels, traffic using the BGP label may not be forwarded properly. PR881571: This issue has been resolved.
  • The VpnId value contains no information, but was being returned as the empty string, when the MIB requires that it be a length 7 octet string. The value (since it contains no information) is now returned as 7 zeros. PR882828: This issue has been resolved.
  • With OSPF overload enabled, the te-metric will be set as 2^32, and the Constrained Shortest Path First (CSPF) process ignores the path with metric value 2^32, causing the ingress LSPs not to come up. PR887929: This issue has been resolved.
  • When a LDP egress router advertises multiple prefixes, by default the prefixes are bound to a single label and aggregated into a single forwarding equivalence class (FEC). If the nexthops of some prefixes in the FEC change (e.g. LDP interface flapping), LDP still tries to bind a single label to all of the prefixes which is incorrect. PR889585: This issue has been resolved.
  • LSP metric will be not correctly changed as the new configured one after committed when cspf finds an Explicit Route Object (ERO) different from the current ERO and the Path State Block (PSB) re-signaling fails. This is because a change in metric is a local PSB change, but after a configuration change (for example, the bandwidth requirement was changed), PSB and associated routes used to get this change only after a cspf computation followed by a session refresh or re-signaling. If the re-signaling fails, the configured metric value is not updated in the existing PSB and the route metric. PR894035: This issue has been resolved.
  • Changing the preference on an LSP was considered a catastrophic event, tearing down the current path and then re-establishing a new one. This PR makes the preference change minor and only needs a new path to be re-signalled in a make-before-break manner. PR897182: This issue has been resolved.
  • With Junos OS Release 12.1R1 or later, any configuration changes in the MPLS stanza, P2MP LSP connection with a single branch, will flap and cause brief traffic drops if allow-fragmentation knob is configured under the MPLS path-mtu stanza. No traffic drops are seen if the P2MP LSP has two or more branches. Any application which is using P2MP RSVP LSP is exposed to this issue, like ccc p2mp-transmit-switch, static route with p2mp-lsp-next-hop, etc. PR905483: This issue has been resolved.

Network Management and Monitoring

  • When we do snmp polling via CLI on a big MIB node that has lots of OIDs and huge data like "show snmp mib walk 1.3.6.1.4.1", CLI might not be able to consume data at the rate it was being generated by snmpd, so the snmpd buffer is occupied more and more. Eventually this would cause snmpd to reach its rlimit then crash. PR864704: This issue has been resolved.
  • SNMP query from valid client on routing-instance-1 with community string that belongs to routing-instance-2 gets the details of routing-instance-2 instead of blocking such queries based on community. PR865023: This issue has been resolved.
  • When you perform the following MIB Walk on interfaces, for some interfaces the ifLastChange value will show a value of zero. show snmp mib get ifLastChange.<SNMP ifIndex> will show a value of zero. ifLastChange.<SNMP ifIndex> = 0. PR886624: This issue has been resolved.
  • A memory leak in the cosd process is seen when both of the following conditions are met: - multiple OIDs from jnxCos MIB, that are under the same logical interface hierarchy, are queried in a single SNMP query sent to the device (i.e. in a single PDU) - either "per-unit-scheduler" or "hierarchical-scheduler" configured on the physical interface The following messages will be logged when the cosd process exceeds 85% of its maximum usable memory: Jun 9 13:16:35.475 2013 router-re0 /kernel: %KERN-5: Process (1457,cosd) has exceeded 85% of RLIMIT_DATA: used 1894060 KB Max 2097152 KB PR893464: This issue has been resolved.

Platform and Infrastructure

  • RMOPD crash is due to sort of buffer overflow crash and library function being used improperly. It is not caused by RPM scaling, This issue happens randomly and hard to point out the specific trigger. PR277900: This issue has been resolved.
  • Junos OS 10.4R8 or later on MX Series platforms, L3VPN application using l3vpn-composite-nexthop when the indirect-next-hop configuration statement is added or removed it might cause traffic traffic drops affecting L3VPN flows. To recover from this condition all the l3vpn prefixes need to get removed and installed new into the forwarding-table, like clearing the bgp peers where the routes are learned from. PR741646: This issue has been resolved.
  • When changing configuration repeatedly, in rare conditions, some internal errors may cause CLI process hogs memory and the utilization keeps on increasing due to memory leak. When the memory usage of CLI process increases to around 85% of system limit, the following logs could be seen: /kernel: Process (1383,cli) has exceeded 85% of RLIMIT_DATA: used 62048 KB Max 65536 KB The memory will be released once user logout from the router. PR813673: This issue has been resolved.
  • In rare case, after no graceful FPC rebooting (i.e. temporary power failure on egress FPC), fabric ASIC on ingress STFPC can run into temporary problematic status. This will cause temporary large delay on fabric traffic from STFPC to the egress FPC. PR831743: This issue has been resolved.
  • Since the AC Power System on MX2020 is a N+N feed redundant and N+1 PSM redundant, there are two separate input stages per PSM, each connected to one of the two different/redundant feeds. However, only one stage is active at a time. This means, the other input stage (unused input stage) may be bad and system will not know about it till it tries to switch to it in case of a feed failure. This is a pretty bad corner case and needs to be addressed. The way to work around this problem is by testing both stages when the power supply is first powered on. This test is done by the system software and an alarm is raised if any feed failure is detected. PR832434: This issue has been resolved.
  • On MPC 3D 16x 10GE, MPC3, and MPC4 platforms, if host outbound traffic is set to any forwarding class which may maps to queue numbers 4 through 7, after configuring "max-queues-per-interface 4" (4-queue mode is enabled), then queues 4-7 will not be configured with proper traffic parameters, but queue is enabled with default config. When physical interface gets oversubscribed the queue which is carrying host originated traffic can starve for bandwidth because of no q-rate for the queue. Eventually in the event of steady state oversubscription causes loss of control traffic and hence control session can flap. Especially on Junos OS release 12.3R3, after configuring "max-queues-per-interface 4", the corresponding interfaces will get only just 4 queues, it causes 100% loss of host originated traffic because the queues from 4-7 are not enabled for traffic transmission. PR868021: This issue has been resolved.
  • When an MX Series router collects with inline jflow, exported IPv6 UDP packets show UDP checksum is incorrectly set to 0x0000, which might be discarded by received node. 12:19:11.513058 In IP6 (hlim 64, next-header: UDP (17), length: 138) 2001:db8:ffff:ffff::20.33068 > 2001:db8:0:100::101.2055: [bad udp cksum 9652!] UDP, length 130 12:19:11.524964 In IP6 (hlim 64, next-header: UDP (17), length: 138) 2001:db8:ffff:ffff::20.33068 > 2001:db8:0:100::101.2055: [bad udp cksum 2086!] UDP, length 130 12:19:16.509978 In IP6 (hlim 64, next-header: UDP (17), length: 138) 2001:db8:ffff:ffff::20.33068 > 2001:db8:0:100::101.2055: [bad udp cksum 1340!] UDP, length 130 PR870172: This issue has been resolved.
  • When check trace route, RSVP-TE Probe status is not shown as success and It is shown as unhelpful. Note: seeing this issue with ip-enhance mode and not seeing this issue without ip-enhance in same setup and same image. PR871015: This issue has been resolved.
  • After restart of an FPC, when it comes online the queue block on another FPC becomes locked up and all traffic into the fabric from this Packet Forwarding Engine is dropped The issue occurs when there is a lot of high-priority traffic, and low-priority traffic gets stuck behind and therefore causes the timeout and queue draining. PR877123: This issue has been resolved.
  • This is a regression issue introduced by the fix of PR801982, which causes DOM MIB values for SFP+ "rx power" related statistics to be incorrect. Note that XFP is not affected. PR878843: This issue has been resolved.
  • If interface flaps of a bridge-domain with igmp-snooping enabled or multicast snooping routes are pruned due to Designated Router changes, LUCHIP might report traps and EDMEM read errors. These conditions are transient and only seen once the system is operating with enhanced-ip mode. PR879158: This issue has been resolved.
  • PHP to PE link with MPLS MTU 1300 allows transit traffic more than 1268 i.e. up 1272. Note : PE to PHP has default MTU in this case i.e. there is MTU mismatch between PHP and PE link. Max packet size allowed is 1300 - 20 (ip) - 8 (icmp) -4 (1 label due to PHP) = 1268 PR879427: This issue has been resolved.
  • Deactive/delete AE interface when route is flapping might cause MX Series routers with MPCs/MICs based Packet Forwarding Engine to crash. PR884837: This issue has been resolved.
  • While configuring a filter with a generic prefix followed by specific one in different terms may lead to incorrect match, which might lead to packet drop. PR886955: This issue has been resolved.
  • In l2circuit connection scenario, when the STFPC interconnect with MX Series based FPC, PPP-CCC l2circuit connection will drop the small packets with Ethernet length error. PR887098: This issue has been resolved.
  • In L2VPN scenario, on the PE router, if the encapsulation of the PE-CE interface is vlan-ccc and there is a COS filter under the interface, when the interface flaps, it can cause all the traffic to different sites via different outgoing interfaces is forwarded incorrectly through one of the interfaces. Meantime, when manually flap the label-switched paths (LSPs) on the router after the problem occurred, the traffic is forwarded incorrectly still but only the egress interface will change to other one. The way to resolve the problem is manually clearing the LSPs on the PE router. PR887838: This issue has been resolved.
  • It is observed that in the setup route nexthop for destination of collector's IP address was of type indexed nexthop. PR889884: This issue has been resolved.
  • Because of the hardware limit, the feature "maximum-labels" on FPC can't exceed 3. Whenever maximum mpls label is configured as 4 or 5 on unsupported FPC, the LDP/RSVP session will go down and cause MPLS traffic black hole for couple of minutes. This dark window will remain till the unicast next hops are installed and attached to the egress interface where the label has been configure. After that MPLS traffic will resume. PR890992: This issue has been resolved.
  • Traffic may be affected after performing an offline/online sequence on the PIC in a T4000 system. This issue is usually seen when the event is performed on PICs carried in a Type 5 FPC. PR892548: This issue has been resolved.
  • High rate of traffic to the Routing Engine may trigger credit overflow within the Traffic Offload Engine (TOE) and prevents further processing of packets destined to or originated by the Routing Engine. High rate of traffic means continued Hardware input drops reported via the show pfe statistics traffic command. The following message might be seen in the system message log-file: member1-fpc2 TOE Pkt Xfer:** WEDGE DETECTED IN PFE 2 stream 0 TOE host packet transfer: reason code 0x1 The following Junos OS software release 12.3R3, 12.3R3-S1, 12.3R3-S2 or 12.3R3-S3 are exposed on MX Series based FPCs and T4000-FPC5-3D FPCs. PR896592: This issue has been resolved.
  • Scheduler with zero guaranteed rate and excess priority none is an invalid class of service configuration. The packet enqueued in the corresponding queue will not be able to dequeued. PR900239: This issue has been resolved.
  • In MX-VC setup using virtual-switch instance type, there can be scenarios where the outer vlan-tag of PPPoE/PADI packets on egress can be stripped off when ingress interface is a LAG with two member links spread across the two Chassis members. PR905667: This issue has been resolved.
  • Junos OS 12.3R3, 12.3R3S1 and 12.3R3S2, interfaces with interface-mode trunk connected on top PFE[0] and with IRB interfaces might corrupt forwarding-state on lowest Packet Forwarding Engine of the FPC. This is applicable to system operating with network-services enhanced-ip mode and systems operating in virtual-chassis mode. PR907291: This issue has been resolved.

Routing Protocols

  • When "passive" and "disable" knobs are both configured under [edit protocols isis interface <intf> level <N>] hierarchy the interface is treated as "passive" instead of being disabled. PR697553: This issue has been resolved.
  • BFD triggered local-repair(RLI9007) not initiating immediately. RLI 9007 is applicable from 12.2 onwards. PR825283: This issue has been resolved.
  • Junos OS checks for mask-length mismatch for OSPF P2P-over-LAN interfaces, but skips the check if an interface has /32 mask configured. In a scenario with OSPF configured between Juniper Networks platform and other venders' platform, if a /32 mask IP address is configured on P2P-over-LAN OSPF interface of Juniper platform and a non /32 mask IP address is configured on the peer, the OSPF neighbor can establish but Kernel Routing Table (KRT) queue gets stuck. PR840122: This issue has been resolved.
  • In BGP scenario, the initial peer flaps and goes down, then a new peer is established, which might cause an rpd core. PR840652: This issue has been resolved.
  • Memory leak after deleting a single BFD session. Observed in show heap command. PR840672: This issue has been resolved.
  • Multicast packets coming with source address as 0.0.0.0 may cause the RPD to crash. PR866800: This issue has been resolved.
  • If the SNMP MIB for BGP is walked, the AFI=1, SAFI=5 entries are missing. If an SNMP "get" is performed, the values can be retrieved. PR868424: This issue has been resolved.
  • In an IS-IS scenario, when a large number of routes are distributed into IS-IS, IS-IS overload bit will be set due to maximum LSP fragment exhaustion, this is correct. Then delete the IS-IS export policy, after that, the IS-IS overload bit should be cleared. But the number of exported prefix might be incorrect even though the number of export prefix is zero actually. This can cause overload bit to be set always. This is because local-data for prefixes is not freed up and leads to some memory leak. PR874015: This issue has been resolved.
  • If a static route is configured and exported into OSPF, and if the static route has the same subnet as an OSPF interface address, then committing configuration changes (even unrelated to OSPF, such as a device's hostname) results in the removal of the static route related to OSPF type-5 link-state advertisement (LSA) from the OSPF database. PR875481: This issue has been resolved.
  • Returned attribute values are not in the defined value range of the mib bgp4PathAttrASPathSegment. PR882407: This issue has been resolved.
  • RPD CPU utilization keeps 100% due to "BGP resync" task when BGP is configured with no neighbor and NSR is configured. id@router> show configure routing-options nonstop-routing; id@router> show configure protocols bgp { group bgp-group { type internal; inactive: neighbor 1.0.0.1; } } PR884602: This issue has been resolved.
  • RPD may crash on the new master Routing Engine after Routing Engine switchover. The issue is NSR related, and it happens due to the bad BGP route data structure on the backup Routing Engine. PR885305: This issue has been resolved.
  • When used JUNOScript to run command 'get-pim-neighbors-information instance=' (with NULL instance name), which triggered core dump even though there are no routing-instances with pim enabled. It won't trigger core file if JUNOScript command includes any instance name. PR887070: This issue has been resolved.
  • In a scenario with graceful restart (GR) enabled for BGP between Cisco platform and Juniper Networks platform, Junos OS is helper (default) and Cisco being restarting router, when Cisco restarts BGP process, Juniper deletes all BGP routes due to doesn't receive End Of RIB (EOR) markers for all configured NLRIs from Cisco. PR890737: This issue has been resolved.
  • The downstream PE router's RPF_neighbor(S) on the MDT reverts back to mRIB.next_hop(S) rather than the Assert(S,G)Winner when their PPT expires. Bug identified in the code and is fixed. PR896898: This issue has been resolved.

Services Applications

  • Memory leak in key management daemon (kmd) causes some IPsec VPN tunnels to be dropped and don't get re-negotiated for over 10 minutes. Before issue happens, the following logs could be observed: /kernel: Process (1466,kmd) attempted to exceed RLIMIT_DATA: attempted 131080 KB Max 131072 KB /kernel: Process (1466,kmd) has exceeded 85% of RLIMIT_DATA: used 132008 KB Max 131072 KB PR814156: This issue has been resolved.
  • In L2TP subscriber management environment, after issuing CLI command "commit full", jl2tpd process (l2tp daemon) deletes all tunnel profiles and brings down all L2TP subscribers. Even though there are no configuration changes. PR834504: This issue has been resolved.
  • MIB module in file "mib-jnx-sp.txt" contains a coding error, which may lead to a loop. PR866166: This issue has been resolved.
  • If RSP1 and RSP10 interfaces are configured on the same box, issuing the "request interface switchover rs1" or "request interface revert rsp1" causes both RSP1 and RSP10 to switchover or revert. PR877569: This issue has been resolved.
  • In a CGNAT environment when sp interfaces, which are underlying rsp interface, are present in the configuration, sp interfaces service-options may incorrectly overwrite rsp interfaces service-options and syslog stopped working and inactivity-timeout values were reset to the default values. PR881792: This issue has been resolved.
  • AAPID list configuration not copied to Backup Routing Engine // 12.3R2.5 PR885833: This issue has been resolved.
  • The jl2tpd process generates a core file as follows: "./../src/bsd/lib/libc/stdlib/abort.c:69." PR887662: This issue has been resolved.
  • SIP ALG - Service PIC might crash when SIP flows are cleared. PR890193: This issue has been resolved.
  • Output interface shown as 'Unknown' under show services accounting flow-detail.issue has been analyzed RCA;-At the time when a flow is created in PIC memory, if the route to the destination IP(in the flow) is not known, we set a flag indicating that there is no route to Destination IP in the flow structure. When the flows are queried using "show service accounting flow-detail", picinfo daemon inspects this flag for each flow and prints the Output interface as "Unknown" if this flag is set. Now, after route record for that flow is downloaded to the Service PIC, the flow structure is updated to reflect the corresponding output interface, but, the above flag is NOT UNSET. So, picinfo daemon continues to print the output interface as "unknown" whenever "show services accounting flow-detail" is executed. PR890324: This issue has been resolved.
  • L2TP session on MS-PIC may fail and following error is observed "L2TPD_RADIUS_SERVER_NOT_FOUND" after a test access profile <ppp-profile> is issued. PR898872: This issue has been resolved.
  • When the 'learn-sip-register' knob is enabled for the SIP ALG (it is by default), for a SIP request in slow path implicitly denied by the firewall or NAT rules, a look up is done to see if the SIP request has a target that corresponds to any current registration state, in which case the corresponding reverse flows get created. While service PIC creating the corresponding reverse flows, an internal error may occur, causing service PIC to crash and generate a core file. PR899195: This issue has been resolved.
  • In a L2TP scenario, after performing an SNMP walk of "jnxL2tpTunnel" or "jnxL2tpSession" MIBs, the SNMP reply message fails to be written because write buffer is exceeding MTU, causing Routing Engine CPU spikes to 100%. PR905218: This issue has been resolved.

Subscriber Access Management

  • In DHCP/PPPoE subscriber management environment, after terminating subscribers, authd process might crash and generate a core file due to an invalid pointer being used. PR821639: This issue has been resolved.
  • In situation when CoA message includes both LI attributes and CoA attributes, authd process fails to respond to CoA. PR821876: This issue has been resolved.
  • DTCP - First 127 triggers are applied. PR873013: This issue has been resolved.
  • PPoE was not supported for the 802.1ad 0x88a8 TPID on the outer tags of dual-tagged VLANs:
    [edit interfaces interface-name gigether-options ethernet-switch-profile]set tag-protocol-id [0x88a8]

    PR874603: This issue has been resolved.

  • The authdlib logout/terminate release notify request might experience a processing loop. PR888281: This issue has been resolved.

User Interface and Configuration

  • In an aggressive provisioning scenario using scripts or automated tools, we recommend that you do not use rollback immediately after a successful commit. PR874677: This issue has been resolved.

VPNs

  • In this release Ngen-MVPN does not support NSR. But the commit check when Ngen-MVPN and NSR is configured does not fail. In previous releases this commit would fail. The commit check not failing for this configuration is expected to be fixed in release 12.3 R4. In Release 12.3 R3 config with NSR and Ngen-MVPN configuration should not be committed. Doing this commit can lead to routing application crashes (like PR 864439) as it is an unsupported feature. PR827519: This issue has been resolved.
  • Wrong data type for MIB object "mplsL3VpnVrfRteXCPointer". PR866259: This issue has been resolved.
  • If SNMP "get" tries to retrieve local and direct routes from mplsL3VpnVrfRteTable, they are not found. SNMP walk does walk the local and direct routes. PR874365: This issue has been resolved.
  • In VPLS scenario, if CE facing interface is aggregated Ethernet with multiple member ports (more than two members), BUM (broadcast, unicast unknown, and multicast) traffic from MPLS core will be replicated on all child link of aggregated Ethernet interface, and BUM from CE will be replicated at sending out from MPLS core facing interface. The problem is specific to M10i and M7i routers running with I chip based CFEB. PR880422: This issue has been resolved.
  • RPD might experience software exception during clear pim join on routing-instance. Typically seen in scenario where PIM load balancing is implemented over eibgp sessions. PR891586: This issue has been resolved.

Resolved Issues in Release 12.3R3

Class of Service

  • A few memory leaks have been fixed in the class of service process. PR811613: This issue has been resolved.
  • This cosmetic issue is specific of 3D linecards, based on MX Series routers with MPCs/MICs. In these cards, the logical interfaces with family mpls do not have any EXP rewrite rule applied by default. In other words, EXP value is copied from the previous codepoints: for example, from IP Precedence in IPv4->MPLS next hops. However, the command "show class-of-service interface" still shows the exp-default rule as if it was applied (in fact, it isn't): user@router> show class-of-service interface ge-2/3/1.204 | match rewrite Rewrite exp-default exp (mpls-any) 33. PR824791: This issue has been resolved.
  • When 'scheduler-map-chassis derived' configuration is used under class-of-service, interface related configuration changes can lead to cosd process crash. PR863734: This issue has been resolved.

Forwarding and Sampling

  • There is always a chance to see this issue if any daemon adds a blob size which comes closer to 65520(after IDR encoding). PR700635: This issue has been resolved.
  • Memory leak could happen to pfed, dcd, cosd, cfmd and dfcd processes if user frequently and repeatedly executes "show interface extensive" command from multiple telnet sessions under the conditions below. 1. Set screen-length value to small value. Screen length can be changed by the command "set cli screen-length <n> ". 2. User enters "show interface extensive" command simultaneously from multiple telnet sessions. And cancel the output of the command with "q" as soon as "---(more)---" shows up at the end of the output. PR843145: This issue has been resolved.
  • MPLS forwarding table filter (ftf) not getting linked in JTREE after router or FPC reboot. PR851599: This issue has been resolved.
  • When committing a firewall filter with a "then decapsulate" action, the router might throw the following errors Feb 19 11:20:59 user@host dfwd[45123]: DFWD_FW_PGM_READ_ERR: Read of segment 0/0 in filter 2 failed: Unknown error: 0 Feb 19 11:21:01 user@host dfwd[45123]: DFWD_CONFIG_WRITE_FAILED: Failed to write firewall filter configuration for FILTER idx=2 owned by CLI. Error: Message too long This issue happens on an MX Series router that has at least one I-chip board (MX with DPC). This happens because the firewall daemon fails to properly update the Packet Forwarding Engine firewall configuration. PR857708: This issue has been resolved.
  • In T4000 platforms with ES-FPC, for IPv6 firewall filters with match conditions on address prefixes longer than 64 bits, in some corner cases, the filter might not be correctly evaluated and packet loss might occur. PR879829: This issue has been resolved.

General Routing

  • Prior to this change, the L2TP sessions with cos/ firewall attachments fail to come up when the L2TP Access Concentrator (LAC) is reachable over a unilist nexthop. PR660208: This issue has been resolved.
  • The 'RL-dropped' lines of "> show interfaces queue" will be missing when the PIC is bounced. PR749283: This issue has been resolved.
  • If dynamic profile versioning is configured and In-service software upgrade (ISSU) is performed from 11.4x27.35 (GA build) to 11.4x27.38(Nov-2012), exiting subscribers might either lose traffic or might get terminated. PR817018: This issue has been resolved.
  • VPLS traffic gets flooded back over the ingress interface on the local PE as the split-horizon gets disabled upon interface flap. PR818926: This issue has been resolved.
  • In a race condition where multiple interrupts are asserted, timer tick might not get well handled and remain asserted. This caused panic and core. PR828496: This issue has been resolved.
  • RPD on the backup Routing Engine might crash when it receives a malformed message from the master. This can occur at high scale with nonstop active routing enabled when a large flood of updates are being sent to the backup. There is no workaround to avoid the problem, but it is rare and backup RPD will restart and the system will recover without intervention. PR830057: This issue has been resolved.
  • Since the AC Power System on the MX2020 is a N+N feed redundant and N+1 PSM redundant, there are two separate input stages per PSM, each connected to one of the two different/redundant feeds. However, only one stage is active at a time. This means, the other input stage (unused input stage) might be bad and system will not know about it till it tries to switch to it in case of a feed failure. This is a pretty bad corner case and needs to be addressed. The way to work around this problem is by testing both stages when the power supply is first powered on. This test is done by the system software and an alarm is raised if any feed failure is detected. PR832434: This issue has been resolved.
  • Memory leak is observed in authd process, with a churn of 1000 subscribers over 3 min period. PR835204: This issue has been resolved.
  • It is possible that RPD's higher priority tasks (HPTs) are scheduled to run such that lower priority tasks (LPTs) might not be able to complete until HPTs are completed. PR836197: This issue has been resolved.
  • Enabling PIM - Bidir feature (possibly pim rp with 224.0.0.0/4 group) and rpd restart triggers this issue is hit during regression test for PIM bidir. 2) HW type of chassis/linecard/Routing Engine. If it affects all, just say ?all?. =>all. 3) Suspected software feature combination. (If customer turns on feature X along with Y, they may hit, etc) =>PIM - Bidir feature (rp configured) and rpd restart is causing the issue. 4) Describe if any behavior/ change to existing function =>None. PR836629: This issue has been resolved.
  • After a Routing Engine switchover with graceful Routing Engine switchover enabled, and then deactivate and activate a routing-instance, 4xOC48 IQE PIC might reboot unexpectedly. This is caused by a problem in channel allocation for the 4xOC48 PIC logical interfaces in kernel. PR841822: This issue has been resolved.
  • LMNR Chipset type FPC generates a core file with copy-plp-all enabled when adding link to existing AE interface, which is part of downstream interface list of a multicast route. PR842046: This issue has been resolved.
  • When MX Series router running with DPC is upgraded by ISSU, some of interface might show incorrect input packet/byte count. And the incorrect count is also seen to the related interface MIB. The value will be a large number. Physical interface: xe-3/1/0, Enabled, Physical link is Up Interface index: 138, SNMP ifIndex: 5449, Generation: 141 Link-level type: Ethernet, MTU: 1514, LAN-PHY mode, Speed: 10Gbps, BPDU Error: None, Loopback: Local, Source filtering: Disabled, Flow control: Enabled Device flags : Present Running Loop-Detected Interface flags: SNMP-Traps Internal: 0x4000 Link flags : None CoS queues : 8 supported, 8 maximum usable queues Hold-times : Up 0 ms, Down 0 ms Current address: 00:24:dc:9c:7c:30, Hardware address: 00:24:dc:9c:7c:30 Last flapped : 2013-01-13 14:36:25 JST (02:07:52 ago) Statistics last cleared: Never Traffic statistics: Input bytes : 3867797326912475 0 bps Output bytes : 0 0 bps Input packets: 15108583308733 0 pps Output packets: 0 0 pps ~snip~ Logical interface xe-3/1/0.0 (Index 196614) (SNMP ifIndex 5450) (Generation 140) Flags: SNMP-Traps 0x4004000 Encapsulation: ENET2 Traffic statistics: Input bytes : 3867797326912475 Output bytes : 0 Input packets: 15108583308733 Output packets: 0 Local statistics: Input bytes : 0 Output bytes : 0 Input packets: 0 Output packets: 0 Transit statistics: Input bytes : 3867797326912475 0 bps Output bytes : 0 0 bps Input packets: 15108583308733 0 pps Output packets: 0 0 pps Protocol inet, MTU: 1500, Generation: 160, Route table: 0 Flags: Sendbcast-pkt-to-re Addresses, Flags: Is-Preferred Is-Primary Destination: 10.3.1/24, Local: 10.3.1.1, Broadcast: 10.3.1.255, Generation: 141 Protocol multiservice, MTU: Unlimited, Generation: 161, Route table: 0 Policer: Input: __default_arp_policer__ gladiolus:Desktop$ grep .5449 mib_value_after_issu.txt ifName.5449 = xe-3/1/0 ifInMulticastPkts.5449 = 0 ifInBroadcastPkts.5449 = 0 ifOutMulticastPkts.5449 = 0 ifOutBroadcastPkts.5449 = 0 ifHCInOctets.5449 = 3867797326912475 ifHCInUcastPkts.5449 = 0 ifHCInMulticastPkts.5449 = 0 ifHCInBroadcastPkts.5449 = 0 ifHCOutOctets.5449 = 0 ifHCOutUcastPkts.5449 = 0 ifHCOutMulticastPkts.5449 = 0 ifHCOutBroadcastPkts.5449 = 0 gladiolus:Desktop$ grep .5450 mib_value_after_issu.txt ifName.5450 = xe-3/1/0.0 ifInMulticastPkts.5450 = 0 ifInBroadcastPkts.5450 = 0 ifOutMulticastPkts.5450 = 0 ifOutBroadcastPkts.5450 = 0 ifHCInOctets.5450 = 3867797326912475 ifHCInUcastPkts.5450 = 15108583308733 ifHCInMulticastPkts.5450 = 0 ifHCInBroadcastPkts.5450 = 0 ifHCOutOctets.5450 = 0 ifHCOutUcastPkts.5450 = 0 ifHCOutMulticastPkts.5450 = 0 ifHCOutBroadcastPkts.5450 = 0. PR847106: This issue has been resolved.
  • It is possible for RPD core when the following conditions are met: - VRF with multipath knob configured - static routes with next-hops which are indirect type and needs further resolution - the numerically lowest (smallest IP) next-hop of indirect type becomes unreachable RPD core is NOT triggered in either of the following scenarios: - no multipath under VRF - if there is no static route entry - static route whose next-hops are indirect type requiring further resolution multipath under VRF is supported only for BGP configurations. multipath in other conditions are not supported, and a bug in this detection phase is fixed in this PR. PR847214: This issue has been resolved.
  • mlfr/mlppp interface are not reachable after restart FPC (primary MSPIC) followed by deactivate and activate R.I or GRES followed by deactivate and activate R.I. This is because link FPC does not have the interfaces programmed towards the bundle. PR847278: This issue has been resolved.
  • In certain graceful Routing Engine switchover scenarios, with IPv6 address configured on at least two interfaces, Solicited node multicast addresses (SNMA) and link local addresses with same prefix might be created on the two interfaces. There is a possibility that there could be inconsistency in the Next Hop database between Master and Backup Routing Engines. When the Backup becomes Master in these scenarios, it'll try to program the Packet Forwarding Engines with the bad Next Hop data. This might cause undesired forwarding behavior on the Packet Forwarding Engines. PR850625: This issue has been resolved.
  • Ptsp failed to append policy with multi-rules since 'msg over size limit'. PR852224: This issue has been resolved.
  • FPC or PIC connects to the Routing Engine Kernel for the first time when it comes up or reconnects during connection trip. After the connection is established with the Routing Engine, if FPC/PIC does not respond kernel for 300 seconds, a timer is triggered to disconnect the Routing Engine from FPC/PIC. In a particular race condition between kernel processing received data on the connection and the fired timer trying to close the connection, kernel crashes and creates a core file. FPC/PIC's slow response may be attributed to high traffic or a faulty hardware. Before kernel crash, the following logs could be seen: fpc3 LCHIP(3): 1 new Lin SIF ins eope errors fpc3 LIN(3): PIC HSR is not OK, LCHIP(3) <- PIC 3 HSR 1. PR853296: This issue has been resolved.
  • If routing-instance is popping the mpls label through vt tunnel interface and the egress interface MTU of the vrf needs fragmentation and the dont-fragment bit is set in the ipv4 header, the egress vrf interface might stop forwarding traffic. The following syslog message will be reported fpc4 LCHIP(3): 1 new errors in LSIF To recover from this condition you can either bring the interface down via disable knob or deactivate/activate the interface from the configuration. The following platforms are exposed to this condition: M320 (excluding E3 FPCs),T/TX systems (excluding ES FPCs and FPC Type 5). PR854806: This issue has been resolved.
  • In the T4000 Type 5 FPC platform, aperture management can lead to a collision between the sched tick timer and asic driver interrupt handlers, which will result in FPC crashes. PR857167: This issue has been resolved.
  • In a virtual chassis environment in the event power is loss on the Master virtual chassis the standby chassis has potential to experience slot resets during transition period. PR859717: This issue has been resolved.
  • BOOTP request packets might get dropped because of the DDOS protection feature in old MX Series router with MPCs/MICs. In this case, the bootp packets is coming with 1 byte option. So the length of bootp become 241 which is larger than 240. Then Packet Forwarding Engine will identify it not as BOOTP as per the current DDOS algorithm, and tries to parse it as DHCP. Since the packet lacks the options fields which need for DHCP, then pfe_nhdb_dhcpv4_msg_type() mark it as DHCPNOMSGTYPE. PR862206: This issue has been resolved.
  • When a prefix next-hop address resolution requires a recursive lookup, the next-hop might not be updated correctly after an egress interface is disabled. PR862989: This issue has been resolved.
  • When using BGP Flow Spec with rate-limit option, even though the value is in Bytes/second, the value being programmed is in bits/second. PR864496: This issue has been resolved.
  • Output of show subscribers physical-interface aex displays multiple AE links. PR864555: This issue has been resolved.
  • "set chassis fru-poweron-sequence .... " configuration is not supported for T4000 platform in Junos OS Release 12.3R2 . PR868035: This issue has been resolved.
  • On T Series platforms with ES-FPC equipped, while adding and deleting source-class usage (scu) or unicast Reverse path forwarding (uRPF) configuration, Jtree memory leak and the following error messages could be observed: fpc0 nh_jtree_fe_posthandler: RNH_TABLE 1 missing ext rnh. PR869651: This issue has been resolved.
  • In subscriber management environment, with scaling subscribers login (110K DHCP and 20K PPPoE), after restarting one of the line cards which has subscribers, autoconf process might crash and generate a core file due to memory corruption or memory double free. Only 11.4X27.45 is affected by this issue. PR870661: This issue has been resolved.
  • In a scenario with scale Routing Instances (RIs) configured, after deactivating/activating two RIs, routing protocol process (rpd) might try to free a specific pointer pointing to an incorrect structure that is actively in use. Then rpd process crashes and generate core files. PR870683: This issue has been resolved.
  • When configuration stanza: [protocols router-advertisement] starts as: ## ## inactive: protocols router-advertisement ## interface ge-0/0/1.1 { virtual-router-only; } Then perform the following actions: Step 1 - activate protocols router-advertisement Step 2 - deactivate protocols router-advertisement interface ge-0/0/1.1 Step 3 - set protocols router-advertisement interface ge-0/0/1.2 After issuing "commit check", there are no problems. But after issuing "commit", routing protocol process (rpd) crashes and generates core with following logs: rpd[1422]: RPD_RA_CFG_UNKNOWN_ACTION: Unknown configuration action 3 received. PR871359: This issue has been resolved.
  • Adding a routing-instance with "/" in its name will cause the router not to boot properly if logical-systems were previously configured. PR871392: This issue has been resolved.
  • Under high scale, expiration of a Kernelside reconnect timer would cause it to send a non-serviceable message to the Packet Forwarding Engine (asking the line cards to restart and resync since reconnect failed). Since there is no ack- to this Kernel message, Kernel thought it sent the message and untoggles the GRES flag. The Packet Forwarding Engine wasn't expecting anything so it continued along. The EFFECT: The system is permanently not ready for GRES... CLI GRES check will always report: [cmd] request chassis routing-engine master switch check Apr 14 19:03:13 [INFO ] warning: Standby Routing Engine is not ready for graceful switchover. PR873679: This issue has been resolved.
  • On MX Series routers with DPC (I-Chip based) type FPCs running a 11.4 (or newer) Junos OS release, disabling uRPF on a logical interface might result in another logical interface on the router to drop all incoming packets. This problem happens only when the following conditions are met concurrently: a) 2 different logical interfaces share the same lookup index b) both logical interfaces have uRPF enabled c) these 2 different logical interfaces belong to 2 different FPCs d) at least one of the logical interfaces belongs to a DPC (ICHIP based) type FPC The lookup index is calculated by taking the lower 16 bits of the logical interface index (also called the IFL index). In other words lookup index = IFL index MOD 65536 . It is normal, valid and expected to have logical interfaces which share the same lookup index. The problem described in this PR is _not_ the fact that the lookup indexes are the same. Here is an example of 2 different logical interfaces on 2 different FPCs which share the same lookup index: Interface ge-0/1/0.945 has an IFL index of 1774 and a lookup index 1774: user@router-re1> show interfaces ge-0/1/0.945 Logical interface ge-0/1/0.945 (Index 1774) (SNMP ifIndex 1635) ^^^^^^^^^^ Flags: Device-Down SNMP-Traps 0x4000 VLAN-Tag [ 0x8100.945 ] Encapsulation: ENET2 Input packets : 0 Output packets: 0 Protocol inet, MTU: 4462 Flags: Sendbcast-pkt-to-re, uRPF, uRPF-loose Addresses, Flags: Dest-route-down Is-Preferred Is-Primary Destination: 52.3.168.216/29, Local: 52.3.168.217, Broadcast: 52.3.168.223 Protocol multiservice, MTU: Unlimited And interface xe-2/2/0.0 has an IFL index of 198382 and a lookup index of 198382 MOD 65536 = 1774: user@router-re1> show interfaces xe-2/2/0.0 Logical interface xe-2/2/0.0 (Index 198382) (SNMP ifIndex 698) ^^^^^^^^^^^^ Flags: SNMP-Traps 0x4004000 Encapsulation: ENET2 Input packets : 381 Output packets: 376 Protocol inet, MTU: 1500 Flags: Sendbcast-pkt-to-re, uRPF, uRPF-loose Addresses, Flags: Is-Preferred Is-Primary Destination: 155.154.153.0/30, Local: 155.154.153.1, Broadcast: 155.154.153.3 Protocol multiservice, MTU: Unlimited In the example above if uRPF is disabled on ge-0/1/0.945 then xe-2/2/0.0 will start dropping all incoming packets due to RPF failure. When this condition occurs the only way to recover is to disable, commit and re-enable uRPF on the broken interface. When this is done the following error messages are generated: Apr 15 16:02:53 router-re1 fpc2 rt_iff_generic_topo_handler: jtree error Not found for disconnect on iff-post-src Apr 15 16:02:54 router-re1 fpc2 RT(rt_rpf_jtree_drt_remove_ifl): Unable to remove logical interface 198382 from drt(4) Apr 15 16:02:54 router-re1 fpc2 RT(rt_rpf_jtree_drt_remove_ifl): Unable to remove logical interfacel 198382 from loose(7). PR873709: This issue has been resolved.
  • The default setting for the sysctl "net.pfe.relayg_merge_enabled" is 0 (off), this results in a support limit of 16 line-cards within the VC. Even with the group merge disabled, line-cards may have been grouped at system start-up only presenting an issue after they restart. PR874791: This issue has been resolved.

High Availability and Resiliency

  • The backup Routing Engine sends Arp 128.0.0.6 to the Packet Forwarding Engine, then they are counted as "unknown" on show pfe statistics traffic. PR830661: This issue has been resolved.
  • This issue is seen on IQ2 PICs during ISSU on TX platform. When upgrading to 12.3R2 from releases prior to 12.3R2 through ISSU, IQ2 PICs will report error. This error is due to IQ2 PICs not able to download image during ISSU. PR855661: This issue has been resolved.

Infrastructure

  • The root cause of the problem was IFADDR change in VRRP context was not replicated to GRES backup. PR790485: This issue has been resolved.
  • Kernel fails to generate ICMP ttl expired when IP packet len is a multiple of 256. PR829567: This issue has been resolved.
  • Aggregate Bundle interface with IPV6 Interface stuck in Tentative state. Trigger was deactivation/activation of ae-interface. PR844177: This issue has been resolved.
  • Delay in bringing online an FPC after it is inserted into the chassis. PR853304: This issue has been resolved.
  • TCP is mistakenly enabling re-transmit timer for pure ACK's which is causing the FPC to reboot. PR858489: This issue has been resolved.
  • With nonstop active routing (NSR) enabled, while performing graceful Routing Engine switchover, the Junos OS fails to restore BGP peers' TCP connections on the new master Routing Engine's replicated socket due to it is not able to find the BGP peer address's route, causing BGP peers to flap with following logs: /kernel: jsr_sdrl_merge: PSRM merge failed 65 rpd[xx]: RPD_BGP_NEIGHBOR_STATE_CHANGED: BGP peer a.b.c.d (Internal AS X) changed state from Established to Idle (event TcpSocketReplicationError). PR862796: This issue has been resolved.
  • When a sonet interface with PPP encapsulation is used as forwarding next hop for the IPv6 remote router loopback address on IPv6 BGP sessions, if the sonet link is down, the IPv6 BGP session might flap at same time although there is valid route via other interface. PR863462: This issue has been resolved.
  • After enabling firewall filter of IPv6 on Aggregated Ethernet (AE) interface to block Micro BFD Packets (Dst Port 6784), kernel crashes continually on Master and the backup Routing Engine due to double free of memory. PR864112: This issue has been resolved.
  • IPv6 Neighbor discovery(ND) failed after multiple GRES. Nexthop getting stuck in hold state forever. We also see that the neighbor state is in NO_STATE and it is on ND timer queue. In this condition, on ND timer expiry it never sends neighbor solicitation (NS) out and it never transitions to known ND states. PR864133: This issue has been resolved.

Interfaces and Chassis

  • When MAC address filters are configured on an AE, MAC filters might not be programmed on the child link of the AE if and only if the following sequence of events occur: AE is disabled via a configuration change, a graceful Routing Engine switchover occurred and AE is subsequently enabled on the new master Routing Engine. PR561106: This issue has been resolved.
  • There can be a mismatch between the ifIndex value on IF-MIB-ifName and the ifIndex value on SONET-APS-MIB-apsMapGroupName and apsMapEntry. PR771877: This issue has been resolved.
  • This issue is specific to the M120 hardware since there are two independent FRU's from where the PIC needs to be detached/attached. This IPC messages goes out-of-order due to the additional control-plane messages related to routing-change as a result of PIC restart which happens in this case due to the buffer configuration change. When PIC needs to be detached and at the same time there are still a lot of protocol information which should be process as well, the detached messages will NOT be able to be delivered in time. After PIC restarts it request to be attached again but obviously this action failed because from other FRU?s perspective the PIC has NOT been detached at all. PR773081: This issue has been resolved.
  • With LSQ interface, the MLPPP fragments cannot use the egress queue 4 to 7 on the MLPPP member links. There is no workaround. PR805307: This issue has been resolved.
  • Incorrect Detection timestamp in show chassis fabric reachability. PR811846: This issue has been resolved.
  • Faulty SCG causes continuous interrupts to HCFPC making its CPU Utilization 100% and unusable for any service. As a fix the monitoring mode for the SCG is changed to polling status of SCG device rather then interrupts based awake and monitoring system. PR827489: This issue has been resolved.
  • In Integrated Routing and Bridging (IRB) interface over Aggregate Ethernet (AE) interface scenario, if there is an MAC Move event or an L2 IFL change event with IRB, the Junos OS will remove the IRB nexthops on the backup Routing Engine and Packet Forwarding Engines first and then remove it from the master Routing Engine. During this phase, if an logical interface change event of the underlying AE interface occurs, the Junos OS might try to access a stale pointer which was freed already and cause memory corruption. In some conditions, the memory corruption occurs in kernel, hence cause kernel crash and generate a core file. PR829093: This issue has been resolved.
  • The kernel "devbuf" memory leaks when fxp0 interface is in down state (admin up). PR829521: This issue has been resolved.
  • A request(like snmp query) for collecting input ipv6 stats of ae logical interface on abc chipset is not working properly. PR831811: This issue has been resolved.
  • Removing IP address on ATM interface after adding another IP address from the common subnet can lead to a race condition. New IP address configured on the interface still refering to shared broadcast-nexthop. Then when TCP/IP access this broadcast-nexthop kernel panic might happen. PR833015: This issue has been resolved.
  • When packet has to be forwarded over NH topology unilist->indirect-indexed and when the packet size is greater than egress interface MTU w/ DF set, then we might log the following message and not send the message back to source indicating "frag needed and DF set". fpc0 NH: Can not find logical interface for nh 1048590 fpc0 NH(nh_get_mtu_iff) : get unilist mtu failed. PR844987: This issue has been resolved.
  • In a scenario of PPP sessions over L2TP tunnels, on L2TP network server (LNS), if authentication is none or if authentication is enabled but radius does not return any Framed-IP-Address/Framed-Pool, jpppd process is not setting the IP address key of subscriber to "255.255.255.254" thereby resulting in address allocation failure in authd process. Then the L2TP tunnels can not be established, hence subscribers can not login. When issue happens, the following logs of authd process could be seen: client type jpppd client type REQUESTING: OldStyle 0 OldStyleFilled 0 hint null network null client pool name. PR849191: This issue has been resolved.
  • Whenever tunnel interface -pe/-pd got created using the MS-DPC instead of the MPC, it was not able to process register messages. Because of MPC and MS-DPC have different multicast architecture and they are incompatible if chassis is configured in "enhanced-ip" mode, this issue will be seen. Necessary changes have been made to code so that these interfaces will not be created on MS-DPC. PR853995: This issue has been resolved.
  • SDG : After rebooting both the Routing Engines together, the FPCs and MS-DPCs might come online, go offline (with "Chassis connection dropped" and "Chassis Manager terminated" error messages) and come back online again automatically. This issue is seen only when both Routing Engines are rebooting at once. There is exactly one additional reboot of the FPCs when this happens, and the FPCs come back up online, and system stabilized by itself within 2 to 3 additional minutes. PR854519: This issue has been resolved.
  • In certain topology set up such as multiple trunks are used on a PE with P and the CE-PE interface is MLFR, and enhanced-ip and MS-DPC route-localization are configured, if the active trunk FPC is offlined, VRF traffic from PE towards CE using the mlfr interface might get blackholed. PR854623: This issue has been resolved.
  • Multilink Frame-relay (MLFR) stuck in ready state after restarting FPC and then graceful Routing Engine switchover (some of the MLFR bundles will show "ready" although the interfaces are in up/up state which causes data loss). PR857648: This issue has been resolved.
  • The backup Routing Engine might log the following often in chassisd: Feb 17 12:40:01 CB:1 need not to sync information Feb 17 12:40:21 CB:1 need not to sync information Feb 17 12:40:41 CB:1 need not to sync information Feb 17 12:41:01 CB:1 need not to sync information. This is a harmless message that can be ignored. PR857698: This issue has been resolved.
  • In PPPoE subscriber management environment, PPPoE deamon might crash and generate a core file in following two scenarios: 1 - Firewall Filter/Policer is not configured on Broadband Remote Access Server (BRAS) side, and AAA pushes the filter name in "Ingress Policy Name/Egress Policy Name" which will expire the lockout timer waiting to create required dynamic interface, and eventually causes pppoed process crash. 2 - When IPv6 only capable modem is trying to connect and the configuration does not contain IPv6 dynamic configuration; i.e. under PPPoE dynamic profile/family inet6 stanza; PPPoE dynamic profile/protocols/router-advertisement, this will again expires lockout timer waiting for dynamic interface creation, which crashes pppoed process. PR859000: This issue has been resolved.
  • Interface hold-time-down is not working properly for PIC type 10x10GE(LAN/WAN) SFPP. PR859102: This issue has been resolved.
  • Enables maximum-links CLI knob which specifies the maximum number of links in an aggregated ethernet bundle. This can take a value of 16, 32 or 64 depending on the platform. PR860152: This issue has been resolved.
  • ISSU does not support VRRP. PR862052: This issue has been resolved.
  • MX Series router is sending RADIUS Acct-Start, in spite of the fact that IPCP/IPv6CP is not established. PR867084: This issue has been resolved.
  • The chassisd crashes when enable route-localization with MPC2E. PR872500: This issue has been resolved.
  • If both "startup-silent-period" and "delegate-processing" are configured under protocols vrrp, both vrrp routers keep backup-backup state until "startup-silent-period" expires when trying to revert. PR873488: This issue has been resolved.

Layer 2 Ethernet Services

  • In the rare case of one GRES is performed after another GRES, without logging out and logging in subscribers, some ipv4 access routes will not reinstalled. This will result in traffic loss for the affected dhcp v4 subscribers. PR808932: This issue has been resolved.
  • DHCPv6 fails for clients using DUID type 2 (Vendor-assigned unique ID), the software was using the DUID to extract MAC address information. This behavior is fixed and tested. PR838404: This issue has been resolved.
  • MXVC-DHCP bindings stuck in a "RELEASE(RELAY_STATE_WAIT_AUTH_REQ_RELEASE" state. PR850187: This issue has been resolved.
  • For MXVC, the derivation of the dhcp server-id has changed from using hardware serial number to lacp mac addr. The reason is that the lacp mac address is guaranteed to be reflected across the chassis so upon GRES, the same dhcp server id can be built. However, upon ISSU, the old software will derive server-id from hardware serial number and the new software will derive it from lacp mac address and they will not match. After the ISSU, DHCP packets may be dropped by a dhcp server because the serverid in the client packet will not match that of the server. This will only happen when transition to the new method of building the serverid. Once that has happened, all future ISSU should work as before. PR853329: This issue has been resolved.
  • In DHCP subscriber management environment, while DHCP subscribers login, in rare conditions, system calls of these subscribers fail, due to only on success does system free the memory, resulting in a memory leak for the jdhcpd process. If memory usage of jdhcpd process goes to its limit, no new DHCP subscribers can login. When issue happens, high weighted CPU usage of jdhcpd process and following logs could be observed. /kernel: %KERN-5: Process (31403,jdhcpd) has exceeded 85% of RLIMIT_DATA: used 2825132 KB Max 3145728 KB jdhcpd: %USER-3-DH_SVC_RTSOCK_FAILURE: Error with rtsock: rtslib: ERROR Failed to allocate new block of size 16384 jdhcpd: %USER-3-DH_SVC_RTSOCK_FAILURE: Error with rtsock: rtslib: ERROR Failed to allocate new block of size 16384 jdhcpd: %USER-3-DH_SVC_RTSOCK_FAILURE: Error with rtsock: rtslib: ERROR Allocation Failure for (16384) bytes authd[1822]: %DAEMON-3: ../../../../../src/junos/usr.sbin/authd/plugin/radius/authd_plugin_radius_m odule.cc:1090 Failed to get SDB snapshot for session-id:3549005. PR856024: This issue has been resolved.
  • DHCP relay functionality over IRB performing dhcp v4 relay functionality, and configured with both inet and inet6 address families. The removal of the IPv6 address family configuration from the IRB can cause the IPv4 dhcp relay functionality on that IRB to break. This happens regardless of whether the 'family inet6' is configured directly under the IRB or applied through a 'apply-group' configuration. In versions that do not have the fix for this PR, the workarounds to get the dhcp relay functionality working again over the IRB are *either* of the following:
    • Deactivate/activate the IRB configuration.
    • Restart the dhcp process using the restart dhcp-service command.

    Note: This workaround has to be applied everytime any configuration change (as explained in the trigger) is applied that could potentially get the dhcp-relay functionality to break.

    PR870543: This issue has been resolved.

  • When IPv6 is configured on integrated routing & bridging(IRB) interfaces which has AE interfaces as child links, after GRES was enabled and one child link failure or removal, the kernel get crashed. PR878470: This issue has been resolved.

MPLS

  • In a RSVP P2MP crossover/pass-through scenario, more than one sub-LSP can use the same PHOP and NHOP. If link protection is enabled in the above mentioned scenario, when a 'primary link up' event immediately followed by a Path Tear message, disassociation of the routes/nexthops are sequential in nature. When the routes/nexthops dissociation is in progress if a sub-LSP receives a path tear/PSB delete will lead to this core. PR739375: This issue has been resolved.
  • The cleanup procedures may leave transient inconsistent references when the interface address of an MPLS enabled GRE or IPIP tunnel is being deleted or the action taken implies an internal reconfiguration of the interface address (for example MTU change). During this period, if these references are being reused by a particular task, the kernel may report an invalid memory access and restart. PR844790: This issue has been resolved.
  • The routing protocol process (rpd) might leak memory when there are MPLS LSP changes, the memory leak could eventually cause rpd process to crash. PR847354: This issue has been resolved.
  • There appears an unsupported feature warning missing for mLDP+NSR while doing ISSU. PR849178: This issue has been resolved.
  • RPD generates a core file on the backup Routing Engine with rsvp_mirror_telink_attempt_resolve. PR859602: This issue has been resolved.
  • ASBR might not rewrite EXP correctly for egress MPLS packets on the Inter-AS link for the eBGP-LU LSP if the eBGP session is a multihop BGP session. PR868945: This issue has been resolved.
  • In a scenario when scaled MPLS tag labels exist, configure and delete ospf overload configuration. After committing the configuration changes, routing protocol process (rpd) might crash and generate a core file due to system tries to delete an already freed MPLS tag label Element. PR878443: This issue has been resolved.

Network Management and Monitoring

  • When snmp unknown PDUs are received, the appropriate counter in (show snmp statistics) is not incremented. PR865121: This issue has been resolved.
  • Polling an snmp oid that was excluded from the snmp view in configuration might trigger an increase in CPU load related to SNMP and RPD processes. PR866541: This issue has been resolved.

Platform and Infrastructure

  • XML tags for get-software-information output missing some elements of the new Junos OS service release naming convention. PR783653: This issue has been resolved.
  • Due to a bug in logical interface localization, a DPC restart/offline may cause a removal of legitimate CCC routes on other DPC's. This can also be triggered by removal of an unrelated family CCC logical unit. PR835216: This issue has been resolved.
  • When a junoscript get-configuration RPC query, by default the query is done on candidate DB, a MGD process is spawned to handle this request. Now at the same time via another session if the configuration is deleted it is possible for the above spawned MGD process performing the junoscript query to crash. MGD process crashes while accessing a NULL parent which contained an object previously which was deleted. The fix addresses this by not exporting the object which has no parent. PR844795: This issue has been resolved.
  • On a device that is in the configuration private mode, when you attempt to deactivate a previously defined VLAN members list and then commit the change, the mgd process creates a core file. PR855990: This issue has been resolved.
  • Packet dropped with reject route is currently subjected to loopback filter processing on MPCs, as a result the packet dropped by a reject route might be seen in the output of "show firewall log". This behavior will be changed so that this traffic is no longer subjected to loopback filter processing to bring it in line with other line cards. PR858511: This issue has been resolved.
  • Once ingress queuing is enabled on MX Series routers L2 control traffic had no default classifier assigned and used best-effort queue. Under queue congestion, L2 control traffic like IS-IS might get behind and trigger an adjacency flap. L3 control traffic and MPLS control traffic are not affected. PR858882: This issue has been resolved.
  • In IPFIX context: 1. In an IPv6 single stack environment, when exporting Data and Template records for family IPv6, the Template records sequence number is not initialized and is always == 0 for all records. This is because the Template sequence numbers are blindly copied from family IPv4 and if this is not configured for IPFIX, then the Sequence Number is always 0. 2. In an IPv4 + IPv6 dual stack environment, since the Template records sequence numbers will be identical for both families, we will get Data and Template records sequence numbers being interleaved when exported. This could confuse the Flow Collector and mislead it into reporting random missing flows. PR859169: This issue has been resolved.
  • On MX Series routers with MPCs/MICs, error message "LUCHIP(x) has no shadow data for IDMEM[0x00xxxx]" might be seen. PR859424: This issue has been resolved.
  • In some corner cases SPMB can stuck in READY state. Restarting the SPMB does not help to recover from the problem state. PR866127: This issue has been resolved.
  • The mgd crashed with core-dump after executing show configiration | display rfc5952. PR869650: This issue has been resolved.
  • On MX Series based line cards, after repeated firewall filter delete/change operations (which might occur with interface flaps, e.g.), memory might leak which can cause ASIC memory exhaustion, causing MX Series based line cards to crash and generate a core file. PR875276: This issue has been resolved.
  • When we are deleting a configuration hierarchy which has no groups applied, the corresponding group object hierarchy is also marked as changed in commit script view. PR878940: This issue has been resolved.
  • While configuring a filter with a generic prefix followed by specific one in different terms might lead to incorrect match, this might lead to packet drop. PR886955: This issue has been resolved.

Routing Policy and Firewall Filters

  • If RPF and/or SCU is enabled then any change to an ingress firewall table filter will trigger RPF/SCU reconfiguration for every prefix in the routing table. This may cause transient high CPU utilization on the fpc which may result in SNMP stats request being time out. PR777082: This issue has been resolved.

Routing Protocols

  • If you have configured PIM nonstop active routing (NSR), a core file might be created on an upstream router because of high churn in unicast routes or a continuous clearing of PIM join-distribution in the downstream router. To prevent this possibility, disable NSR for PIM. PR707900: This issue has been resolved.
  • On a device that is running Protocol Independent Multicast (PIM) and with nonstop active routing (NSR) enabled on the device, if a PIM corresponding interface flaps continuously, a PIM thread might attempt to free a pointer that has already been freed, causing the routing protocol process (rpd) to crash and create a core file. PR801104: This issue has been resolved.
  • With OSPFv3, PIMv6 or LDP configured, the periodic packet management daemon (ppmd) takes responsibility for these protocols' adjacencies. In a rare condition, kernel might send an invalid packet with a null destination in the message header to ppmd process, causing ppmd process to crash and generate a core file. PR802231: This issue has been resolved.
  • In subscriber management environment, routing protocol process (rpd) might crash and create a core file due to snmpwalk fails at mplsL3VpnVrfRteInetCidrDestType when a subscriber access-internal route in a VRF has a datalink nexthop (such as when DHCP subscriber connects into a VRF). When issue happens, the following behaviors could be observed: user@router> show snmp mib walk ascii mplsL3VpnVrfRteInetCidr | no-more Request failed: Could not resolve 'mplsL3VpnVrfRteInetCidr' to an OID user@router> show snmp mib walk ascii mplsL3VpnVrfRteInetCidrDest | no-more Request failed: General error. PR840323: This issue has been resolved.
  • In IS-IS scenario, with graceful Routing Engine switchover and nonstop active routing (NSR) enabled, after Routing Engine switchover, in very rare case, routing protocol process (rpd) might crash and generate a core file on new master (old backup) Routing Engine . This crash happens upon IS-IS lsp generation due to memory corruption. PR841558: This issue has been resolved.
  • Under certain conditions moving a link that has BFD clients can cause stale BFD entry for the old link. PR846981: This issue has been resolved.
  • The upstream interface of multicast rpf not matching multicast route in Inter-AS PIM. PR847370: This issue has been resolved.
  • In multicast environment with PIM configured, in RP-on-a-stick scenario (aka one-legged RP), if the rendezvous point (rp) receives multicast traffic but there are no receivers, RP's kernel will keep sending resolve requests to routing protocol process (rpd). These resolve requests might get stuck in resolve queue delaying other (S,G) resolves and thereby multicast traffic will be blackholed. PR851210: This issue has been resolved.
  • When an import-policy change rejects a BGP-route previously contributing to BGP-Multipath formation, the Peer Active-route-counters in "show bgp neighbor" may not get updated correctly. PR855857: This issue has been resolved.
  • If an invalid PIM-SSM multicast group is configured on the routing device, then when you issue the "commit" or "commit check" command, a routing protocol process (rpd) core file is created. There is no traffic impact because the main rpd process spawns another rpd process to parse the corresponding configuration changes, and the new rpd process crashes and creates a core file. When this problem occurs, you might see the following messages: user@router# commit check error: Check-out pass for Routing protocols process (/usr/sbin/rpd) dumped core(0x86) error: configuration check-out failed user@router# commit error: Check-out pass for Routing protocols process (/usr/sbin/rpd) dumped core(0x86) error: configuration check-out failed. PR856925: This issue has been resolved.
  • Routing protocol process (rpd) crashes and generates core files when non-bgp routes (e.g. static route) being advertised as add-path route. PR859307: This issue has been resolved.
  • RPD generates a core file. PR863148: This issue has been resolved.
  • Multicast packets coming with source address as 0.0.0.0, might cause the RPD to crash. PR866800: This issue has been resolved.
  • In VPLS multi-homing environment, with same route-distinguisher configured for the VPLS primary PE and backup PE, routing protocol process (rpd) may crash and generate a core file in each of following two scenarios: 1 - On VPLS backup PE, enable "advertise-external" knob, then rpd process crashes and generates a core file on backup PE. 2 - On VPLS primary PE, enable "advertise-external" knob, after disabling the VPLS interface, rpd process crashes and generates a core file on primary PE. When issue happens, the following behavior could be obseved: user@router> show bgp neighbor error: the routing subsystem is not running user@router> show vpls connections error: the routing subsystem is not running. PR869013: This issue has been resolved.
  • In Release 12.1 MPLS OAM programs BFD, it does not provide the source address (no change in behavior). In BFD before programming PPMD it queries kernel for the source address matching the prefix of the destination address on a interface. BFD programs PPMD with this source address. PPMD will construct BFD packet with BFD provided source address in the IP header. PR870421: This issue has been resolved.
  • In inter-AS Option-B L2VPN scenario, the ASBR might create a L2VPN cloned transit route incorrectly due to a cloned route is a Juniper specific mpls.0 route which the Junos OS creates on the penultimate hop router. Then in a rare case, routing protocol process (rpd) tries to delete the L2VPN cloned transit route (in mpls.0 table) multiple times. After this, routing protocol process (rpd) crashes and generates a core file. PR878437: This issue has been resolved.

Services Applications

  • Extensive CLI requests associated with l2tp (show services l2tp < switch >) might result in l2tpd process crash. PR755948: This issue has been resolved.
  • Only 94 GRE(plain) sessions are in Established state after chassisd restart. PR801931: This issue has been resolved.
  • Memory leak in key management daemon (kmd) causes some IPSec VPN tunnels to be dropped and don't get re-negotiated for over 10 minutes. Before issue happens, the following logs could be observed: /kernel: Process (1466,kmd) attempted to exceed RLIMIT_DATA: attempted 131080 KB Max 131072 KB /kernel: Process (1466,kmd) has exceeded 85% of RLIMIT_DATA: used 132008 KB Max 131072 KB. PR814156: This issue has been resolved.
  • The jnxNatSrcNumPortInuse counter is not refreshing when polling the jnxNatSrcNumPortInuse OID via SNMP after RSP switchover. PR829778: This issue has been resolved.
  • MAC Flow-control asserted and MS-DPC reboot is needed. PR835341: This issue has been resolved.
  • 1) corrected the log to state 4 bundles per tunnel to have been exhaused. 2) change the log level from INFO to DEBUG 3) Add more context to previous log: New IPSec SA install time 1356027092 is less than old IPSec SA install time 1356027092 new log = Tunnel:< tunnel-id > < Local_gw, Remote_gw >: < local-gw-ip-addr, remote-gw-ip-addr > New IPSec SA install time 1356027092 is less than old IPSec SA install time 1356027092 4) added more context to previous log: SA to be deleted with index 3 is not present new log = SA to be deleted with index 3 is not present < Local_gw, Remote_gw >: < local-gw-ip-addr, remote-gw-ip-addr > 5) added a counter to show the number of times each of these messages occur per tunnel. PR843172: This issue has been resolved.
  • Syslog is not sent to remote host when rsp interface is used. PR849995: This issue has been resolved.
  • When allocate the memory from shared memory for bitmaps used in port blocks , the Junos OS requests as many bytes as the size of the block. If customers assign like 10K block size for deterministic nat or PBA then the Junos OS allocates 10K bytes for that bitmap. However, it only needs 10K/8 bytes as one byte can represent 8 ports. These huge allocations are leading to memory depletion when many source addresses are behind the NAT, and port blocks are big. PR851724: This issue has been resolved.
  • The jnxNatSrcNumSessions SNMP OID is broken in 11.4R6-S1 release. PR851989: This issue has been resolved.
  • Defining an application with destination-port range starting at 0 can cause TCP handshake to fail through NAT. As a workaround, specify the application with destination-port range starting at 1 instead of 0. PR854645: This issue has been resolved.
  • The number of terms per NAT rule cannot exceed 200 for the inline-service si- interface. This constraint check is not applicable for other type of service interfaces like sp-, AMS and ms- etc. Following error message will be displayed when there are more than 200 terms per NAT rule: user@host# commit [edit services] 'service-set ss8' NAT rule rule_8 with more than 200 terms is disallowed for si-0/0/0.8 error: configuration check-out failed. PR855683: This issue has been resolved.
  • MS-DPC might crash in certain scenarios when using CGNAT PBA and junos-rsh, junos-rlogin, junos-rpc-services-udp and junos-rpc-services-tcp ALGs (either one) in combination with EIM. PR862756: This issue has been resolved.
  • When DHCP subscribers log in and radius hands down flow-tap variables the following errors are seen in the log:"/kernel: rts_gencfg_dependency_ifstate(): dependency type (2) is not supported." PR864444: This issue has been resolved.
  • Service PIC might crash in corner cases when SIP ALG media flows are deleted. PR871638: This issue has been resolved.
  • The issue is seen because of receiving malformed LCP configure-request packet with bad option length from PPP client. In this case when router tries to generate configure-nak it crashed. As a fix, check is added to discard such malformed configure-request packets. PR872289: This issue has been resolved.

Subscriber Access Management

  • Subnet mask option is not returned to DHCP client when framed-ip-address is used with dhcp-local-server. PR851589: This issue has been resolved.
  • Some requests internally sent to AUTHD process experience a timeout state which may cause the subscribers to remain as either release or terminated. PR853239: This issue has been resolved.
  • Authd core experienced when multiple DHCP subscriber connection attempts require SRC for subscriber authentication. PR862037: This issue has been resolved.
  • Fixed the misbehavior of 'accounting-stop-on-failure' configuration knob. PR865305: This issue has been resolved.
  • PPPoE subscribers do not always get disconnected after the client-session-timeout expires. PR869559: This issue has been resolved.

User Interface and Configuration

  • The blank set command while indicating the configuration by show | display inheritance | display set. PR816722: This issue has been resolved.
  • If a commit sync error occurs for a commit performed in "edit private" mode and later it is followed by another commit in global mode (without private or exclusive mode), the configuration file may remain unzipped after the global commit is complete. PR823555: This issue has been resolved.

VPNs

  • Deleted logical interfaces may not be freed due to references in MVPN. PR851265: This issue has been resolved.
  • When "multicast omit-wildcard-address" is configured on a route-reflector for the MVPN address families, Leaf-AD route NLRIs are not reflected correctly in the newer, standardized format. The Leaf-AD routes transmitted from the RR in the new format will have invalid Leaf-IP fields in the NLRI set to 0.0.0.0. As a result, ingress PEs may fail to properly identify all egress PEs and thus fail to update provider-tunnel state to deliver traffic to those egress PEs. PR854096: This issue has been resolved.
  • While l2circuit/l2vpn is not configured, if user requests for PW object info through mib, L2circuit/l2vpn is creating invalid job, which can lead to rpd crash. The fix exists in: 12.3R3, 11.4R8, 13.1R2, 12.2R5, 12.1R7 and later releases. PR854416: This issue has been resolved.
  • When the egress PEs are on a NGMVPN, which then leads on to the assert being silently ignored when dual forwarders are setup over the PE-CE segment. Eventually duplicate traffic being delivered by PE routers onto the ethernet where receiver is connected. PR862586: This issue has been resolved.
  • RPD can crash when a cmcast leave is received after disabling the internet-multicast. PR864304: This issue has been resolved.
  • Sample topology: multicast +--+ CE_R +--+ PE_R +--MPLS core--+ PE_S +--+ C-BSR +--+ C-RP +--+multicast receiver source With the NG MVPN setup, when RP failed, there could be a delay on RP timeout between PE_S (multicast traffic ingress) and PE_R (multicast traffic egress). And suppose that PE_S removed RP from the PR list and PE_R still learned RP. Under the condition above, when RP came back and BSR informed RP info with generating bootstrap message, PE_R would advertise type 6 routes to PE_S across MPLS core via MPBGP. If a RP is learned on PE_S after PE_S receives the type 6 routes from the core, PE_S neither creates PIM (*,G) join nor sends the join to C-RP. PR866962: This issue has been resolved.
  • If a logical interface is taken out of VPLS or L2VPN Pseudowire Routing Instance and placed in protocol l2circuit, after the above configuration changes are done in one commit, routing protocol process (rpd) crashes and generates a core file. PR872631: This issue has been resolved.

Resolved Issues in Release 12.3R2

Class of Service

  • When rate limit is enabled and disabled on port cos scheduler configuration leaves rate limit configuration on queues in effect. This causes the rate limit feature in effect even after rate limit is removed. This PR addresses this issue in lieu with PR 843603. PR833431: This issue has been resolved.
  • Traffic-control-profile-remaining is not working for logical interface in interface-set. PR835933: This issue has been resolved.
  • In PPPoE/DHCP subscriber management environment, with "burst-size $junos-cos-shaping-rate-burst" configured in subscriber dynamic-profiles, while logging in/out subscribers, the class-of-service daemon (cosd) memory leak due to cosd process doesn't free up memory used for parsing busrt attributes of a traffic-control-profiles (tcp) guaranteed rate. The memory usage of cosd process can be monitored by following CLI command: user@router> show system processes extensive | match "PID | cosd" (Note: The "RES" field means "Current amount of resident memory, in kilobytes") PID USERNAME THR PRI NICE SIZE RES STATE TIME WCPU COMMAND 1326 root 1 96 0 14732K 4764K select 0:01 0.00% cosd PR846615: This issue has been resolved.
  • This seems to be hard to reproduce and noticed only once after GRES. When the cosd restarts (due to the GRES test you performed), cosd reconciles the configurations pushed to the Packet Forwarding Engine with config read from CLI and tries to reuse the object ID. In this case, it was trying to insert the same ID twice. PR848666: This issue has been resolved.
  • Commit throws an error "Invalid rewrite rule rule-name for logical interface <ifl-name>. Ifd <ifd-name> is not capable to rewrite inner vlan tag 802.1p bits" even though there is no rewrite configuration related to inner-vlan tag. PR849710: This issue has been resolved.
  • Configuring Classifiers under groups might result in Class-of-service daemon to core. Work-around is to avoid configuring Classifiers under groups. PR863109: This issue has been resolved.

Forwarding and Sampling

  • With more than four archive-sites configured under [system archival configuration archive-sites] hierarchy, after committing the configuration changes, pfed process crashes and generates a core file due to memory corruption or double free. The core files could be seen by executing the CLI command show system core-dumps. PR849465: This issue has been resolved.

General Routing

  • Prior to this change, the L2TP sessions with cos/ firewall attachments fail to come up when the L2TP Access Concentrator (LAC) is reachable over a unilist nexthop. PR660208: This issue has been resolved.
  • Reconfiguring a deleted interface with BFD sessions can take up to 20 minutes for the BFD sessions to initialize. PR786907: This issue has been resolved.
  • With l3vpn composite next-hops configured and 3 or more odd number of core uplinks every l3vpn route deletion will syslog the following error messages. [LOG: Err] JTREE: (jt_mem_free) size 0 for addr 1595452, seg 1, inst 0 [LOG: Emergency] Multiple Free :jt_mem_free There is no operational impact. An even number of core-uplinks will not trigger such error logs. PR786993: This issue has been resolved.
  • MPLS LDP traceroute does not work if you have a default route 0/0 pointing to discard on the Egress router with DPC cards. PR790935: This issue has been resolved.
  • On T1600-FPC4-ES, T640-FPC3, T640-FPC3-E and T640-FPC3-E2 platforms which have multiple Packet Forwarding Engines, with auto-bandwidth enabled on LSPs where CoS-based forwarding (CBF) is configured, auto-bandwidth might trigger minor changes on LSP nexthops. After this, flapping corresponding interface or any nexthop changes may result in FPC crash and create a core file. The core files can be seen by executing CLI command show system core-dumps. This issue will be seen with auto-bw configuration where there will continuous minor/major changes on LSP nexthops based on traffic conditions. When issue happens, the following logs could be seen: fpc3 PDP(pdp_free): %PFE-3: Invalid PDP 0x4e01d7d0 fpc3 PDP(pdp_free): %PFE-3: Error while removing PDP (0x4df4c068) fpc3 PDP(pdp_free): %PFE-3: Error while removing PDP (0x525b3f78) fpc3 PDP(pdp_free): %PFE-3: Invalid PDP 0x4de522b0' PR818021: This issue has been resolved.
  • icmp redirects are not disabled even after configuring no-redirects on irb interface. PR819722: This issue has been resolved.
  • When an MS-DPC PIC reboots due to a crash or manual intervention, it might get stuck in a booting loop if the MS-DPC up-time is more than 49 days and 17 hours. After 5 consecutive boot failures, the MS-DPC PIC will go offline automatically and gives the following error message: [ 15:21:22.344 LOG: Err] ICHIP(0): SPI4 Training failed while waiting for PLL to get locked, ichip_sra_spi4_rx_snk_init_status_clk [ 15:21:22.344 LOG: Err] CMSPC: I-Chip(0) SPI4 Rx Sink init status clock failed, cmsdpc_spi4_init [ 15:21:22.344 LOG: Err] CMX: I(0) ASIC SPI4 init failed [ 15:21:22.379 LOG: Err] Node for service control logical interface 68, is already present [ 15:21:23.207 LOG: Err] ASER0 SPI-4 XLR source core OOF did not go low in 20ms. [ 15:21:23.208 LOG: Err] ASER/XLR0 spi4 stop src train failed! [ 15:21:23.208 LOG: Err] ASER0 XLR SPI-4 sink core DPA incomplete in 20ms. [ 15:21:23.208 LOG: Err] ASER/XLR0 spi4 sink core init failed! [ 15:21:24.465 LOG: Err] ICHIP(0): SPI4 Stats Unexpected 2'b 11 Error, isra_spi4_parse_panic_errors [ 15:21:24.465 LOG: Err] ICHIP(0): SPI4 Tx Lost Sync Error, isra_spi4_parse_panic_errors In order to recover from this state the whole MS-DPC needs to be rebooted. PR828649: This issue has been resolved.
  • PPPoE sessions cannot be established as rpd is unable to read or access profile database during access-internal route creation via "dynamic-profile->routing-instances->routing-options->access-internal" stanza PR830779: This issue has been resolved.
  • An FPC may reboot when a live-core is requested and the /var partition does not have sufficient space to store the live-core. PR835047: This issue has been resolved.
  • On T4000 systems where the following conditions are met: - the "forwarding-options sampling input maximum-packet-length" knob is configured to a non-zero value - packets are sent to be sampled from a Type 5 FPC to an ES-Type FPC housing the Multiservices PIC used for sampling then an incorrect format of the notification header sent to the destination ES-Type FPC will trigger a packet loss in the packets sent to be sampled. The following message will be logged in the syslog on the destination FPC: [Jan 17 12:43:25.388 LOG: Err] SRCHIP(0): 1 Bad packets on p1 [Jan 17 12:43:25.389 LOG: Err] SRCHIP(0): 1 SONN errors on p1 The outcome is that the respective packets will be dropped and they will not be sampled. PR839696: This issue has been resolved.
  • When you configure tunnel interface in MXVC, the tunnel interface is set to harddown, unfortunately, there is no workaround at this point >High level problem description of the problem Problem: tunnel interface is set to harddown in MXVC >When does it occur When a tunnel interface is configured, it is always set to harddown >Is there a workaround, and if yes, what it is... Unfortunately, there is not workaround to bring this interface up. A fix is planned for R2. PR839784: This issue has been resolved.
  • When the transit traceroute packets with ttl=1 are received on the LSI interface, you may retrieve the Source Address from the LSI interface to reply ICMP. As LSI does not have any IFA, it will use first the IFA in routing-instance to reply. So Source Address used was the first IFA added in VPN routing-instance. As a workaround, if the incoming interface is LSI, then retrieve Source Address from the logical interface which is having the Destination IP Address. This will make sure we reply with Source Address from CE-facing the logical interface. PR839920: This issue has been resolved.
  • Dynamic arp or routing does not work when using ether-over-atn-llc in the new PIC. PR840159: This issue has been resolved.
  • mlfr/mlppp interface are not reachable after restart FPC (primary MSPIC) followed by deactivate and activate R.I or GRES followed by deactivate and activate R.I. This is because link FPC does not have the interfaces programmed towards the bundle PR847278: This issue has been resolved.
  • Maximum power required for SFBs is changed from 250W to 220W. Maximum power required for 172mm Fan Trays is increased from 1500W to 1700W. The power requirement for MX2010's upper fan trays is not changed. It is still 500W. With this change, the Reserved Power for critical FRUs (CB/Routing Engine, SFB and FanTrays) changes from 7000W to 7360W for MX2020 and from 6500W to 6660W for MX2010. PR848358: This issue has been resolved.
  • Distributed protocol adjacencies (LFM/BFD/etc) may experience a delay in keepalives transmission and/or processing due to a prolonged CPU usage on the FPC microkernel on T4000 Type 5-3D FPCs. The delay in keepalive transmission/processing may result in a mis-diagnosis of a link fault by the peer devices. The issue is seen several seconds after an Routing Engine mastership switch with NSR enabled and the fault condition will clear after a couple of minutes. PR849148: This issue has been resolved.
  • FPC/PICs usually have high response time when there are loaded with high traffic. Kernel cores when a PIC/FPC stops responding kernel after a new connection or a reconnection. PR853296: This issue has been resolved.
  • After configuring MX as ingress for RSVP LSP's, all the FPC's start throwing the error message "TOPO_FLAVOR_IFF_HW_OUT before family (1) : 0x0", this is a cosmetic issue with no impact to any protocol functionality. PR854499: This issue has been resolved.

High Availability and Resiliency

  • On TX Matrix routers with four LCCs and IQ2 PICs, in-service software upgrade (ISSU) from 12.3R1.7 to a newer release results in traffic loss and a FRU upgrade error. PR768502: This issue has been resolved.

Infrastructure

  • A kernel crash may occur on routers running 10.4 or higher (which does not have fix for this PR), with "targeted-broadcast" knob configured on a broadcast interface. If this knob is configured, MAC address will be learned for subnet broadcast IP (configured on that interface). When this ARP table entry gets timed out, it corrupts an internal data structure, leading to kernel crash. This MAC learning will happen with one of the following : 1. Mismatched IP subnet is configured on one of the connected devices 2. A malformed packet (ARP request to subnet broadcast IP) is received on that interface NOTE: - MAC address learned for the subnet broadcast IP can't be seen using "show arp" command. - This issue is platform independent. PR814507: This issue has been resolved.

Interfaces and Chassis

  • Under certain circumstances,MX80 may crash when using the command "request system snapshot". PR603468: This issue has been resolved.
  • Kernel can cache a high incorrect value for stats and is rejecting the correct subsequently stats coming from the PIC. The fix consists in checking if the difference of what is cached in kernel and what is reported by the PIC is less than an acceptable value. If the answer is not kernel does not gets stuck permanently and recovers while fetching stats next time. PR806015: This issue has been resolved.
  • "show interfaces redundancy" may display secondary as down upon following sequence: deactivate R.I.(that contains entire mfr logical interfaces)-->restart fpc(that holds secondary MS pic)--> activate the R.I. back PR816595: This issue has been resolved.
  • Warning message added is syslog when external sync is not supported. PR817049: This issue has been resolved.
  • Hash Key configuration not programmed in the Packet Forwarding Engine correctly after system reboot. PR818035: This issue has been resolved.
  • Prior to this PR, the speed of a GE interface capable of working at FE speeds was set to 'auto' in the Packet Forwarding Engine level. This causes a problem when manually setting the speed on the Routing Engine. Now the behavior is to set the speed to '1 g' in the Packet Forwarding Engine. For automatic speed detection the interface should be set to 'speed auto' in the configuration. PR821512: This issue has been resolved.
  • MX Series chassis-control interrupt storm may be falsely reported when a Field Replaceable Unit (FRU) is removed, inserted, or FPM button pushed. A FRU may not be recognized/booted, resulting in chassis operational failure. PR823969: This issue has been resolved.
  • IEEE 802.3 ah LFM stats counter "OAM current frame error event information" is not cleared correctly by CLI operation. PR827270: This issue has been resolved.
  • If per-unit-schedular is configured under a physical interface(ifd), and trying to delete this ifd and its sub-interfaces (logical interface) in one single commmit, ksyncd may core in the backup Routing Engine which will cause GRES malfunction. PR827772: This issue has been resolved.
  • Although physical interface is disabled, reseating 1GbE SFP on MPC/MIC restores its output optical power, hence the opposite router interface turns Up(Near-end interface is still down). Only 1g-SFP on MPC/MIC has the problem, but 1g-SFP on DPC/MX, EX Series and 10G-XFP on DPC/MX don't have the problem. When the sfp is reseated, then the sfp periodic is going ahead and enabling the laser irrespective of the fact that interface has been enabled or disabled. Driver needs to store the state for each sfp link and enable laser based on that. This software problem is fixed in 11.4R7, 12.1R6, 12.2R4, 12.3R2 and later release. PR836604: This issue has been resolved.
  • Configuring 100-Gigabit Ethernet Link Down Notification for Optics Options Alarm or Warning. The "optics-options" alarm/warning "low-light"; the syslog action was not taking effect on T1600 and T4k for 100 GE PICs. This was fixed as part of this PR. PR836709: This issue has been resolved.
  • The Logical Interfaces are marked with 0 (null) after deactivate system commit synchronize and deactivate chassis redundancy which result backup Routing Engine to core. PR840167: This issue has been resolved.
  • ERA events are not credited back by jpppoed. ERA has a purge timer of 10 minutes which reclaims stale events so new connections are allowed after the purge timer fires. In a high scaled scenario this can lead to slow PPPoE connections. PR842935: This issue has been resolved.
  • The device configuration daemon (dcd) may crash when a partial demux subinterface configuration is attempted to be committed. There is no impact to traffic forwarding but before the configuration can be committed, it must provide a valid 'underlying-interface' for the demux subinterface. PR852162: This issue has been resolved.

Layer 2 Ethernet Services

  • It can happen that when changing an interface framing from lan-phy (default) to wan-phy and back a few times, the interface doesn't show up any more in "show interfaces terse". PR836382: This issue has been resolved.
  • DHCPv6 relay terminates the client if DHCPv6-REPLY message from server contains status-code option. PR845365: This issue has been resolved.
  • In certain cases when MX Series is configured as DHCPv6 server and servicing DHCPv6 clients through LDRA relay it may send advertisements with UDP port 546 instead of 547. PR851642: This issue has been resolved.

Network Management and Monitoring

  • The default maximum log file size depends on the platform type for TX Matrix or TX Matrix Plus routers it is expected to be 10 MB. However, due to a software defect, this file size was only 1 MB. PR823143: This issue has been resolved.
  • On a router with interfaces with Frame Relay encapsulation a SNMP WALK operation will cause a MIB daemon (mib2d) crash and will generate a mib2d core-dump. The crash itself does not cause any impact on the router as the MIB daemon is restarted automatically. The only effect is that a SNMP WALK will never complete successfully. user@router-re1> show snmp mib walk 1 | no-more sysDescr.0 = Juniper Networks, Inc. mx480 internet router, kernel JUNOS OS 11.4R6.5 #0: 2012-11-28 21:57:12 UTC builderdevicex.example.com:/volume/build/junos/11.4/release/11.4R6.5/obj-i 386/bsd/kernels/JUNIPER/kernel Build date: 2012-11-28 21:39:15 UTC Copyright (c sysObjectID.0 = jnxProductNameMX480 sysUpTime.0 = 339594 sysContact.0 < .................................... > dot3OutPauseFrames.942 = 0 dot3OutPauseFrames.943 = 0 dot3OutPauseFrames.953 = 0 dot3OutPauseFrames.954 = 0 frDlcmiIfIndex.153 = 153 frDlcmiIfIndex.512 = 512 frDlcmiIfIndex.513 = 513 frDlcmiState.153 = 6 Request failed: General error user@router-re1> show log messages Dec 20 09:23:20 router-re1 clear-log[8240]: logfile cleared Dec 20 09:23:38.683 router-re1 /kernel: %KERN-3-BAD_PAGE_FAULT: pid 7382 (mib2d), uid 0: pc 0x810fe09 got a read fault at 0x7c, x86 fault flags = 0x4 Dec 20 09:23:38.683 router-re1 /kernel: %KERN-3: Trapframe Register Dump: Dec 20 09:23:38.683 router-re1 /kernel: %KERN-3: eax: 00000000 ecx: bfbeda88 edx: 00000000 ebx: bfbeda7c Dec 20 09:23:38.683 router-re1 /kernel: %KERN-3: esp: bfbeda60 ebp: bfbeda98 esi: 089de834 edi: 089fb680 Dec 20 09:23:38.683 router-re1 /kernel: %KERN-3: eip: 0810fe09 eflags: 00010297 Dec 20 09:23:38.683 router-re1 /kernel: %KERN-3: cs: 0033 ss: 003b ds: bfbe003b es: 003b Dec 20 09:23:38.683 router-re1 /kernel: %KERN-3: fs: 003b trapno: 0000000c err: 00000004 Dec 20 09:23:38.683 router-re1 /kernel: %KERN-3: Page table info for PC address 0x810fe09: PDE = 0x42e60067, PTE = 5290c425 Dec 20 09:23:38.683 router-re1 /kernel: %KERN-3: Dumping 16 bytes starting at PC address 0x810fe09: Dec 20 09:23:38.683 router-re1 /kernel: %KERN-3: 8b 40 7c 89 04 24 e8 5a 3f 2f 00 89 45 ec 8b 55 Dec 20 09:23:40.787 router-re1 init: %AUTH-3: mib-process (PID 7382) terminated by signal number 11. Core dumped! Dec 20 09:23:40.787 router-re1 init: %AUTH-6: mib-process (PID 8247) started Dec 20 09:23:40.809 router-re1 mib2d[8247]: %DAEMON-5-LIBJSNMP_SA_IPC_REG_ROWS: ns_subagent_register_mibs: registering 88 rows Dec 20 09:23:41.595 router-re1 mib2d[8247]: %DAEMON-6-LIBJSNMP_NS_LOG_INFO: INFO: ns_subagent_open_session: NET-SNMP version 5.3.1 AgentX subagent connected Dec 20 09:23:43.533 router-re1 dumpd: %USER-5: Core and context for mib2d saved in /var/tmp/mib2d.core-tarball.0.tgz Dec 20 09:23:43.793 router-re1 mib2d[8247]: %DAEMON-6-SNMP_TRAP_LINK_UP: ifIndex 5, ifAdminStatus up(1), ifOperStatus up(1), ifName dsc < .................................... > user@router-re1> show system core-dumps /var/crash/*core*: No such file or directory -rw------- 1 root field 680417 Dec 20 09:23 /var/tmp/mib2d.core-tarball.0.tgz /var/tmp/pics/*core*: No such file or directory /var/crash/kernel.*: No such file or directory /tftpboot/corefiles/*core*: No such file or directory total 1 PR835722: This issue has been resolved.

Platform and Infrastructure

  • When using configure private with large group definition and high number of groups the commit process can spend a lot of time to merge the configuration change with the global configuration. PR828005: This issue has been resolved.
  • This applies to all Juniper M, MX, and T Series routers. In certain GRES scenarios, the backup Routing Engine may not have the complete state of the NH database from the active Routing Engine and may send duplicate NH add messages to Packet Forwarding Engine with same NH Ids when it becomes active. This could potentially cause undesirable behavior in forwarding resulting in broken forwarding state and/or FPC cores. To limit the affect of these duplicate NH add messages, only certain duplicate NH adds messages which can be handled gracefully are allowed and all other duplicate add messages are rejected. There is no work-around for this problem. PR843907: This issue has been resolved.
  • On MX Series and T4000, when output Filter-Based Forwarding (FBF) destined to a routing-instance is configured, the packets matched by the FBF filter may be discarded or sent to the unintended Packet Forwarding Engine. PR845700: This issue has been resolved.

Routing Protocols

  • If maximum-paths or maximum-prefixes is configured for a route table, these limits are displayed in the output of "show route summary". In affected releases, these limits were omitted from the output of "show route summary". PR753013: This issue has been resolved.
  • Due to duplication of the traffic, assert will be triggered. *,G and S,G assert is not handled properly hence few assert entries will not be deleted due to Routing Engine switchover which result in a core file. ?HW type of chassis/linecard/RE. "ALL" ?Suspected software feature combination. Multicast feature ?Describe if any behavior/ change to existing function - Handle the *,G and S,G assert properly. PR809338: This issue has been resolved.
  • This PR fixes a bug by which the receiving EBGP speaker mistakenly accepts a session establishment attempt from an EBGP peer address that is not directly connected because it did not check to see if the address to which peer wants to establish a session belongs to the receiving interface or not. PR816531: This issue has been resolved.
  • Changes to add-path prefix-policy do not get absorbed automatically, and require a manual soft-clearing of the BGP session. PR818789: This issue has been resolved.
  • RPD on the backup Routing Engine might crash when it receives a malformed message from the master. This can occur at high scale with nonstop active routing enabled when a large flood of updates are being sent to the backup. There is no workaround to avoid the problem, but it is rare and backup RPD will restart and the system will recover without intervention. PR830057: This issue has been resolved.
  • Multiple route nexthops will not be returned via SNMP for ipCidrRouteTable object. PR831553: This issue has been resolved.
  • If LDP-SYNC <hold-down> timer is configured under IS-IS interfaces after configuration change the IS-IS interfaces can go to <hold-down> state. PR831871: This issue has been resolved.
  • 1) What triggers the bug to be happened? =>Enabling PIM - Bidir feature (possibly pim rp with 224.0.0.0/4 group) and rpd restart. This issue is hit during regression test for PIM bidir. 2) HW type of chassis/linecard/Routing Engine. If it affects all, just say ?all?. =>all. 3) Suspected software feature combination. (If customer turns on feature X along with Y, they might hit, etc) =>PIM - Bidir feature (rp configured) and rpd restart is causing the issue. 4) Describe if any behavior/ change to existing function =>None. PR836629: This issue has been resolved.
  • On EX8200 switches, multiple rpd process core files might be created on the backup Routing Engine after a nonstop software upgrade (NSSU) has been performed while multicast traffic is on the switch. PR841848: This issue has been resolved.
  • IS-IS reports prefix-export-limit exceeded even though the number of exported routes is smaller than the configured value of prefix-export-limit. PR844224: This issue has been resolved.
  • In scenarios that use BGP to distribute traffic flow specifications, if the recieved flow-spec Network Layer Reachability Information (NLRI) contains invalid argument (such as dscp is larger that 63), routing protocol process (rpd) will generate flow-spec routes and install them in the routing table for these NLRIs; but these flow routes with invalid match conditions are rejected by dynamic firewall daemon (dfwd) from being added to the flowspec filters. When issue happens, the following errors could be seen: krt_flow_trans_match_config: Failed defining match conditions 10.0.1.1,1.0.0.1,proto=6,dscp=81 krt_flow_trans_term_add: Failed adding term 10.0.1.1,1.0.0.1,proto=6,dscp=81 to filter 0x9504000 - Unknown error: 0 krt_flow_trans_filter_add: Failed sending transaction (ADD FILTER SINGLE TERM) for filter 0x9504000 __flowspec_default_inet__ to add term 10.0.1.1,1.0.0.1,proto=6,dscp=81 - Invalid argument When the bgp peer withdraws these flow routes, they will only be deleted but not freed, hence cause memory leak. PR845039: This issue has been resolved.
  • In BGP scenario with multipath configured, if a static route which has table nexthop (such as inet.0) is configured in the same routing-instance as BGP, when an interconnect link between BGP pees is brought down or flapping, the corresponding BGP session takes 90 seconds to timeout. During this period routes received over the BGP session will stay there. For a multipath transit route recieved from both BGP sessions, initially both paths are resolved over the interconnect links directly. When one of the interconnect link is brought down or flapping, that path will be resolved over the static default route which has table nexthop (such as inet.0). So now, one path is resolved over a router nexthop and the other path is resolved over a table nexthop. This will cause routing protocol process (rpd) crash and generate a core file. This issue usually occurs in BGP/L3VPN environment. The core files could be seen by executing CLI command show system core-dumps. PR851807: This issue has been resolved.

Services Applications

  • SIP ALG was not allowing SIP 603 decline message. PR822679: This issue has been resolved.
  • jl2tpd crash _thr_send_sig (thread=0x8a5e000, sig=6) at ../../../../src/bsd/lib/libthr/thread/thr_kern.c:91 jl2tpd crash exhibited in environment where MX480 was configured as LAC and terminating 500 l2tp subscribers. PR824760: This issue has been resolved.
  • When MX Series uses MS-DPC to provide the tunnelling service for flow-tap traffic, if there is SCU/DCU configured on the same slot of the flow-tap traffic ingress interface, all the flow-taped sampled packets will be dropped. It is caused by the wrong nexthop linking when DCU is confiured. PR825958: This issue has been resolved.
  • This issue is seen when two l2tp users get connected to same routing-instance and they get same framed routes. When last connected user disconnects this issue can be seen. PR832034: This issue has been resolved.
  • In the case of a stateful proxy, two SIP users behind the NAT device (so-called SIP hairpinning) will be unable to signal the call. PR832364: This issue has been resolved.
  • With RTSP ALG enabled, RTSP keep-alive packets might be dropped if it's already Ack'ed by the receiver. PR834198: This issue has been resolved.
  • In Carrier Grade NAT (CGNAT) scenario, without any configuration change, under some conditions, MS-DPC PIC might crash and create a core file when encountering unknown flow-type. Service will be impacted during the period. When issue happens, the following logs could be seen: chassisd[1477]: CHASSISD_SNMP_TRAP10: SNMP trap generated: FRU power off (jnxFruContentsIndex 8, jnxFruL1Index 6, jnxFruL2Index 2, jnxFruL3Index 0, jnxFruName PIC: MS-DPC PIC @ 5/1/*, jnxFruType 11, jnxFruSlot 5, jnxFruOfflineReason 8, jnxFruLastPowerOff 192338801, jnxFruLastPowerOn 33404122) chassisd[1477]: CHASSISD_SNMP_TRAP10: SNMP trap generated: FRU power on (jnxFruContentsIndex 8, jnxFruL1Index 6, jnxFruL2Index 2, jnxFruL3Index 0, jnxFruName PIC: MS-DPC PIC @ 5/1/*, jnxFruType 11, jnxFruSlot 5, jnxFruOfflineReason 2, jnxFruLastPowerOff 192338801, jnxFruLastPowerOn 192338924) PR834899: This issue has been resolved.
  • In scenarios which use sp interface, such as IPSec VPN, multiservice process (mspd) will memory leak during sp interface flapping. The memory usage of mspd process can be checked by following CLI command: user@router> show system processes extensive | match "PID | mspd" (Note: The "RES" field means "Current amount of resident memory, in kilobytes") PID USERNAME THR PRI NICE SIZE RES STATE TIME WCPU COMMAND 2048 root 1 96 0 36216K 34820K select 0:10 0.00% mspd When the memory usage of mspd process increases to system limit(about 131072KB), the following logs could be seen: /kernel: %KERN-5: Process (2048,mspd) attempted to exceed RLIMIT_DATA: attempted 131076 KB Max 131072 KB PR836735: This issue has been resolved.
  • When DHCP subscribers login and radius hands down flow-tap variables the following errors are seen in the log: "/kernel: GENCFG: op 24 (Lawful Intercept) failed; err 5 (Invalid)." PR837877: This issue has been resolved.
  • The "hot-standby" CLI knob under [edit interfaces <RSP-interface-name> redundancy-options] is made hidden for the Redundant Service PIC (RSP). PR838762: This issue has been resolved.
  • If flow-tap or radius-flow-tap is configured and logging, dfcd might be leaking file descriptors. RPD may crash and write a core with a signature like "kern.maxfiles limit exceeded by uid 0" due to this issue. PR842124: This issue has been resolved.
  • Service PIC might crash under certain race conditions when receiving sip invite packets. PR843047: This issue has been resolved.
  • Service PIC might crash in corner cases when receiving specific SIP REGISTER. PR843479: This issue has been resolved.
  • Service PIC might crash in corner cases when EIM is enabled for SIP ALG. PR847124: This issue has been resolved.
  • spd core generated during switchover when CGAT config is there.Issue is well understood now and has been fixed in later releases. PR854206: This issue has been resolved.

Subscriber Access Management

  • Snmpwalk requests sent to MX Series returns multiple duplicate records for jnxUserAAAAccessPool. PR840640: This issue has been resolved.

VPNs

  • In a scaled Multicast VPN setup, where many selective provider tunnels are used, and the MVPN instance is deleted, RPD can sometimes crash. PR801667: This issue has been resolved.
  • In BGP-MVPN, when the number of multicast routes falls below the threshold, the earlier suppressed MVPN multicast routes because of limit are not added back again. For MVPN, there was no mechanism to trigger the processing of cmcast entries that were not added earlier. The fix is to queue the cmcast entries that are suppressed for multicast route addition in a new list. When the reuse limit is reached, this list is walked and used to add back the entries. PR841105: This issue has been resolved.
  • In l2circuit scenarios where a backup neighbor is being defined along the 'standby' knob, after deleting this backup neighbor from configuration, its associated vc-route is not being eliminated. Later if user deletes the l2circuit neighbor or restarts routing protocol process (rpd), rpd process will crash and core dumped. PR841522: This issue has been resolved.

Related Documentation

Modified: 2016-06-09