Known Behavior

CEM-16030 Contrail Installation on Red Hat environment has a dependency on net-snmp package version 48. The repository should provide the following files during the installation.

CEM 10726 Reboot the compute nodes manually after a kernel upgrade.

JCB-1681680 / JCB-161567 When the DPDK vRouter fragments packets before sending them on the wire, the reassembly of the fragments on the receiver might time out in some cases.

1694343 / JCB-177784 In DPDK vRouter use-cases (SNAT, LBaaS) that require netns to be launched, do not set Jumbo frames. Use MTU <= 1500 bytes.

1711256 Project isolation is not supported in nested mode. In nested mode, Namespaces-isolation results in a virtual-network creation and doesn't create a new project.

1716308/ JCB-184771 When the head fragment is received in the vRouter, the head fragment is enqueued to the assembler immediately upon arrival. The flow is created as hold flow and then trapped to the agent. If fragments corresponding to this head fragment are already in the assembler or if new fragments arrive immediately after the head fragment, the assembler releases them to flow module. If agent does not write flow action by the time the assembler releases fragments to the flow module, fragments get enqueued in the hold queue. As a maximum of only three fragments are enqueued in the hold queue, rest of the fragments from the assembler get dropped in the flow module. This leads to the whole packet being dropped on the receive side leading to the first packet loss.

1718807 / JCB-186624 In OVSDB case, the routes are exported from the ToR Agent where the SG is appropriately updated so that inter-virtual network traffic doesn’t require any explicit SG to be configured for it to pass. When TOR is peering with control node, this SG has to be explicitly configured and this behavior is expected.

1720990 / JCB-195914 With policy-based mirroring with ECMP destinations, one of the destination vRouter drops packet with invalid NH.

1724357 / JCB-215307 While provisioning a RHOSP10 cluster with DPDK nodes, the DPDK node power state goes offline during introspection stage. As a workaround, delete the DPDK nodes from Ironic configuration and add them with the right configuration.

1. ironic node-delete node-name

2. Create a JSON file configuration as shown in the following example for all DPDK nodes:

3. openstack baremetal import --json path to .json

4. openstack baremetal introspection bulk start

1728802 / JCB-166617 Session logging: Incorrect VN information seen for sessions on transparent SI VMIs. As a workaround, ensure that VLAN NH inherits the policy status from its associated interface.

1796812 In Netronome based Smart NIC deployments, launching a VM directly using net-ids is not supported. Port-ids need to be used instead.

CEM-13620 While deploying RHOSP 10, under rare scenarios, contrail-named service will stay down. As a one-time workaround, restart the service using service contrail-named restart command from the command prompt.

CEM 10126 When three analytics db nodes are configured, the Cassandra service on one of the analytics db nodes is down after provisioning. This was observed rarely during testing. The cassandra service on the other two analytics db nodes are functional and active. As a workaround, restart the 'contrail-database' service.

CEM 10158 On a Red Hat 7.7 OSP10 scale cluster, Keystone is not issuing tokens, causing OpenStack commands to fail. This is attributed to the following Red Hat KBs and it is recommended to deploy the cluster as per the following documentation from Red Hat.

• https://access.redhat.com/solutions/2262771

• https://access.redhat.com/solutions/3451691

CEM 10311 After provisioning, sometimes the kafka server does not come up due to ZooKeeperClientTimeoutException. This happens if kafka tries to connect to zookeeper before it is up. As a workaround, restart kafka once zookeeper is up.

CEM 10726 Reboot the compute nodes manually after a kernel upgrade.

JCB-1681680 / JCB-161567 When the DPDK vRouter fragments packets before sending them on the wire, the reassembly of the fragments on the receiver might time out in some cases.

1694343 / JCB-177784 In DPDK vRouter use-cases (SNAT, LBaaS) that require netns to be launched, do not set Jumbo frames. Use MTU <= 1500 bytes.

1711256 Project isolation is not supported in nested mode. In nested mode, Namespaces-isolation results in a virtual-network creation and doesn't create a new project.

1716308/ JCB-184771 When the head fragment is received in the vRouter, the head fragment is enqueued to the assembler immediately upon arrival. The flow is created as hold flow and then trapped to the agent. If fragments corresponding to this head fragment are already in the assembler or if new fragments arrive immediately after the head fragment, the assembler releases them to flow module. If agent does not write flow action by the time the assembler releases fragments to the flow module, fragments get enqueued in the hold queue. As a maximum of only three fragments are enqueued in the hold queue, rest of the fragments from the assembler get dropped in the flow module. This leads to the whole packet being dropped on the receive side leading to the first packet loss.

1718807 / JCB-186624 In OVSDB case, the routes are exported from the ToR Agent where the SG is appropriately updated so that inter-virtual network traffic doesn’t require any explicit SG to be configured for it to pass. When TOR is peering with control node, this SG has to be explicitly configured and this behavior is expected.

1720990 / JCB-195914 With policy-based mirroring with ECMP destinations, one of the destination vRouter drops packet with invalid NH.

1724357 / JCB-215307 While provisioning a RHOSP10 cluster with DPDK nodes, the DPDK node power state goes offline during introspection stage. As a workaround, delete the DPDK nodes from Ironic configuration and add them with the right configuration.

1. ironic node-delete node-name

2. Create a JSON file configuration as shown in the following example for all DPDK nodes:

3. openstack baremetal import --json path to .json

4. openstack baremetal introspection bulk start

1728802 / JCB-166617 Session logging: Incorrect VN information seen for sessions on transparent SI VMIs. As a workaround, ensure that VLAN NH inherits the policy status from its associated interface.

1796812 In Netronome based Smart NIC deployments, launching a VM directly using net-ids is not supported. Port-ids need to be used instead.

CEM 10126 When three analytics db nodes are configured, the cassandra service on one of the analytics db nodes is down after provisioning. This was observed rarely during testing. The cassandra service on the other two analytics db nodes are functional and active. As a workaround, restart the 'contrail-database' service.

CEM 10158 On a Red Hat 7.7 OSP10 scale cluster, Keystone is not issuing tokens, causing OpenStack commands to fail. This is attributed to the following Red Hat KBs and it is recommended to deploy the cluster as per the following documentation from Red Hat.

• https://access.redhat.com/solutions/2262771

• https://access.redhat.com/solutions/3451691

CEM 10311 After provisioning, sometimes the kafka server does not come up due to ZooKeeperClientTimeoutException. This happens if kafka tries to connect to zookeeper before it is up. As a workaround, restart kafka once zookeeper is up.

CEM 10726 Reboot the compute nodes manually after a kernel upgrade.

JCB-1681680 / JCB-161567 When the DPDK vRouter fragments packets before sending them on the wire, the reassembly of the fragments on the receiver might time out in some cases.

1694343 / JCB-177784 In DPDK vRouter use-cases (SNAT, LBaaS) that require netns to be launched, do not set Jumbo frames. Use MTU <= 1500 bytes.

1711256 Project isolation is not supported in nested mode. In nested mode, Namespaces-isolation results in a virtual-network creation and doesn't create a new project.

1716308/ JCB-184771 When the head fragment is received in the vRouter, the head fragment is enqueued to the assembler immediately upon arrival. The flow is created as hold flow and then trapped to the agent. If fragments corresponding to this head fragment are already in the assembler or if new fragments arrive immediately after the head fragment, the assembler releases them to flow module. If agent does not write flow action by the time the assembler releases fragments to the flow module, fragments get enqueued in the hold queue. As a maximum of only three fragments are enqueued in the hold queue, rest of the fragments from the assembler get dropped in the flow module. This leads to the whole packet being dropped on the receive side leading to the first packet loss.

1718807 / JCB-186624 In OVSDB case, the routes are exported from the ToR Agent where the SG is appropriately updated so that inter-virtual network traffic doesn’t require any explicit SG to be configured for it to pass. When TOR is peering with control node, this SG has to be explicitly configured and this behavior is expected.

1720990 / JCB-195914 With policy-based mirroring with ECMP destinations, one of the destination vRouter drops packet with invalid NH.

1724357 / JCB-215307 While provisioning a RHOSP10 cluster with DPDK nodes, the DPDK node power state goes offline during introspection stage. As a workaround, delete the DPDK nodes from Ironic configuration and add them with the right configuration.

1. ironic node-delete node-name

2. Create a JSON file configuration as shown in the following example for all DPDK nodes:

3. openstack baremetal import --json path to .json

4. openstack baremetal introspection bulk start

1728802 / JCB-166617 Session logging: Incorrect VN information seen for sessions on transparent SI VMIs. As a workaround, ensure that VLAN NH inherits the policy status from its associated interface.

1796812 In Netronome based Smart NIC deployments, launching a VM directly using net-ids is not supported. Port-ids need to be used instead.

Redhat BZ 1684271: Installation of Contrail Release 4.1.3 release on Red Hat 7.6 does not work. Contrail Release 4.1.3 is tested on Red Hat 7.5.

1797358 In a Netronome based Smart NIC deployment, gateway less forwarding feature does not work with nfp_p1 interface. This works with nfp_p0 interface.

1796812 In Netronome based Smart NIC deployments, launching a VM directly using net-ids is not supported. Port-ids need to be used instead.

1681680 When the DPDK vRouter fragments packets before sending them on the wire, the reassembly of the fragments on the receiver might time out in some cases.

1694343 In DPDK vRouter use-cases (SNAT, LBaaS) that require netns to be launched, do not set Jumbo frames. Use MTU <= 1500 bytes.

1705795 On an RHOSP10 provisioned cluster, if the vrouter-agent gets restarted, vhost0 interface does not come up. The service supervisor-vrouter restart command brings the service back online.

1709974 TSN support in RHOSP-based clusters are supported upto RHOSP10. As a workaround, deploy the stack with computes and DPDK first. Then change the VrouterPhysicalInterface, add the TSN nodes, and update the stack.

1711256 Project isolation is not supported in nested mode. In nested mode, Namespaces-isolation results in a virtual-network creation and doesn't create a new project.

1716308 When the head fragment is received in the vRouter, the head fragment is enqueued to the assembler immediately upon arrival. The flow is created as hold flow and then trapped to the agent. If fragments corresponding to this head fragment are already in the assembler or if new fragments arrive immediately after the head fragment, the assembler releases them to flow module. If agent does not write flow action by the time the assembler releases fragments to the flow module, fragments get enqueued in the hold queue. As a maximum of only three fragments are enqueued in the hold queue, rest of the fragments from the assembler get dropped in the flow module. This leads to the whole packet being dropped on the receive side leading to the first packet loss.

1718807 In OVSDB case, the routes are exported from the ToR Agent where the SG is appropriately updated so that inter-virtual network traffic doesn’t require any explicit SG to be configured for it to pass. When TOR is peering with control node, this SG has to be explicitly configured and this behavior is expected.

1720990 With policy-based mirroring with ECMP destinations, one of the destination vRouter drops packet with invalid NH.

1721620 VNC API sends an update of all subfields in a field, like virtual_network_properties and not the updated subfields, like allow_transit of vn_properties alone, hence matching against all the subfields due to which you might not be able to update a subfield. As a workaround, perform the following steps:

1. Delete the attributes which are not updated from the field class. For example, all the attributes of virtual_network_properties except allow_transit.

2. Call vn_obj.set_virtual_network_properties().

3. Use RestApi or Contrail-UI instead of vnc_api.

1724357 While provisioning a RHOSP10 cluster with DPDK nodes, the DPDK node power state goes offline during introspection stage. As a workaround, delete the DPDK nodes from Ironic configuration and add them with the right configuration.

1. ironic node-delete node-name

2. Create a JSON file configuration as shown in the following example for all DPDK nodes:

3. openstack baremetal import --json path to .json

4. openstack baremetal introspection bulk start

1728802 Session logging: Incorrect VN information seen for sessions on transparent SI VMIs. As a workaround, ensure that VLAN NH inherits the policy status from its associated interface.

1729059 You must use Ansible version 2.3 to install Contrail using contrail-ansible for Kubernetes and OpenShift deployments.

1735057 When bringing up Contrail cluster on Red Hat container, manually install docker-py on all the target nodes.

• To install Pip, use the following command:

• To install docker-py, use the following command:

1794702 - vRouter crashes if network policy contains large number of Community Tags. Recommended number for 4.1.2 code is less than 50.

1797358 - In a Netronome based Smart NIC deployment, gateway less forwarding feature does not work with nfp_p1 interface. This works with nfp_p0 interface.

1796812 - In Netronome based Smart NIC deployments, Launching a VM directly using net-ids is not supported. Port-ids need to be used instead.

1681680 When the DPDK vRouter fragments packets before sending them on the wire, the reassembly of the fragments on the receiver might time out in some cases.

1694343 In DPDK vRouter use-cases (SNAT, LBaaS) that require netns to be launched, do not set Jumbo frames. Use MTU <= 1500 bytes.

1705795 On an RHOSP10 provisioned cluster, if the vrouter-agent gets restarted, vhost0 interface does not come up. The service supervisor-vrouter restart command brings the service back online.

1709974 TSN support in RHOSP-based clusters are supported upto RHOSP10. As a workaround, deploy the stack with computes and DPDK first. Then change the VrouterPhysicalInterface, add the TSN nodes, and update the stack.

1711256 Project isolation is not supported in nested mode. In nested mode, Namespaces-isolation results in a virtual-network creation and doesn't create a new project.

1716308 When the head fragment is received in the vRouter, the head fragment is enqueued to the assembler immediately upon arrival. The flow is created as hold flow and then trapped to the agent. If fragments corresponding to this head fragment are already in the assembler or if new fragments arrive immediately after the head fragment, the assembler releases them to flow module. If agent does not write flow action by the time the assembler releases fragments to the flow module, fragments get enqueued in the hold queue. As a maximum of only three fragments are enqueued in the hold queue, rest of the fragments from the assembler get dropped in the flow module. This leads to the whole packet being dropped on the receive side leading to the first packet loss.

1718807 In OVSDB case, the routes are exported from the ToR Agent where the SG is appropriately updated so that inter-virtual network traffic doesn’t require any explicit SG to be configured for it to pass. When TOR is peering with control node, this SG has to be explicitly configured and this behavior is expected.

1720990 With policy-based mirroring with ECMP destinations, one of the destination vRouter drops packet with invalid NH.

1721620 VNC API sends an update of all subfields in a field, like virtual_network_properties and not the updated subfields, like allow_transit of vn_properties alone, hence matching against all the subfields due to which you might not be able to update a subfield. As a workaround, perform the following steps:

1. Delete the attributes which are not updated from the field class. For example, all the attributes of virtual_network_properties except allow_transit.

2. Call vn_obj.set_virtual_network_properties().

3. Use RestApi or Contrail-UI instead of vnc_api.

1724357 While provisioning a RHOSP10 cluster with DPDK nodes, the DPDK node power state goes offline during introspection stage. As a workaround, delete the DPDK nodes from Ironic configuration and add them with the right configuration.

1. ironic node-delete node-name

2. Create a JSON file configuration as shown in the following example for all DPDK nodes:

3. openstack baremetal import --json path to .json

4. openstack baremetal introspection bulk start

1728802 Session logging: Incorrect VN information seen for sessions on transparent SI VMIs. As a workaround, ensure that VLAN NH inherits the policy status from its associated interface.

1729059 You must use Ansible version 2.3 to install Contrail using contrail-ansible for Kubernetes and OpenShift deployments.

1735057 When bringing up Contrail cluster on Red Hat container, manually install docker-py on all the target nodes.

• To install Pip, use the following command:

• To install docker-py, use the following command:

1681680 When the DPDK vRouter fragments packets before sending them on the wire, the reassembly of the fragments on the receiver might time out in some cases.

1694343 In DPDK vRouter use-cases (SNAT, LBaaS) that require netns to be launched, do not set Jumbo frames. Use MTU <= 1500 bytes.

1705795 On an RHOSP10 provisioned cluster, if the vrouter-agent gets restarted, vhost0 interface does not come up. The service supervisor-vrouter restart command brings the service back online.

1709974 TSN support in RHOSP-based clusters are supported upto RHOSP10. As a workaround, deploy the stack with computes and DPDK first. Then change the VrouterPhysicalInterface, add the TSN nodes, and update the stack.

1711256 Project isolation is not supported in nested mode. In nested mode, Namespaces-isolation results in a virtual-network creation and doesn't create a new project.

1716297 Provisioning fails for OpenStack HA with SMLite on Ocata.

1716308 When the head fragment is received in the vRouter, the head fragment is enqueued to the assembler immediately upon arrival. The flow is created as hold flow and then trapped to the agent. If fragments corresponding to this head fragment are already in the assembler or if new fragments arrive immediately after the head fragment, the assembler releases them to flow module. If agent does not write flow action by the time the assembler releases fragments to the flow module, fragments get enqueued in the hold queue. As a maximum of only three fragments are enqueued in the hold queue, rest of the fragments from the assembler get dropped in the flow module. This leads to the whole packet being dropped on the receive side leading to the first packet loss.

1718807 In OVSDB case, the routes are exported from the ToR Agent where the SG is appropriately updated so that inter-virtual network traffic doesn’t require any explicit SG to be configured for it to pass. When TOR is peering with control node, this SG has to be explicitly configured and this behavior is expected.

1720990 With policy-based mirroring with ECMP destinations, one of the destination vRouter drops packet with invalid NH.

1721564 Contrail Ocata: ansible_hostname does not handle “-” in hostname. As a workaround, copy the cert and key files with the expected name without the “-” in the same path as /etc/contrail_smgr/puppet/ssl/.

1721620 VNC API sends an update of all subfields in a field, like virtual_network_properties and not the updated subfields, like allow_transit of vn_properties alone, hence matching against all the subfields due to which you might not be able to update a subfield. As a workaround, perform the following steps:

1. Delete the attributes which are not updated from the field class. For example, all the attributes of virtual_network_properties except allow_transit.

2. Call vn_obj.set_virtual_network_properties().

3. Use RestApi or Contrail-UI instead of vnc_api.

1724357 While provisioning a RHOSP10 cluster with DPDK nodes, the DPDK node power state goes offline during introspection stage. As a workaround, delete the DPDK nodes from Ironic configuration and add them with the right configuration.

1. ironic node-delete node-name

2. Create a JSON file configuration as shown in the following example for all DPDK nodes:

3. openstack baremetal import --json path to .json

4. openstack baremetal introspection bulk start

1728802 Session logging: Incorrect VN information seen for sessions on transparent SI VMIs. As a workaround, ensure that VLAN NH inherits the policy status from its associated interface.

1729059 You must use Ansible version 2.3 to install Contrail using contrail-ansible for Kubernetes and OpenShift deployments.

1735057 When bringing up Contrail cluster on Red Hat container, manually install docker-py on all the target nodes.

• To install Pip, use the following command:

• To install docker-py, use the following command:

1735874 Kubernetes: Analytics services fail when 3 node HA setup is brought up using single yaml.

1681680 When the DPDK vRouter fragments packets before sending them on the wire, the reassembly of the fragments on the receiver might time out in some cases.

1694343 In DPDK vRouter use-cases (SNAT, LBaaS) that require netns to be launched, do not set Jumbo frames. Use MTU <= 1500 bytes.

1705795 On an RHOSP10 provisioned cluster, if the vrouter-agent gets restarted, vhost0 interface does not come up. The service supervisor-vrouter restart command brings the service back online.

1709974 TSN support in RHOSP-based clusters are supported upto RHOSP10. As a workaround, deploy the stack with computes and DPDK first. Then change the VrouterPhysicalInterface, add the TSN nodes, and update the stack.

1711256 Project isolation is not supported in nested mode. In nested mode, Namespaces-isolation results in a virtual-network creation and doesn't create a new project.

1716297 Provisioning fails for OpenStack HA with SMLite on Ocata.

1716308 When the head fragment is received in the vRouter, the head fragment is enqueued to the assembler immediately upon arrival. The flow is created as hold flow and then trapped to the agent. If fragments corresponding to this head fragment are already in the assembler or if new fragments arrive immediately after the head fragment, the assembler releases them to flow module. If agent does not write flow action by the time the assembler releases fragments to the flow module, fragments get enqueued in the hold queue. As a maximum of only three fragments are enqueued in the hold queue, rest of the fragments from the assembler get dropped in the flow module. This leads to the whole packet being dropped on the receive side leading to the first packet loss.

1718807 In OVSDB case, the routes are exported from the ToR Agent where the SG is appropriately updated so that inter-virtual network traffic doesn’t require any explicit SG to be configured for it to pass. When TOR is peering with control node, this SG has to be explicitly configured and this behavior is expected.

1719430 While upgrading OpenShift cluster from build 31 to 32 in Redhat base OS, contrail kube-manager fails to come up for permission due to an issue with contrail-kube-manager.log. As a workaround, change the permission of the log file inside contrail-kube-manager docker and restart the service.

1720118 Configuration of Allowed Address Pair (AAP) with prefix length less than 24 is not allowed.

1720990 With policy-based mirroring with ECMP destinations, one of the destination vRouter drops packet with invalid NH.

1721564 Contrail Ocata: ansible_hostname does not handle “-” in hostname. As a workaround, copy the cert and key files with the expected name without the “-” in the same path as /etc/contrail_smgr/puppet/ssl/.

1721620 VNC API sends an update of all subfields in a field, like virtual_network_properties and not the updated subfields, like allow_transit of vn_properties alone, hence matching against all the subfields due to which you might not be able to update a subfield. As a workaround, perform the following steps:

1. Delete the attributes which are not updated from the field class. For example, all the attributes of virtual_network_properties except allow_transit.

2. Call vn_obj.set_virtual_network_properties().

3. Use RestApi or Contrail-UI instead of vnc_api.

1722877 There is no automated provisioning method through director for SRIOV.

1724357 While provisioning a RHOSP10 cluster with DPDK nodes, the DPDK node power state goes offline during introspection stage. As a workaround, delete the DPDK nodes from Ironic configuration and add them with the right configuration.

1. ironic node-delete node-name

2. Create a JSON file configuration as shown in the following example for all DPDK nodes:

3. openstack baremetal import --json path to .json

4. openstack baremetal introspection bulk start

1728802 Session logging: Incorrect VN information seen for sessions on transparent SI VMIs. As a workaround, ensure that VLAN NH inherits the policy status from its associated interface.

1729059 You must use Ansible version 2.3 to install Contrail using contrail-ansible for Kubernetes and OpenShift deployments.

1733684 ContrailSecurity: Addressgroup match performs the OR function of both subnet and label rather than the AND function.

1734110 Provisioning SSL for metadata fetch does not work if the OpenStack SKU is Ocata.

1734790 In RBAC-enabled deployments, SM-Lite doesn’t enable RBAC in api-paste.ini for Neutron. As a workaround, after provisioning the cluster, manually update /etc/neutron/api-paste.ini as described in RBAC.

1735054 RBAC for analytics alarms doesn’t work.

1735057 When bringing up Contrail cluster on Red Hat container, manually install docker-py on all the target nodes.

• To install Pip, use the following command:

• To install docker-py, use the following command:

1735081 While deploying using Contrail Networking package, the dependency for liblua5.3-0 must be resolved explicitly by the user.

1735590 In Kubernetes and OpenShift-based deployments when we crate SNAT router and extend cluster-network to that SNAT router host is losing all connectivity.

As a workaround, if you want to use the SNAT feature in Contrail, disassociate the ip-fabric-cluster-network-default policy and delete it.

1773294 In a RHOSP deployment incase of DPDK vRouter crash the vRouter does not restart itself. As a workaround, use the service supervisor-vrouter restart command.