Example: Configuring Graceful Restart
Figure 7 shows a standard MPLS VPN network. Routers CE1 and CE2 are customer edge routers, PE1 and PE2 are provider edge routers, and P0 is a provider core router. Several Layer 3 VPNs are configured across this network, as well as one Layer 2 VPN. Interfaces are shown in the diagram and are not included in the configuration example that follows.
Figure 7: Layer 3 VPN Graceful Restart Topology

Router CE1
On Router CE1, configure the following protocols on the logical interfaces of t3-3/1/0: OSPF on unit 101, RIP on unit 102, BGP on unit 103, and IS-IS on unit 512. Also configure graceful restart, BGP, IS-IS, OSPF, and RIP on the main instance to be able to connect to the routing instances on Router PE1.
Router PE1
On Router PE1, configure graceful restart in the master instance, along with BGP, OSPF, MPLS, and LDP. Next, configure several protocol-specific instances of graceful restart. By including instances for BGP, OSPF, Layer 2 VPNs, RIP, and static routes, you can observe the wide range of options available when you implement graceful restart. Configure the following protocols in individual instances on the logical interfaces of t3-0/0/0: a static route on unit 100, OSPF on unit 101, RIP on unit 102, BGP on unit 103, and Frame Relay on unit 512 for the Layer 2 VPN instance.
Router P0
On Router P0, configure graceful restart in the main instance, along with OSPF, MPLS, and LDP. This allows the protocols on the PE routers to reach one another.
Router PE2
On Router PE2, configure BGP, OSPF, MPLS, LDP, and graceful restart in the master instance. Configure the following protocols in individual instances on the logical interfaces of t1-0/1/3: a static route on unit 200, OSPF on unit 201, RIP on unit 202, BGP on unit 203, and Frame Relay on unit 612 for the Layer 2 VPN instance. Also configure protocol-specific graceful restart in all routing instances, except the Layer 2 VPN instance.
Router CE2
On Router CE2, complete the Layer 2 and Layer 3 VPN configuration by mirroring the protocols already set on Routers PE2 and CE1. Specifically, configure the following on the logical interfaces of t1-0/0/3: OSPF on unit 201, RIP on unit 202, BGP on unit 203, and IS-IS on unit 612. Finally, configure graceful restart, BGP, IS-IS, OSPF, and RIP on the main instance to be able to connect to the routing instances on Router PE2.
Router PE1 Status Before a Restart
The following example displays neighbor relationships on Router PE1 before a restart happens:
user@PE1> show bgp neighborPeer: 10.96.103.2+3785 AS 65100 Local: 10.96.103.1+179 AS 65103
Type: External State: Established Flags: <>
Last State: OpenConfirm Last Event: RecvKeepAlive
Last Error: None
Export: [ BGP-INET-import ]
Options: <Preference LocalAddress HoldTime GracefulRestart AddressFamily PeerAS Refresh>
Address families configured: inet-unicast
Local Address: 10.96.103.1 Holdtime: 90 Preference: 170
Number of flaps: 0
Peer ID: 10.96.110.1 Local ID: 10.96.103.1 Active Holdtime: 90
Keepalive Interval: 30
Local Interface: t3-0/0/0.103
NLRI for restart configured on peer: inet-unicast
NLRI advertised by peer: inet-unicast
NLRI for this session: inet-unicast
Peer supports Refresh capability (2)
Restart time configured on the peer: 120
Stale routes from peer are kept for: 300
Restart time requested by this peer: 120
NLRI that peer supports restart for: inet-unicast
NLRI peer can save forwarding state: inet-unicast
NLRI that peer saved forwarding for: inet-unicast
NLRI that restart is negotiated for: inet-unicast
NLRI of all end-of-rib markers sent: inet-unicast
Table BGP-INET.inet.0 Bit: 30001
RIB State: BGP restart is complete
RIB State: VPN restart is complete
Send state: in sync
Active prefixes: 0
Received prefixes: 0
Suppressed due to damping: 0
Last traffic (seconds): Received 8 Sent 3 Checked 3
Input messages: Total 15 Updates 0 Refreshes 0 Octets 321
Output messages: Total 18 Updates 2 Refreshes 0 Octets 450
Output Queue[2]: 0
Peer: 10.245.14.182+4701 AS 69 Local: 10.245.14.176+179 AS 69
Type: Internal State: Established Flags: <>
Last State: OpenConfirm Last Event: RecvKeepAlive
Last Error: None
Options: <Preference LocalAddress HoldTime GracefulRestart AddressFamily
Rib-group Refresh>
Address families configured: inet-vpn-unicast l2vpn
Local Address: 10.245.14.176 Holdtime: 90 Preference: 170
Number of flaps: 1
Peer ID: 10.245.14.182 Local ID: 10.245.14.176 Active Holdtime: 90
Keepalive Interval: 30
NLRI for restart configured on peer: inet-vpn-unicast l2vpn
NLRI advertised by peer: inet-vpn-unicast l2vpn
NLRI for this session: inet-vpn-unicast l2vpn
Peer supports Refresh capability (2)
Restart time configured on the peer: 120
Stale routes from peer are kept for: 300
Restart time requested by this peer: 120
NLRI that peer supports restart for: inet-vpn-unicast l2vpn
NLRI peer can save forwarding state: inet-vpn-unicast l2vpn
NLRI that peer saved forwarding for: inet-vpn-unicast l2vpn
NLRI that restart is negotiated for: inet-vpn-unicast l2vpn
NLRI of all end-of-rib markers sent: inet-vpn-unicast l2vpn
Table bgp.l3vpn.0 Bit: 10000
RIB State: BGP restart is complete
RIB State: VPN restart is complete
Send state: in sync
Active prefixes: 0
Received prefixes: 0
Suppressed due to damping: 0
Table bgp.l2vpn.0 Bit: 20000
RIB State: BGP restart is complete
RIB State: VPN restart is complete
Send state: in sync
Active prefixes: 1
Received prefixes: 1
Suppressed due to damping: 0
Table BGP-INET.inet.0 Bit: 30000
RIB State: BGP restart is complete
RIB State: VPN restart is complete
Send state: in sync
Active prefixes: 0
Received prefixes: 0
Suppressed due to damping: 0
Table OSPF.inet.0 Bit: 60000
RIB State: BGP restart is complete
RIB State: VPN restart is complete
Send state: in sync
Active prefixes: 0
Received prefixes: 0
Suppressed due to damping: 0
Table RIP.inet.0 Bit: 70000
RIB State: BGP restart is complete
RIB State: VPN restart is complete
Send state: in sync
Active prefixes: 0
Received prefixes: 0
Suppressed due to damping: 0
Table STATIC.inet.0 Bit: 80000
RIB State: BGP restart is complete
RIB State: VPN restart is complete
Send state: in sync
Active prefixes: 0
Received prefixes: 0
Suppressed due to damping: 0
Table L2VPN.l2vpn.0 Bit: 90000
RIB State: BGP restart is complete
RIB State: VPN restart is complete
Send state: in sync
Active prefixes: 1
Received prefixes: 1
Suppressed due to damping: 0
Last traffic (seconds): Received 28 Sent 28 Checked 28
Input messages: Total 2 Updates 0 Refreshes 0 Octets 86
Output messages: Total 13 Updates 10 Refreshes 0 Octets 1073
Output Queue[0]: 0
Output Queue[1]: 0
Output Queue[2]: 0
Output Queue[3]: 0
Output Queue[4]: 0
Output Queue[5]: 0
Output Queue[6]: 0
Output Queue[7]: 0
Output Queue[8]: 0
user@PE1> show route instance detail
master:
Router ID: 10.245.14.176
Type: forwarding State: Active
Restart State: Complete Path selection timeout: 300
Tables:
inet.0 : 17 routes (15 active, 0 holddown, 1 hidden)
Restart Complete
inet.3 : 2 routes (2 active, 0 holddown, 0 hidden)
Restart Complete
iso.0 : 1 routes (1 active, 0 holddown, 0 hidden)
Restart Complete
mpls.0 : 19 routes (19 active, 0 holddown, 0 hidden)
Restart Complete
bgp.l3vpn.0 : 10 routes (10 active, 0 holddown, 0 hidden)
Restart Complete
inet6.0 : 2 routes (2 active, 0 holddown, 0 hidden)
Restart Complete
bgp.l2vpn.0 : 1 routes (1 active, 0 holddown, 0 hidden)
Restart Complete
BGP-INET:
Router ID: 10.96.103.1
Type: vrf State: Active
Restart State: Complete Path selection timeout: 300
Interfaces:
t3-0/0/0.103
Route-distinguisher: 10.245.14.176:103
Vrf-import: [ BGP-INET-import ]
Vrf-export: [ BGP-INET-export ]
Tables:
BGP-INET.inet.0 : 4 routes (4 active, 0 holddown, 0 hidden)
Restart Complete
L2VPN:
Router ID: 0.0.0.0
Type: l2vpn State: Active
Restart State: Complete Path selection timeout: 300
Interfaces:
t3-0/0/0.512
Route-distinguisher: 10.245.14.176:512
Vrf-import: [ L2VPN-import ]
Vrf-export: [ L2VPN-export ]
Tables:
L2VPN.l2vpn.0 : 2 routes (2 active, 0 holddown, 0 hidden)
Restart Complete
OSPF:
Router ID: 10.96.101.1
Type: vrf State: Active
Restart State: Complete Path selection timeout: 300
Interfaces:
t3-0/0/0.101
Route-distinguisher: 10.245.14.176:101
Vrf-import: [ OSPF-import ]
Vrf-export: [ OSPF-export ]
Tables:
OSPF.inet.0 : 8 routes (7 active, 0 holddown, 0 hidden)
Restart Complete
RIP:
Router ID: 10.96.102.1
Type: vrf State: Active
Restart State: Complete Path selection timeout: 300
Interfaces:
t3-0/0/0.102
Route-distinguisher: 10.245.14.176:102
Vrf-import: [ RIP-import ]
Vrf-export: [ RIP-export ]
Tables:
RIP.inet.0 : 6 routes (6 active, 0 holddown, 0 hidden)
Restart Complete
STATIC:
Router ID: 10.96.100.1
Type: vrf State: Active
Restart State: Complete Path selection timeout: 300
Interfaces:
t3-0/0/0.100
Route-distinguisher: 10.245.14.176:100
Vrf-import: [ STATIC-import ]
Vrf-export: [ STATIC-export ]
Tables:
STATIC.inet.0 : 4 routes (4 active, 0 holddown, 0 hidden)
Restart Complete
__juniper_private1__:
Router ID: 0.0.0.0
Type: forwarding State: Active
user@PE1> show route protocol l2vpn
inet.0: 16 destinations, 17 routes (15 active, 0 holddown, 1 hidden)
Restart Complete
inet.3: 2 destinations, 2 routes (2 active, 0 holddown, 0 hidden)
Restart Complete
BGP-INET.inet.0: 5 destinations, 6 routes (5 active, 0 holddown, 0 hidden)
Restart Complete
OSPF.inet.0: 7 destinations, 8 routes (7 active, 0 holddown, 0 hidden)
Restart Complete
RIP.inet.0: 6 destinations, 6 routes (6 active, 0 holddown, 0 hidden)
Restart Complete
STATIC.inet.0: 4 destinations, 4 routes (4 active, 0 holddown, 0 hidden)
Restart Complete
iso.0: 1 destinations, 1 routes (1 active, 0 holddown, 0 hidden)
Restart Complete
mpls.0: 20 destinations, 20 routes (20 active, 0 holddown, 0 hidden)
Restart Complete
+ = Active Route, - = Last Active, * = Both
800003 *[L2VPN/7] 00:06:00
> via t3-0/0/0.512, Pop Offset: 4
t3-0/0/0.512 *[L2VPN/7] 00:06:00
> via t1-0/1/0.0, Push 800003, Push 100004(top) Offset: -4
bgp.l3vpn.0: 10 destinations, 10 routes (10 active, 0 holddown, 0 hidden)
Restart Complete
inet6.0: 2 destinations, 2 routes (2 active, 0 holddown, 0 hidden)
Restart Complete
L2VPN.l2vpn.0: 2 destinations, 2 routes (2 active, 0 holddown, 0 hidden)
Restart Complete
+ = Active Route, - = Last Active, * = Both
10.245.14.176:512:512:611/96
*[L2VPN/7] 00:06:01
Discard
bgp.l2vpn.0: 1 destinations, 1 routes (1 active, 0 holddown, 0 hidden)
Restart Complete
Router PE1 Status During a Restart
Before you can verify that graceful restart is working, you must simulate a router restart. To cause the routing process to refresh and simulate a restart, use the restart routing operational mode command:
user@PE1> restart routingRouting protocol daemon started, pid 3558
The following sample output is captured during the router restart:
user@PE1> show bgp neighborPeer: 10.96.103.2 AS 65100 Local: 10.96.103.1 AS 65103
Type: External State: Active Flags: <ImportEval>
Last State: Idle Last Event: Start
Last Error: None
Export: [ BGP-INET-import ]
Options: <Preference LocalAddress HoldTime GracefulRestart AddressFamily PeerAS Refresh>
Address families configured: inet-unicast
Local Address: 10.96.103.1 Holdtime: 90 Preference: 170
Number of flaps: 0
Peer: 10.245.14.182+179 AS 69 Local: 10.245.14.176+2131 AS 69
Type: Internal State: Established Flags: <ImportEval>
Last State: OpenConfirm Last Event: RecvKeepAlive
Last Error: None
Options: <Preference LocalAddress HoldTime GracefulRestart AddressFamily Rib-group Refresh>
Address families configured: inet-vpn-unicast l2vpn
Local Address: 10.245.14.176 Holdtime: 90 Preference: 170
Number of flaps: 0
Peer ID: 10.245.14.182 Local ID: 10.245.14.176 Active Holdtime: 90
Keepalive Interval: 30
NLRI for restart configured on peer: inet-vpn-unicast l2vpn
NLRI advertised by peer: inet-vpn-unicast l2vpn
NLRI for this session: inet-vpn-unicast l2vpn
Peer supports Refresh capability (2)
Restart time configured on the peer: 120
Stale routes from peer are kept for: 300
Restart time requested by this peer: 120
NLRI that peer supports restart for: inet-vpn-unicast l2vpn
NLRI peer can save forwarding state: inet-vpn-unicast l2vpn
NLRI that peer saved forwarding for: inet-vpn-unicast l2vpn
NLRI that restart is negotiated for: inet-vpn-unicast l2vpn
NLRI of received end-of-rib markers: inet-vpn-unicast l2vpn
Table bgp.l3vpn.0 Bit: 10000
RIB State: BGP restart in progress
RIB State: VPN restart in progress
Send state: in sync
Active prefixes: 10
Received prefixes: 10
Suppressed due to damping: 0
Table bgp.l2vpn.0 Bit: 20000
RIB State: BGP restart in progress
RIB State: VPN restart in progress
Send state: in sync
Active prefixes: 1
Received prefixes: 1
Suppressed due to damping: 0
Table BGP-INET.inet.0 Bit: 30000
RIB State: BGP restart in progress
RIB State: VPN restart in progress
Send state: in sync
Active prefixes: 2
Received prefixes: 2
Suppressed due to damping: 0
Table OSPF.inet.0 Bit: 60000
RIB State: BGP restart is complete
RIB State: VPN restart in progress
Send state: in sync
Active prefixes: 2
Received prefixes: 2
Suppressed due to damping: 0
Table RIP.inet.0 Bit: 70000
RIB State: BGP restart is complete
RIB State: VPN restart in progress
Send state: in sync
Active prefixes: 2
Received prefixes: 2
Suppressed due to damping: 0
Table STATIC.inet.0 Bit: 80000
RIB State: BGP restart is complete
RIB State: VPN restart in progress
Send state: in sync
Active prefixes: 1
Received prefixes: 1
Suppressed due to damping: 0
Table L2VPN.l2vpn.0 Bit: 90000
RIB State: BGP restart is complete
RIB State: VPN restart in progress
Send state: in sync
Active prefixes: 1
Received prefixes: 1
Suppressed due to damping: 0
Last traffic (seconds): Received 0 Sent 0 Checked 0
Input messages: Total 14 Updates 13 Refreshes 0 Octets 1053
Output messages: Total 3 Updates 0 Refreshes 0 Octets 105
Output Queue[0]: 0
Output Queue[1]: 0
Output Queue[2]: 0
Output Queue[3]: 0
Output Queue[4]: 0
Output Queue[5]: 0
Output Queue[6]: 0
Output Queue[7]: 0
Output Queue[8]: 0
user@PE1> show route instance detail
master:
Router ID: 10.245.14.176
Type: forwarding State: Active
Restart State: Pending Path selection timeout: 300
Tables:
inet.0 : 17 routes (15 active, 1 holddown, 1 hidden)
Restart Pending: OSPF LDP
inet.3 : 2 routes (2 active, 0 holddown, 0 hidden)
Restart Pending: OSPF LDP
iso.0 : 1 routes (1 active, 0 holddown, 0 hidden)
Restart Complete
mpls.0 : 23 routes (23 active, 0 holddown, 0 hidden)
Restart Pending: LDP VPN
bgp.l3vpn.0 : 10 routes (10 active, 0 holddown, 0 hidden)
Restart Pending: BGP VPN
inet6.0 : 2 routes (2 active, 0 holddown, 0 hidden)
Restart Complete
bgp.l2vpn.0 : 1 routes (1 active, 0 holddown, 0 hidden)
Restart Pending: BGP VPN
BGP-INET:
Router ID: 10.96.103.1
Type: vrf State: Active
Restart State: Pending Path selection timeout: 300
Interfaces:
t3-0/0/0.103
Route-distinguisher: 10.245.14.176:103
Vrf-import: [ BGP-INET-import ]
Vrf-export: [ BGP-INET-export ]
Tables:
BGP-INET.inet.0 : 6 routes (5 active, 0 holddown, 0 hidden)
Restart Pending: VPN
L2VPN:
Router ID: 0.0.0.0
Type: l2vpn State: Active
Restart State: Pending Path selection timeout: 300
Interfaces:
t3-0/0/0.512
Route-distinguisher: 10.245.14.176:512
Vrf-import: [ L2VPN-import ]
Vrf-export: [ L2VPN-export ]
Tables:
L2VPN.l2vpn.0 : 2 routes (2 active, 0 holddown, 0 hidden)
Restart Pending: VPN L2VPN
OSPF:
Router ID: 10.96.101.1
Type: vrf State: Active
Restart State: Pending Path selection timeout: 300
Interfaces:
t3-0/0/0.101
Route-distinguisher: 10.245.14.176:101
Vrf-import: [ OSPF-import ]
Vrf-export: [ OSPF-export ]
Tables:
OSPF.inet.0 : 8 routes (7 active, 1 holddown, 0 hidden)
Restart Pending: OSPF VPN
RIP:
Router ID: 10.96.102.1
Type: vrf State: Active
Restart State: Pending Path selection timeout: 300
Interfaces:
t3-0/0/0.102
Route-distinguisher: 10.245.14.176:102
Vrf-import: [ RIP-import ]
Vrf-export: [ RIP-export ]
Tables:
RIP.inet.0 : 8 routes (6 active, 2 holddown, 0 hidden)
Restart Pending: RIP VPN
STATIC:
Router ID: 10.96.100.1
Type: vrf State: Active
Restart State: Pending Path selection timeout: 300
Interfaces:
t3-0/0/0.100
Route-distinguisher: 10.245.14.176:100
Vrf-import: [ STATIC-import ]
Vrf-export: [ STATIC-export ]
Tables:
STATIC.inet.0 : 4 routes (4 active, 0 holddown, 0 hidden)
Restart Pending: VPN
__juniper_private1__:
Router ID: 0.0.0.0
Type: forwarding State: Active
user@PE1> show route instance summaryInstance Type Primary rib Active/holddown/hidden
master forwarding
inet.0 15/0/1
iso.0 1/0/0
mpls.0 35/0/0
l3vpn.0 0/0/0
inet6.0 2/0/0
l2vpn.0 0/0/0
l2circuit.0 0/0/0
BGP-INET vrf
BGP-INET.inet.0 5/0/0
BGP-INET.iso.0 0/0/0
BGP-INET.inet6.0 0/0/0
L2VPN l2vpn
L2VPN.inet.0 0/0/0
L2VPN.iso.0 0/0/0
L2VPN.inet6.0 0/0/0
L2VPN.l2vpn.0 2/0/0
OSPF vrf
OSPF.inet.0 7/0/0
OSPF.iso.0 0/0/0
OSPF.inet6.0 0/0/0
RIP vrf
RIP.inet.0 6/0/0
RIP.iso.0 0/0/0
RIP.inet6.0 0/0/0
STATIC vrf
STATIC.inet.0 4/0/0
STATIC.iso.0 0/0/0
STATIC.inet6.0 0/0/0
__juniper_private1__ forwarding
__juniper_priva.inet.0 0/0/0
__juniper_privat.iso.0 0/0/0
__juniper_priv.inet6.0 0/0/0
user@PE1> show route protocol l2vpn
inet.0: 16 destinations, 17 routes (15 active, 1 holddown, 1 hidden)
Restart Pending: OSPF LDP
inet.3: 2 destinations, 2 routes (2 active, 0 holddown, 0 hidden)
Restart Pending: OSPF LDP
BGP-INET.inet.0: 5 destinations, 6 routes (5 active, 0 holddown, 0 hidden)
Restart Pending: VPN
OSPF.inet.0: 7 destinations, 8 routes (7 active, 1 holddown, 0 hidden)
Restart Pending: OSPF VPN
RIP.inet.0: 6 destinations, 8 routes (6 active, 2 holddown, 0 hidden)
Restart Pending: RIP VPN
STATIC.inet.0: 4 destinations, 4 routes (4 active, 0 holddown, 0 hidden)
Restart Pending: VPN
iso.0: 1 destinations, 1 routes (1 active, 0 holddown, 0 hidden)
Restart Complete
mpls.0: 24 destinations, 24 routes (24 active, 0 holddown, 0 hidden)
Restart Pending: LDP VPN
+ = Active Route, - = Last Active, * = Both
800001 *[L2VPN/7] 00:00:13
> via t3-0/0/0.512, Pop Offset: 4
t3-0/0/0.512 *[L2VPN/7] 00:00:13
> via t1-0/1/0.0, Push 800003, Push 100004(top) Offset: -4
bgp.l3vpn.0: 10 destinations, 10 routes (10 active, 0 holddown, 0 hidden)
Restart Pending: BGP VPN
inet6.0: 2 destinations, 2 routes (2 active, 0 holddown, 0 hidden)
Restart Complete
L2VPN.l2vpn.0: 2 destinations, 2 routes (2 active, 0 holddown, 0 hidden)
Restart Pending: VPN L2VPN
+ = Active Route, - = Last Active, * = Both
10.245.14.176:512:512:611/96
*[L2VPN/7] 00:00:13
Discard
bgp.l2vpn.0: 1 destinations, 1 routes (1 active, 0 holddown, 0 hidden)
Restart Pending: BGP VPN
Hide Navigation Pane
Show Navigation Pane
Download
SHA1