NSX-T EVPN North/South routing with NX-OS EVPN-based Datacenter fabric

From NSX Ninja
Jump to: navigation, search

Summary

With the release of version 3.0, NSX-T supports now VRF Lite and EVPN. VRF support was a long-waited feature to overcome the scalability of NSX-T for multitenancy support. The purpose of this wiki article is to show how to extend NX-OS EVPN based DC network down to the NSX-T Edge level to enable VRF routing between them.

Introduction

With the release of version 3.0, NSX-T supports now VRF Lite and EVPN. VRF support was a long-waited feature to overcome the scalability of NSX-T for multitenancy support.

To support multiple independent routing domains with possible overlapping IP addresses, a Tier-0 Gateway per domain needs to be created. Limitation of a single Tier-0 Gateway SR (Service Router) instance per Edge Node posed scalability problems. E.g., the most used Edge VM of Large form factor requires 32GB RAM, eight vCPU, and 200GB Disk space and can have only one Tier-0 SR. In the case of Bare Metal Edge, only a single Tier-0 SR instance can be hosted on the entire host.

With the introduction of VRF Gateways, 100 VRFs can be associated with a single Tier-0 Gateway, thus essentially boosting the scale of routing domains number that can run on a single Edge Node (both VM and Bare Metal). Tier-0 Gateway routing instance could be compared to a global routing table of the physical router, while VRF Gateway routing instance with a VRF.

To connect individual VRF Gateways northbound with a physical router, VRF Lite can be used. There is a certain administrative overhead configuring VRF Lite northbound routing, as it requires several configurations per VRF Gateway, which includes VLAN, uplink IP addresses, and BGP configuration per individual VRF (This is explained here). It is here EVPN feature can help to decrease administrative overhead.

EVPN is often used in modern Datacenters as a fabric technology like shown in the picture.

NSXT-EVPN-NXOS-1.png

As EVPN is already used in the physical underlay of the DC fabric, is it possible to extend it a bit further from Border GWs to NSX-T Edges and use it for VRF routing exchange?

A lab setup has been built to simulate the above network. The DC EVPN fabric is configured on Nexus 9K virtual switches. NSX-T v3.0.1 has been used to provide the NSX-T domain, and the EVPN feature is used to exchange VRF routing information between the NSX-T domain and DC EVPN Fabric. DC EVPN Border Gateways exchange also routing information with the WAN block.

Here is the lab diagram at its final configuration state:

NSXT-EVPN-NXOS-2.png

The purpose of this wiki article is to show how to extend NX-OS EVPN based DC network down to the NSX-T Edge level to enable VRF routing between them.

  1. Create an EVPN base configuration with one Spine ad three Leaf switches. Two of the Leaf switches will be used as Border GWs. Create 2 VRFs in the EVPN and configure routing to an external to WAN router for these two VRFs.
  2. Configure Base NSX-T setup with a single NSX-T Manager and two Edge VMs nodes running in a combined Management/Edge 3-node ESXi cluster. Prepare a 2-node ESXi Compute cluster for NSX-T.
  3. Create Parent Tier-0 Gateway.
  4. Create Uplink Segments in NSX-T for BGP and uplink interfaces on the Border Gateway Switches (NX-OS).
  5. Create EVPN TEP interfaces on the Parent Tier-0 Gateway (NSX-T) and Multi-site EVPN configuration on the Border Gateway Switches (NX-OS).
  6. Configure BGP between the Parent Tier-0 Gateway (NSX-T) and the Border Gateway Switches (NX-OS).
  7. Create VRF Gateway in NSX-T.
  8. Create Tier-1 Gateway and connect it to the VRF Gateway. Create a test segment and attach it to the Tier-1 Gateway. Attach a test VM to this segment and test connectivity.


Let's dive into more detail into the configuration steps below.

Create EVPN base configuration

The network components of this step are shown on the following diagram:

NSXT-EVPN-NXOS-3.png

This is a classic EVPN setup with IGP unicast-routing (OSPF is used here) and PIM BiDir multicast routing in the underlay. MP-BGP with L2VPN EVPN AFI is configured between Loopback 200 interfaces, where the Spine switch is used as a BGP Route Reflector. Loopback 100 on each switch is used as VXLAN TEP (VTEP). The configuration of this part is out-of-scope of this document. There are plenty of references on the Internet on how to build NX-OS based EVPN fabric. The base configurations of the network are given in Appendix B. at the end of the document. When the base configuration is in place, the Bare Metal servers connected each to own VRF can reach corresponding VRFs in WAN outside the EVPN fabric.

Configure base NSX-T setup

The Base NSX-T setup consists of a single NSX-T Manager (dc1-nsxm-01) and two Edge VMs nodes (dc1-edge-05 and dc1-edge-06) running on a Management/Edge 3-node ESXi cluster. 2-node ESXi Compute cluster is prepared for NSX-T. Here is the view from vCenter:

NSXT-EVPN-NXOS-4.png

Again, the base NSX-T setup creation is out-of-scope of this document. At the base setup, all NSX-T transport nodes (edge and host) are created, but no routing or GENEVE overlay segments exist yet.

Create Parent Tier-0 Gateway

Parent Tier-0 gateway is similar to a global routing instance. Let us create Active/Active Tier-0 named “T0” in the Edge Cluster “dc1-edgecluster-03” (the cluster consists of two edge nodes “dc1-edge-05” and “dc1-edge-06”):

NSXT-EVPN-NXOS-5.png

Notice one new parameter named RD Admin Address with the assigned value “172.16.255.99”. It can be used to automatically assign Route Distinguisher per VRF when advertising VRF routes via MP-BGP. This address is just an ID. It does not belong to any interface and is not reachable. One has to click “Save” before any other Tier-0 Gateway parameters can be configured.

Create link nets between T0 and BGWs (NX-OS)

First, we need VLAN-backed segments on Edge nodes for the Tier-0 Gateway uplinks. Here is an example of segment “sv-nxos-t0-u1-v105” creation, backed by VLAN 105:

NSXT-EVPN-NXOS-6.png

Here is the list of the four created VLAN-backed segments. Some more details are shown for the last created segment:

NSXT-EVPN-NXOS-7.png

Second, let us create Interfaces on the parent Tier-0 Gateway “T0”. Here is an example of interface “edge6-uplink2” creation:

NSXT-EVPN-NXOS-8.png

Note MTU size of 1600 bytes is configured. It is necessary to provide higher MTU than standard 1500 bytes to accommodate VXLAN encapsulation for VRF data plane forwarding between NSX-T and NX-OS EVPN domains. Note also that URPF Mode has to be set to “None.”

Here is the list of the created interfaces on the parent Tier-0 Gateway “T0” with some details shown for 2 of the interfaces:

NSXT-EVPN-NXOS-9.png

And now the corresponding interface has to be created on the Border Gateways BGW-1 and BGW-2:

Tip

In this article, it is explained how you set up the NX-OS based Nexus Switches on the ESXi Hypervisor

Important

The NX-OS version needs to be 9.2(4) (or higher) on the Border Gateways (BGW-X). Cisco Documentation states that Cisco NX-OS Software Release 7.0(3)I7(1) or later can be used, but version 7 was missing EVPN Multisite commands.

BGW-1

interface Ethernet1/3
  no switchport
  mtu 1600
  no shutdown
 
interface Ethernet1/3.105
 mtu 1600
 encapsulation dot1q 105
 ip address 172.16.105.1/24
 no shutdown
 
interface Ethernet1/4
  no switchport
  mtu 1600
  no shutdown
 
interface Ethernet1/4.106
 mtu 1600
 encapsulation dot1q 106
 ip address 172.16.106.1/24
 no shutdown

BGW-2

interface Ethernet1/3
  no switchport
  mtu 1600
  no shutdown
 
 interface Ethernet1/3.107
  mtu 1600
  encapsulation dot1q 107
  ip address 172.16.107.1/24
  no shutdown
 
interface Ethernet1/4
  no switchport
  mtu 1600
  no shutdown
 
 interface Ethernet1/4.108
  mtu 1600
  encapsulation dot1q 108
  ip address 172.16.108.1/24
  no shutdown

Note MTU of 1600 bytes is configured as on the T0 uplinks.

Here is the network diagram at this stage:

NSXT-EVPN-NXOS-10.png

Prepare T0 and BGWs for EVPN

Some extra configuration steps are needed to prepare T0 and BGWs for EVPN and VRF routing.

First, we need a VNI pool for VXLANs used for VRF data plane forwarding between NSX-T and NX-OS EVPN domains. Here a pool named “nxos-vni-pool” consisting of 100 VNIs is created:

NSXT-EVPN-NXOS-11.png

Second, EVPN VNI pool need to be assigned to the parent Tier-0 GW “T0”:

NSXT-EVPN-NXOS-12.png

Third, EVPN Tunnel Endpoint interfaces are needed on the parent Tier-0 GW “T0”:

NSXT-EVPN-NXOS-13.png

Now let us prepare the NX-OS part on the BGW-1 and BGW-2. We need to add “Multi-site behavior” to the base configuration of these leaf switches. The Multi-site configuration has been derived from “VXLAN EVPN Multi-Site Design and Deployment White Paper”. Commands added to the baseline configuration are Bold/Italic and yellow highlighted:

BGW-1

evpn multisite border-gateway 12
  delay-restore time 30

interface loopback112
  ip address 192.168.100.112/32 tag 112
  ip router ospf 0 area 0.0.0.0

interface nve1
  no shutdown
  host-reachability protocol bgp
  source-interface loopback100
  multisite border-gateway interface loopback112
  member vni 200003 associate-vrf
  member vni 200004 associate-vrf
 
interface Ethernet1/1
  no switchport
  mtu 9000
  medium p2p
  ip unnumbered loopback0
  ip ospf network point-to-point
  ip router ospf 0 area 0.0.0.0
  ip pim sparse-mode
  evpn multisite fabric-tracking
  no shutdown

BGW-2

evpn multisite border-gateway 12
  delay-restore time 30

interface loopback112
  ip address 192.168.100.112/32 tag 112
  ip router ospf 0 area 0.0.0.0

interface nve1
  no shutdown
  host-reachability protocol bgp
  source-interface loopback100
  multisite border-gateway interface loopback112
  member vni 200003 associate-vrf
  member vni 200004 associate-vrf
 
interface Ethernet1/1
  no switchport
  mtu 9000
  medium p2p
  ip unnumbered loopback0
  ip ospf network point-to-point
  ip router ospf 0 area 0.0.0.0
  ip pim sparse-mode
  evpn multisite fabric-tracking
  no shutdown

Configure BGP between T0 and BGWs

Parent Tier-0 GW BGP configuration does not differ much from the usual case. Here T0 is configured in ASN 65022, the rest is default (Graceful Restart is chosen here to be disabled as it does not give any benefit in the lab environment):

NSXT-EVPN-NXOS-14.png

There are, however, a couple of extra parameters when configuring BGP neighbors. First, it is necessary to change the Max Hop Limit from default value of 1 due to the fact that VXLAN TEP interfaces are now several hops away from T0 to BGW-1 and BGW-2 (max value of 255 is used here, but 3 would be enough):

NSXT-EVPN-NXOS-15.png

Second, it is necessary in Router Filter configuration add IPV4 and L2VPN EVPN address families to enable MP-BGP:

NSXT-EVPN-NXOS-16.png

Here is the list of T0 neighbors with some more details shown for one of the neighbors:

NSXT-EVPN-NXOS-17.png

The last important step configuring routing between T0 and BGW-1 and BGW-2 is the announcement of EVPN TEP IP from T0 in BGP IPv4 address family:

NSXT-EVPN-NXOS-18.png

Its time to configure BGW-1 and BGW-2 now.

BGW-1

! Prepare for T0 VTEP IPs redistribution into OSPF: 
ip prefix-list NSXT-TEPS seq 5 permit 172.16.255.105/32
ip prefix-list NSXT-TEPS seq 6 permit 172.16.255.106/32
route-map BGP2OSPF permit 10
  match ip address prefix-list NSXT-TEPS

router ospf 0
  router-id 192.168.0.101
  ! Redistribute T0 VTEP IPs into OSPF:
  redistribute bgp 64999 route-map BGP2OSPF

router bgp 64999
  router-id 192.168.200.101
  address-family ipv4 unicast
    ! Announce BGW-1 TEP IP to T0:
    network 192.168.100.101/32
    maximum-paths 8
  address-family l2vpn evpn
    maximum-paths 8
  neighbor 172.16.105.2
    remote-as 65022
    update-source Ethernet1/3.105
    ! eBGP Multihop is necessary for VTEP next-hop reachability:
    ebgp-multihop 255
    timers 4 12
    address-family ipv4 unicast
    address-family l2vpn evpn
      send-community
      send-community extended
      ! This will rewrite auto-assigned RT 64999:VNI <-> 65022:VNI
      rewrite-evpn-rt-asn
  neighbor 172.16.106.2
    remote-as 65022
    update-source Ethernet1/4.106
    ! eBGP Multihop is necessary for VTEP next-hop reachability:
    ebgp-multihop 255
    timers 4 12
    address-family ipv4 unicast
    address-family l2vpn evpn
      send-community
      send-community extended
      ! This will rewrite auto-assigned RT 64999:VNI <-> 65022:VNI
      rewrite-evpn-rt-asn

BGW-2

! Prepare for T0 VTEP IPs redistribution into OSPF: 
ip prefix-list NSXT-TEPS seq 5 permit 172.16.255.105/32
ip prefix-list NSXT-TEPS seq 6 permit 172.16.255.106/32
route-map BGP2OSPF permit 10
  match ip address prefix-list NSXT-TEPS

router ospf 0
  router-id 192.168.0.102
  ! Redistribute T0 VTEP IPs into OSPF:
  redistribute bgp 64999 route-map BGP2OSPF

router bgp 64999
  router-id 192.168.200.102
  address-family ipv4 unicast
    ! Announce BGW-2 TEP IP to T0:
    network 192.168.100.102/32
    maximum-paths 8
  address-family l2vpn evpn
    maximum-paths 8
  neighbor 172.16.107.2
    remote-as 65022
    update-source Ethernet1/3.107
    ! eBGP Multihop is necessary for VTEP next-hop reachability:
    ebgp-multihop 255
    timers 4 12
    address-family ipv4 unicast
    address-family l2vpn evpn
      send-community
      send-community extended
      ! This will rewrite auto-assigned RT 64999:VNI <-> 65022:VNI
      rewrite-evpn-rt-asn
  neighbor 172.16.108.2
    remote-as 65022
    update-source Ethernet1/4.108
    ! eBGP Multihop is necessary for VTEP next-hop reachability:
    ebgp-multihop 255
    timers 4 12
    address-family ipv4 unicast
    address-family l2vpn evpn
      send-community
      send-community extended
      ! This will rewrite auto-assigned RT 64999:VNI <-> 65022:VNI
      rewrite-evpn-rt-asn

Complete final configurations of BGW-1 and BGW-2 are given in Appendix C. at the end of this article.

Here is the lab network diagram after configuring BGP:

NSXT-EVPN-NXOS-19.png

Note the full redundant connectivity between the NSX-T and NX-OS EVPN domain. Failure or maintenance of one of the BGW nodes and simultaneous loss or maintenance of one of the Edge Nodes do not affect North/South connectivity between the two domains. Failover relies on the dynamic routing protocol convergence.

Create VRF in NSX-T

VRF in NSX-T are configured as VRF Gateway, which is a child Tier-0 object to a parent Tier-0 Gateway. As a child object, VRF Gateway inherits many configuration parameters from parent Tier-0 that cannot be changed in an individual VRF.

Let us create VRF Gateway named “T0-vrf3” and associated with parent Tier-0 “T0” (corresponding VRF in NX-OS EVPN fabric is named “VRF3”):

NSXT-EVPN-NXOS-20.png

Not entering Route Distinguisher value auto-assigns it. Alternatively, it can be assigned manually.

The value of EVPN transit VNI must match the VNI value used in NX-OS EVPN fabric for VRF3. This value can be found on any EVPN border gateway switch configured for VRF3 (e.g. from BGW-1):

NSXT-EVPN-NXOS-21.png

Note that VRF VNI in NX-OS EVPN fabric must be within the earlier defined “nxos-vni-pool”.

Similar to NX-OS VRF configuration it is necessary to define Route Target (RT) in the VRF Gateway “T0-vrf3”:

NSXT-EVPN-NXOS-22.png

RT can be set manually or assigned automatically. Auto-assigned RT has a format <Parent Tier-0 ASN>:<VRF VNI>. With the values chosen for the T0 and VRF3, the auto-assigned RT is 65022:200003.

Need to extend another VRF to NSX-T? Let’s say VRF4? Just find out VRF VNI configured in the NX-OS EVPN fabric (in our case it is 200004), use auto-assignment of RD and RT, and with a couple of clicks you create VRF Gateway named e.g. “T0-vrf4”. That easy it is! All routing is established automatically without any extra configuration steps!

Connect Tier-1 GW, segment and a VM to VRF in NSX-T

The procedure of connecting the Tier-1 Gateway to the VRF Gateway is the same as connecting it to Tier-0 Gateway. If Tier-1 does not exist yet, let us create one named “T1-vrf3-01” and connect it to the VRF Gateway “T0-vrf3”:

NSXT-EVPN-NXOS-23.png

Optionally, if there is a plan to use services on the Tier-1 (e.g. NAT, Load Balancing, etc.), it can be associated with an edge cluster. For the connectivity tests let us advertise segments connected to this Tier-1 Gateway.

Now we create an overlay segment named “so-dc1-04”, connect it to the “T1-vrf3-01” and assign an IP/subnet “172.16.40.1/24”:

NSXT-EVPN-NXOS-24.png

Finally, let us connect a VM named “dc1-app2-web1” configured with IP 172.16.40.11/24 and Default GW 172.16.40.1 to this segment:

NSXT-EVPN-NXOS-25.png

Simple ping test from this VM show data plane connectivity within VRF3 to a server (IP address 10.3.31.11) attached to the leaf L-103 of NX-OS EVPN fabric and to an endpoint (IP address 10.3.100.1) in WAN block outside Datacenter connected via router “lab-csr1k”:

NSXT-EVPN-NXOS-26.png

Here is the lab diagram at this stage:

NSXT-EVPN-NXOS-27.png

Similar to VRF3 a Tier-1 Gateway, a segment, and a VM can be added to the VRF4 inside the NSX-T domain and the lab will reach its final look as given in the “LAB Network Infrastructure” diagram at the beginning of this document.

Conclusion

I have shown you how to interconnect NX-OS EVPN based DC network with NSX-T using EVPN and enable VRF routing between the two domains. EVPN interconnect configuration is not too simple but configured once allows very easily and quickly add additional VRFs to the NSX-T domain. If a VRF already exists in the Data Center EVPN fabric, extending this VRF to the NSX-T domain requires only a single parameter - VXLAN VNI, when auto-assignment of Route Distinguisher and Route Target is used. This dramatically simplifies operations of creation, configuration, and maintenance of VRFs in NSX-T.

Appendices

Appendix A. Control Plane “show” commands output.

In this appendix, the output of some “show” commands is given focusing on VRF3 and Global routing table (VRF “default” in NX-OS terms).

For VRF3, three IP prefixes are of interest will be highlighted yellow:

  • 172.16.40.0/24 is an overlay segment located in the NSX-T domain
  • 10.3.31.11/32 of BM server attached to leaf L-103 in NX-OS EVPN domain
  • 10.3.100.0/22 of WAN block outside Data Center.


For the Global routing table, IP addresses of VXLAN TEPs are of interest and will be highlighted blue:

  • 192.168.100.101 – NX-OS border gateway BGW-1
  • 192.168.100.102 – NX-OS border gateway BGW-2
  • 192.168.100.103 – NX-OS leaf switch L-103
  • 172.16.255.105 – NSX-T SR component of T0 gateway on edge node dc1-edge-05
  • 172.16.255.106 – NSX-T SR component of T0 gateway on edge node dc1-edge-06

BGW-1

BGW-1# show ip route vrf VRF3
IP Route Table for VRF "VRF3"
'*' denotes best ucast next-hop
'**' denotes best mcast next-hop
'[x/y]' denotes [preference/metric]
'%<string>' in via output denotes VRF <string>

10.3.0.101/32, ubest/mbest: 2/0, attached
    *via 10.3.0.101, Lo3, [0/0], 3w5d, local
    *via 10.3.0.101, Lo3, [0/0], 3w5d, direct
10.3.0.102/32, ubest/mbest: 1/0
    *via 192.168.100.102%default, [200/0], 2w4d, bgp-64999, internal, tag 64999 (evpn) segid: 200003 tunnelid: 0xc0a86466 encap: VXLAN

10.3.0.103/32, ubest/mbest: 1/0
    *via 192.168.100.103%default, [200/0], 2w4d, bgp-64999, internal, tag 64999 (evpn) segid: 200003 tunnelid: 0xc0a86467 encap: VXLAN

10.3.30.0/24, ubest/mbest: 1/0, attached
    *via 10.3.30.1, Eth1/2.30, [0/0], 2w4d, direct
10.3.30.1/32, ubest/mbest: 1/0, attached
    *via 10.3.30.1, Eth1/2.30, [0/0], 2w4d, local
10.3.31.11/32, ubest/mbest: 1/0
    *via 192.168.100.103%default, [200/0], 2w4d, bgp-64999, internal, tag 64999 (evpn) segid: 200003 tunnelid: 0xc0a86467 encap: VXLAN

10.3.100.0/22, ubest/mbest: 1/0
    *via 10.3.30.3, [20/0], 2w4d, bgp-64999, external, tag 100
172.16.40.0/24, ubest/mbest: 2/0
    *via 172.16.255.105%default, [20/0], 1d22h, bgp-64999, external, tag 65022 (evpn) segid: 200003 tunnelid: 0xac10ff69 encap: VXLAN

    *via 172.16.255.106%default, [20/0], 1d22h, bgp-64999, external, tag 65022 (evpn) segid: 200003 tunnelid: 0xac10ff6a encap: VXLAN

BGW-1# show bgp vrf VRF3 all
BGP routing table information for VRF VRF3, address family IPv4 Unicast
BGP table version is 69, Local Router ID is 10.3.0.101
Status: s-suppressed, x-deleted, S-stale, d-dampened, h-history, *-valid, >-best
Path type: i-internal, e-external, c-confed, l-local, a-aggregate, r-redist, I-injected
Origin codes: i - IGP, e - EGP, ? - incomplete, - - multipath, & - backup, 2 - best2

   Network            Next Hop            Metric     LocPrf     Weight Path
*>l10.3.0.101/32      0.0.0.0                           100      32768 i
*>i10.3.0.102/32      192.168.100.102                   100          0 i
*>i10.3.0.103/32      192.168.100.103                   100          0 i
*>i10.3.31.11/32      192.168.100.103                   100          0 i
*>e10.3.100.0/22      10.3.30.3                0                     0 100 i
* i                   192.168.100.102          0        100          0 100 i
*/e172.16.40.0/24     172.16.255.106           0                     0 65022 ?
*>e                   172.16.255.105           0                     0 65022 ?

BGW-1# show bgp l2vpn evpn
BGP routing table information for VRF default, address family L2VPN EVPN
BGP table version is 7424, Local Router ID is 192.168.200.101
Status: s-suppressed, x-deleted, S-stale, d-dampened, h-history, *-valid, >-best
Path type: i-internal, e-external, c-confed, l-local, a-aggregate, r-redist, I-injected
Origin codes: i - IGP, e - EGP, ? - incomplete, - - multipath, & - backup, 2 - best2

   Network            Next Hop            Metric     LocPrf     Weight Path
Route Distinguisher: 172.16.255.99:50797
*/e[5]:[0]:[0]:[24]:[172.16.40.0]/224
                      172.16.255.106           0                     0 65022 ?
*>e                   172.16.255.105           0                     0 65022 ?

Route Distinguisher: 192.168.200.102:5
*>i[5]:[0]:[0]:[22]:[10.3.100.0]/224
                      192.168.100.102          0        100          0 100 i
*>i[5]:[0]:[0]:[32]:[10.3.0.102]/224
                      192.168.100.102                   100          0 i

Route Distinguisher: 192.168.200.102:6
*>i[5]:[0]:[0]:[22]:[10.4.100.0]/224
                      192.168.100.102          0        100          0 100 i
*>i[5]:[0]:[0]:[32]:[10.4.0.102]/224
                      192.168.100.102                   100          0 i

Route Distinguisher: 192.168.200.103:5
*>i[5]:[0]:[0]:[32]:[10.3.0.103]/224
                      192.168.100.103                   100          0 i

Route Distinguisher: 192.168.200.103:6
*>i[5]:[0]:[0]:[32]:[10.4.0.103]/224
                      192.168.100.103                   100          0 i

Route Distinguisher: 192.168.200.103:32798
*>i[2]:[0]:[0]:[48]:[0050.56be.84bd]:[32]:[10.3.31.11]/272
                      192.168.100.103                   100          0 i

Route Distinguisher: 192.168.200.101:5    (L3VNI 200003)
*>l[5]:[0]:[0]:[22]:[10.3.100.0]/224
                      192.168.100.101          0                     0 100 i
*>l[5]:[0]:[0]:[32]:[10.3.0.101]/224
                      192.168.100.101                   100      32768 i

Route Distinguisher: 192.168.200.101:6    (L3VNI 200004)
*>l[5]:[0]:[0]:[22]:[10.4.100.0]/224
                      192.168.100.101          0                     0 100 i
*>l[5]:[0]:[0]:[32]:[10.4.0.101]/224
                      192.168.100.101                   100      32768 i

BGW-1# show ip route
IP Route Table for VRF "default"
'*' denotes best ucast next-hop
'**' denotes best mcast next-hop
'[x/y]' denotes [preference/metric]
'%<string>' in via output denotes VRF <string>

172.16.105.0/24, ubest/mbest: 1/0, attached
    *via 172.16.105.1, Eth1/3.105, [0/0], 3w2d, direct
172.16.105.1/32, ubest/mbest: 1/0, attached
    *via 172.16.105.1, Eth1/3.105, [0/0], 3w2d, local
172.16.106.0/24, ubest/mbest: 1/0, attached
    *via 172.16.106.1, Eth1/4.106, [0/0], 1d22h, direct
172.16.106.1/32, ubest/mbest: 1/0, attached
    *via 172.16.106.1, Eth1/4.106, [0/0], 1d22h, local
172.16.107.0/24, ubest/mbest: 1/0
    *via 172.16.105.2, [20/0], 3w2d, bgp-64999, external, tag 65022
172.16.108.0/24, ubest/mbest: 1/0
    *via 172.16.106.2, [20/0], 1d22h, bgp-64999, external, tag 65022
172.16.255.105/32, ubest/mbest: 1/0
    *via 172.16.105.2, [20/0], 3w2d, bgp-64999, external, tag 65022
172.16.255.106/32, ubest/mbest: 1/0
    *via 172.16.106.2, [20/0], 1d22h, bgp-64999, external, tag 65022
192.168.0.11/32, ubest/mbest: 1/0
    *via 192.168.0.11, Eth1/1, [110/41], 3w5d, ospf-0, intra
192.168.0.101/32, ubest/mbest: 2/0, attached
    *via 192.168.0.101, Lo0, [0/0], 3w5d, local
    *via 192.168.0.101, Lo0, [0/0], 3w5d, direct
192.168.0.102/32, ubest/mbest: 1/0
    *via 192.168.0.11, Eth1/1, [110/81], 3w5d, ospf-0, intra
192.168.0.103/32, ubest/mbest: 1/0
    *via 192.168.0.11, Eth1/1, [110/81], 3w5d, ospf-0, intra
192.168.100.101/32, ubest/mbest: 2/0, attached
    *via 192.168.100.101, Lo100, [0/0], 3w5d, local
    *via 192.168.100.101, Lo100, [0/0], 3w5d, direct
192.168.100.102/32, ubest/mbest: 1/0
    *via 192.168.0.11, Eth1/1, [110/81], 3w5d, ospf-0, intra
192.168.100.103/32, ubest/mbest: 1/0
    *via 192.168.0.11, Eth1/1, [110/81], 3w5d, ospf-0, intra
192.168.200.11/32, ubest/mbest: 1/0
    *via 192.168.0.11, Eth1/1, [110/41], 3w5d, ospf-0, intra
192.168.200.101/32, ubest/mbest: 2/0, attached
    *via 192.168.200.101, Lo200, [0/0], 3w5d, local
    *via 192.168.200.101, Lo200, [0/0], 3w5d, direct
192.168.200.102/32, ubest/mbest: 1/0
    *via 192.168.0.11, Eth1/1, [110/81], 3w5d, ospf-0, intra
192.168.200.103/32, ubest/mbest: 1/0
    *via 192.168.0.11, Eth1/1, [110/81], 3w5d, ospf-0, intra
192.168.255.0/30, ubest/mbest: 1/0
    *via 192.168.0.11, Eth1/1, [110/41], 3w5d, ospf-0, intra


BGW-1# show bgp ipv4 unicast
BGP routing table information for VRF default, address family IPv4 Unicast
BGP table version is 58, Local Router ID is 192.168.200.101
Status: s-suppressed, x-deleted, S-stale, d-dampened, h-history, *-valid, >-best
Path type: i-internal, e-external, c-confed, l-local, a-aggregate, r-redist, I-injected
Origin codes: i - IGP, e - EGP, ? - incomplete, - - multipath, & - backup, 2 - best2

   Network            Next Hop            Metric     LocPrf     Weight Path
*>e172.16.105.0/24    172.16.105.2             0                     0 65022 ?
*>e172.16.106.0/24    172.16.106.2             0                     0 65022 ?
*>e172.16.107.0/24    172.16.105.2             0                     0 65022 ?
*>e172.16.108.0/24    172.16.106.2             0                     0 65022 ?
*>e172.16.255.105/32  172.16.105.2             0                     0 65022 ?
*>e172.16.255.106/32  172.16.106.2             0                     0 65022 ?
*>l192.168.100.101/32 0.0.0.0                           100      32768 i

BGW-2

BGW-2# show ip route vrf VRF3
IP Route Table for VRF "VRF3"
'*' denotes best ucast next-hop
'**' denotes best mcast next-hop
'[x/y]' denotes [preference/metric]
'%<string>' in via output denotes VRF <string>

10.3.0.101/32, ubest/mbest: 1/0
    *via 192.168.100.101%default, [200/0], 2w4d, bgp-64999, internal, tag 64999 (evpn) segid: 200003 tunnelid: 0xc0a86465 encap: VXLAN

10.3.0.102/32, ubest/mbest: 2/0, attached
    *via 10.3.0.102, Lo3, [0/0], 3w5d, local
    *via 10.3.0.102, Lo3, [0/0], 3w5d, direct
10.3.0.103/32, ubest/mbest: 1/0
    *via 192.168.100.103%default, [200/0], 2w4d, bgp-64999, internal, tag 64999 (evpn) segid: 200003 tunnelid: 0xc0a86467 encap: VXLAN

10.3.30.0/24, ubest/mbest: 1/0, attached
    *via 10.3.30.2, Eth1/2.30, [0/0], 3w1d, direct
10.3.30.2/32, ubest/mbest: 1/0, attached
    *via 10.3.30.2, Eth1/2.30, [0/0], 3w1d, local
10.3.31.11/32, ubest/mbest: 1/0
    *via 192.168.100.103%default, [200/0], 2w4d, bgp-64999, internal, tag 64999 (evpn) segid: 200003 tunnelid: 0xc0a86467 encap: VXLAN

10.3.100.0/22, ubest/mbest: 1/0
    *via 10.3.30.3, [20/0], 2w4d, bgp-64999, external, tag 100
172.16.40.0/24, ubest/mbest: 2/0
    *via 172.16.255.105%default, [20/0], 1w1d, bgp-64999, external, tag 65022 (evpn) segid: 200003 tunnelid: 0xac10ff69 encap: VXLAN

    *via 172.16.255.106%default, [20/0], 1w1d, bgp-64999, external, tag 65022 (evpn) segid: 200003 tunnelid: 0xac10ff6a encap: VXLAN

BGW-2# show bgp vrf VRF3 all
BGP routing table information for VRF VRF3, address family IPv4 Unicast
BGP table version is 74, Local Router ID is 10.3.0.102
Status: s-suppressed, x-deleted, S-stale, d-dampened, h-history, *-valid, >-best
Path type: i-internal, e-external, c-confed, l-local, a-aggregate, r-redist, I-injected
Origin codes: i - IGP, e - EGP, ? - incomplete, - - multipath, & - backup, 2 - best2

   Network            Next Hop            Metric     LocPrf     Weight Path
*>i10.3.0.101/32      192.168.100.101                   100          0 i
*>l10.3.0.102/32      0.0.0.0                           100      32768 i
*>i10.3.0.103/32      192.168.100.103                   100          0 i
*>i10.3.31.11/32      192.168.100.103                   100          0 i
* i10.3.100.0/22      192.168.100.101          0        100          0 100 i
*>e                   10.3.30.3                0                     0 100 i
*/e172.16.40.0/24     172.16.255.106           0                     0 65022 ?
*>e                   172.16.255.105           0                     0 65022 ?

BGW-2# show bgp l2vpn evpn
BGP routing table information for VRF default, address family L2VPN EVPN
BGP table version is 7379, Local Router ID is 192.168.200.102
Status: s-suppressed, x-deleted, S-stale, d-dampened, h-history, *-valid, >-best
Path type: i-internal, e-external, c-confed, l-local, a-aggregate, r-redist, I-injected
Origin codes: i - IGP, e - EGP, ? - incomplete, - - multipath, & - backup, 2 - best2

   Network            Next Hop            Metric     LocPrf     Weight Path
Route Distinguisher: 172.16.255.99:50797
*/e[5]:[0]:[0]:[24]:[172.16.40.0]/224
                      172.16.255.106           0                     0 65022 ?
*>e                   172.16.255.105           0                     0 65022 ?
* i                   172.16.255.105           0        100          0 65022 ?

Route Distinguisher: 192.168.200.101:5
*>i[5]:[0]:[0]:[22]:[10.3.100.0]/224
                      192.168.100.101          0        100          0 100 i
*>i[5]:[0]:[0]:[32]:[10.3.0.101]/224
                      192.168.100.101                   100          0 i

Route Distinguisher: 192.168.200.101:6
*>i[5]:[0]:[0]:[22]:[10.4.100.0]/224
                      192.168.100.101          0        100          0 100 i
*>i[5]:[0]:[0]:[32]:[10.4.0.101]/224
                      192.168.100.101                   100          0 i

Route Distinguisher: 192.168.200.103:5
*>i[5]:[0]:[0]:[32]:[10.3.0.103]/224
                      192.168.100.103                   100          0 i

Route Distinguisher: 192.168.200.103:6
*>i[5]:[0]:[0]:[32]:[10.4.0.103]/224
                      192.168.100.103                   100          0 i

Route Distinguisher: 192.168.200.103:32798
*>i[2]:[0]:[0]:[48]:[0050.56be.84bd]:[32]:[10.3.31.11]/272
                      192.168.100.103                   100          0 i

Route Distinguisher: 192.168.200.102:5    (L3VNI 200003)
*>l[5]:[0]:[0]:[22]:[10.3.100.0]/224
                      192.168.100.102          0                     0 100 i
*>l[5]:[0]:[0]:[32]:[10.3.0.102]/224
                      192.168.100.102                   100      32768 i

Route Distinguisher: 192.168.200.102:6    (L3VNI 200004)
*>l[5]:[0]:[0]:[22]:[10.4.100.0]/224
                      192.168.100.102          0                     0 100 i
*>l[5]:[0]:[0]:[32]:[10.4.0.102]/224
                      192.168.100.102                   100      32768 i

BGW-2# show ip route
IP Route Table for VRF "default"
'*' denotes best ucast next-hop
'**' denotes best mcast next-hop
'[x/y]' denotes [preference/metric]
'%<string>' in via output denotes VRF <string>

172.16.105.0/24, ubest/mbest: 1/0
    *via 172.16.107.2, [20/0], 3w1d, bgp-64999, external, tag 65022
172.16.106.0/24, ubest/mbest: 1/0
    *via 172.16.108.2, [20/0], 1w1d, bgp-64999, external, tag 65022
172.16.107.0/24, ubest/mbest: 1/0, attached
    *via 172.16.107.1, Eth1/3.107, [0/0], 3w1d, direct
172.16.107.1/32, ubest/mbest: 1/0, attached
    *via 172.16.107.1, Eth1/3.107, [0/0], 3w1d, local
172.16.108.0/24, ubest/mbest: 1/0, attached
    *via 172.16.108.1, Eth1/4.108, [0/0], 3w1d, direct
172.16.108.1/32, ubest/mbest: 1/0, attached
    *via 172.16.108.1, Eth1/4.108, [0/0], 3w1d, local
172.16.255.105/32, ubest/mbest: 1/0
    *via 172.16.107.2, [20/0], 3w1d, bgp-64999, external, tag 65022
172.16.255.106/32, ubest/mbest: 1/0
    *via 172.16.108.2, [20/0], 1w1d, bgp-64999, external, tag 65022
192.168.0.11/32, ubest/mbest: 1/0
    *via 192.168.0.11, Eth1/1, [110/41], 3w5d, ospf-0, intra
192.168.0.101/32, ubest/mbest: 1/0
    *via 192.168.0.11, Eth1/1, [110/81], 3w5d, ospf-0, intra
192.168.0.102/32, ubest/mbest: 2/0, attached
    *via 192.168.0.102, Lo0, [0/0], 3w5d, local
    *via 192.168.0.102, Lo0, [0/0], 3w5d, direct
192.168.0.103/32, ubest/mbest: 1/0
    *via 192.168.0.11, Eth1/1, [110/81], 3w5d, ospf-0, intra
192.168.100.101/32, ubest/mbest: 1/0
    *via 192.168.0.11, Eth1/1, [110/81], 3w5d, ospf-0, intra
192.168.100.102/32, ubest/mbest: 2/0, attached
    *via 192.168.100.102, Lo100, [0/0], 3w5d, local
    *via 192.168.100.102, Lo100, [0/0], 3w5d, direct
192.168.100.103/32, ubest/mbest: 1/0
    *via 192.168.0.11, Eth1/1, [110/81], 3w5d, ospf-0, intra
192.168.200.11/32, ubest/mbest: 1/0
    *via 192.168.0.11, Eth1/1, [110/41], 3w5d, ospf-0, intra
192.168.200.101/32, ubest/mbest: 1/0
    *via 192.168.0.11, Eth1/1, [110/81], 3w5d, ospf-0, intra
192.168.200.102/32, ubest/mbest: 2/0, attached
    *via 192.168.200.102, Lo200, [0/0], 3w5d, local
    *via 192.168.200.102, Lo200, [0/0], 3w5d, direct
192.168.200.103/32, ubest/mbest: 1/0
    *via 192.168.0.11, Eth1/1, [110/81], 3w5d, ospf-0, intra
192.168.255.0/30, ubest/mbest: 1/0
    *via 192.168.0.11, Eth1/1, [110/41], 3w5d, ospf-0, intra

BGW-2# sho bgp ipv4 unicast
BGP routing table information for VRF default, address family IPv4 Unicast
BGP table version is 80, Local Router ID is 192.168.200.102
Status: s-suppressed, x-deleted, S-stale, d-dampened, h-history, *-valid, >-best
Path type: i-internal, e-external, c-confed, l-local, a-aggregate, r-redist, I-injected
Origin codes: i - IGP, e - EGP, ? - incomplete, - - multipath, & - backup, 2 - best2

   Network            Next Hop            Metric     LocPrf     Weight Path
*>e172.16.105.0/24    172.16.107.2             0                     0 65022 ?
*>e172.16.106.0/24    172.16.108.2             0                     0 65022 ?
*>e172.16.107.0/24    172.16.107.2             0                     0 65022 ?
*>e172.16.108.0/24    172.16.108.2             0                     0 65022 ?
*>e172.16.255.105/32  172.16.107.2             0                     0 65022 ?
*>e172.16.255.106/32  172.16.108.2             0                     0 65022 ?
*>l192.168.100.102/32 0.0.0.0                           100      32768 i

L-103

L-103# show ip route vrf VRF3
IP Route Table for VRF "VRF3"
'*' denotes best ucast next-hop
'**' denotes best mcast next-hop
'[x/y]' denotes [preference/metric]
'%<string>' in via output denotes VRF <string>

10.3.0.101/32, ubest/mbest: 1/0
    *via 192.168.100.101%default, [200/0], 2w4d, bgp-64999, internal, tag 64999 (evpn) segid: 200003 tunnelid: 0xc0a86465 encap: VXLAN

10.3.0.102/32, ubest/mbest: 1/0
    *via 192.168.100.102%default, [200/0], 2w4d, bgp-64999, internal, tag 64999 (evpn) segid: 200003 tunnelid: 0xc0a86466 encap: VXLAN

10.3.0.103/32, ubest/mbest: 2/0, attached
    *via 10.3.0.103, Lo3, [0/0], 2w4d, local
    *via 10.3.0.103, Lo3, [0/0], 2w4d, direct
10.3.31.0/24, ubest/mbest: 1/0, attached
    *via 10.3.31.1, Vlan31, [0/0], 3w3d, direct
10.3.31.1/32, ubest/mbest: 1/0, attached
    *via 10.3.31.1, Vlan31, [0/0], 3w3d, local
10.3.31.11/32, ubest/mbest: 1/0, attached
    *via 10.3.31.11, Vlan31, [190/0], 3w2d, hmm
10.3.32.0/24, ubest/mbest: 1/0, attached
    *via 10.3.32.1, Vlan32, [0/0], 2w4d, direct
10.3.32.1/32, ubest/mbest: 1/0, attached
    *via 10.3.32.1, Vlan32, [0/0], 2w4d, local
10.3.100.0/22, ubest/mbest: 1/0
    *via 192.168.100.101%default, [200/0], 2w4d, bgp-64999, internal, tag 100 (evpn) segid: 200003 tunnelid: 0xc0a86465 encap: VXLAN

172.16.40.0/24, ubest/mbest: 1/0
    *via 172.16.255.105%default, [200/0], 2w4d, bgp-64999, internal, tag 65022 (evpn) segid: 200003 tunnelid: 0xac10ff69 encap: VXLAN

L-103# show bgp vrf VRF3 all
BGP routing table information for VRF VRF3, address family IPv4 Unicast
BGP table version is 55, Local Router ID is 10.3.0.103
Status: s-suppressed, x-deleted, S-stale, d-dampened, h-history, *-valid, >-best
Path type: i-internal, e-external, c-confed, l-local, a-aggregate, r-redist, I-injected
Origin codes: i - IGP, e - EGP, ? - incomplete, - - multipath, & - backup

   Network            Next Hop            Metric     LocPrf     Weight Path
*>i10.3.0.101/32      192.168.100.101                   100          0 i
*>i10.3.0.102/32      192.168.100.102                   100          0 i
*>l10.3.0.103/32      0.0.0.0                           100      32768 i
*>i10.3.100.0/22      192.168.100.101          0        100          0 100 i
* i                   192.168.100.102          0        100          0 100 i
*>i172.16.40.0/24     172.16.255.105           0        100          0 65022 ?

L-103# show bgp l2vpn evpn
BGP routing table information for VRF default, address family L2VPN EVPN
BGP table version is 65096, Local Router ID is 192.168.200.103
Status: s-suppressed, x-deleted, S-stale, d-dampened, h-history, *-valid, >-best
Path type: i-internal, e-external, c-confed, l-local, a-aggregate, r-redist, I-injected
Origin codes: i - IGP, e - EGP, ? - incomplete, - - multipath, & - backup

   Network            Next Hop            Metric     LocPrf     Weight Path
Route Distinguisher: 172.16.255.99:50797
*>i[5]:[0]:[0]:[24]:[172.16.40.0]:[0.0.0.0]/224
                      172.16.255.105           0        100          0 65022 ?

Route Distinguisher: 192.168.200.101:5
*>i[5]:[0]:[0]:[22]:[10.3.100.0]:[0.0.0.0]/224
                      192.168.100.101          0        100          0 100 i
*>i[5]:[0]:[0]:[32]:[10.3.0.101]:[0.0.0.0]/224
                      192.168.100.101                   100          0 i

Route Distinguisher: 192.168.200.101:6
*>i[5]:[0]:[0]:[22]:[10.4.100.0]:[0.0.0.0]/224
                      192.168.100.101          0        100          0 100 i
*>i[5]:[0]:[0]:[32]:[10.4.0.101]:[0.0.0.0]/224
                      192.168.100.101                   100          0 i

Route Distinguisher: 192.168.200.102:5
*>i[5]:[0]:[0]:[22]:[10.3.100.0]:[0.0.0.0]/224
                      192.168.100.102          0        100          0 100 i
*>i[5]:[0]:[0]:[32]:[10.3.0.102]:[0.0.0.0]/224
                      192.168.100.102                   100          0 i

Route Distinguisher: 192.168.200.102:6
*>i[5]:[0]:[0]:[22]:[10.4.100.0]:[0.0.0.0]/224
                      192.168.100.102          0        100          0 100 i
*>i[5]:[0]:[0]:[32]:[10.4.0.102]:[0.0.0.0]/224
                      192.168.100.102                   100          0 i

Route Distinguisher: 192.168.200.103:32798    (L2VNI 30031)
*>l[2]:[0]:[0]:[48]:[0050.56be.84bd]:[0]:[0.0.0.0]/216
                      192.168.100.103                   100      32768 i
*>l[2]:[0]:[0]:[48]:[0050.56be.84bd]:[32]:[10.3.31.11]/272
                      192.168.100.103                   100      32768 i

Route Distinguisher: 192.168.200.103:5    (L3VNI 200003)
*>i[5]:[0]:[0]:[22]:[10.3.100.0]:[0.0.0.0]/224
                      192.168.100.101          0        100          0 100 i
* i                   192.168.100.102          0        100          0 100 i
*>i[5]:[0]:[0]:[24]:[172.16.40.0]:[0.0.0.0]/224
                      172.16.255.105           0        100          0 65022 ?
*>i[5]:[0]:[0]:[32]:[10.3.0.101]:[0.0.0.0]/224
                      192.168.100.101                   100          0 i
*>i[5]:[0]:[0]:[32]:[10.3.0.102]:[0.0.0.0]/224
                      192.168.100.102                   100          0 i
*>l[5]:[0]:[0]:[32]:[10.3.0.103]:[0.0.0.0]/224
                      192.168.100.103                   100      32768 i

Route Distinguisher: 192.168.200.103:6    (L3VNI 200004)
* i[5]:[0]:[0]:[22]:[10.4.100.0]:[0.0.0.0]/224
                      192.168.100.102          0        100          0 100 i
*>i                   192.168.100.101          0        100          0 100 i
*>i[5]:[0]:[0]:[32]:[10.4.0.101]:[0.0.0.0]/224
                      192.168.100.101                   100          0 i
*>i[5]:[0]:[0]:[32]:[10.4.0.102]:[0.0.0.0]/224
                      192.168.100.102                   100          0 i
*>l[5]:[0]:[0]:[32]:[10.4.0.103]:[0.0.0.0]/224
                      192.168.100.103                   100      32768 i

L-103# sho ip route
IP Route Table for VRF "default"
'*' denotes best ucast next-hop
'**' denotes best mcast next-hop
'[x/y]' denotes [preference/metric]
'%<string>' in via output denotes VRF <string>

172.16.255.105/32, ubest/mbest: 1/0
    *via 192.168.0.11, Eth1/1, [110/1], 3w2d, ospf-0, type-2, tag 65022
172.16.255.106/32, ubest/mbest: 1/0
    *via 192.168.0.11, Eth1/1, [110/1], 1w1d, ospf-0, type-2, tag 65022
192.168.0.11/32, ubest/mbest: 1/0
    *via 192.168.0.11, Eth1/1, [110/41], 3w5d, ospf-0, intra
192.168.0.101/32, ubest/mbest: 1/0
    *via 192.168.0.11, Eth1/1, [110/81], 3w5d, ospf-0, intra
192.168.0.102/32, ubest/mbest: 1/0
    *via 192.168.0.11, Eth1/1, [110/81], 3w5d, ospf-0, intra
192.168.0.103/32, ubest/mbest: 2/0, attached
    *via 192.168.0.103, Lo0, [0/0], 3w5d, local
    *via 192.168.0.103, Lo0, [0/0], 3w5d, direct
192.168.100.101/32, ubest/mbest: 1/0
    *via 192.168.0.11, Eth1/1, [110/81], 3w5d, ospf-0, intra
192.168.100.102/32, ubest/mbest: 1/0
    *via 192.168.0.11, Eth1/1, [110/81], 3w5d, ospf-0, intra
192.168.100.103/32, ubest/mbest: 2/0, attached
    *via 192.168.100.103, Lo100, [0/0], 3w5d, local
    *via 192.168.100.103, Lo100, [0/0], 3w5d, direct
192.168.200.11/32, ubest/mbest: 1/0
    *via 192.168.0.11, Eth1/1, [110/41], 3w5d, ospf-0, intra
192.168.200.101/32, ubest/mbest: 1/0
    *via 192.168.0.11, Eth1/1, [110/81], 3w5d, ospf-0, intra
192.168.200.102/32, ubest/mbest: 1/0
    *via 192.168.0.11, Eth1/1, [110/81], 3w5d, ospf-0, intra
192.168.200.103/32, ubest/mbest: 2/0, attached
    *via 192.168.200.103, Lo200, [0/0], 3w5d, local
    *via 192.168.200.103, Lo200, [0/0], 3w5d, direct
192.168.255.0/30, ubest/mbest: 1/0
    *via 192.168.0.11, Eth1/1, [110/41], 3w5d, ospf-0, intra

SP-11

SP-11# show bgp l2vpn evpn
BGP routing table information for VRF default, address family L2VPN EVPN
BGP table version is 65034, Local Router ID is 192.168.200.11
Status: s-suppressed, x-deleted, S-stale, d-dampened, h-history, *-valid, >-best
Path type: i-internal, e-external, c-confed, l-local, a-aggregate, r-redist, I-injected
Origin codes: i - IGP, e - EGP, ? - incomplete, - - multipath, & - backup

   Network            Next Hop            Metric     LocPrf     Weight Path
Route Distinguisher: 172.16.255.99:50797
* i[5]:[0]:[0]:[24]:[172.16.40.0]:[0.0.0.0]/224
                      172.16.255.105           0        100          0 65022 ?
*>i                   172.16.255.105           0        100          0 65022 ?

Route Distinguisher: 192.168.200.101:5
*>i[5]:[0]:[0]:[22]:[10.3.100.0]:[0.0.0.0]/224
                      192.168.100.101          0        100          0 100 i
*>i[5]:[0]:[0]:[32]:[10.3.0.101]:[0.0.0.0]/224
                      192.168.100.101                   100          0 i

Route Distinguisher: 192.168.200.101:6
*>i[5]:[0]:[0]:[22]:[10.4.100.0]:[0.0.0.0]/224
                      192.168.100.101          0        100          0 100 i
*>i[5]:[0]:[0]:[32]:[10.4.0.101]:[0.0.0.0]/224
                      192.168.100.101                   100          0 i

Route Distinguisher: 192.168.200.102:5
*>i[5]:[0]:[0]:[22]:[10.3.100.0]:[0.0.0.0]/224
                      192.168.100.102          0        100          0 100 i
*>i[5]:[0]:[0]:[32]:[10.3.0.102]:[0.0.0.0]/224
                      192.168.100.102                   100          0 i

Route Distinguisher: 192.168.200.102:6
*>i[5]:[0]:[0]:[22]:[10.4.100.0]:[0.0.0.0]/224
                      192.168.100.102          0        100          0 100 i
*>i[5]:[0]:[0]:[32]:[10.4.0.102]:[0.0.0.0]/224
                      192.168.100.102                   100          0 i

Route Distinguisher: 192.168.200.103:5
*>i[5]:[0]:[0]:[32]:[10.3.0.103]:[0.0.0.0]/224
                      192.168.100.103                   100          0 i

Route Distinguisher: 192.168.200.103:6
*>i[5]:[0]:[0]:[32]:[10.4.0.103]:[0.0.0.0]/224
                      192.168.100.103                   100          0 i

Route Distinguisher: 192.168.200.103:32798
*>i[2]:[0]:[0]:[48]:[0050.56be.84bd]:[0]:[0.0.0.0]/216
                      192.168.100.103                   100          0 i
*>i[2]:[0]:[0]:[48]:[5555.5555.5555]:[0]:[0.0.0.0]/216
                      192.168.100.103                   100          0 i
*>i[2]:[0]:[0]:[48]:[0050.56be.84bd]:[32]:[10.3.31.11]/272
                      192.168.100.103                   100          0 i

Route Distinguisher: 192.168.200.103:32799
*>i[2]:[0]:[0]:[48]:[5555.5555.5555]:[0]:[0.0.0.0]/216
                      192.168.100.103                   100          0 i

Route Distinguisher: 192.168.200.103:32808
*>i[2]:[0]:[0]:[48]:[5555.5555.5555]:[0]:[0.0.0.0]/216
                      192.168.100.103                   100          0 i

Route Distinguisher: 192.168.200.103:32809
*>i[2]:[0]:[0]:[48]:[5555.5555.5555]:[0]:[0.0.0.0]/216
                      192.168.100.103                   100          0 i

SP-11# show ip route
IP Route Table for VRF "default"
'*' denotes best ucast next-hop
'**' denotes best mcast next-hop
'[x/y]' denotes [preference/metric]
'%<string>' in via output denotes VRF <string>

172.16.255.105/32, ubest/mbest: 2/0
    *via 192.168.0.101, Eth1/1, [110/1], 3w1d, ospf-0, type-2, tag 65022
    *via 192.168.0.102, Eth1/2, [110/1], 3w1d, ospf-0, type-2, tag 65022
172.16.255.106/32, ubest/mbest: 2/0
    *via 192.168.0.101, Eth1/1, [110/1], 1d23h, ospf-0, type-2, tag 65022
    *via 192.168.0.102, Eth1/2, [110/1], 1d23h, ospf-0, type-2, tag 65022
192.168.0.11/32, ubest/mbest: 2/0, attached
    *via 192.168.0.11, Lo0, [0/0], 3w5d, local
    *via 192.168.0.11, Lo0, [0/0], 3w5d, direct
192.168.0.101/32, ubest/mbest: 1/0
    *via 192.168.0.101, Eth1/1, [110/41], 3w5d, ospf-0, intra
192.168.0.102/32, ubest/mbest: 1/0
    *via 192.168.0.102, Eth1/2, [110/41], 3w5d, ospf-0, intra
192.168.0.103/32, ubest/mbest: 1/0
    *via 192.168.0.103, Eth1/3, [110/41], 3w5d, ospf-0, intra
192.168.100.101/32, ubest/mbest: 1/0
    *via 192.168.0.101, Eth1/1, [110/41], 3w5d, ospf-0, intra
192.168.100.102/32, ubest/mbest: 1/0
    *via 192.168.0.102, Eth1/2, [110/41], 3w5d, ospf-0, intra
192.168.100.103/32, ubest/mbest: 1/0
    *via 192.168.0.103, Eth1/3, [110/41], 3w5d, ospf-0, intra
192.168.200.11/32, ubest/mbest: 2/0, attached
    *via 192.168.200.11, Lo200, [0/0], 3w5d, local
    *via 192.168.200.11, Lo200, [0/0], 3w5d, direct
192.168.200.101/32, ubest/mbest: 1/0
    *via 192.168.0.101, Eth1/1, [110/41], 3w5d, ospf-0, intra
192.168.200.102/32, ubest/mbest: 1/0
    *via 192.168.0.102, Eth1/2, [110/41], 3w5d, ospf-0, intra
192.168.200.103/32, ubest/mbest: 1/0
    *via 192.168.0.103, Eth1/3, [110/41], 3w5d, ospf-0, intra
192.168.255.0/30, ubest/mbest: 1/0, attached
    *via 192.168.255.2, Lo255, [0/0], 3w5d, direct
192.168.255.2/32, ubest/mbest: 1/0, attached
    *via 192.168.255.2, Lo255, [0/0], 3w5d, local

lab-csr1k

Tip

In this article, it is explained how you set up the IOS based CSR Routers on the ESXi Hypervisor

lab-csr1k#sho ip route vrf VRF3

Routing Table: VRF3
Codes: L - local, C - connected, S - static, R - RIP, M - mobile, B - BGP
       D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
       N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
       E1 - OSPF external type 1, E2 - OSPF external type 2
       i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
       ia - IS-IS inter area, * - candidate default, U - per-user static route
       o - ODR, P - periodic downloaded static route, H - NHRP, l - LISP
       a - application route
       + - replicated route, % - next hop override

Gateway of last resort is not set

      10.0.0.0/8 is variably subnetted, 8 subnets, 3 masks
B        10.3.0.101/32 [20/0] via 10.3.30.2, 2w4d
B        10.3.0.102/32 [20/0] via 10.3.30.2, 2w4d
B        10.3.0.103/32 [20/0] via 10.3.30.2, 2w4d
C        10.3.30.0/24 is directly connected, GigabitEthernet3.30
L        10.3.30.3/32 is directly connected, GigabitEthernet3.30
B        10.3.31.11/32 [20/0] via 10.3.30.2, 2w4d
C        10.3.100.0/22 is directly connected, Loopback3
L        10.3.100.1/32 is directly connected, Loopback3
      172.16.0.0/24 is subnetted, 1 subnets
B        172.16.40.0 [20/0] via 10.3.30.2, 2w4d

lab-csr1k#sho ip bgp vpnv4 vrf VRF3
BGP table version is 99, local router ID is 172.31.255.1
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
              r RIB-failure, S Stale, m multipath, b backup-path, f RT-Filter,
              x best-external, a additional-path, c RIB-compressed,
Origin codes: i - IGP, e - EGP, ? - incomplete
RPKI validation codes: V valid, I invalid, N Not found

     Network          Next Hop            Metric LocPrf Weight Path
Route Distinguisher: 172.31.255.1:3 (default for vrf VRF3)
 *   10.3.0.101/32    10.3.30.1                              0 64999 i
 *>                   10.3.30.2                              0 64999 i
 *   10.3.0.102/32    10.3.30.1                              0 64999 i
 *>                   10.3.30.2                              0 64999 i
 *   10.3.0.103/32    10.3.30.1                              0 64999 i
 *>                   10.3.30.2                              0 64999 i
 *   10.3.31.11/32    10.3.30.1                              0 64999 i
 *>                   10.3.30.2                              0 64999 i
 *>  10.3.100.0/22    0.0.0.0                  0         32768 i
 *   172.16.40.0/24   10.3.30.1                              0 64999 65022 ?
 *>                   10.3.30.2                              0 64999 65022 ?

dc1-edge-05 (T0-vrf3)

dc1-edge-05(tier0_vrf_sr)> get route ipv4

Flags: t0c - Tier0-Connected, t0s - Tier0-Static, b - BGP,
t0n - Tier0-NAT, t1s - Tier1-Static, t1c - Tier1-Connected,
t1n: Tier1-NAT, t1l: Tier1-LB VIP, t1ls: Tier1-LB SNAT,
t1d: Tier1-DNS FORWARDER, t1ipsec: Tier1-IPSec, isr: Inter-SR,
> - selected route, * - FIB route

Total number of routes: 8

b  > * 10.3.0.101/32 [20/0] via 192.168.100.101, br-200003, 03w1d07h
b  > * 10.3.0.101/32 [20/0] via 172.16.107.1, br-200003, 03w1d07h
b  > * 10.3.0.102/32 [20/0] via 172.16.105.1, br-200003, 03w1d07h
b  > * 10.3.0.102/32 [20/0] via 192.168.100.102, br-200003, 03w1d07h
b  > * 10.3.0.103/32 [20/0] via 172.16.105.1, br-200003, 02w4d10h
b  > * 10.3.0.103/32 [20/0] via 172.16.107.1, br-200003, 02w4d10h
b  > * 10.3.31.11/32 [20/0] via 172.16.105.1, br-200003, 03w1d07h
b  > * 10.3.31.11/32 [20/0] via 172.16.107.1, br-200003, 03w1d07h
b  > * 10.3.100.0/22 [20/0] via 172.16.105.1, br-200003, 02w4d08h
b  > * 10.3.100.0/22 [20/0] via 172.16.107.1, br-200003, 02w4d08h
b  > * 10.3.100.0/22 [20/0] via 192.168.100.101, br-200003, 02w4d08h
b  > * 10.3.100.0/22 [20/0] via 192.168.100.102, br-200003, 02w4d08h
t0c> * 100.64.96.0/31 is directly connected, linked-280, 03w5d08h
t0c> * 169.254.0.0/24 is directly connected, downlink-282, 03w2d11h
t1c> * 172.16.40.0/24 [3/0] via 100.64.96.1, linked-280, 03w5d08h

dc1-edge-05(tier0_vrf_sr)> get bgp ipv4
BGP IPv4 table version is 47
Local router ID is 172.16.105.2
Status flags: > - best, I - internal
Origin flags: i - IGP, e - EGP, ? - incomplete

   Network                             Next Hop                            Metric       LocPrf  Weight   Path
   10.3.31.11/32                       172.16.107.1                        0            100     0        64999 65022 i
 > 10.3.31.11/32                       172.16.105.1                        0            100     0        64999 65022 i
   10.3.0.103/32                       172.16.107.1                        0            100     0        64999 65022 i
 > 10.3.0.103/32                       172.16.105.1                        0            100     0        64999 65022 i
   10.3.0.101/32                       172.16.107.1                        0            100     0        64999 65022 i
 > 10.3.0.101/32                       192.168.100.101                     0            100     0        64999 65022 i
 > 172.16.40.0/24                      100.64.96.1                         0            100     32768    65022 ?
   10.3.100.0/22                       172.16.107.1                        0            100     0        64999 100 65022 i
   10.3.100.0/22                       192.168.100.101                     0            100     0        64999 100 65022 i
 > 10.3.100.0/22                       172.16.105.1                        0            100     0        64999 100 65022 i
   10.3.100.0/22                       192.168.100.102                     0            100     0        64999 100 65022 i
   10.3.0.102/32                       192.168.100.102                     0            100     0        64999 65022 i
 > 10.3.0.102/32                       172.16.105.1                        0            100     0        64999 65022 i

dc1-edge-05 (T0)

dc1-edge-05(tier0_sr)> get bgp evpn
BGP table version is 1, local router ID is 172.16.105.2
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal
Origin codes: i - IGP, e - EGP, ? - incomplete
EVPN type-2 prefix: [2]:[EthTag]:[MAClen]:[MAC]:[IPlen]:[IP]
EVPN type-3 prefix: [3]:[EthTag]:[IPlen]:[OrigIP]
EVPN type-4 prefix: [4]:[ESI]:[IPlen]:[OrigIP]
EVPN type-5 prefix: [5]:[EthTag]:[IPlen]:[IP]

   Network          Next Hop            Metric LocPrf Weight Path
Route Distinguisher: 172.16.255.99:50797
*> [5]:[0]:[24]:[172.16.40.0]
                    172.16.255.105           0         32768 ?
Route Distinguisher: 192.168.200.101:5
*> [5]:[0]:[22]:[10.3.100.0]
                    172.16.107.1                           0 64999 100 i
*  [5]:[0]:[22]:[10.3.100.0]
                    192.168.100.101          0             0 64999 100 i
*> [5]:[0]:[32]:[10.3.0.101]
                    172.16.107.1                           0 64999 i
*  [5]:[0]:[32]:[10.3.0.101]
                    192.168.100.101                        0 64999 i
Route Distinguisher: 192.168.200.101:6
*> [5]:[0]:[22]:[10.4.100.0]
                    172.16.107.1                           0 64999 100 i
*  [5]:[0]:[22]:[10.4.100.0]
                    192.168.100.101          0             0 64999 100 i
*> [5]:[0]:[32]:[10.4.0.101]
                    172.16.107.1                           0 64999 i
*  [5]:[0]:[32]:[10.4.0.101]
                    192.168.100.101                        0 64999 i
Route Distinguisher: 192.168.200.102:5
*> [5]:[0]:[22]:[10.3.100.0]
                    172.16.105.1                           0 64999 100 i
*  [5]:[0]:[22]:[10.3.100.0]
                    192.168.100.102          0             0 64999 100 i
*  [5]:[0]:[32]:[10.3.0.102]
                    192.168.100.102                        0 64999 i
*> [5]:[0]:[32]:[10.3.0.102]
                    172.16.105.1                           0 64999 i
Route Distinguisher: 192.168.200.102:6
*> [5]:[0]:[22]:[10.4.100.0]
                    172.16.105.1                           0 64999 100 i
*  [5]:[0]:[22]:[10.4.100.0]
                    192.168.100.102          0             0 64999 100 i
*> [5]:[0]:[32]:[10.4.0.102]
                    172.16.105.1                           0 64999 i
*  [5]:[0]:[32]:[10.4.0.102]
                    192.168.100.102                        0 64999 i
Route Distinguisher: 192.168.200.103:5
*  [5]:[0]:[32]:[10.3.0.103]
                    172.16.107.1                           0 64999 i
*> [5]:[0]:[32]:[10.3.0.103]
                    172.16.105.1                           0 64999 i
Route Distinguisher: 192.168.200.103:6
*  [5]:[0]:[32]:[10.4.0.103]
                    172.16.107.1                           0 64999 i
*> [5]:[0]:[32]:[10.4.0.103]
                    172.16.105.1                           0 64999 i
Route Distinguisher: 192.168.200.103:32798
*  [2]:[0]:[48]:[00:50:56:be:84:bd]:[32]:[10.3.31.11]
                    172.16.107.1                           0 64999 i
*> [2]:[0]:[48]:[00:50:56:be:84:bd]:[32]:[10.3.31.11]
                    172.16.105.1                           0 64999 i

Displayed 12 prefixes (23 paths)

dc1-edge-05(tier0_sr)> get vrf vni
VRF                                   VNI        VxLAN IF             L3-SVI               State Rmac
VRF-3084                              200003     vxlan-200003         br-200003            Up    02:50:56:00:68:02
VRF-4107                              200004     vxlan-200004         br-200004            Up    02:50:56:00:68:02

dc1-edge-05(tier0_sr)> get route ipv4

Flags: t0c - Tier0-Connected, t0s - Tier0-Static, b - BGP,
t0n - Tier0-NAT, t1s - Tier1-Static, t1c - Tier1-Connected,
t1n: Tier1-NAT, t1l: Tier1-LB VIP, t1ls: Tier1-LB SNAT,
t1d: Tier1-DNS FORWARDER, t1ipsec: Tier1-IPSec, isr: Inter-SR,
> - selected route, * - FIB route

Total number of routes: 13

isr> * 169.254.0.128/25 is directly connected, inter-sr-286, 03w5d09h
t0c> * 172.16.105.0/24 is directly connected, uplink-279, 03w5d09h
isr> * 172.16.105.2/32 is directly connected, uplink-279, 03w3d23h
isr> * 172.16.106.0/24 [200/0] via 169.254.0.131, inter-sr-286, 03w4d00h
isr> * 172.16.106.2/32 [200/0] via 169.254.0.131, inter-sr-286, 03w4d00h
t0c> * 172.16.107.0/24 is directly connected, uplink-308, 03w5d00h
isr> * 172.16.107.2/32 is directly connected, uplink-308, 03w3d23h
isr> * 172.16.108.0/24 [200/0] via 169.254.0.131, inter-sr-286, 03w4d00h
isr> * 172.16.108.2/32 [200/0] via 169.254.0.131, inter-sr-286, 03w4d00h
t0c> * 172.16.255.105/32 is directly connected, loopback-285, 03w5d09h
isr> * 172.16.255.106/32 [200/0] via 169.254.0.131, inter-sr-286, 03w4d00h
b  > * 192.168.100.101/32 [20/0] via 172.16.105.1, uplink-279, 03w2d12h
b  > * 192.168.100.102/32 [20/0] via 172.16.107.1, uplink-308, 03w1d07h

dc1-edge-05(tier0_sr)> get bgp ipv4
BGP IPv4 table version is 13
Local router ID is 172.16.105.2
Status flags: > - best, I - internal
Origin flags: i - IGP, e - EGP, ? - incomplete

   Network                             Next Hop                            Metric       LocPrf  Weight   Path
 > 192.168.100.102/32                  172.16.107.1                        0            100     0        64999 65022 i
 > 192.168.100.101/32                  172.16.105.1                        0            100     0        64999 65022 i
 > 172.16.105.0/24                     0.0.0.0                             0            100     32768    65022 ?
 > 172.16.255.105/32                   0.0.0.0                             0            100     32768    65022 ?
 > 172.16.107.0/24                     0.0.0.0                             0            100     32768    65022 ?

dc1-edge-06 (T0-vrf3)

dc1-edge-06(tier0_vrf_sr)> get route ipv4

Flags: t0c - Tier0-Connected, t0s - Tier0-Static, b - BGP,
t0n - Tier0-NAT, t1s - Tier1-Static, t1c - Tier1-Connected,
t1n: Tier1-NAT, t1l: Tier1-LB VIP, t1ls: Tier1-LB SNAT,
t1d: Tier1-DNS FORWARDER, t1ipsec: Tier1-IPSec, isr: Inter-SR,
> - selected route, * - FIB route

Total number of routes: 8

b  > * 10.3.0.101/32 [20/0] via 172.16.108.1, br-200003, 2d00h25m
b  > * 10.3.0.101/32 [20/0] via 192.168.100.101, br-200003, 2d00h25m
b  > * 10.3.0.102/32 [20/0] via 192.168.100.102, br-200003, 2d00h25m
b  > * 10.3.0.102/32 [20/0] via 172.16.106.1, br-200003, 2d00h25m
b  > * 10.3.0.103/32 [20/0] via 172.16.108.1, br-200003, 2d00h25m
b  > * 10.3.0.103/32 [20/0] via 172.16.106.1, br-200003, 2d00h25m
b  > * 10.3.31.11/32 [20/0] via 172.16.108.1, br-200003, 2d00h25m
b  > * 10.3.31.11/32 [20/0] via 172.16.106.1, br-200003, 2d00h25m
b  > * 10.3.100.0/22 [20/0] via 172.16.108.1, br-200003, 2d00h25m
b  > * 10.3.100.0/22 [20/0] via 172.16.106.1, br-200003, 2d00h25m
b  > * 10.3.100.0/22 [20/0] via 192.168.100.101, br-200003, 2d00h25m
b  > * 10.3.100.0/22 [20/0] via 192.168.100.102, br-200003, 2d00h25m
t0c> * 100.64.96.0/31 is directly connected, linked-294, 03w4d01h
t0c> * 169.254.0.0/24 is directly connected, downlink-293, 01w1d14h
t1c> * 172.16.40.0/24 [3/0] via 100.64.96.1, linked-294, 03w4d01h

dc1-edge-06(tier0_vrf_sr)> get bgp ipv4
BGP IPv4 table version is 63
Local router ID is 172.16.106.2
Status flags: > - best, I - internal
Origin flags: i - IGP, e - EGP, ? - incomplete

   Network                             Next Hop                            Metric       LocPrf  Weight   Path
   10.3.31.11/32                       172.16.106.1                        0            100     0        64999 65022 i
 > 10.3.31.11/32                       172.16.108.1                        0            100     0        64999 65022 i
   10.3.0.103/32                       172.16.106.1                        0            100     0        64999 65022 i
 > 10.3.0.103/32                       172.16.108.1                        0            100     0        64999 65022 i
   10.3.0.101/32                       192.168.100.101                     0            100     0        64999 65022 i
 > 10.3.0.101/32                       172.16.108.1                        0            100     0        64999 65022 i
 > 172.16.40.0/24                      100.64.96.1                         0            100     32768    65022 ?
   10.3.100.0/22                       192.168.100.101                     0            100     0        64999 100 65022 i
   10.3.100.0/22                       172.16.106.1                        0            100     0        64999 100 65022 i
   10.3.100.0/22                       192.168.100.102                     0            100     0        64999 100 65022 i
 > 10.3.100.0/22                       172.16.108.1                        0            100     0        64999 100 65022 i
   10.3.0.102/32                       172.16.106.1                        0            100     0        64999 65022 i
 > 10.3.0.102/32                       192.168.100.102                     0            100     0        64999 65022 i

dc1-edge-06 (T0)

dc1-edge-06(tier0_sr)> get bgp evpn
BGP table version is 1, local router ID is 172.16.106.2
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal
Origin codes: i - IGP, e - EGP, ? - incomplete
EVPN type-2 prefix: [2]:[EthTag]:[MAClen]:[MAC]:[IPlen]:[IP]
EVPN type-3 prefix: [3]:[EthTag]:[IPlen]:[OrigIP]
EVPN type-4 prefix: [4]:[ESI]:[IPlen]:[OrigIP]
EVPN type-5 prefix: [5]:[EthTag]:[IPlen]:[IP]

   Network          Next Hop            Metric LocPrf Weight Path
Route Distinguisher: 172.16.255.99:50797
*> [5]:[0]:[24]:[172.16.40.0]
                    172.16.255.106           0         32768 ?
Route Distinguisher: 192.168.200.101:5
*  [5]:[0]:[22]:[10.3.100.0]
                    192.168.100.101          0             0 64999 100 i
*> [5]:[0]:[22]:[10.3.100.0]
                    172.16.108.1                           0 64999 100 i
*  [5]:[0]:[32]:[10.3.0.101]
                    192.168.100.101                        0 64999 i
*> [5]:[0]:[32]:[10.3.0.101]
                    172.16.108.1                           0 64999 i
Route Distinguisher: 192.168.200.101:6
*  [5]:[0]:[22]:[10.4.100.0]
                    192.168.100.101          0             0 64999 100 i
*> [5]:[0]:[22]:[10.4.100.0]
                    172.16.108.1                           0 64999 100 i
*  [5]:[0]:[32]:[10.4.0.101]
                    192.168.100.101                        0 64999 i
*> [5]:[0]:[32]:[10.4.0.101]
                    172.16.108.1                           0 64999 i
Route Distinguisher: 192.168.200.102:5
*> [5]:[0]:[22]:[10.3.100.0]
                    172.16.106.1                           0 64999 100 i
*  [5]:[0]:[22]:[10.3.100.0]
                    192.168.100.102          0             0 64999 100 i
*> [5]:[0]:[32]:[10.3.0.102]
                    172.16.106.1                           0 64999 i
*  [5]:[0]:[32]:[10.3.0.102]
                    192.168.100.102                        0 64999 i
Route Distinguisher: 192.168.200.102:6
*> [5]:[0]:[22]:[10.4.100.0]
                    172.16.106.1                           0 64999 100 i
*  [5]:[0]:[22]:[10.4.100.0]
                    192.168.100.102          0             0 64999 100 i
*> [5]:[0]:[32]:[10.4.0.102]
                    172.16.106.1                           0 64999 i
*  [5]:[0]:[32]:[10.4.0.102]
                    192.168.100.102                        0 64999 i
Route Distinguisher: 192.168.200.103:5
*> [5]:[0]:[32]:[10.3.0.103]
                    172.16.106.1                           0 64999 i
*  [5]:[0]:[32]:[10.3.0.103]
                    172.16.108.1                           0 64999 i
Route Distinguisher: 192.168.200.103:6
*> [5]:[0]:[32]:[10.4.0.103]
                    172.16.106.1                           0 64999 i
*  [5]:[0]:[32]:[10.4.0.103]
                    172.16.108.1                           0 64999 i
Route Distinguisher: 192.168.200.103:32798
*> [2]:[0]:[48]:[00:50:56:be:84:bd]:[32]:[10.3.31.11]
                    172.16.106.1                           0 64999 i
*  [2]:[0]:[48]:[00:50:56:be:84:bd]:[32]:[10.3.31.11]
                    172.16.108.1                           0 64999 i

Displayed 12 prefixes (23 paths)

dc1-edge-06(tier0_sr)> get vrf vni
VRF                                   VNI        VxLAN IF             L3-SVI               State Rmac
VRF-3084                              200003     vxlan-200003         br-200003            Up    02:50:56:00:68:0b
VRF-4107                              200004     vxlan-200004         br-200004            Up    02:50:56:00:68:0b

dc1-edge-06(tier0_sr)> get route ipv4

Flags: t0c - Tier0-Connected, t0s - Tier0-Static, b - BGP,
t0n - Tier0-NAT, t1s - Tier1-Static, t1c - Tier1-Connected,
t1n: Tier1-NAT, t1l: Tier1-LB VIP, t1ls: Tier1-LB SNAT,
t1d: Tier1-DNS FORWARDER, t1ipsec: Tier1-IPSec, isr: Inter-SR,
> - selected route, * - FIB route

Total number of routes: 13

isr> * 169.254.0.128/25 is directly connected, inter-sr-279, 03w4d01h
isr> * 172.16.105.0/24 [200/0] via 169.254.0.130, inter-sr-279, 03w4d01h
isr> * 172.16.105.2/32 [200/0] via 169.254.0.130, inter-sr-279, 03w4d01h
t0c> * 172.16.106.0/24 is directly connected, uplink-277, 03w4d01h
isr> * 172.16.106.2/32 is directly connected, uplink-277, 03w4d00h
isr> * 172.16.107.0/24 [200/0] via 169.254.0.130, inter-sr-279, 03w4d01h
isr> * 172.16.107.2/32 [200/0] via 169.254.0.130, inter-sr-279, 03w4d01h
t0c> * 172.16.108.0/24 is directly connected, uplink-304, 03w4d01h
isr> * 172.16.108.2/32 is directly connected, uplink-304, 01w1d14h
isr> * 172.16.255.105/32 [200/0] via 169.254.0.130, inter-sr-279, 03w4d01h
t0c> * 172.16.255.106/32 is directly connected, vxlan-200003, 03w4d01h
b  > * 192.168.100.101/32 [20/0] via 172.16.106.1, uplink-277, 2d00h38m
b  > * 192.168.100.102/32 [20/0] via 172.16.108.1, uplink-304, 01w1d14h

dc1-edge-06(tier0_sr)> get bgp ipv4
BGP IPv4 table version is 16
Local router ID is 172.16.106.2
Status flags: > - best, I - internal
Origin flags: i - IGP, e - EGP, ? - incomplete

   Network                             Next Hop                            Metric       LocPrf  Weight   Path
 > 172.16.106.0/24                     0.0.0.0                             0            100     32768    65022 ?
 > 172.16.108.0/24                     0.0.0.0                             0            100     32768    65022 ?
 > 172.16.255.106/32                   0.0.0.0                             0            100     32768    65022 ?
 > 192.168.100.101/32                  172.16.106.1                        0            100     0        64999 65022 i
 > 192.168.100.102/32                  172.16.108.1                        0            100     0        64999 65022 i

Appendix B. Base configurations

BGW-1

CLICK ON EXPAND ===> ON THE RIGHT ===> TO SEE THE OUTPUT ===> :

BGW-1# show run

version 9.2(4) Bios:version
hostname BGW-1
vdc BGW-1 id 1
  limit-resource vlan minimum 16 maximum 4094
  limit-resource vrf minimum 2 maximum 4096
  limit-resource port-channel minimum 0 maximum 511
  limit-resource u4route-mem minimum 248 maximum 248
  limit-resource u6route-mem minimum 96 maximum 96
  limit-resource m4route-mem minimum 58 maximum 58
  limit-resource m6route-mem minimum 8 maximum 8

nv overlay evpn
feature ospf
feature bgp
feature pim
feature fabric forwarding
feature interface-vlan
feature vn-segment-vlan-based
feature lacp
feature nv overlay

username admin password 5 $5$1U2YRfmP$we1GazfktNgcPpCr5NYRPGit/5GQz2LS79zECxOM4D0  role network-admin
ip domain-lookup
copp profile lenient
snmp-server user admin network-admin auth md5 0x9e739edc4cfa2ec898b5e3858d15226f priv 0x9e739edc4cfa2ec898b5e3858d15226f localizedkey
rmon event 1 description FATAL(1) owner PMON@FATAL
rmon event 2 description CRITICAL(2) owner PMON@CRITICAL
rmon event 3 description ERROR(3) owner PMON@ERROR
rmon event 4 description WARNING(4) owner PMON@WARNING
rmon event 5 description INFORMATION(5) owner PMON@INFO

ip pim rp-address 192.168.255.1 group-list 230.0.0.0/8 bidir
ip pim ssm range 232.0.0.0/8
vlan 1,930,940
vlan 930
  vn-segment 200003
vlan 940
  vn-segment 200004

vrf context VRF3
  vni 200003
  rd auto
  address-family ipv4 unicast
    route-target both auto
    route-target both auto evpn
vrf context VRF4
  vni 200004
  rd auto
  address-family ipv4 unicast
    route-target both auto
    route-target both auto evpn
vrf context management
hardware access-list tcam region racl 512
hardware access-list tcam region arp-ether 256 double-wide


interface Vlan1

interface Vlan930
  no shutdown
  vrf member VRF3
  ip forward

interface Vlan940
  no shutdown
  vrf member VRF4
  ip forward

interface nve1
  no shutdown
  host-reachability protocol bgp
  source-interface loopback100
  member vni 200003 associate-vrf
  member vni 200004 associate-vrf

interface Ethernet1/1
  no switchport
  mtu 9000
  medium p2p
  ip unnumbered loopback0
  ip ospf network point-to-point
  ip router ospf 0 area 0.0.0.0
  ip pim sparse-mode
  no shutdown

interface Ethernet1/2
  no switchport
  no shutdown

interface Ethernet1/2.30
  encapsulation dot1q 30
  vrf member VRF3
  ip address 10.3.30.1/24
  no shutdown

interface Ethernet1/2.40
  encapsulation dot1q 40
  vrf member VRF4
  ip address 10.4.40.1/24
  no shutdown

interface Ethernet1/3
  no switchport

interface Ethernet1/4
  no switchport

interface Ethernet1/5
  no switchport

interface Ethernet1/6
  no switchport

interface mgmt0
  vrf member management
  ip address 10.1.7.13/24

interface loopback0
  ip address 192.168.0.101/32
  ip router ospf 0 area 0.0.0.0

interface loopback3
  vrf member VRF3
  ip address 10.3.0.101/32

interface loopback4
  vrf member VRF4
  ip address 10.4.0.101/32

interface loopback100
  ip address 192.168.100.101/32
  ip router ospf 0 area 0.0.0.0
  ip pim sparse-mode

interface loopback200
  ip address 192.168.200.101/32
  ip router ospf 0 area 0.0.0.0
line console
line vty
  exec-timeout 0
boot nxos bootflash:/nxos.9.2.4.bin
router ospf 0
  router-id 192.168.0.101
router bgp 64999
  router-id 192.168.200.101
  address-family ipv4 unicast
  address-family l2vpn evpn
  neighbor 192.168.200.11
    remote-as 64999
    update-source loopback200
    address-family l2vpn evpn
      send-community
      send-community extended
  vrf VRF3
    address-family ipv4 unicast
      network 10.3.0.101/32
      advertise l2vpn evpn
      maximum-paths 8
    neighbor 10.3.30.3
      remote-as 100
      update-source Ethernet1/2.30
      address-family ipv4 unicast
  vrf VRF4
    address-family ipv4 unicast
      network 10.4.0.101/32
      advertise l2vpn evpn
      maximum-paths 8
    neighbor 10.4.40.3
      remote-as 100
      update-source Ethernet1/2.40
      address-family ipv4 unicast
evpn

no logging console

BGW-2

CLICK ON EXPAND ===> ON THE RIGHT ===> TO SEE THE OUTPUT ===> :

BGW-2# show run

version 9.2(4) Bios:version
hostname BGW-2
vdc BGW-2 id 1
  limit-resource vlan minimum 16 maximum 4094
  limit-resource vrf minimum 2 maximum 4096
  limit-resource port-channel minimum 0 maximum 511
  limit-resource u4route-mem minimum 248 maximum 248
  limit-resource u6route-mem minimum 96 maximum 96
  limit-resource m4route-mem minimum 58 maximum 58
  limit-resource m6route-mem minimum 8 maximum 8

nv overlay evpn
feature ospf
feature bgp
feature pim
feature fabric forwarding
feature interface-vlan
feature vn-segment-vlan-based
feature lacp
feature nv overlay

username admin password 5 $5$ql0GFYYU$innLQVZZMjLR/35.BWW7j533rUyxh0n7BRPGzqzt3f5  role network-admin
ip domain-lookup
snmp-server user admin network-admin auth md5 0xa9e8ac1596e8a7efb9863ba23e93394d priv 0xa9e8ac1596e8a7efb9863ba23e93394d localizedkey
rmon event 1 description FATAL(1) owner PMON@FATAL
rmon event 2 description CRITICAL(2) owner PMON@CRITICAL
rmon event 3 description ERROR(3) owner PMON@ERROR
rmon event 4 description WARNING(4) owner PMON@WARNING
rmon event 5 description INFORMATION(5) owner PMON@INFO

ip pim rp-address 192.168.255.1 group-list 230.0.0.0/8 bidir
ip pim ssm range 232.0.0.0/8
vlan 1,930,940
vlan 930
  vn-segment 200003
vlan 940
  vn-segment 200004

vrf context VRF3
  vni 200003
  rd auto
  address-family ipv4 unicast
    route-target both auto
    route-target both auto evpn
vrf context VRF4
  vni 200004
  rd auto
  address-family ipv4 unicast
    route-target both auto
    route-target both auto evpn
vrf context management
hardware access-list tcam region racl 512
hardware access-list tcam region arp-ether 256 double-wide


interface Vlan1

interface Vlan930
  no shutdown
  vrf member VRF3
  ip forward

interface Vlan940
  no shutdown
  vrf member VRF4
  ip forward

interface nve1
  no shutdown
  host-reachability protocol bgp
  source-interface loopback100
  member vni 200003 associate-vrf
  member vni 200004 associate-vrf

interface Ethernet1/1
  no switchport
  mtu 9000
  medium p2p
  ip unnumbered loopback0
  ip ospf network point-to-point
  ip router ospf 0 area 0.0.0.0
  ip pim sparse-mode
  no shutdown

interface Ethernet1/2
  no switchport
  no shutdown

interface Ethernet1/2.30
  encapsulation dot1q 30
  vrf member VRF3
  ip address 10.3.30.2/24
  no shutdown

interface Ethernet1/2.40
  encapsulation dot1q 40
  vrf member VRF4
  ip address 10.4.40.2/24
  no shutdown

interface Ethernet1/3
  no switchport

interface Ethernet1/4
  no switchport

interface Ethernet1/5
  no switchport

interface Ethernet1/6
  no switchport

interface mgmt0
  vrf member management
  ip address 10.1.7.14/24

interface loopback0
  ip address 192.168.0.102/32
  ip router ospf 0 area 0.0.0.0

interface loopback3
  vrf member VRF3
  ip address 10.3.0.102/32

interface loopback4
  vrf member VRF4
  ip address 10.4.0.102/32

interface loopback100
  ip address 192.168.100.102/32
  ip router ospf 0 area 0.0.0.0
  ip pim sparse-mode

interface loopback200
  ip address 192.168.200.102/32
  ip router ospf 0 area 0.0.0.0
line console
line vty
  exec-timeout 0
boot nxos bootflash:/nxos.9.2.4.bin
router ospf 0
  router-id 192.168.0.102
router bgp 64999
  router-id 192.168.200.102
  address-family ipv4 unicast
  address-family l2vpn evpn
  neighbor 192.168.200.11
    remote-as 64999
    update-source loopback200
    address-family l2vpn evpn
      send-community
      send-community extended
  vrf VRF3
    address-family ipv4 unicast
      network 10.3.0.102/32
      advertise l2vpn evpn
      maximum-paths 8
    neighbor 10.3.30.3
      remote-as 100
      update-source Ethernet1/2.30
      address-family ipv4 unicast
  vrf VRF4
    address-family ipv4 unicast
      network 10.4.0.102/32
      advertise l2vpn evpn
      maximum-paths 8
    neighbor 10.4.40.3
      remote-as 100
      update-source Ethernet1/2.40
      address-family ipv4 unicast
evpn

no logging console

L-103

CLICK ON EXPAND ===> ON THE RIGHT ===> TO SEE THE OUTPUT ===> :

version 7.0(3)I7(8) Bios:version
hostname L-103
vdc L-103 id 1
  limit-resource vlan minimum 16 maximum 4094
  limit-resource vrf minimum 2 maximum 4096
  limit-resource port-channel minimum 0 maximum 511
  limit-resource u4route-mem minimum 248 maximum 248
  limit-resource u6route-mem minimum 96 maximum 96
  limit-resource m4route-mem minimum 58 maximum 58
  limit-resource m6route-mem minimum 8 maximum 8

nv overlay evpn
feature ospf
feature bgp
feature pim
feature fabric forwarding
feature interface-vlan
feature vn-segment-vlan-based
feature nv overlay

username admin password 5 $5$3RswCBro$XTof0QUdxeH8FyCKrfee27CtkYOQUDXMdf9/Sdm0Ox6  role network-admin
ip domain-lookup
copp profile strict
snmp-server user admin network-admin auth md5 0x4ea612fd490dfc94117f5f80ca7985b7 priv 0x4ea612fd490dfc94117f5f80ca7985b7 localizedkey
rmon event 1 description FATAL(1) owner PMON@FATAL
rmon event 2 description CRITICAL(2) owner PMON@CRITICAL
rmon event 3 description ERROR(3) owner PMON@ERROR
rmon event 4 description WARNING(4) owner PMON@WARNING
rmon event 5 description INFORMATION(5) owner PMON@INFO

fabric forwarding anycast-gateway-mac 0001.0001.0001
ip pim rp-address 192.168.255.1 group-list 230.0.0.0/8 bidir
ip pim ssm range 232.0.0.0/8
vlan 1,31-32,41-42,930,940
vlan 31
  vn-segment 30031
vlan 32
  vn-segment 30032
vlan 41
  vn-segment 40041
vlan 42
  vn-segment 40042
vlan 930
  vn-segment 200003
vlan 940
  vn-segment 200004

vrf context VRF3
  vni 200003
  rd auto
  address-family ipv4 unicast
    route-target both auto
    route-target both auto evpn
vrf context VRF4
  vni 200004
  rd auto
  address-family ipv4 unicast
    route-target both auto
    route-target both auto evpn
vrf context management
hardware access-list tcam region racl 512
hardware access-list tcam region arp-ether 256 double-wide


interface Vlan1

interface Vlan31
  no shutdown
  vrf member VRF3
  ip address 10.3.31.1/24
  fabric forwarding mode anycast-gateway

interface Vlan32
  no shutdown
  vrf member VRF3
  ip address 10.3.32.1/24
  fabric forwarding mode anycast-gateway

interface Vlan41
  no shutdown
  vrf member VRF4
  ip address 10.3.41.1/24
  fabric forwarding mode anycast-gateway

interface Vlan42
  no shutdown
  vrf member VRF4
  ip address 10.3.42.1/24
  fabric forwarding mode anycast-gateway

interface Vlan930
  no shutdown
  vrf member VRF3
  ip forward

interface Vlan940
  no shutdown
  vrf member VRF4
  ip forward

interface nve1
  no shutdown
  host-reachability protocol bgp
  source-interface loopback100
  member vni 30031
    suppress-arp
    mcast-group 230.3.0.31
  member vni 30032
    suppress-arp
    mcast-group 230.3.0.32
  member vni 40041
    suppress-arp
    mcast-group 230.4.0.41
  member vni 40042
    suppress-arp
    mcast-group 230.4.0.42
  member vni 200003 associate-vrf
  member vni 200004 associate-vrf

interface Ethernet1/1
  no switchport
  mtu 9000
  medium p2p
  ip unnumbered loopback0
  ip ospf network point-to-point
  ip router ospf 0 area 0.0.0.0
  ip pim sparse-mode
  no shutdown

interface Ethernet1/2
  switchport mode trunk
  switchport trunk allowed vlan 31-32,41-42
  spanning-tree port type edge trunk

interface Ethernet1/3
  no switchport

interface Ethernet1/4
  no switchport

interface Ethernet1/5
  no switchport

interface Ethernet1/6
  no switchport

interface mgmt0
  vrf member management
  ip address 10.1.7.17/24

interface loopback0
  ip address 192.168.0.103/32
  ip router ospf 0 area 0.0.0.0

interface loopback3
  vrf member VRF3
  ip address 10.3.0.103/32

interface loopback4
  vrf member VRF4
  ip address 10.4.0.103/32

interface loopback100
  ip address 192.168.100.103/32
  ip router ospf 0 area 0.0.0.0
  ip pim sparse-mode

interface loopback200
  ip address 192.168.200.103/32
  ip router ospf 0 area 0.0.0.0
line console
line vty
  exec-timeout 0
boot nxos bootflash:/nxos.7.0.3.I7.8.bin
router ospf 0
  router-id 192.168.0.103
router bgp 64999
  router-id 192.168.200.103
  address-family l2vpn evpn
  neighbor 192.168.200.11
    remote-as 64999
    update-source loopback200
    address-family l2vpn evpn
      send-community
      send-community extended
  vrf VRF3
    address-family ipv4 unicast
      network 10.3.0.103/32
      advertise l2vpn evpn
  vrf VRF4
    address-family ipv4 unicast
      network 10.4.0.103/32
      advertise l2vpn evpn
evpn
  vni 30031 l2
    rd auto
    route-target import auto
    route-target export auto
  vni 30032 l2
    rd auto
    route-target import auto
    route-target export auto
  vni 40041 l2
    rd auto
    route-target import auto
    route-target export auto
  vni 40042 l2
    rd auto
    route-target import auto
    route-target export auto

no logging console

SP-11

CLICK ON EXPAND ===> ON THE RIGHT ===> TO SEE THE OUTPUT ===> :

SP-11# show run

version 7.0(3)I7(8) Bios:version
hostname SP-11
vdc SP-11 id 1
  limit-resource vlan minimum 16 maximum 4094
  limit-resource vrf minimum 2 maximum 4096
  limit-resource port-channel minimum 0 maximum 511
  limit-resource u4route-mem minimum 248 maximum 248
  limit-resource u6route-mem minimum 96 maximum 96
  limit-resource m4route-mem minimum 58 maximum 58
  limit-resource m6route-mem minimum 8 maximum 8

nv overlay evpn
feature ospf
feature bgp
feature pim
feature nv overlay

username admin password 5 $5$8jg4fbWJ$XtXigovAW3ieSzVMn9Nswx2wiqnjRbrG9L3uQNl3t81  role network-admin
ip domain-lookup
copp profile strict
snmp-server user admin network-admin auth md5 0xb8e04e133ba5ce998dc62ddb3c70da64 priv 0xb8e04e133ba5ce998dc62ddb3c70da64 localizedkey
rmon event 1 description FATAL(1) owner PMON@FATAL
rmon event 2 description CRITICAL(2) owner PMON@CRITICAL
rmon event 3 description ERROR(3) owner PMON@ERROR
rmon event 4 description WARNING(4) owner PMON@WARNING
rmon event 5 description INFORMATION(5) owner PMON@INFO

ip pim rp-address 192.168.255.1 group-list 230.0.0.0/8 bidir
ip pim ssm range 232.0.0.0/8
vlan 1

vrf context management

interface Ethernet1/1
  no switchport
  mtu 9000
  medium p2p
  ip unnumbered loopback0
  ip ospf network point-to-point
  ip router ospf 0 area 0.0.0.0
  ip pim sparse-mode
  no shutdown

interface Ethernet1/2
  no switchport
  mtu 9000
  medium p2p
  ip unnumbered loopback0
  ip ospf network point-to-point
  ip router ospf 0 area 0.0.0.0
  ip pim sparse-mode
  no shutdown

interface Ethernet1/3
  no switchport
  mtu 9000
  medium p2p
  ip unnumbered loopback0
  ip ospf network point-to-point
  ip router ospf 0 area 0.0.0.0
  ip pim sparse-mode
  no shutdown

interface Ethernet1/4
  no switchport

interface Ethernet1/5
  no switchport

interface Ethernet1/6
  no switchport

interface mgmt0
  vrf member management
  ip address 10.1.7.18/24

interface loopback0
  ip address 192.168.0.11/32
  ip router ospf 0 area 0.0.0.0

interface loopback200
  ip address 192.168.200.11/32
  ip router ospf 0 area 0.0.0.0

interface loopback255
  ip address 192.168.255.2/30
  ip ospf network point-to-point
  ip router ospf 0 area 0.0.0.0
  ip pim sparse-mode
line console
line vty
  exec-timeout 0
boot nxos bootflash:/nxos.7.0.3.I7.8.bin
router ospf 0
  router-id 192.168.0.11
router bgp 64999
  router-id 192.168.200.11
  address-family l2vpn evpn
    maximum-paths ibgp 8
  template peer LEAF
    remote-as 64999
    update-source loopback200
    address-family l2vpn evpn
      send-community
      send-community extended
      route-reflector-client
  neighbor 192.168.200.101
    inherit peer LEAF
  neighbor 192.168.200.102
    inherit peer LEAF
  neighbor 192.168.200.103
    inherit peer LEAF

no logging console

lab-csr1k

CLICK ON EXPAND ===> ON THE RIGHT ===> TO SEE THE OUTPUT ===> :

lab-csr1k#show run

version 15.4
service timestamps debug datetime msec
service timestamps log datetime msec
no platform punt-keepalive disable-kernel-core
platform console virtual
!
hostname lab-csr1k
!
boot-start-marker
boot-end-marker
!
!
vrf definition VRF3
 rd 172.31.255.1:3
 !
 address-family ipv4
 exit-address-family
!
vrf definition VRF4
 rd 172.31.255.1:4
 !
 address-family ipv4
 exit-address-family
!
enable secret 5 $1$Rjcp$jA94UgP2/GyojqLqrXN2/0
!
no aaa new-model
!
!
!
!
!
!
!


ip domain name lab.local
!
!
!
!
!
!
!
!
!
!
subscriber templating
multilink bundle-name authenticated
!
!
license udi pid CSR1000V sn 9HALGV4K29M
!
username admin privilege 15 secret 5 $1$XBBh$Lbb6NISmg5khpBsyR8Wah/
!
redundancy
 mode none
!
!
!
ip ssh rsa keypair-name ssh-key
ip ssh version 2
ip scp server enable
!
!
!
!
interface Loopback0
 ip address 172.31.255.1 255.255.255.255
!
interface Loopback3
 vrf forwarding VRF3
 ip address 10.3.100.1 255.255.252.0
!
interface Loopback4
 vrf forwarding VRF4
 ip address 10.4.100.1 255.255.252.0
!
interface VirtualPortGroup0
 ip unnumbered GigabitEthernet1
!
interface GigabitEthernet1
 ip address 10.1.7.7 255.255.255.0
 negotiation auto
!
interface GigabitEthernet2
 no ip address
 negotiation auto

!
interface GigabitEthernet3
 no ip address
 negotiation auto
!
interface GigabitEthernet3.30
 encapsulation dot1Q 30
 vrf forwarding VRF3
 ip address 10.3.30.3 255.255.255.0
!
interface GigabitEthernet3.40
 encapsulation dot1Q 40
 vrf forwarding VRF4
 ip address 10.4.40.3 255.255.255.0
!
router bgp 100
 bgp log-neighbor-changes
 maximum-paths 8
 !
 address-family ipv4 vrf VRF3
  network 10.3.100.0 mask 255.255.252.0
  neighbor 10.3.30.1 remote-as 64999
  neighbor 10.3.30.1 activate
  neighbor 10.3.30.2 remote-as 64999
  neighbor 10.3.30.2 activate
 exit-address-family
 !
 address-family ipv4 vrf VRF4
  network 10.4.100.0 mask 255.255.252.0
  neighbor 10.4.40.1 remote-as 64999
  neighbor 10.4.40.1 activate
  neighbor 10.4.40.2 remote-as 64999
  neighbor 10.4.40.2 activate
 exit-address-family
!
!
virtual-service csr_mgmt
!
ip forward-protocol nd
!
no ip http server
ip http secure-server
!
!
!
!
!
control-plane
!
!
line con 0
 stopbits 1
line aux 0
 stopbits 1
line vty 0 4
 exec-timeout 0 0
 login local
 transport input ssh
!
ntp server 10.1.7.10
!
end

Appendix C. Final configurations

Note that only configurations of BGW-1 and BGW-2 have been modified from the base and are given in this Appendix. The base configuration of L-103, SP-11, and lab-asr1k given in Appendix B. have not been modified and also are final.

BGW-1

CLICK ON EXPAND ===> ON THE RIGHT ===> TO SEE THE OUTPUT ===> :

BGW-1# show run

version 9.2(4) Bios:version
hostname BGW-1
vdc BGW-1 id 1
  limit-resource vlan minimum 16 maximum 4094
  limit-resource vrf minimum 2 maximum 4096
  limit-resource port-channel minimum 0 maximum 511
  limit-resource u4route-mem minimum 248 maximum 248
  limit-resource u6route-mem minimum 96 maximum 96
  limit-resource m4route-mem minimum 58 maximum 58
  limit-resource m6route-mem minimum 8 maximum 8

nv overlay evpn
feature ospf
feature bgp
feature pim
feature fabric forwarding
feature interface-vlan
feature vn-segment-vlan-based
feature lacp
feature nv overlay

username admin password 5 $5$1U2YRfmP$we1GazfktNgcPpCr5NYRPGit/5GQz2LS79zECxOM4D0  role network-admin
ip domain-lookup
copp profile lenient
evpn multisite border-gateway 12
  delay-restore time 30
snmp-server user admin network-admin auth md5 0x9e739edc4cfa2ec898b5e3858d15226f priv 0x9e739edc4cfa2ec898b5e3858d15226f localizedkey
rmon event 1 description FATAL(1) owner PMON@FATAL
rmon event 2 description CRITICAL(2) owner PMON@CRITICAL
rmon event 3 description ERROR(3) owner PMON@ERROR
rmon event 4 description WARNING(4) owner PMON@WARNING
rmon event 5 description INFORMATION(5) owner PMON@INFO

ip pim rp-address 192.168.255.1 group-list 230.0.0.0/8 bidir
ip pim ssm range 232.0.0.0/8
vlan 1,930,940
vlan 930
  vn-segment 200003
vlan 940
  vn-segment 200004

ip prefix-list NSXT-TEPS seq 5 permit 172.16.255.105/32
ip prefix-list NSXT-TEPS seq 6 permit 172.16.255.106/32
route-map BGP2OSPF permit 10
  match ip address prefix-list NSXT-TEPS
route-map REDIST-TO-SITE-EXT-DCI permit 10
  match tag 112
vrf context VRF3
  vni 200003
  rd auto
  address-family ipv4 unicast
    route-target both auto
    route-target both auto evpn
vrf context VRF4
  vni 200004
  rd auto
  address-family ipv4 unicast
    route-target both auto
    route-target both auto evpn
vrf context management
hardware access-list tcam region racl 512
hardware access-list tcam region arp-ether 256 double-wide


interface Vlan1

interface Vlan930
  no shutdown
  vrf member VRF3
  ip forward

interface Vlan940
  no shutdown
  vrf member VRF4
  ip forward

interface nve1
  no shutdown
  host-reachability protocol bgp
  source-interface loopback100
  multisite border-gateway interface loopback112
  member vni 200003 associate-vrf
  member vni 200004 associate-vrf

interface Ethernet1/1
  no switchport
  mtu 9000
  medium p2p
  ip unnumbered loopback0
  ip ospf network point-to-point
  ip router ospf 0 area 0.0.0.0
  ip pim sparse-mode
  evpn multisite fabric-tracking
  no shutdown

interface Ethernet1/2
  no switchport
  no shutdown

interface Ethernet1/2.30
  encapsulation dot1q 30
  vrf member VRF3
  ip address 10.3.30.1/24
  no shutdown

interface Ethernet1/2.40
  encapsulation dot1q 40
  vrf member VRF4
  ip address 10.4.40.1/24
  no shutdown

interface Ethernet1/3
  no switchport
  mtu 1600
  no shutdown

interface Ethernet1/3.105
  mtu 1600
  encapsulation dot1q 105
  ip address 172.16.105.1/24
  no shutdown

interface Ethernet1/4
  no switchport
  mtu 1600
  no shutdown

interface Ethernet1/4.106
  mtu 1600
  encapsulation dot1q 106
  ip address 172.16.106.1/24
  no shutdown

interface Ethernet1/5
  no switchport

interface Ethernet1/6
  no switchport

interface mgmt0
  vrf member management
  ip address 10.1.7.13/24

interface loopback0
  ip address 192.168.0.101/32
  ip router ospf 0 area 0.0.0.0

interface loopback3
  vrf member VRF3
  ip address 10.3.0.101/32

interface loopback4
  vrf member VRF4
  ip address 10.4.0.101/32

interface loopback100
  ip address 192.168.100.101/32
  ip router ospf 0 area 0.0.0.0
  ip pim sparse-mode

interface loopback112
  ip address 192.168.100.112/32 tag 112
  ip router ospf 0 area 0.0.0.0

interface loopback200
  ip address 192.168.200.101/32
  ip router ospf 0 area 0.0.0.0
line console
line vty
  exec-timeout 0
boot nxos bootflash:/nxos.9.2.4.bin
router ospf 0
  router-id 192.168.0.101
  redistribute bgp 64999 route-map BGP2OSPF
router bgp 64999
  router-id 192.168.200.101
  address-family ipv4 unicast
    network 192.168.100.101/32
    redistribute direct route-map REDIST-TO-SITE-EXT-DCI
    maximum-paths 8
  address-family l2vpn evpn
    maximum-paths 8
  neighbor 172.16.105.2
    remote-as 65022
    update-source Ethernet1/3.105
    ebgp-multihop 255
    timers 4 12
    address-family ipv4 unicast
    address-family l2vpn evpn
      send-community
      send-community extended
      rewrite-evpn-rt-asn
  neighbor 172.16.106.2
    remote-as 65022
    update-source Ethernet1/4.106
    ebgp-multihop 255
    timers 4 12
    address-family ipv4 unicast
    address-family l2vpn evpn
      send-community
      send-community extended
      rewrite-evpn-rt-asn
  neighbor 192.168.200.11
    remote-as 64999
    update-source loopback200
    address-family l2vpn evpn
      send-community
      send-community extended
  vrf VRF3
    address-family ipv4 unicast
      network 10.3.0.101/32
      advertise l2vpn evpn
      maximum-paths 8
    neighbor 10.3.30.3
      remote-as 100
      update-source Ethernet1/2.30
      address-family ipv4 unicast
  vrf VRF4
    address-family ipv4 unicast
      network 10.4.0.101/32
      advertise l2vpn evpn
      maximum-paths 8
    neighbor 10.4.40.3
      remote-as 100
      update-source Ethernet1/2.40
      address-family ipv4 unicast
evpn

no logging console

BGW-2

CLICK ON EXPAND ===> ON THE RIGHT ===> TO SEE THE OUTPUT ===> :

version 9.2(4) Bios:version
hostname BGW-2
vdc BGW-2 id 1
  limit-resource vlan minimum 16 maximum 4094
  limit-resource vrf minimum 2 maximum 4096
  limit-resource port-channel minimum 0 maximum 511
  limit-resource u4route-mem minimum 248 maximum 248
  limit-resource u6route-mem minimum 96 maximum 96
  limit-resource m4route-mem minimum 58 maximum 58
  limit-resource m6route-mem minimum 8 maximum 8

nv overlay evpn
feature ospf
feature bgp
feature pim
feature fabric forwarding
feature interface-vlan
feature vn-segment-vlan-based
feature lacp
feature nv overlay

username admin password 5 $5$ql0GFYYU$innLQVZZMjLR/35.BWW7j533rUyxh0n7BRPGzqzt3f5  role network-admin
ip domain-lookup
evpn multisite border-gateway 12
  delay-restore time 30
snmp-server user admin network-admin auth md5 0xa9e8ac1596e8a7efb9863ba23e93394d priv 0xa9e8ac1596e8a7efb9863ba23e93394d localizedkey
rmon event 1 description FATAL(1) owner PMON@FATAL
rmon event 2 description CRITICAL(2) owner PMON@CRITICAL
rmon event 3 description ERROR(3) owner PMON@ERROR
rmon event 4 description WARNING(4) owner PMON@WARNING
rmon event 5 description INFORMATION(5) owner PMON@INFO

ip pim rp-address 192.168.255.1 group-list 230.0.0.0/8 bidir
ip pim ssm range 232.0.0.0/8
vlan 1,930,940
vlan 930
  vn-segment 200003
vlan 940
  vn-segment 200004

ip prefix-list NSXT-TEPS seq 5 permit 172.16.255.105/32
ip prefix-list NSXT-TEPS seq 6 permit 172.16.255.106/32
route-map BGP2OSPF permit 10
  match ip address prefix-list NSXT-TEPS
route-map REDIST-TO-SITE-EXT-DCI permit 10
  match tag 112
vrf context VRF3
  vni 200003
  rd auto
  address-family ipv4 unicast
    route-target both auto
    route-target both auto evpn
vrf context VRF4
  vni 200004
  rd auto
  address-family ipv4 unicast
    route-target both auto
    route-target both auto evpn
vrf context management
hardware access-list tcam region racl 512
hardware access-list tcam region arp-ether 256 double-wide


interface Vlan1

interface Vlan930
  no shutdown
  vrf member VRF3
  ip forward

interface Vlan940
  no shutdown
  vrf member VRF4
  ip forward

interface nve1
  no shutdown
  host-reachability protocol bgp
  source-interface loopback100
  multisite border-gateway interface loopback112
  member vni 200003 associate-vrf
  member vni 200004 associate-vrf

interface Ethernet1/1
  no switchport
  mtu 9000
  medium p2p
  ip unnumbered loopback0
  ip ospf network point-to-point
  ip router ospf 0 area 0.0.0.0
  ip pim sparse-mode
  evpn multisite fabric-tracking
  no shutdown

interface Ethernet1/2
  no switchport
  no shutdown

interface Ethernet1/2.30
  encapsulation dot1q 30
  vrf member VRF3
  ip address 10.3.30.2/24
  no shutdown

interface Ethernet1/2.40
  encapsulation dot1q 40
  vrf member VRF4
  ip address 10.4.40.2/24
  no shutdown

interface Ethernet1/3
  no switchport
  mtu 1600
  no shutdown

interface Ethernet1/3.107
  mtu 1600
  encapsulation dot1q 107
  ip address 172.16.107.1/24
  no shutdown

interface Ethernet1/4
  no switchport
  mtu 1600
  no shutdown

interface Ethernet1/4.108
  mtu 1600
  encapsulation dot1q 108
  ip address 172.16.108.1/24
  no shutdown

interface Ethernet1/5
  no switchport

interface Ethernet1/6
  no switchport

interface mgmt0
  vrf member management
  ip address 10.1.7.14/24

interface loopback0
  ip address 192.168.0.102/32
  ip router ospf 0 area 0.0.0.0

interface loopback3
  vrf member VRF3
  ip address 10.3.0.102/32

interface loopback4
  vrf member VRF4
  ip address 10.4.0.102/32

interface loopback100
  ip address 192.168.100.102/32
  ip router ospf 0 area 0.0.0.0
  ip pim sparse-mode

interface loopback112
  ip address 192.168.100.112/32 tag 112
  ip router ospf 0 area 0.0.0.0

interface loopback200
  ip address 192.168.200.102/32
  ip router ospf 0 area 0.0.0.0
line console
line vty
  exec-timeout 0
boot nxos bootflash:/nxos.9.2.4.bin
router ospf 0
  router-id 192.168.0.102
  redistribute bgp 64999 route-map BGP2OSPF
router bgp 64999
  router-id 192.168.200.102
  address-family ipv4 unicast
    network 192.168.100.102/32
    redistribute direct route-map REDIST-TO-SITE-EXT-DCI
    maximum-paths 8
  address-family l2vpn evpn
    maximum-paths 8
  neighbor 172.16.107.2
    remote-as 65022
    update-source Ethernet1/3.107
    ebgp-multihop 255
    timers 4 12
    address-family ipv4 unicast
    address-family l2vpn evpn
      send-community
      send-community extended
      rewrite-evpn-rt-asn
  neighbor 172.16.108.2
    remote-as 65022
    update-source Ethernet1/4.108
    ebgp-multihop 255
    timers 4 12
    address-family ipv4 unicast
    address-family l2vpn evpn
      send-community
      send-community extended
      rewrite-evpn-rt-asn
  neighbor 192.168.200.11
    remote-as 64999
    update-source loopback200
    address-family l2vpn evpn
      send-community
      send-community extended
  vrf VRF3
    address-family ipv4 unicast
      network 10.3.0.102/32
      advertise l2vpn evpn
      maximum-paths 8
    neighbor 10.3.30.3
      remote-as 100
      update-source Ethernet1/2.30
      address-family ipv4 unicast
  vrf VRF4
    address-family ipv4 unicast
      network 10.4.0.102/32
      advertise l2vpn evpn
      maximum-paths 8
    neighbor 10.4.40.3
      remote-as 100
      update-source Ethernet1/2.40
      address-family ipv4 unicast
evpn

no logging console

Credits

This article has been written by Sergei Ischenko
Senior Consultant at VMware
CCIE #15523 R&S, VCP-NV, VCAP-NV

This article has been reviewed, edited and published by Iwan Hoogendoorn
Staff Consultant at VMware
CCIEx6 #13084, VCP-NV, VCAP-NV.

YouTube Video

Comming soon

Technical Reviews

This article was technically reviewed by the following SME's:

Name LinkedIn Profile
Iwan Hoogendoorn Link

I am always trying to improve the quality of my articles so if you see any errors, mistakes in this article or you have suggestions for improvement, please contact me and I will fix this.