Contents

VXLAN BGP EVPN Layer 2 Fabric

Last updated: April 25, 2017

VXLAN BGP EVPN Layer 2 Fabric

Before proceeding, ensure that the Nexus 5600 Leaf switches are operating in Store-and-Forward mode. This is required for VXLAN encapsulation support on the Nexus 5600s. This command is not required on the 7K/9K platforms. This command requires a reload before taking effect.

hardware ethernet store-and-fwd-switching 
Copy run start
reload
  • Disable all East/West Connectivity between the Spines

  • Disable all East/West Connectivity between the Leafs

  • Enable the necessary features

    • The following functionality will be required (Spine switches only require a subset of these features, do not enable more than is necessary):
      • OSPF
      • BGP
      • PIM
      • BFD
      • EVPN Address-family
      • VXLAN encapsulation
      • VXLAN troubleshooting tools
      • Distributed Anycast Gateway
      • NXAPI
      • Mapping of VLANs to VNIDs
  • Build Underlay Topology

    • Provision Loopback0 as the Overlay Loopback

    • Provision Loopback 1 as the Underlay Loopback

    • Do not assign any IP addresses on the links between the Spine and Leaf switches

    • Leverage OSPF as the Underlay IGP; All links should be in Area 0

      • All OSPF Router-IDs should be the Underlay Loopback address
      • No Network LSAs should exist within the LSDB
    • Implement PIM in the Underlay to enable the Multicast distribution of BUM traffic within the Overlay

      • The creation of an additional loopback on the Spines is acceptable for this task
      • Ensure redundancy is in place for the RP
      • There should not be any (S,G) state on any switches in the fabric
    • Implement BFD for fast failure detection in the underlay for both OSPF and PIM

    • Once the underlay is completed, all devices should be able to ping both Underlay and Overlay Loopback interfaces

  • Create VLANs and VNIDs

    • On the appropriate switches, provision the following (VLAN and VNID values are based on Rack Number, the numbers below are based on Rack-3):

      • VLAN 103

      • VLAN 103 should map to VNID 10103

      • Server-5 should be connected to N5K5 in VLAN 103 and configured as follows:

        • IP address 10.103.103.100/24
        • MAC address: aaaa.aaaa.aa01
        • Gateway 10.103.103.1
      • Server-6 should be connected to N5K6 in VLAN 103 and configured as follows:

        • IP address: 10.103.103.200/24
        • MAC address: cccc.cccc.cc01
        • Gateway 10.103.103.1
      • Interfaces where servers are connected should immediately transition to a STP forwarding state

  • Establish Overlay BGP Peering and Configuration

    • All devices within the fabric will operate within BGP Autonomous System 65000

    • All BGP router-IDs should be the Overlay Loopback address

    • Establish BGP adjacencies in the required address-families i. Leaf switches must not peer with each other ii. All BGP adjacencies must be established using the OVERLAY Loopback iii. Bonus: Implement BGP on the Spines using no more than 10 lines of BGP configuration

    • Ensure communities are included in BGP Updates for the appropriate address families

    • Do not establish any BGP adjacencies in the IPv4 Unicast address-family

    • Configure EVPN L2VNIs on the required switches i. Route-Distinguishers and Route-Targets should be automatically generated

  • Creation of VXLAN Overlay Interface

    • On each VTEP switch, configure the interface required for VXLAN tunnel sourcing and termination

    • The Overlay Loopback IP should be used for addressing of this interface

    • BGP must be used as the control-plane for end-host advertisement and reachability

    • Add the appropriate L2VNIs to this interface

      • VNID 103 should use 239.103.103.103 for BUM traffic forwarding
  • Verify

    • Server-5 should now be able to ping Server-6

Configuration

N5K5:
install feature-set fabric
feature-set fabric
!
hostname N5K5
!
feature fabric forwarding
nv overlay evpn
feature ospf
feature bgp
feature pim
feature bfd
feature nv overlay
feature nxapi
feature vn-segment-vlan-based
!
feature ngoam
!
hardware ethernet store-and-fwd-switching
!
ip pim rp-address 7.7.7.7 group-list 224.0.0.0/4 bidir
ip pim ssm range 232.0.0.0/8
ip pim bfd
!
vlan 103
  vn-segment 10103
!
interface nve1
  no shutdown
  source-interface loopback0
  host-reachability protocol bgp
  member vni 10103
    suppress-arp
    mcast-group 239.103.103.103
!
interface Ethernet1/1
  description SRV5
  switchport access vlan 103
  spanning-tree port type edge
!
interface Ethernet1/5
  no switchport
  medium p2p
  bfd interval 250 min_rx 250 multiplier 4
  no ip redirects
  ip unnumbered loopback1
  ip ospf network point-to-point
  ip router ospf UNDERLAY area 0.0.0.0
  ip pim sparse-mode

interface Ethernet1/6
  no switchport
  medium p2p
  bfd interval 250 min_rx 250 multiplier 4
  no ip redirects
  ip unnumbered loopback1
  ip ospf network point-to-point
  ip router ospf UNDERLAY area 0.0.0.0
  ip pim sparse-mode
!
interface loopback0
  description OVERLAY LOOPBACK
  ip address 2.2.2.55/32
  ip router ospf UNDERLAY area 0.0.0.0
  ip pim sparse-mode

interface loopback1
  description UNDERLAY LOOPBACK
  ip address 1.1.1.55/32
  ip router ospf UNDERLAY area 0.0.0.0
  ip pim sparse-mode
!
router ospf UNDERLAY
  bfd
  router-id 1.1.1.55
!
router bgp 65000
  router-id 2.2.2.55
  address-family l2vpn evpn
  template peer SPINE_PEERS
    update-source loopback0
    address-family l2vpn evpn
      send-community both
  neighbor 2.2.2.75 remote-as 65000
    inherit peer SPINE_PEERS
    description 7K5
  neighbor 2.2.2.76 remote-as 65000
    inherit peer SPINE_PEERS
    description 7K6
evpn
  vni 10103 l2
    rd auto
    route-target import auto
    route-target export auto
!
ngoam install acl

N5K6:
install feature-set fabric
feature-set fabric
!
hostname N5K6
!
feature fabric forwarding
nv overlay evpn
feature ospf
feature bgp
feature pim
feature lldp
feature bfd
feature nv overlay
feature nxapi
feature vn-segment-vlan-based
!
feature ngoam
!
hardware ethernet store-and-fwd-switching
!
ip pim rp-address 7.7.7.7 group-list 224.0.0.0/4 bidir
ip pim ssm range 232.0.0.0/8
ip pim bfd
!
vlan 103
  vn-segment 10103
!
interface nve1
  no shutdown
  source-interface loopback0
  host-reachability protocol bgp
  member vni 10103
    suppress-arp
    mcast-group 239.103.103.103
!
interface Ethernet1/2
  description SRV6
  switchport access vlan 103
  spanning-tree port type edge
!
interface Ethernet1/5
  no switchport
  medium p2p
  bfd interval 250 min_rx 250 multiplier 4
  no ip redirects
  ip unnumbered loopback1
  ip ospf network point-to-point
  ip router ospf UNDERLAY area 0.0.0.0
  ip pim sparse-mode
!
interface Ethernet1/6
  no switchport
  medium p2p
  bfd interval 250 min_rx 250 multiplier 4
  no ip redirects
  ip unnumbered loopback1
  ip ospf network point-to-point
  ip router ospf UNDERLAY area 0.0.0.0
  ip pim sparse-mode
!
interface loopback0
  description OVERLAY LOOPBACK
  ip address 2.2.2.56/32
  ip router ospf UNDERLAY area 0.0.0.0
  ip pim sparse-mode
!
interface loopback1
  description UNDERLAY LOOPBACK
  ip address 1.1.1.56/32
  ip router ospf UNDERLAY area 0.0.0.0
  ip pim sparse-mode
!
router ospf UNDERLAY
  bfd
  router-id 1.1.1.56
!
router bgp 65000
  router-id 2.2.2.56
  address-family l2vpn evpn
  template peer SPINE_PEERS
    update-source loopback0
    address-family l2vpn evpn
      send-community both
  neighbor 2.2.2.75 remote-as 65000
    inherit peer SPINE_PEERS
    description 7K5
  neighbor 2.2.2.76 remote-as 65000
    inherit peer SPINE_PEERS
    description 7K6
evpn
  vni 10103 l2
    rd auto
    route-target import auto
    route-target export auto
!
ngoam install acl

N7K5:
hostname N7K5
!
nv overlay evpn
feature ospf
feature bgp
feature pim
feature udld
feature bfd
feature nxapi
!
feature ngoam
!
ip pim rp-address 7.7.7.7 group-list 224.0.0.0/4 bidir
ip pim ssm range 232.0.0.0/8
ip pim bfd
!
interface Ethernet1/19
  medium p2p
  bfd interval 250 min_rx 250 multiplier 4
  no ip redirects
  ip unnumbered loopback1
  ip ospf network point-to-point
  ip router ospf UNDERLAY area 0.0.0.0
  ip pim sparse-mode
  no shutdown
!
interface Ethernet1/20
  medium p2p
  bfd interval 250 min_rx 250 multiplier 4
  no ip redirects
  ip unnumbered loopback1
  ip ospf network point-to-point
  ip router ospf UNDERLAY area 0.0.0.0
  ip pim sparse-mode
  no shutdown
!
interface loopback0
  description OVERLAY LOOPBACK
  ip address 2.2.2.75/32
  ip router ospf UNDERLAY area 0.0.0.0
  ip pim sparse-mode
!
interface loopback1
  description UNDERLAY LOOPBACK
  ip address 1.1.1.75/32
  ip router ospf UNDERLAY area 0.0.0.0
  ip pim sparse-mode
!
interface loopback7
  description Anycast RP Address
  ip address 7.7.7.6/31
  ip ospf network point-to-point
  ip router ospf UNDERLAY area 0.0.0.0
  ip pim sparse-mode
!
router ospf UNDERLAY
  bfd
  router-id 1.1.1.75
!
router bgp 65000
  router-id 2.2.2.75
  address-family l2vpn evpn
  template peer LEAF_PEERS
    update-source loopback0
    address-family l2vpn evpn
      send-community both
      route-reflector-client
  neighbor 2.2.2.0/24 remote-as 65000
    inherit peer LEAF_PEERS


N7K6:
hostname N7K6
!
feature telnet
nv overlay evpn
feature ospf
feature bgp
feature pim
feature udld
feature bfd
feature nxapi
!
feature ngoam
!
ip pim rp-address 7.7.7.7 group-list 224.0.0.0/4 bidir
ip pim ssm range 232.0.0.0/8
ip pim bfd
!
interface Ethernet1/19
  medium p2p
  bfd interval 250 min_rx 250 multiplier 4
  no ip redirects
  ip unnumbered loopback1
  ip ospf network point-to-point
  ip router ospf UNDERLAY area 0.0.0.0
  ip pim sparse-mode
  no shutdown
!
interface Ethernet1/20
  medium p2p
  bfd interval 250 min_rx 250 multiplier 4
  no ip redirects
  ip unnumbered loopback1
  ip ospf network point-to-point
  ip router ospf UNDERLAY area 0.0.0.0
  ip pim sparse-mode
  no shutdown
!
interface loopback0
  description OVERLAY LOOPBACK
  ip address 2.2.2.76/32
  ip router ospf UNDERLAY area 0.0.0.0
  ip pim sparse-mode
!
interface loopback1
  description UNDERLAY LOOPBACK
  ip address 1.1.1.76/32
  ip router ospf UNDERLAY area 0.0.0.0
  ip pim sparse-mode
!
interface loopback7
  description Anycast RP Address
  ip address 7.7.7.5/30
  ip ospf network point-to-point
  ip router ospf UNDERLAY area 0.0.0.0
  ip pim sparse-mode
!
router ospf UNDERLAY
  bfd
  router-id 1.1.1.76
!
router bgp 65000
  router-id 2.2.2.76
  address-family l2vpn evpn
  template peer LEAF_PEERS
    update-source loopback0
    address-family l2vpn evpn
      send-community both
      route-reflector-client
  neighbor 2.2.2.0/24 remote-as 65000
    inherit peer LEAF_PEERS

Verification

^ back to top