In this lab I plan to build on the previous Lab1 by adding more VNIs, Edge Routing and Bridging ( ERB ) and a link aggregation across 2 edge devices.
For me this is where key advantages of IP fabric using EVPN & VXLAN over traditional chassis fabric designs become apparent.
When I refer to chassis fabric design I really mean what was called stacking or any vendor proprietary fabric solution where a number of devices appear as one, this could be via stacking cables or by converting conventional ports to Stacking or VCP ports. These usually have some disadvantages the vendors tend not to emphasise these are.
Disadvantages of a traditional fabric:
- Vendor Lock in, the devices must all be the same vendor making migration strategies at EOL difficult
- Single device is acting as a Master, this means one device acts as a routing engine ( or at least the control plane ) whilst others are just slave devices acting as line cards, the transition to RE from a backup device usually takes some seconds but more importantly software failures in the Routing Engines do happen and this can bring down the whole fabric.
- Firmware versions must usually be identical, and usually upgrades must take place on the full fabric at on time, Yes ISSU does exist but in my experience when asked “will this defiantly not be service effecting ” the vendors back peddle and say they have only tested between exact version A-> B and they recommend a reboot of the of the whole fabric which in any case with makes ISSU rather pointless.
Advantages of EVPN for a fabric
- EVPN Standards based vendor neutral protocol is in use.
- No single point of failure, the underlay just needs to take care of the Loopback – Loopback connectivity.
- No need for identical firmware versions, devices can be taken down individually for upgrades as required.
Back to the next iteration of my lab,
In this lab I intend to demonstrate distributed IP gateways and a link aggregation across 2 leaf switches.
To simulate hosts with multiple endpoint addresses I used a vSRX in packet mode. I used a Switch ( vQFX ) to test the link aggregation as I could not get this to work direct from the vSRX, on this note Junipers feature explorer does not show this as supported but also does not show it as supported on the vQFX ( but this worked ).
Above is the EVE-NG screen shot of the same lab.
root@LEAF5# show switch-options
vtep-source-interface lo0.0;
route-distinguisher 10.0.255.5:1;
vrf-target target:65200:1;
{master:0}[edit]
root@LEAF5# show protocols evpn
vni-options {
vni 2010 {
vrf-target target:65200:2010;
}
vni 2020 {
vrf-target target:65200:2020;
}
vni 3020 {
vrf-target target:65200:3020;
}
}
encapsulation vxlan;
extended-vni-list [ 2010 2020 3020 ];
root@LEAF5# show vlans
v10 {
vlan-id 10;
l3-interface irb.10;
vxlan {
vni 2010;
}
}
v110 {
vlan-id 110;
l3-interface irb.110;
vxlan {
vni 3020;
}
}
v20 {
vlan-id 20;
l3-interface irb.20;
vxlan {
vni 2020;
}
}
Above is the VRF target and extra VNI config
{master:0}[edit interfaces ae0]
root@LEAF5# show
esi {
00:45:45:45:45:45:45:45:45:45;
all-active;
}
aggregated-ether-options {
lacp {
system-id 00:45:45:45:45:45;
}
}
unit 0 {
family ethernet-switching {
interface-mode trunk;
vlan {
members [ v10 v20 v110 ];
}
}
}
{master:0}[edit interfaces xe-0/0/0]
root@LEAF5# show
gigether-options {
802.3ad ae0;
}
Above is the code for Leaf5 , both the ESI and the LACP system ID must be identical between Leaf4 And Leaf5
{master:0}[edit]
root@LEAF5# show interfaces irb
unit 10 {
virtual-gateway-accept-data;
family inet {
address 192.168.10.205/24 {
primary;
virtual-gateway-address 192.168.10.254;
}
}
}
unit 20 {
virtual-gateway-accept-data;
family inet {
address 192.168.20.205/24 {
primary;
virtual-gateway-address 192.168.20.254;
}
}
}
unit 110 {
virtual-gateway-accept-data;
family inet {
address 192.168.10.205/24 {
primary;
virtual-gateway-address 192.168.10.254;
}
}
}
root@LEAF5# show routing-instances
CustomerB {
instance-type virtual-router;
interface irb.110;
}Above is the essential code for the distributed Gateways, the routing instance was only required because I was experimenting with a separate customer VRF. The only required change between the Leafs would be the Physical IP addresses. The full config is here.
root@LEAF3> ping 192.168.10.3
PING 192.168.10.3 (192.168.10.3): 56 data bytes
64 bytes from 192.168.10.3: icmp_seq=0 ttl=63 time=569.238 ms
64 bytes from 192.168.10.3: icmp_seq=1 ttl=63 time=403.902 ms
root@HOST1> ping 192.168.20.3
PING 192.168.20.3 (192.168.20.3): 56 data bytes
64 bytes from 192.168.20.3: icmp_seq=0 ttl=63 time=185.607 ms
64 bytes from 192.168.20.3: icmp_seq=1 ttl=63 time=311.518 ms
64 bytes from 192.168.20.3: icmp_seq=2 ttl=63 time=248.938 msPings to both the same and different subnets are working
So lets take a look at some of the key tables
these are the commands we will be using for quick reference
# useful show commands
show ethernet-switching table
show evpn database
show ethernet-switching vxlan-tunnel-end-point remote mac-table
show interfaces vtep
show route table bgp.evpn.0
show route table default-switch.evpn.0
show route advertising-protocol bgp
show route receive-protocol bgp
# useful show commands QFX
show ethernet-switching vxlan-tunnel-end-point source
show ethernet-switching vxlan-tunnel-end-point remote
# useful show commands MX
show l2-learning vxlan-tunnel-end-point source
show l2-learning vxlan-tunnel-end-point remote
# Link Aggregations
show lacp statistics
show interfaces ae0 details
I have changes the MAC addresses on the hosts to 00:00:00:11:11:11 ect to make the output more readable.
I just focus on v10 , subnet 192.168.0.0/24 for now for readability
root@LEAF3# run show ethernet-switching table
Vlan MAC MAC Logical SVLBNH/ Active
name address flags interface VENH Index source
v10 00:00:00:11:11:11 D xe-0/0/0.0
v10 00:00:00:22:22:22 D vtep.32769 10.0.255.4
v10 00:00:00:33:33:33 DR esi.1831 00:45:45:45:45:45:45:45:45:45 Host 1 is local
Host2 is via a VTEP
Host3 is on a ethernet segment “ESI”
root@LEAF3# run show ethernet-switching vxlan-tunnel-end-point remote mac-table
MAC flags (S -static MAC, D -dynamic MAC, L -locally learned, C -Control MAC
SE -Statistics enabled, NM -Non configured MAC, R -Remote PE MAC, P -Pinned MAC)
Logical system : <default>
Routing instance : default-switch
Bridging domain : v10+10, VLAN : 10, VNID : 2010
MAC MAC Logical Remote VTEP
address flags interface IP address
00:00:5e:00:01:01 DR esi.1817 10.0.255.5 10.0.255.4
00:00:00:33:33:33 DR esi.1831 10.0.255.4 10.0.255.5
00:00:00:22:22:22 D vtep.32769 10.0.255.4
02:05:86:71:fe:00 D vtep.32769 10.0.255.4
50:00:00:09:00:01 D vtep.32769 10.0.255.4
02:05:86:71:06:00 D vtep.32770 10.0.255.5 We know our VTEP must be up but here is the output notice the ESI of 00:00:00:00:00:00:00:00:00:00 this is for a single homed link.
root@LEAF3# run show interfaces vtep
Physical interface: vtep, Enabled, Physical link is Up
Interface index: 641, SNMP ifIndex: 514
Type: Software-Pseudo, Link-level type: VxLAN-Tunnel-Endpoint, MTU: Unlimited, Speed: Unlimited
Device flags : Present Running
Link type : Full-Duplex
Link flags : None
Last flapped : Never
Input packets : 0
Output packets: 0
Logical interface vtep.32768 (Index 552) (SNMP ifIndex 532)
Flags: Up SNMP-Traps 0x4000 Encapsulation: ENET2
Ethernet segment value: 00:00:00:00:00:00:00:00:00:00, Mode: single-homed, Multi-homed status: Forwarding
VXLAN Endpoint Type: Source, VXLAN Endpoint Address: 10.0.255.3, L2 Routing Instance: default-switch, L3 Routing Instance: default
Input packets : 0
Output packets: 0
Logical interface vtep.32769 (Index 574) (SNMP ifIndex 538)
Flags: Up SNMP-Traps Encapsulation: ENET2
VXLAN Endpoint Type: Remote, VXLAN Endpoint Address: 10.0.255.4, L2 Routing Instance: default-switch, L3 Routing Instance: default
Input packets : 829
Output packets: 831
Protocol eth-switch, MTU: Unlimited
Flags: Trunk-Mode
Logical interface vtep.32770 (Index 575) (SNMP ifIndex 541)
Flags: Up SNMP-Traps Encapsulation: ENET2
VXLAN Endpoint Type: Remote, VXLAN Endpoint Address: 10.0.255.5, L2 Routing Instance: default-switch, L3 Routing Instance: default
Input packets : 11
Output packets: 22
Protocol eth-switch, MTU: Unlimited
Flags: Trunk-Mode
{master:0}[edit]
root@LEAF3# LAB Files:
EVE-NG Files : EVE-NG:
Awesome content!