Juniper EVPN-VXLAN & DCI
Goal
In this tutorial we are going to setup 2 sites: Site A and site B and connect both sites together using a MPLS backbone. Both sites will be configured using EVPN-VXLAN centrally Routed Bridging (CRB).
Prerequisites
For this tutorial I will be using :
- EVE-NG 2.0.3-112
- VQFX model: vqfx-10000 running JUNOS 19.4R1.10 for spines and leaves
- VMX running Junos: 18.2R1.9 for the core routers
- Debian VM's servers
Diagram
Setup and configuration
Site A
| Devices | Role | lo.0 IP | xe-0/0/0 IP | xe-0/0/1 IP | xe-0/0/2 IP | xe-0/0/9 IP | mgmt | AS | local-as |
| qfxssw27 | spine1 | 10.179.0.1/32 | 172.16.1.49/30 | 172.16.2.2/30 | 172.16.3.1/30 | 10.193.0.78/24 | 64601 | 65030 | |
| qfxssw28 | spine2 | 10.179.0.2/32 | 172.16.1.6/30 | 172.16.2.6/30 | 172.16.3.5/30 | 10.193.0.79/24 | 64601 | 65031 | |
| qfxlsw27 | leaf1 | 10.179.0.5/32 | 172.16.1.50/30 | 172.16.1.5/30 | 10.193.0.80/24 | 64601 | 65032 | ||
| qfxlsw28 | leaf2 | 10.179.0.6/32 | 172.16.2.1/30 | 172.16.2.5/30 | 10.193.0.81/24 | 64601 | 65033 | ||
| cr1 | core router | 10.179.0.7/32 | 172.16.3.2/30 | 172.16.3.6/30 | 172.16.4.0/31 | 10.193.0.86/24 | 100 |
| vlan | vlan-id | vni | Network | irb IP | VG |
| private1-a-dal | 2005 | 102005 | 10.192.64.0/22 | 10.192.64.2/22 | 10.192.64.1 |
| private1-b-dal | 2006 | 102006 | 10.192.80.0/22 | 10.192.80.2/22 | 10.192.80.1 |
Spine1 configuration
set system host-name qfxssw27-dfw set interfaces xe-0/0/0 description link_leaf1 set interfaces xe-0/0/0 unit 0 family inet address 172.16.1.49/30 set interfaces xe-0/0/1 description link_leaf2 set interfaces xe-0/0/1 unit 0 family inet address 172.16.2.2/30 set interfaces xe-0/0/2 description link_cr1 set interfaces xe-0/0/2 unit 0 family inet address 172.16.3.1/30 set interfaces em0 unit 0 family inet address 10.193.0.78/24 set interfaces irb unit 2006 proxy-macip-advertisement set interfaces irb unit 2006 virtual-gateway-accept-data set interfaces irb unit 2006 family inet address 10.192.80.2/22 virtual-gateway-address 10.192.80.1 set interfaces irb unit 3502 proxy-macip-advertisement set interfaces irb unit 3502 virtual-gateway-accept-data set interfaces irb unit 3502 family inet address 10.192.64.2/22 virtual-gateway-address 10.192.64.1 set interfaces lo0 unit 0 family inet address 10.179.0.1/32 primary set policy-options policy-statement EVPN-IMPORT term VNI102005 from community com2006 set policy-options policy-statement EVPN-IMPORT term VNI102005 then accept set policy-options policy-statement EVPN-IMPORT2006 term 1 from community com2005 set policy-options policy-statement EVPN-IMPORT2006 term 1 then accept set policy-options policy-statement PFE-ECMP then load-balance per-packet set policy-options policy-statement export-dc-routes term fabric from protocol static set policy-options policy-statement export-dc-routes term fabric from prefix-list-filter fab-lo0s exact set policy-options policy-statement export-dc-routes term fabric then accept set policy-options policy-statement export-dc-routes then reject set policy-options policy-statement import-overlay term 1 from family evpn set policy-options policy-statement import-overlay term 1 from next-hop 10.179.1.1 set policy-options policy-statement import-overlay term 1 from nlri-route-type 1 set policy-options policy-statement import-overlay term 1 from nlri-route-type 2 set policy-options policy-statement import-overlay term 1 then reject set policy-options policy-statement import-overlay then accept set policy-options policy-statement send-direct term send-lo0 from protocol direct set policy-options policy-statement send-direct term send-lo0 from interface lo0.0 set policy-options policy-statement send-direct term send-lo0 then accept set policy-options community com2005 members target:64601:2 set policy-options community com2006 members target:64601:3 set routing-instances vlan2005 routing-options auto-export set routing-instances vlan2005 instance-type vrf set routing-instances vlan2005 interface irb.3502 set routing-instances vlan2005 route-distinguisher 10.179.0.1:2 set routing-instances vlan2005 vrf-import EVPN-IMPORT set routing-instances vlan2005 vrf-target target:64601:2 set routing-instances vlan2006 routing-options auto-export set routing-instances vlan2006 instance-type vrf set routing-instances vlan2006 interface irb.2006 set routing-instances vlan2006 route-distinguisher 10.179.0.1:3 set routing-instances vlan2006 vrf-import EVPN-IMPORT2006 set routing-instances vlan2006 vrf-target target:64601:3 set routing-options static route 0.0.0.0/0 next-hop 10.193.0.1 set routing-options static route 0.0.0.0/0 no-readvertise set routing-options static route 10.179.0.0/24 discard set routing-options forwarding-table export PFE-ECMP set routing-options forwarding-table ecmp-fast-reroute set routing-options router-id 10.179.0.1 set routing-options autonomous-system 64601 set protocols evpn vni-options vni 102005 vrf-target target:1:2005 set protocols evpn vni-options vni 102006 vrf-target target:1:2006 set protocols evpn encapsulation vxlan set protocols evpn multicast-mode ingress-replication set protocols evpn default-gateway no-gateway-community set protocols evpn extended-vni-list all set protocols bgp group UNDERLAY type external set protocols bgp group UNDERLAY mtu-discovery set protocols bgp group UNDERLAY export send-direct set protocols bgp group UNDERLAY local-as 65030 set protocols bgp group UNDERLAY multipath multiple-as set protocols bgp group UNDERLAY bfd-liveness-detection minimum-interval 1000 set protocols bgp group UNDERLAY bfd-liveness-detection multiplier 3 set protocols bgp group UNDERLAY bfd-liveness-detection session-mode automatic set protocols bgp group UNDERLAY neighbor 172.16.2.1 description leaf2 set protocols bgp group UNDERLAY neighbor 172.16.2.1 peer-as 65033 set protocols bgp group UNDERLAY neighbor 172.16.1.50 description leaf1 set protocols bgp group UNDERLAY neighbor 172.16.1.50 peer-as 65032 set protocols bgp group EVPN-IBGP type internal set protocols bgp group EVPN-IBGP local-address 10.179.0.1 set protocols bgp group EVPN-IBGP import import-overlay set protocols bgp group EVPN-IBGP family evpn signaling set protocols bgp group EVPN-IBGP cluster 1.1.1.1 set protocols bgp group EVPN-IBGP multipath set protocols bgp group EVPN-IBGP bfd-liveness-detection minimum-interval 350 set protocols bgp group EVPN-IBGP bfd-liveness-detection multiplier 3 set protocols bgp group EVPN-IBGP bfd-liveness-detection session-mode automatic set protocols bgp group EVPN-IBGP neighbor 10.179.0.5 description lfsw01 set protocols bgp group EVPN-IBGP neighbor 10.179.0.6 description lfsw02 set protocols bgp group EVPN-IBGP neighbor 10.179.0.2 description spine2 set protocols bgp group EVPN-IBGP neighbor 10.179.1.1 description dc2_spine1 set protocols bgp group core type external set protocols bgp group core export export-dc-routes set protocols bgp group core peer-as 100 set protocols bgp group core local-as 65030 set protocols bgp group core neighbor 172.16.3.2 set protocols igmp-snooping vlan default set switch-options vtep-source-interface lo0.0 set switch-options route-distinguisher 10.179.0.1:1 set switch-options vrf-target target:64512:1111 set vlans default vlan-id 1 set vlans private1-a-dal vlan-id 2005 set vlans private1-a-dal l3-interface irb.3502 set vlans private1-a-dal vxlan vni 102005 set vlans private1-b-dal vlan-id 2006 set vlans private1-b-dal l3-interface irb.2006 set vlans private1-b-dal vxlan vni 102006
Spine2 configuration
Leaf1 configuration
set system host-name qfxlsw27-dfw set interfaces xe-0/0/0 description link_spine1 set interfaces xe-0/0/0 unit 0 family inet address 172.16.1.50/30 set interfaces xe-0/0/1 description link_spine2 set interfaces xe-0/0/1 unit 0 family inet address 172.16.1.5/30 set interfaces xe-0/0/10 description srv101 set interfaces xe-0/0/10 unit 0 family ethernet-switching interface-mode access set interfaces xe-0/0/10 unit 0 family ethernet-switching vlan members private1-b-dal set interfaces xe-0/0/11 description srv100 set interfaces xe-0/0/11 unit 0 family ethernet-switching interface-mode access set interfaces xe-0/0/11 unit 0 family ethernet-switching vlan members private1-a-dal set interfaces em0 unit 0 family inet address 10.193.0.80/24 set interfaces lo0 unit 0 family inet address 10.179.0.5/32 set policy-options policy-statement PFE-ECMP then load-balance per-packet set policy-options policy-statement send-direct term send-lo0 from protocol direct set policy-options policy-statement send-direct term send-lo0 then accept set routing-options static route 0.0.0.0/0 next-hop 10.193.0.1 set routing-options static route 0.0.0.0/0 no-readvertise set routing-options forwarding-table export PFE-ECMP set routing-options forwarding-table ecmp-fast-reroute set routing-options router-id 10.179.0.5 set routing-options autonomous-system 64601 set protocols evpn vni-options vni 102005 vrf-target target:1:2005 set protocols evpn vni-options vni 102006 vrf-target target:1:2006 set protocols evpn encapsulation vxlan set protocols evpn multicast-mode ingress-replication set protocols evpn extended-vni-list all set protocols bgp group UNDERLAY type external set protocols bgp group UNDERLAY export send-direct set protocols bgp group UNDERLAY local-as 65032 set protocols bgp group UNDERLAY multipath multiple-as set protocols bgp group UNDERLAY bfd-liveness-detection minimum-interval 1000 set protocols bgp group UNDERLAY bfd-liveness-detection multiplier 3 set protocols bgp group UNDERLAY bfd-liveness-detection session-mode automatic set protocols bgp group UNDERLAY neighbor 172.16.1.6 description spine2 set protocols bgp group UNDERLAY neighbor 172.16.1.6 peer-as 65031 set protocols bgp group UNDERLAY neighbor 172.16.1.49 description spine set protocols bgp group UNDERLAY neighbor 172.16.1.49 peer-as 65030 set protocols bgp group EVPN-IBGP type internal set protocols bgp group EVPN-IBGP local-address 10.179.0.5 set protocols bgp group EVPN-IBGP family evpn signaling set protocols bgp group EVPN-IBGP multipath set protocols bgp group EVPN-IBGP bfd-liveness-detection minimum-interval 350 set protocols bgp group EVPN-IBGP bfd-liveness-detection multiplier 3 set protocols bgp group EVPN-IBGP bfd-liveness-detection session-mode automatic set protocols bgp group EVPN-IBGP neighbor 10.179.0.1 description spsw01 set protocols bgp group EVPN-IBGP neighbor 10.179.0.2 description spsw02 set protocols lldp disable set protocols lldp port-id-subtype interface-name set protocols lldp interface all set protocols lldp-med interface all set protocols igmp-snooping vlan default set switch-options service-id 1 set switch-options vtep-source-interface lo0.0 set switch-options route-distinguisher 10.179.0.5:1 set switch-options vrf-target target:64512:1111 set vlans private1-a-dal vlan-id 2005 set vlans private1-a-dal vxlan vni 102005 set vlans private1-b-dal vlan-id 2006 set vlans private1-b-dal vxlan vni 102006