Juniper Collapsed Spine with EVPN
Jump to navigation
Jump to search
Goal
In this tutorial, We will be setting up a collapsed Spine EVPN/VXLAN with multi-homing.We will use two SRX's in cluster mode to route traffic between the 2 Tenants that we will configure. We will be using BGP for the underlay and overlay.
Prerequisites
For this tutorial we will be using:
- EVE-NG 2.0.3-112
- VQFX model: vqfx-10000 running JUNOS 19.4R1.10 for spines and leaves
- VSRX 3.0
- Debian VM's servers
Diagram
Setup and configuration
| Devices | Role | lo.0 IP | VLAN | IP address | ae0 IP | ae0 interfaces | ae1 interfaces | ae2 interfaces | ae5 interfaces | ae6 interfaces | mgmt | local-as underlay | local-as overlay |
| cc1 | core swtich1 | 10.179.1.1/32 | 172.16.3.1/3 | xe-0/0[10-11] | xe-0/0/0 | xe-0/0/1 | xe-0/0/9 | xe-0/0/8 | 10.193.0.105/24 | 65012 | 65100 | ||
| cc2 | core switch2 | 10.179.1.2/32 | 172.16.3.2/30 | xe-0/0[10-11] | xe-0/0/0 | xe-0/0/1 | xe-0/0/9 | xe-0/0/8 | 10.193.0.106/24 | 65013 | 65100 | ||
| sw1 | switch 1 | xe-0/0[0-1] | 10.193.0.107/24 | ||||||||||
| sw2 | switch 2 | xe-0/0[0-1] | 10.193.0.108/24 | ||||||||||
| srx | Firewall | ||||||||||||
| srv1 | Server1 | private1-a-dfw | 10.192.144.100/22 | ||||||||||
| srv2 | Server2 | private1-b-dfw | 10.192.160.100/22 | ||||||||||
| srv3 | Server3 | private1-d-dfw | 10.192.192.100/22 | ||||||||||
| srv4 | Server4 | private1-c-dfw | 10.192.176.100/22 | ||||||||||
| srv5 | Server5 | private1-a-dfw | 10.192.144.200/22 | ||||||||||
| srv6 | Server6 | private1-b-dfw | 10.192.160.200/22 | ||||||||||
| srv7 | Server7 | private1-c-dfw | 10.192.176.200/22 | ||||||||||
| srv8 | Server8 | private1-d-dfw | 10.192.192.200/22 |
core switch 1 configuration
set system host-name sswecc1-dfw set system root-authentication encrypted-password "$5$bSgF2gnxBS/rA$sYP/f1pWJhl5d1VN0hHzjxd0jZhmnwGLCiwVm3hE8Z." set system login user homer uid 2002 set system login user homer class super-user set system login user homer authentication ssh-rsa "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC4xpjWJoQhCf176i77ni9//mcYO3bBWu7necWZwJNVkFsvvT6XuWfkKVMUFnTjTMr1erv8WRDze7le9Jl2a/xMIgo9Cf71SU9faPbd /ukvaLl5VUeGvHKFg9d+7GUGx1z9K1qKY2VOBO5EQCht8+4o4mMaizoXoxHvkNolswAa5Jv/EPwnfCeDyV7TsG+Se1k7 /1h1VFOwW7Dbxno1aCnMDYbcfiBnzGnLSZQGjehok6cqYTjsNIIdAiZYSpH77pnAGglFhxNUSlqj0qRIJZdG3nhPlvIRPjn7fouq3BJEmiWPP8ru67H1J2mdSkix4xOxdUWfGB9eJlENfnobJjBr pp@U18" set system login user ppaul uid 2003 set system login user ppaul class super-user set system login user ppaul authentication ssh-rsa "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDST4EbXJc9l/AdrVmOZEPl3sxi6qjGIZyPwkupthSdooFHxPxUIh/a5PC9bMk5go6KvRoChpc4L8XuMRsxLTd6Ro6DsWIZieGHFuO /AL9SRUtmevGiSC2q4ibR7ACosJBUvKPRVK8anYnMSL9YWd7lnmVLnW5mvOM3Alhd8aTNKE3/H9ogDt9UfndEJXmieMTLJzGvx65sw6riqa5hh6iOcw02qb3QQCKLSRJmUJQuToY4oo/ZdLl/prEDKQ0I9DSnOxRYIvZxvUsTzwoXVq9X9dWGkKAAMDw7f2DJfa/4uCNT2dKPydApeN0ea2/69VRL3fmTz47y0CC1RTEd8j1j U18pc " set chassis aggregated-devices ethernet device-count 10 set interfaces xe-0/0/0 description Link_leaf1-xe-0/0/0 set interfaces xe-0/0/0 gigether-options 802.3ad ae1 set interfaces xe-0/0/1 description link_leaf2_xe-0/0/0 set interfaces xe-0/0/1 gigether-options 802.3ad ae2 set interfaces xe-0/0/8 description vsrx_node1_ge-7/0/4 set interfaces xe-0/0/8 gigether-options 802.3ad ae6 set interfaces xe-0/0/9 description vsrx_node0_ge-0/0/4 set interfaces xe-0/0/9 gigether-options 802.3ad ae5 set interfaces xe-0/0/10 gigether-options 802.3ad ae0 set interfaces xe-0/0/11 gigether-options 802.3ad ae0 set interfaces ae0 description link_spine2 set interfaces ae0 mtu 9216 set interfaces ae0 aggregated-ether-options lacp active set interfaces ae0 aggregated-ether-options lacp periodic fast set interfaces ae0 unit 0 family inet address 172.16.3.1/30 set interfaces ae1 description sw1_ae1 set interfaces ae1 mtu 9192 set interfaces ae1 esi 00:00:00:ab:cd:00:01:00:00:03 set interfaces ae1 esi all-active set interfaces ae1 aggregated-ether-options lacp active set interfaces ae1 aggregated-ether-options lacp periodic fast set interfaces ae1 aggregated-ether-options lacp system-id 00:11:00:00:00:01 set interfaces ae1 unit 0 family ethernet-switching interface-mode trunk set interfaces ae1 unit 0 family ethernet-switching vlan members private1-a-dfw set interfaces ae1 unit 0 family ethernet-switching vlan members private1-b-dfw set interfaces ae1 unit 0 family ethernet-switching vlan members private1-c-dfw set interfaces ae1 unit 0 family ethernet-switching vlan members private1-d-dfw set interfaces ae2 description sw2_ae1 set interfaces ae2 mtu 9216 set interfaces ae2 esi 00:00:00:ab:cd:00:01:00:00:04 set interfaces ae2 esi all-active set interfaces ae2 aggregated-ether-options lacp active set interfaces ae2 aggregated-ether-options lacp periodic fast set interfaces ae2 aggregated-ether-options lacp system-id 00:22:00:00:00:02 set interfaces ae2 unit 0 family ethernet-switching interface-mode trunk set interfaces ae2 unit 0 family ethernet-switching vlan members private1-a-dfw set interfaces ae2 unit 0 family ethernet-switching vlan members private1-b-dfw set interfaces ae2 unit 0 family ethernet-switching vlan members private1-c-dfw set interfaces ae2 unit 0 family ethernet-switching vlan members private1-d-dfw set interfaces ae5 description "to SRX cluster node0" set interfaces ae5 mtu 9216 set interfaces ae5 esi 00:00:00:00:00:00:00:00:01:11 set interfaces ae5 esi all-active set interfaces ae5 aggregated-ether-options lacp active set interfaces ae5 aggregated-ether-options lacp periodic fast set interfaces ae5 aggregated-ether-options lacp system-id 00:00:00:00:01:11 set interfaces ae5 unit 0 family ethernet-switching interface-mode trunk set interfaces ae5 unit 0 family ethernet-switching vlan members VLAN_42 set interfaces ae5 unit 0 family ethernet-switching vlan members VLAN_46 set interfaces ae6 description "to SRx Cluster" set interfaces ae6 mtu 9216 set interfaces ae6 esi 00:00:00:00:00:00:00:00:01:12 set interfaces ae6 esi all-active set interfaces ae6 aggregated-ether-options lacp active set interfaces ae6 aggregated-ether-options lacp periodic fast set interfaces ae6 aggregated-ether-options lacp system-id 00:00:00:00:01:12 set interfaces ae6 unit 0 family ethernet-switching interface-mode trunk set interfaces ae6 unit 0 family ethernet-switching vlan members VLAN_42 set interfaces ae6 unit 0 family ethernet-switching vlan members VLAN_46 set interfaces em0 unit 0 family inet address 10.193.0.105/24 set interfaces irb unit 20 virtual-gateway-accept-data set interfaces irb unit 20 description "l3 interface for vlan private1-a-dfw" set interfaces irb unit 20 family inet address 10.192.144.3/22 preferred set interfaces irb unit 20 family inet address 10.192.144.3/22 virtual-gateway-address 10.192.144.1 set interfaces irb unit 30 virtual-gateway-accept-data set interfaces irb unit 30 description "l3 interface for vlan private1-b-dfw" set interfaces irb unit 30 family inet address 10.192.160.3/22 preferred set interfaces irb unit 30 family inet address 10.192.160.3/22 virtual-gateway-address 10.192.160.1 set interfaces irb unit 40 virtual-gateway-accept-data set interfaces irb unit 40 description "l3 inteface for vlan private1-c-dfw" set interfaces irb unit 40 family inet address 10.192.176.3/22 preferred set interfaces irb unit 40 family inet address 10.192.176.3/22 virtual-gateway-address 10.192.176.1 set interfaces irb unit 42 description "Tenant1 SRX Interconnect" set interfaces irb unit 42 family inet address 172.16.4.2/29 set interfaces irb unit 46 description "Tenant2 SRx Interconnect" set interfaces irb unit 46 family inet address 172.16.5.2/29 set interfaces irb unit 50 virtual-gateway-accept-data set interfaces irb unit 50 description "l3 interface for vlan private1-d-dfw" set interfaces irb unit 50 family inet address 10.192.192.3/22 preferred set interfaces irb unit 50 family inet address 10.192.192.3/22 virtual-gateway-address 10.192.192.1 set interfaces lo0 unit 0 family inet address 10.179.1.1/32 set interfaces lo0 unit 5 family inet address 10.179.1.3/32 set interfaces lo0 unit 6 family inet address 10.179.1.5/32 set forwarding-options vxlan-routing next-hop 32768 set forwarding-options vxlan-routing overlay-ecmp set policy-options policy-statement ECMP-POLICY then load-balance per-packet set policy-options policy-statement Interconnect_Tenant1_Export term Tenant_Routes from route-filter 10.192.128.0/17 orlonger set policy-options policy-statement Interconnect_Tenant1_Export term Tenant_Routes from route-filter 10.179.0.0/16 orlonger set policy-options policy-statement Interconnect_Tenant1_Export term Tenant_Routes then accept set policy-options policy-statement Interconnect_Tenant1_Export term DEFAULT then reject set policy-options policy-statement Interconnect_Tenant1_Import term Tenant_Routes from route-filter 10.179.0.0/16 orlonger set policy-options policy-statement Interconnect_Tenant1_Import term Tenant_Routes from route-filter 10.192.128.0/17 orlonger set policy-options policy-statement Interconnect_Tenant1_Import term Tenant_Routes then accept set policy-options policy-statement Interconnect_Tenant1_Import term DEFAULT then reject set policy-options policy-statement Interconnect_Tenant2_Export term Tenant_Routes from route-filter 10.192.128.0/17 orlonger set policy-options policy-statement Interconnect_Tenant2_Export term Tenant_Routes from route-filter 10.179.0.0/16 orlonger set policy-options policy-statement Interconnect_Tenant2_Export term Tenant_Routes then accept set policy-options policy-statement Interconnect_Tenant2_Export term DEFAULT then reject set policy-options policy-statement Interconnect_Tenant2_Import term Tenant_Routes from route-filter 10.179.0.0/16 orlonger set policy-options policy-statement Interconnect_Tenant2_Import term Tenant_Routes from route-filter 10.192.128.0/17 orlonger set policy-options policy-statement Interconnect_Tenant2_Import term Tenant_Routes then accept set policy-options policy-statement Interconnect_Tenant2_Import term DEFAULT then reject set policy-options policy-statement T5_EXPORT term 1 from protocol direct set policy-options policy-statement T5_EXPORT term 1 then accept set policy-options policy-statement T5_EXPORT term 2 from protocol bgp set policy-options policy-statement T5_EXPORT term 2 then accept set policy-options policy-statement UNDERLAY-EXPORT term LOOPBACK from route-filter 10.179.1.0/24 orlonger set policy-options policy-statement UNDERLAY-EXPORT term LOOPBACK then accept set policy-options policy-statement UNDERLAY-EXPORT term DEFAULT then reject set policy-options policy-statement UNDERLAY-IMPORT term LOOPBACK from route-filter 10.179.1.0/24 orlonger set policy-options policy-statement UNDERLAY-IMPORT term LOOPBACK then accept set policy-options policy-statement UNDERLAY-IMPORT term DEFAULT then reject set routing-instances Tennat1 routing-options multipath set routing-instances Tennat1 protocols evpn ip-prefix-routes advertise direct-nexthop set routing-instances Tennat1 protocols evpn ip-prefix-routes encapsulation vxlan set routing-instances Tennat1 protocols evpn ip-prefix-routes vni 1101 set routing-instances Tennat1 protocols evpn ip-prefix-routes export T5_EXPORT set routing-instances Tennat1 protocols bgp group INTERCONNECT type external set routing-instances Tennat1 protocols bgp group INTERCONNECT import Interconnect_Tenant1_Import set routing-instances Tennat1 protocols bgp group INTERCONNECT family inet unicast set routing-instances Tennat1 protocols bgp group INTERCONNECT authentication-key "$9$JNZi.Pfz6CuTzlMX-2gTz3n/tuO1" set routing-instances Tennat1 protocols bgp group INTERCONNECT export Interconnect_Tenant1_Export set routing-instances Tennat1 protocols bgp group INTERCONNECT local-as 65112 set routing-instances Tennat1 protocols bgp group INTERCONNECT multipath multiple-as set routing-instances Tennat1 protocols bgp group INTERCONNECT bfd-liveness-detection minimum-interval 1000 set routing-instances Tennat1 protocols bgp group INTERCONNECT bfd-liveness-detection multiplier 3 set routing-instances Tennat1 protocols bgp group INTERCONNECT neighbor 172.16.4.1 peer-as 65200 set routing-instances Tennat1 instance-type vrf set routing-instances Tennat1 interface irb.20 set routing-instances Tennat1 interface irb.30 set routing-instances Tennat1 interface irb.42 set routing-instances Tennat1 interface lo0.5 set routing-instances Tennat1 route-distinguisher 10.179.1.3:1101 set routing-instances Tennat1 vrf-target target:64701:20 set routing-instances Tennat1 vrf-table-label set routing-instances Tennat2 routing-options multipath set routing-instances Tennat2 protocols evpn ip-prefix-routes advertise direct-nexthop set routing-instances Tennat2 protocols evpn ip-prefix-routes encapsulation vxlan set routing-instances Tennat2 protocols evpn ip-prefix-routes vni 1102 set routing-instances Tennat2 protocols evpn ip-prefix-routes export T5_EXPORT set routing-instances Tennat2 protocols bgp group INTERCONNECT type external set routing-instances Tennat2 protocols bgp group INTERCONNECT import Interconnect_Tenant2_Import set routing-instances Tennat2 protocols bgp group INTERCONNECT family inet unicast set routing-instances Tennat2 protocols bgp group INTERCONNECT authentication-key "$9$JNZi.Pfz6CuTzlMX-2gTz3n/tuO1" set routing-instances Tennat2 protocols bgp group INTERCONNECT export Interconnect_Tenant2_Export set routing-instances Tennat2 protocols bgp group INTERCONNECT local-as 65112 set routing-instances Tennat2 protocols bgp group INTERCONNECT multipath multiple-as set routing-instances Tennat2 protocols bgp group INTERCONNECT bfd-liveness-detection minimum-interval 1000 set routing-instances Tennat2 protocols bgp group INTERCONNECT bfd-liveness-detection multiplier 3 set routing-instances Tennat2 protocols bgp group INTERCONNECT neighbor 172.16.5.1 peer-as 65200 set routing-instances Tennat2 instance-type vrf set routing-instances Tennat2 interface irb.40 set routing-instances Tennat2 interface irb.46 set routing-instances Tennat2 interface irb.50 set routing-instances Tennat2 interface lo0.6 set routing-instances Tennat2 route-distinguisher 10.179.1.5:1102 set routing-instances Tennat2 vrf-target target:64701:4050 set routing-instances Tennat2 vrf-table-label set routing-options static route 0.0.0.0/0 next-hop 10.193.0.1 set routing-options static route 0.0.0.0/0 no-readvertise set routing-options forwarding-table export ECMP-POLICY set routing-options forwarding-table ecmp-fast-reroute set routing-options forwarding-table chained-composite-next-hop ingress evpn set routing-options router-id 10.179.1.1 set routing-options graceful-restart set protocols evpn encapsulation vxlan set protocols evpn default-gateway do-not-advertise set protocols evpn extended-vni-list 5020 set protocols evpn extended-vni-list 5030 set protocols evpn extended-vni-list 5040 set protocols evpn extended-vni-list 5042 set protocols evpn extended-vni-list 5046 set protocols evpn extended-vni-list 5050 set protocols evpn no-core-isolation set protocols bgp group RR-OVERLAY type internal set protocols bgp group RR-OVERLAY local-address 10.179.1.1 set protocols bgp group RR-OVERLAY family evpn signaling set protocols bgp group RR-OVERLAY local-as 65100 set protocols bgp group RR-OVERLAY multipath set protocols bgp group RR-OVERLAY bfd-liveness-detection minimum-interval 1000 set protocols bgp group RR-OVERLAY bfd-liveness-detection multiplier 3 set protocols bgp group RR-OVERLAY neighbor 10.179.1.2 set protocols bgp group RR-OVERLAY vpn-apply-export set protocols bgp group UNDERLAY type external set protocols bgp group UNDERLAY import UNDERLAY-IMPORT set protocols bgp group UNDERLAY family inet unicast set protocols bgp group UNDERLAY authentication-key "$9$5TnCtpBESe0BVYoGq.0BIRhrevW" set protocols bgp group UNDERLAY export UNDERLAY-EXPORT set protocols bgp group UNDERLAY local-as 65012 set protocols bgp group UNDERLAY multipath multiple-as set protocols bgp group UNDERLAY neighbor 172.16.3.2 peer-as 65013 set protocols bgp graceful-restart restart-time 30 set protocols l2-learning global-mac-table-aging-time 600 set protocols l2-learning global-mac-ip-table-aging-time 300 set switch-options vtep-source-interface lo0.0 set switch-options route-distinguisher 10.179.1.1:64701 set switch-options vrf-target target:64701:9999 set switch-options vrf-target auto set vlans VLAN_42 vlan-id 42 set vlans VLAN_42 l3-interface irb.42 set vlans VLAN_42 vxlan vni 5042 set vlans VLAN_46 vlan-id 46 set vlans VLAN_46 l3-interface irb.46 set vlans VLAN_46 vxlan vni 5046 set vlans private1-a-dfw vlan-id 20 set vlans private1-a-dfw l3-interface irb.20 set vlans private1-a-dfw vxlan vni 5020 set vlans private1-b-dfw vlan-id 30 set vlans private1-b-dfw l3-interface irb.30 set vlans private1-b-dfw vxlan vni 5030 set vlans private1-c-dfw vlan-id 40 set vlans private1-c-dfw l3-interface irb.40 set vlans private1-c-dfw vxlan vni 5040 set vlans private1-d-dfw description Admin set vlans private1-d-dfw vlan-id 50 set vlans private1-d-dfw l3-interface irb.50 set vlans private1-d-dfw vxlan vni 5050
core switch 2 configuration
set system host-name sswecc2-dfw set system root-authentication encrypted-password "$5$bSgF2gnxBS/rA$sYP/f1pWJhl5d1VN0hHzjxd0jZhmnwGLCiwVm3hE8Z." set system login user homer uid 2002 set system login user homer class super-user set system login user homer authentication ssh-rsa "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC4xpjWJoQhCf176i77ni9//mcYO3bBWu7necWZwJNVkFsvvT6XuWfkKVMUFnTjTMr1erv8WRDze7le9Jl2a/xMIgo9Cf71SU9faPbd /ukvaLl5VUeGvHKFg9d+7GUGx1z9K1qKY2VOBO5EQCht8+4o4mMaizoXoxHvkNolswAa5Jv/EPwnfCeDyV7TsG+Se1k7 /1h1VFOwW7Dbxno1aCnMDYbcfiBnzGnLSZQGjehok6cqYTjsNIIdAiZYSpH77pnAGglFhxNUSlqj0qRIJZdG3nhPlvIRPjn7fouq3BJEmiWPP8ru67H1J2mdSkix4xOxdUWfGB9eJlENfnobJjBr pp@U18" set system login user ppaul uid 2003 set system login user ppaul class super-user set system login user ppaul authentication ssh-rsa "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDST4EbXJc9l/AdrVmOZEPl3sxi6qjGIZyPwkupthSdooFHxPxUIh/a5PC9bMk5go6KvRoChpc4L8XuMRsxLTd6Ro6DsWIZieGHFuO /AL9SRUtmevGiSC2q4ibR7ACosJBUvKPRVK8anYnMSL9YWd7lnmVLnW5mvOM3Alhd8aTNKE3/H9ogDt9UfndEJXmieMTLJzGvx65sw6riqa5hh6iOcw02qb3QQCKLSRJmUJQuToY4oo/ZdLl/prEDKQ0I9DSnOxRYIvZxvUsTzwoXVq9X9dWGkKAAMDw7f2DJfa/4uCNT2dKPydApeN0ea2/69VRL3fmTz47y0CC1RTEd8j1j U18pc " set chassis aggregated-devices ethernet device-count 10 set interfaces xe-0/0/0 description link_leaf1_xe-0/0/1 set interfaces xe-0/0/0 gigether-options 802.3ad ae1 set interfaces xe-0/0/1 description link_leaf2_xe-0/0/1 set interfaces xe-0/0/1 gigether-options 802.3ad ae2 set interfaces xe-0/0/8 description vsrx_node1_ge-7/0/5 set interfaces xe-0/0/8 gigether-options 802.3ad ae6 set interfaces xe-0/0/9 description vsrx_node0_ge-0/0/5 set interfaces xe-0/0/9 gigether-options 802.3ad ae5 set interfaces xe-0/0/10 gigether-options 802.3ad ae0 set interfaces xe-0/0/11 gigether-options 802.3ad ae0 set interfaces ae0 description link-spine1 set interfaces ae0 mtu 9216 set interfaces ae0 aggregated-ether-options lacp active set interfaces ae0 aggregated-ether-options lacp periodic fast set interfaces ae0 unit 0 family inet address 172.16.3.2/30 set interfaces ae1 description leaf1_ae1 set interfaces ae1 mtu 9192 set interfaces ae1 esi 00:00:00:ab:cd:00:01:00:00:03 set interfaces ae1 esi all-active set interfaces ae1 aggregated-ether-options lacp active set interfaces ae1 aggregated-ether-options lacp periodic fast set interfaces ae1 aggregated-ether-options lacp system-id 00:11:00:00:00:01 set interfaces ae1 unit 0 family ethernet-switching interface-mode trunk set interfaces ae1 unit 0 family ethernet-switching vlan members private1-a-dfw set interfaces ae1 unit 0 family ethernet-switching vlan members private1-b-dfw set interfaces ae1 unit 0 family ethernet-switching vlan members private1-c-dfw set interfaces ae1 unit 0 family ethernet-switching vlan members private1-d-dfw set interfaces ae2 mtu 9216 set interfaces ae2 esi 00:00:00:ab:cd:00:01:00:00:04 set interfaces ae2 esi all-active set interfaces ae2 aggregated-ether-options lacp active set interfaces ae2 aggregated-ether-options lacp periodic fast set interfaces ae2 aggregated-ether-options lacp system-id 00:22:00:00:00:02 set interfaces ae2 unit 0 family ethernet-switching interface-mode trunk set interfaces ae2 unit 0 family ethernet-switching vlan members private1-a-dfw set interfaces ae2 unit 0 family ethernet-switching vlan members private1-b-dfw set interfaces ae2 unit 0 family ethernet-switching vlan members private1-c-dfw set interfaces ae2 unit 0 family ethernet-switching vlan members private1-d-dfw set interfaces ae5 description "to SRX cluster node0" set interfaces ae5 mtu 9216 set interfaces ae5 esi 00:00:00:00:00:00:00:00:01:11 set interfaces ae5 esi all-active set interfaces ae5 aggregated-ether-options lacp active set interfaces ae5 aggregated-ether-options lacp periodic fast set interfaces ae5 aggregated-ether-options lacp system-id 00:00:00:00:01:11 set interfaces ae5 unit 0 family ethernet-switching interface-mode trunk set interfaces ae5 unit 0 family ethernet-switching vlan members VLAN_42 set interfaces ae5 unit 0 family ethernet-switching vlan members VLAN_46 set interfaces ae6 description "to SRx Cluster" set interfaces ae6 mtu 9216 set interfaces ae6 esi 00:00:00:00:00:00:00:00:01:12 set interfaces ae6 esi all-active set interfaces ae6 aggregated-ether-options lacp active set interfaces ae6 aggregated-ether-options lacp periodic fast set interfaces ae6 aggregated-ether-options lacp system-id 00:00:00:00:01:12 set interfaces ae6 unit 0 family ethernet-switching interface-mode trunk set interfaces ae6 unit 0 family ethernet-switching vlan members VLAN_42 set interfaces ae6 unit 0 family ethernet-switching vlan members VLAN_46 set interfaces em0 unit 0 family inet address 10.193.0.106/24 set interfaces irb unit 20 virtual-gateway-accept-data set interfaces irb unit 20 description "l3 interface for vlan private1-a-dfw" set interfaces irb unit 20 family inet address 10.192.144.2/22 preferred set interfaces irb unit 20 family inet address 10.192.144.2/22 virtual-gateway-address 10.192.144.1 set interfaces irb unit 30 virtual-gateway-accept-data set interfaces irb unit 30 description "l3 interface for vlan private1-b-dfw" set interfaces irb unit 30 family inet address 10.192.160.2/22 preferred set interfaces irb unit 30 family inet address 10.192.160.2/22 virtual-gateway-address 10.192.160.1 set interfaces irb unit 40 virtual-gateway-accept-data set interfaces irb unit 40 description "l3 interface for vlan private1-c-dfw" set interfaces irb unit 40 family inet address 10.192.176.2/22 preferred set interfaces irb unit 40 family inet address 10.192.176.2/22 virtual-gateway-address 10.192.176.1 set interfaces irb unit 42 description "Tenant1 SRX Interconnect" set interfaces irb unit 42 family inet address 172.16.4.3/29 set interfaces irb unit 46 description "Tenant2 SRx Interconnect" set interfaces irb unit 46 family inet address 172.16.5.3/29 set interfaces irb unit 50 virtual-gateway-accept-data set interfaces irb unit 50 description "l3 interface for vlan private1-d-dfw" set interfaces irb unit 50 family inet address 10.192.192.2/22 virtual-gateway-address 10.192.192.1 set interfaces irb unit 50 family inet address 10.192.192.3/22 preferred set interfaces lo0 unit 0 family inet address 10.179.1.2/32 set interfaces lo0 unit 5 family inet address 10.179.1.4/32 set interfaces lo0 unit 6 family inet address 10.179.1.6/32 set forwarding-options vxlan-routing next-hop 32768 set forwarding-options vxlan-routing overlay-ecmp set policy-options policy-statement ECMP-POLICY then load-balance per-packet set policy-options policy-statement Interconnect_Tenant1_Export term Tenant_Routes from route-filter 10.179.0.0/16 orlonger set policy-options policy-statement Interconnect_Tenant1_Export term Tenant_Routes from route-filter 10.192.128.0/17 orlonger set policy-options policy-statement Interconnect_Tenant1_Export term Tenant_Routes then accept set policy-options policy-statement Interconnect_Tenant1_Export term DEFAULT then reject set policy-options policy-statement Interconnect_Tenant1_Import term Tenant_Routes from route-filter 10.179.0.0/16 orlonger set policy-options policy-statement Interconnect_Tenant1_Import term Tenant_Routes from route-filter 10.192.128.0/17 orlonger set policy-options policy-statement Interconnect_Tenant1_Import term Tenant_Routes then accept set policy-options policy-statement Interconnect_Tenant1_Import term DEFAULT then reject set policy-options policy-statement Interconnect_Tenant2_Export term Tenant_Routes from route-filter 10.179.0.0/16 orlonger set policy-options policy-statement Interconnect_Tenant2_Export term Tenant_Routes from route-filter 10.192.128.0/17 orlonger set policy-options policy-statement Interconnect_Tenant2_Export term Tenant_Routes then accept set policy-options policy-statement Interconnect_Tenant2_Export term DEFAULT then reject set policy-options policy-statement Interconnect_Tenant2_Import term Tenant_Routes from route-filter 10.179.0.0/16 orlonger set policy-options policy-statement Interconnect_Tenant2_Import term Tenant_Routes from route-filter 10.192.128.0/17 orlonger set policy-options policy-statement Interconnect_Tenant2_Import term Tenant_Routes then accept set policy-options policy-statement Interconnect_Tenant2_Import term DEFAULT then reject set policy-options policy-statement T5_EXPORT term 1 from protocol direct set policy-options policy-statement T5_EXPORT term 1 then accept set policy-options policy-statement T5_EXPORT term 2 from protocol bgp set policy-options policy-statement T5_EXPORT term 2 then accept set policy-options policy-statement UNDERLAY-EXPORT term LOOPBACK from route-filter 10.179.1.0/24 orlonger set policy-options policy-statement UNDERLAY-EXPORT term LOOPBACK then accept set policy-options policy-statement UNDERLAY-EXPORT term DEFAULT then reject set policy-options policy-statement UNDERLAY-IMPORT term LOOPBACK from route-filter 10.179.1.0/24 orlonger set policy-options policy-statement UNDERLAY-IMPORT term LOOPBACK then accept set policy-options policy-statement UNDERLAY-IMPORT term DEFAULT then reject set routing-instances Tennat1 routing-options multipath set routing-instances Tennat1 protocols evpn ip-prefix-routes advertise direct-nexthop set routing-instances Tennat1 protocols evpn ip-prefix-routes encapsulation vxlan set routing-instances Tennat1 protocols evpn ip-prefix-routes vni 1101 set routing-instances Tennat1 protocols evpn ip-prefix-routes export T5_EXPORT set routing-instances Tennat1 protocols bgp group INTERCONNECT type external set routing-instances Tennat1 protocols bgp group INTERCONNECT import Interconnect_Tenant1_Import set routing-instances Tennat1 protocols bgp group INTERCONNECT family inet unicast set routing-instances Tennat1 protocols bgp group INTERCONNECT authentication-key "$9$JNZi.Pfz6CuTzlMX-2gTz3n/tuO1" set routing-instances Tennat1 protocols bgp group INTERCONNECT export Interconnect_Tenant1_Export set routing-instances Tennat1 protocols bgp group INTERCONNECT local-as 65113 set routing-instances Tennat1 protocols bgp group INTERCONNECT multipath multiple-as set routing-instances Tennat1 protocols bgp group INTERCONNECT bfd-liveness-detection minimum-interval 1000 set routing-instances Tennat1 protocols bgp group INTERCONNECT bfd-liveness-detection multiplier 3 set routing-instances Tennat1 protocols bgp group INTERCONNECT neighbor 172.16.4.1 peer-as 65200 set routing-instances Tennat1 instance-type vrf set routing-instances Tennat1 interface irb.20 set routing-instances Tennat1 interface irb.30 set routing-instances Tennat1 interface irb.42 set routing-instances Tennat1 interface lo0.5 set routing-instances Tennat1 route-distinguisher 10.179.1.4:1101 set routing-instances Tennat1 vrf-target target:64701:20 set routing-instances Tennat1 vrf-table-label set routing-instances Tennat2 routing-options multipath set routing-instances Tennat2 protocols evpn ip-prefix-routes advertise direct-nexthop set routing-instances Tennat2 protocols evpn ip-prefix-routes encapsulation vxlan set routing-instances Tennat2 protocols evpn ip-prefix-routes vni 1102 set routing-instances Tennat2 protocols evpn ip-prefix-routes export T5_EXPORT set routing-instances Tennat2 protocols bgp group INTERCONNECT type external set routing-instances Tennat2 protocols bgp group INTERCONNECT import Interconnect_Tenant2_Import set routing-instances Tennat2 protocols bgp group INTERCONNECT family inet unicast set routing-instances Tennat2 protocols bgp group INTERCONNECT authentication-key "$9$JNZi.Pfz6CuTzlMX-2gTz3n/tuO1" set routing-instances Tennat2 protocols bgp group INTERCONNECT export Interconnect_Tenant2_Export set routing-instances Tennat2 protocols bgp group INTERCONNECT local-as 65113 set routing-instances Tennat2 protocols bgp group INTERCONNECT multipath multiple-as set routing-instances Tennat2 protocols bgp group INTERCONNECT bfd-liveness-detection minimum-interval 1000 set routing-instances Tennat2 protocols bgp group INTERCONNECT bfd-liveness-detection multiplier 3 set routing-instances Tennat2 protocols bgp group INTERCONNECT neighbor 172.16.5.1 peer-as 65200 set routing-instances Tennat2 instance-type vrf set routing-instances Tennat2 interface irb.40 set routing-instances Tennat2 interface irb.46 set routing-instances Tennat2 interface irb.50 set routing-instances Tennat2 interface lo0.6 set routing-instances Tennat2 route-distinguisher 10.179.1.6:1102 set routing-instances Tennat2 vrf-target target:64701:4050 set routing-instances Tennat2 vrf-table-label set routing-instances mgmt_junos routing-options static route 0.0.0.0/0 next-hop 10.193.0.1 set routing-options static route 0.0.0.0/0 next-hop 10.193.0.1 set routing-options static route 0.0.0.0/0 no-readvertise set routing-options forwarding-table export ECMP-POLICY set routing-options forwarding-table ecmp-fast-reroute set routing-options forwarding-table chained-composite-next-hop ingress evpn set routing-options router-id 10.179.1.2 set routing-options graceful-restart set protocols evpn encapsulation vxlan set protocols evpn default-gateway do-not-advertise set protocols evpn extended-vni-list 5020 set protocols evpn extended-vni-list 5030 set protocols evpn extended-vni-list 5040 set protocols evpn extended-vni-list 5042 set protocols evpn extended-vni-list 5046 set protocols evpn extended-vni-list 5050 set protocols evpn no-core-isolation set protocols bgp group RR-OVERLAY type internal set protocols bgp group RR-OVERLAY local-address 10.179.1.2 set protocols bgp group RR-OVERLAY family inet-vpn any set protocols bgp group RR-OVERLAY family evpn signaling set protocols bgp group RR-OVERLAY local-as 65100 set protocols bgp group RR-OVERLAY multipath set protocols bgp group RR-OVERLAY bfd-liveness-detection minimum-interval 1000 set protocols bgp group RR-OVERLAY bfd-liveness-detection multiplier 3 set protocols bgp group RR-OVERLAY neighbor 10.179.1.1 set protocols bgp group RR-OVERLAY vpn-apply-export set protocols bgp group UNDERLAY type external set protocols bgp group UNDERLAY import UNDERLAY-IMPORT set protocols bgp group UNDERLAY family inet unicast set protocols bgp group UNDERLAY authentication-key "$9$5TnCtpBESe0BVYoGq.0BIRhrevW" set protocols bgp group UNDERLAY export UNDERLAY-EXPORT set protocols bgp group UNDERLAY local-as 65013 set protocols bgp group UNDERLAY multipath multiple-as set protocols bgp group UNDERLAY neighbor 172.16.3.1 peer-as 65012 set protocols bgp graceful-restart restart-time 30 set protocols l2-learning global-mac-table-aging-time 600 set protocols l2-learning global-mac-ip-table-aging-time 300 set switch-options vtep-source-interface lo0.0 set switch-options route-distinguisher 10.179.1.2:64701 set switch-options vrf-target target:64701:9999 set switch-options vrf-target auto set vlans VLAN_42 vlan-id 42 set vlans VLAN_42 l3-interface irb.42 set vlans VLAN_42 vxlan vni 5042 set vlans VLAN_46 vlan-id 46 set vlans VLAN_46 l3-interface irb.46 set vlans VLAN_46 vxlan vni 5046 set vlans private1-a-dfw vlan-id 20 set vlans private1-a-dfw l3-interface irb.20 set vlans private1-a-dfw vxlan vni 5020 set vlans private1-b-dfw vlan-id 30 set vlans private1-b-dfw l3-interface irb.30 set vlans private1-b-dfw vxlan vni 5030 set vlans private1-c-dfw vlan-id 40 set vlans private1-c-dfw l3-interface irb.40 set vlans private1-c-dfw vxlan vni 5040 set vlans private1-d-dfw description Admin set vlans private1-d-dfw vlan-id 50 set vlans private1-d-dfw l3-interface irb.50 set vlans private1-d-dfw vxlan vni 5050
switch 1 configuration
set system host-name lswecc1-dfw set system login user homer uid 2002 set system login user homer class super-user set system login user homer authentication ssh-rsa "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC4xpjWJoQhCf176i77ni9//mcYO3bBWu7necWZwJNVkFsvvT6XuWfkKVMUFnTjTMr1erv8WRDze7le9Jl2a/xMIgo9Cf71SU9faPbd /ukvaLl5VUeGvHKFg9d+7GUGx1z9K1qKY2VOBO5EQCht8+4o4mMaizoXoxHvkNolswAa5Jv/EPwnfCeDyV7TsG+Se1k7/1h1VFOwW7Dbxno1aCnMDYbcfiBnzGnLSZQGjehok6cqYTjsNIIdAiZYSpH77pnAGglFhxNUSlqj0qRIJZdG3nhPlvIRPjn7fouq3BJEmiWPP8ru67H1J2mdSkix4xOxdUWfGB9eJlENfnobJjBr pp@U18" set system login user ppaul uid 2003 set system login user ppaul class super-user set system login user ppaul authentication ssh-rsa "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDST4EbXJc9l/AdrVmOZEPl3sxi6qjGIZyPwkupthSdooFHxPxUIh/a5PC9bMk5go6KvRoChpc4L8XuMRsxLTd6Ro6DsWIZieGHFuO/AL9SRUtmevGiSC2q4ibR7ACosJBUvKPRVK8anYnMSL9YWd7lnmVLnW5mvOM3Alhd8aTNKE3/H9ogDt9UfndEJXmieMTLJzGvx65sw6riqa5hh6iOcw02qb3QQCKLSRJmUJQuToY4oo/ZdLl/prEDKQ0I9DSnOxRYIvZxvUsTzwoXVq9X9dWGkKAAMDw7f2DJfa/4uCNT2dKPydApeN0ea2/69VRL3fmTz47y0CC1RTEd8j1j U18pc " set chassis aggregated-devices ethernet device-count 3 set interfaces xe-0/0/0 description Link_spine1-xe-0/0/0 set interfaces xe-0/0/0 gigether-options 802.3ad ae1 set interfaces xe-0/0/1 description link_spine2_xe-0/0/0 set interfaces xe-0/0/1 gigether-options 802.3ad ae1 set interfaces xe-0/0/8 description srv3 set interfaces xe-0/0/8 unit 0 family ethernet-switching vlan members private1-d-dfw set interfaces xe-0/0/9 description srv4:eth0 set interfaces xe-0/0/9 gigether-options 802.3ad ae0 set interfaces xe-0/0/10 description srv2 set interfaces xe-0/0/10 unit 0 family ethernet-switching vlan members private1-b-dfw set interfaces xe-0/0/11 description srv1 set interfaces xe-0/0/11 unit 0 family ethernet-switching vlan members private1-a-dfw set interfaces ae0 description lacp_srv4 set interfaces ae0 encapsulation ethernet-bridge set interfaces ae0 esi 00:01:01:01:01:01:01:01:01:01 set interfaces ae0 esi all-active set interfaces ae0 aggregated-ether-options lacp active set interfaces ae0 aggregated-ether-options lacp periodic fast set interfaces ae0 aggregated-ether-options lacp system-id 00:00:00:01:01:01 set interfaces ae0 unit 0 family ethernet-switching interface-mode access set interfaces ae0 unit 0 family ethernet-switching vlan members private1-c-dfw set interfaces ae1 description core:ae1 set interfaces ae1 mtu 9192 set interfaces ae1 aggregated-ether-options lacp active set interfaces ae1 aggregated-ether-options lacp periodic fast set interfaces ae1 unit 0 family ethernet-switching interface-mode trunk set interfaces ae1 unit 0 family ethernet-switching vlan members private1-a-dfw set interfaces ae1 unit 0 family ethernet-switching vlan members private1-b-dfw set interfaces ae1 unit 0 family ethernet-switching vlan members private1-c-dfw set interfaces ae1 unit 0 family ethernet-switching vlan members private1-d-dfw set interfaces em0 unit 0 family inet address 10.193.0.107/24 set routing-instances mgmt_junos routing-options static route 0.0.0.0/0 next-hop 10.193.0.1 set routing-options static route 0.0.0.0/0 next-hop 10.193.0.1 set routing-options static route 0.0.0.0/0 no-readvertise set vlans private1-a-dfw vlan-id 20 set vlans private1-b-dfw vlan-id 30 set vlans private1-c-dfw vlan-id 40 set vlans private1-d-dfw vlan-id 50
switch 2 configuration
set system host-name lswecc2-dfw set system login user homer uid 2002 set system login user homer class super-user set system login user homer authentication ssh-rsa "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC4xpjWJoQhCf176i77ni9//mcYO3bBWu7necWZwJNVkFsvvT6XuWfkKVMUFnTjTMr1erv8WRDze7le9Jl2a/xMIgo9Cf71SU9faPbd /ukvaLl5VUeGvHKFg9d+7GUGx1z9K1qKY2VOBO5EQCht8+4o4mMaizoXoxHvkNolswAa5Jv/EPwnfCeDyV7TsG+Se1k7/1h1VFOwW7Dbxno1aCnMDYbcfiBnzGnLSZQGjehok6cqYTjsNIIdAiZYSpH77pnAGglFhxNUSlqj0qRIJZdG3nhPlvIRPjn7fouq3BJEmiWPP8ru67H1J2mdSkix4xOxdUWfGB9eJlENfnobJjBr pp@U18" set system login user ppaul uid 2003 set system login user ppaul class super-user set system login user ppaul authentication ssh-rsa "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDST4EbXJc9l/AdrVmOZEPl3sxi6qjGIZyPwkupthSdooFHxPxUIh/a5PC9bMk5go6KvRoChpc4L8XuMRsxLTd6Ro6DsWIZieGHFuO/AL9SRUtmevGiSC2q4ibR7ACosJBUvKPRVK8anYnMSL9YWd7lnmVLnW5mvOM3Alhd8aTNKE3/H9ogDt9UfndEJXmieMTLJzGvx65sw6riqa5hh6iOcw02qb3QQCKLSRJmUJQuToY4oo/ZdLl/prEDKQ0I9DSnOxRYIvZxvUsTzwoXVq9X9dWGkKAAMDw7f2DJfa/4uCNT2dKPydApeN0ea2/69VRL3fmTz47y0CC1RTEd8j1j U18pc " set chassis aggregated-devices ethernet device-count 3 set interfaces xe-0/0/0 description link_spine1_xe-0/0/1 set interfaces xe-0/0/0 gigether-options 802.3ad ae1 set interfaces xe-0/0/1 description link_spine2_xe-0/0/1 set interfaces xe-0/0/1 gigether-options 802.3ad ae1 set interfaces xe-0/0/7 description srv8 set interfaces xe-0/0/7 unit 0 family ethernet-switching vlan members private1-d-dfw set interfaces xe-0/0/8 description srv7 set interfaces xe-0/0/8 unit 0 family ethernet-switching vlan members private1-c-dfw set interfaces xe-0/0/9 description srv4:eth1 set interfaces xe-0/0/9 gigether-options 802.3ad ae0 set interfaces xe-0/0/10 description srv6 set interfaces xe-0/0/10 unit 0 family ethernet-switching vlan members private1-b-dfw set interfaces xe-0/0/11 description srv5 set interfaces xe-0/0/11 unit 0 family ethernet-switching vlan members private1-a-dfw set interfaces ae0 description lacp_srv4 set interfaces ae0 encapsulation ethernet-bridge set interfaces ae0 esi 00:01:01:01:01:01:01:01:01:01 set interfaces ae0 esi all-active set interfaces ae0 aggregated-ether-options lacp active set interfaces ae0 aggregated-ether-options lacp periodic fast set interfaces ae0 aggregated-ether-options lacp system-id 00:00:00:01:01:01 set interfaces ae0 unit 0 family ethernet-switching interface-mode access set interfaces ae0 unit 0 family ethernet-switching vlan members private1-c-dfw set interfaces ae1 description core:ae2 set interfaces ae1 mtu 9192 set interfaces ae1 aggregated-ether-options lacp active set interfaces ae1 aggregated-ether-options lacp periodic fast set interfaces ae1 unit 0 family ethernet-switching interface-mode trunk set interfaces ae1 unit 0 family ethernet-switching vlan members private1-a-dfw set interfaces ae1 unit 0 family ethernet-switching vlan members private1-b-dfw set interfaces ae1 unit 0 family ethernet-switching vlan members private1-c-dfw set interfaces ae1 unit 0 family ethernet-switching vlan members private1-d-dfw set interfaces em0 unit 0 family inet address 10.193.0.108/24 set routing-instances mgmt_junos routing-options static route 0.0.0.0/0 next-hop 10.193.0.1 set routing-options static route 0.0.0.0/0 next-hop 10.193.0.1 set routing-options static route 0.0.0.0/0 no-readvertise set vlans private1-a-dfw vlan-id 20 set vlans private1-b-dfw vlan-id 30 set vlans private1-c-dfw vlan-id 40 set vlans private1-d-dfw vlan-id 50
Before we jump into the vSRX configuration that will allow Tenant1 to communicate with Tenant2 let us first make sure that our EVPN/VXLAN works.So private1-a-dfw and private1-b-dfw VLAN's belong to Tenant1 and the other two VLAN's to Tenant2. After all verifications and during the testing step, all servers in routing instance Tenant1 should be able to talk to each other same for servers in routing instance Tenant2. let us start the veritfications.
Verification and Testing
Underlay Verification
- Verify BGP session is established between spine1 and spine2
ppaul@sswecc1-dfw> show bgp neighbor 172.16.3.2 Peer: 172.16.3.2+179 AS 65013 Local: 172.16.3.1+64627 AS 65012 Group: UNDERLAY Routing-Instance: master Forwarding routing-instance: master Type: External State: Established Flags: <Sync> Last State: OpenConfirm Last Event: RecvKeepAlive Last Error: None Export: [ UNDERLAY-EXPORT ] Import: [ UNDERLAY-IMPORT ] ...
- Verify loopback address of spine2 is received by spine1
ppaul@sswecc1-dfw> show route receive-protocol bgp 172.16.3.2 inet.0: 9 destinations, 9 routes (9 active, 0 holddown, 0 hidden) Restart Complete Prefix Nexthop MED Lclpref AS path * 10.179.1.2/32 172.16.3.2 65013 I ppaul@sswecc1-dfw> show route 10.179.1.2 inet.0: 9 destinations, 9 routes (9 active, 0 holddown, 0 hidden) Restart Complete + = Active Route, - = Last Active, * = Both
10.179.1.2/32 *[BGP/170] 2w0d 23:00:15, localpref 100
AS path: 65013 I, validation-state: unverified
> to 172.16.3.2 via ae0.0
- Ping spine2 loopback address
ppaul@sswecc1-dfw> ping 10.179.1.2 PING 10.179.1.2 (10.179.1.2): 56 data bytes 64 bytes from 10.179.1.2: icmp_seq=0 ttl=64 time=230.449 ms 64 bytes from 10.179.1.2: icmp_seq=1 ttl=64 time=107.568 ms 64 bytes from 10.179.1.2: icmp_seq=2 ttl=64 time=107.435 ms
Overlay Verification
- Verify IBGP sessions is established between spine1 and spine2
ppaul@sswecc1-dfw> show bgp neighbor 10.179.1.2 Peer: 10.179.1.2+49585 AS 65100 Local: 10.179.1.1+179 AS 65100 Group: RR-OVERLAY Routing-Instance: master Forwarding routing-instance: master Type: Internal State: Established Flags: <Sync> Last State: OpenConfirm Last Event: RecvKeepAlive Last Error: None Options: <Preference LocalAddress GracefulRestart AddressFamily Multipath LocalAS Rib-group Refresh> Options: <VpnApplyExport BfdEnabled> Options: <GracefulShutdownRcv> Address families configured: evpn
- Verify the source VTEP for the EVPN domain
ppaul@sswecc1-dfw> show ethernet-switching vxlan-tunnel-end-point source
Logical System Name Id SVTEP-IP IFL L3-Idx SVTEP-Mode
<default> 0 10.179.1.1 lo0.0 0
L2-RTT Bridge Domain VNID MC-Group-IP
default-switch private1-a-dfw+20 5020 0.0.0.0
default-switch private1-b-dfw+30 5030 0.0.0.0
default-switch private1-c-dfw+40 5040 0.0.0.0
default-switch private1-d-dfw+50 5050 0.0.0.0
- Verify remote and source VTEPs
ppaul@sswecc1-dfw> show interfaces vtep
Physical interface: vtep, Enabled, Physical link is Up
Interface index: 646, SNMP ifIndex: 509
Type: Software-Pseudo, Link-level type: VxLAN-Tunnel-Endpoint, MTU: Unlimited, Speed: Unlimited
Device flags : Present Running
Link type : Full-Duplex
Link flags : None
Last flapped : Never
Input packets : 0
Output packets: 0
Logical interface vtep.32768 (Index 569) (SNMP ifIndex 569)
Flags: Up SNMP-Traps 0x4000 Encapsulation: ENET2
Ethernet segment value: 00:00:00:00:00:00:00:00:00:00, Mode: single-homed, Multi-homed status: Forwarding
VXLAN Endpoint Type: Source, VXLAN Endpoint Address: 10.179.1.1, L2 Routing Instance: default-switch, L3 Routing Instance: default
Input packets : 0
Output packets: 0
Logical interface vtep.32769 (Index 586) (SNMP ifIndex 568)
Flags: Up SNMP-Traps Encapsulation: ENET2
VXLAN Endpoint Type: Remote, VXLAN Endpoint Address: 10.179.1.2, L2 Routing Instance: default-switch, L3 Routing Instance: default
Input packets : 189684
Output packets: 1650077
Protocol eth-switch, MTU: Unlimited
Flags: Trunk-Mode
Verify EVPN Mutihoming
- Verify ae1 is up
ppaul@sswecc1-dfw> show interfaces ae1 Physical interface: ae1, Enabled, Physical link is Up Interface index: 641, SNMP ifIndex: 558 Description: leaf1_ae1 Link-level type: Ethernet, MTU: 9192, Speed: 10Gbps, BPDU Error: None, Ethernet-Switching Error: None, MAC-REWRITE Error: None, Loopback: Disabled, Source filtering: Disabled, Flow control: Disabled, Minimum links needed: 1, Minimum bandwidth needed: 1bps Device flags : Present Running Interface flags: SNMP-Traps Internal: 0x4000 Current address: 02:05:86:72:13:f0, Hardware address: 02:05:86:72:13:f0 Ethernet segment value: 00:00:00:ab:cd:00:01:00:00:03, Mode: all-active Last flapped : 2023-10-19 17:15:12 UTC (10:43:35 ago) Input rate : 0 bps (0 pps) Output rate : 984 bps (0 pps)
ppaul@sswecc1-dfw> show lacp interfaces ae1
Aggregated interface: ae1
LACP state: Role Exp Def Dist Col Syn Aggr Timeout Activity
xe-0/0/0 Actor No No Yes Yes Yes Yes Fast Active
xe-0/0/0 Partner No No Yes Yes Yes Yes Fast Active
LACP protocol: Receive State Transmit State Mux State
xe-0/0/0 Current Fast periodic Collecting distributing
ppaul@sswecc1-dfw> show evpn instance extensive
Instance: __default_evpn__
Route Distinguisher: 10.179.1.1:0
Number of bridge domains: 0
Number of neighbors: 1
Address MAC MAC+IP AD IM ES Leaf-label
10.179.1.2 0 0 0 0 3
Instance: default-switch
Route Distinguisher: 10.179.1.1:64701
Encapsulation type: VXLAN
Duplicate MAC detection threshold: 5
Duplicate MAC detection window: 180
MAC database status Local Remote
MAC advertisements: 13 10
MAC+IP advertisements: 15 8
Default gateway MAC advertisements: 10 0
Number of local interfaces: 5 (4 up)
Interface name ESI Mode Status AC-Role
.local..9 00:00:00:00:00:00:00:00:00:00 single-homed Up Root
ae1.0 00:00:00:ab:cd:00:01:00:00:03 all-active Up Root
ae2.0 00:00:00:ab:cd:00:01:00:00:04 all-active Up Root
...
Number of neighbors: 1
Address MAC MAC+IP AD IM ES Leaf-label
10.179.1.2 10 8 10 6 0
Number of ethernet segments: 10
ESI: 00:00:00:ab:cd:00:01:00:00:03
Status: Resolved by IFL ae1.0
Local interface: ae1.0, Status: Up/Forwarding
Number of remote PEs connected: 1
Remote PE MAC label Aliasing label Mode
10.179.1.2 5040 0 all-active
DF Election Algorithm: MOD based
Designated forwarder: 10.179.1.1
Backup forwarder: 10.179.1.2
Last designated forwarder update: Oct 19 17:15:12
ESI: 00:00:00:ab:cd:00:01:00:00:04
Status: Resolved by IFL ae2.0
Local interface: ae2.0, Status: Up/Forwarding
Number of remote PEs connected: 1
Remote PE MAC label Aliasing label Mode
10.179.1.2 5030 0 all-active
DF Election Algorithm: MOD based
Designated forwarder: 10.179.1.1
Backup forwarder: 10.179.1.2
Last designated forwarder update: Oct 17 17:58:12
- Verify all the member of ae1 are collecting and distributing on sw1
show evpn instance extensive
Aggregated interface: ae1
LACP state: Role Exp Def Dist Col Syn Aggr Timeout Activity
xe-0/0/0 Actor No No Yes Yes Yes Yes Fast Active
xe-0/0/0 Partner No No Yes Yes Yes Yes Fast Active
xe-0/0/1 Actor No No Yes Yes Yes Yes Fast Active
xe-0/0/1 Partner No No Yes Yes Yes Yes Fast Active
LACP protocol: Receive State Transmit State Mux State
xe-0/0/0 Current Fast periodic Collecting distributing
xe-0/0/1 Current Fast periodic Collecting distributing
Server Multihoming
srv4 has two(2) connections, one to sw1 and the other on sw2 same port 9 on both switches. on the server we setup a bond interface using eth0 and eth1 and the bond interface MAC address is:00:50:00:00:0b:00
Verify that the same mac address is on both sw1 and sw2
- sw1
root@lswecc1-dfw> show ethernet-switching table
MAC flags (S - static MAC, D - dynamic MAC, L - locally learned, P - Persistent static, C - Control MAC
SE - statistics enabled, NM - non configured MAC, R - remote PE MAC, O - ovsdb MAC)
Ethe rnet switching table : 9 entries, 9 learned
Routing instance : default-switch
Vlan MAC MAC Age Logical NH RTR
name address flags interface Index ID
private1-a-dfw 00:50:00:00:09:00 D - xe-0/0/11.0 0 0
private1-a-dfw 02:05:86:71:18:00 D - ae1.0 0 0
private1-a-dfw 02:05:86:71:35:00 D - ae1.0 0 0
private1-b-dfw 00:50:00:00:10:00 D - ae1.0 0 0
private1-b-dfw 02:05:86:71:35:00 D - ae1.0 0 0
private1-c-dfw 00:50:00:00:0b:00 D - ae0.0 0 0
private1-c-dfw 02:05:86:71:35:00 D - ae1.0 0 0
private1-d-dfw 00:50:00:00:0d:00 D - ae1.0 0 0
private1-d-dfw 02:05:86:71:35:00 D - ae1.0 0 0
- sw2
root@lswecc2-dfw> show ethernet-switching table
MAC flags (S - static MAC, D - dynamic MAC, L - locally learned, P - Persistent static, C - Control MAC
SE - statistics enabled, NM - non configured MAC, R - remote PE MAC, O - ovsdb MAC)
Ethernet switching table : 10 entries, 10 learned
Routing instance : default-switch
Vlan MAC MAC Age Logical NH RTR
name address flags interface Index ID
private1-a-dfw 00:50:00:00:09:00 D - ae1.0 0 0
private1-b-dfw 00:50:00:00:10:00 D - xe-0/0/10.0 0 0
private1-b-dfw 02:05:86:71:18:00 D - ae1.0 0 0
private1-b-dfw 02:05:86:71:35:00 D - ae1.0 0 0
private1-c-dfw 00:50:00:00:0b:00 D - ae0.0 0 0
private1-c-dfw 02:05:86:71:18:00 D - ae1.0 0 0
private1-c-dfw 02:05:86:71:35:00 D - ae1.0 0 0
private1-d-dfw 00:50:00:00:0d:00 D - xe-0/0/7.0 0 0
private1-d-dfw 02:05:86:71:18:00 D - ae1.0 0 0
private1-d-dfw 02:05:86:71:35:00 D - ae1.0 0 0
Verify that the same mac address is on both spines as well
{master:0}
root@sswecc1-dfw> show ethernet-switching table
MAC flags (S - static MAC, D - dynamic MAC, L - locally learned, P - Persistent static
SE - statistics enabled, NM - non configured MAC, R - remote PE MAC, O - ovsdb MAC)
Ethernet switching table : 14 entries, 14 learned
Routing instance : default-switch
Vlan MAC MAC Logical Active
name address flags interface source
VLAN_42 00:10:db:ff:10:01 DL ae6.0
VLAN_42 4c:96:14:76:1a:04 DL ae6.0
VLAN_42 4c:96:14:76:1a:05 DR ae6.0
VLAN_46 00:10:db:ff:10:01 DL ae6.0
VLAN_46 4c:96:14:76:1a:04 DL ae6.0
VLAN_46 4c:96:14:76:1a:05 DR ae6.0
private1-a-dfw 00:00:5e:00:01:01 DR esi.1856 05:00:00:00:00:00:00:13:9c:00
private1-a-dfw 00:50:00:00:09:00 DLR ae1.0
private1-b-dfw 00:00:5e:00:01:01 DR esi.1855 05:00:00:00:00:00:00:13:a6:00
private1-b-dfw 00:50:00:00:10:00 DLR ae2.0
private1-c-dfw 00:00:5e:00:01:01 DR esi.1854 05:00:00:00:00:00:00:13:b0:00
private1-c-dfw 00:50:00:00:0b:00 DL ae2.0
private1-d-dfw 00:00:5e:00:01:01 DR esi.1853 05:00:00:00:00:00:00:13:ba:00
private1-d-dfw 00:50:00:00:0d:00 DLR ae2.0
root@sswecc2-dfw> show ethernet-switching table
MAC flags (S - static MAC, D - dynamic MAC, L - locally learned, P - Persistent static
SE - statistics enabled, NM - non configured MAC, R - remote PE MAC, O - ovsdb MAC)
Ethernet switching table : 14 entries, 14 learned
Routing instance : default-switch
Vlan MAC MAC Logical Active
name address flags interface source
VLAN_42 00:10:db:ff:10:01 DR ae6.0
VLAN_42 4c:96:14:76:1a:04 DR ae6.0
VLAN_42 4c:96:14:76:1a:05 DL ae6.0
VLAN_46 00:10:db:ff:10:01 DR ae6.0
VLAN_46 4c:96:14:76:1a:04 DR ae6.0
VLAN_46 4c:96:14:76:1a:05 DL ae6.0
private1-a-dfw 00:00:5e:00:01:01 DR esi.1854 05:00:00:00:00:00:00:13:9c:00
private1-a-dfw 00:50:00:00:09:00 DLR ae1.0
private1-b-dfw 00:00:5e:00:01:01 DR esi.1853 05:00:00:00:00:00:00:13:a6:00
private1-b-dfw 00:50:00:00:10:00 DLR ae2.0
private1-c-dfw 00:00:5e:00:01:01 DR esi.1852 05:00:00:00:00:00:00:13:b0:00
private1-c-dfw 00:50:00:00:0b:00 DR ae2.0
private1-d-dfw 00:00:5e:00:01:01 DR esi.1851 05:00:00:00:00:00:00:13:ba:00
private1-d-dfw 00:50:00:00:0d:00 DLR ae2.0
Verify that the same MAC address is in the EVPN database on both spines
root@sswecc1-dfw> show evpn database mac-address 00:50:00:00:0b:00 extensive
Instance: default-switch
VN Identifier: 5040, MAC address: 00:50:00:00:0b:00
State: 0x0
Source: 00:00:00:ab:cd:00:01:00:00:03, Rank: 1, Status: Active
Local origin: ae1.0
Mobility sequence number: 0 (minimum origin address 10.179.1.1)
Timestamp: Oct 24 17:15:16 (0x6537fba4)
State: <Local-MAC-Only Local-To-Remote-Adv-Allowed>
MAC advertisement route status: Created
History db:
Time Event
Oct 24 17:15:16 2023 00:00:00:ab:cd:00:01:00:00:03 : Created
Oct 24 17:15:16 2023 Updating output state (change flags 0x20 <ESI-Added>)
Oct 24 17:15:16 2023 Active ESI changing (not assigned -> 00:00:00:ab:cd:00:01:00:00:03)
papaul@sswecc2-dfw> show evpn database mac-address 00:50:00:00:0b:00 extensive
Instance: default-switch
VN Identifier: 5040, MAC address: 00:50:00:00:0b:00
State: 0x0
Source: 00:00:00:ab:cd:00:01:00:00:03, Rank: 1, Status: Active
Remote origin: 10.179.1.1
Mobility sequence number: 0 (minimum origin address 10.179.1.1)
Timestamp: Oct 24 17:15:16 (0x6537fba4)
State: <Remote-To-Local-Adv-Done>
MAC advertisement route status: Not created (no local state present)
History db:
Time Event
Oct 24 16:38:36 2023 00:00:00:ab:cd:00:01:00:00:04 : Remote peer 10.179.1.1 created
Oct 24 16:38:36 2023 00:00:00:ab:cd:00:01:00:00:04 : Created
Oct 24 16:38:36 2023 Updating output state (change flags 0x60 <ESI-Added ESI-Deleted>)
Oct 24 16:38:36 2023 Active ESI changing (not assigned -> 00:00:00:ab:cd:00:01:00:00:04)
Oct 24 17:15:16 2023 00:00:00:ab:cd:00:01:00:00:04 : Remote peer 10.179.1.1 deleted
Oct 24 17:15:16 2023 00:00:00:ab:cd:00:01:00:00:04 : Deleting
Oct 24 17:15:16 2023 00:00:00:ab:cd:00:01:00:00:03 : Remote peer 10.179.1.1 created
Oct 24 17:15:16 2023 00:00:00:ab:cd:00:01:00:00:03 : Created
Oct 24 17:15:16 2023 Updating output state (change flags 0x60 <ESI-Added ESI-Deleted>)
Oct 24 17:15:16 2023 Active ESI changing (not assigned -> 00:00:00:ab:cd:00:01:00:00:03)
ppaul@sswecc1-dfw> show ethernet-switching table vlan-id 40
MAC flags (S - static MAC, D - dynamic MAC, L - locally learned, P - Persistent static
SE - statistics enabled, NM - non configured MAC, R - remote PE MAC, O - ovsdb MAC)
Ethernet switching table : 2 entries, 2 learned
Routing instance : default-switch
Vlan MAC MAC Logical Active
name address flags interface source
private1-c-dfw 00:00:5e:00:01:01 DR esi.1854 05:00:00:00:00:00:00:13:b0:00
private1-c-dfw 00:50:00:00:0b:00 DL ae2.0
ppaul@sswecc2-dfw> show ethernet-switching table vlan-id 40
MAC flags (S - static MAC, D - dynamic MAC, L - locally learned, P - Persistent static
SE - statistics enabled, NM - non configured MAC, R - remote PE MAC, O - ovsdb MAC)
Ethernet switching table : 2 entries, 2 learned
Routing instance : default-switch
Vlan MAC MAC Logical Active
name address flags interface source
private1-c-dfw 00:00:5e:00:01:01 DR esi.1852 05:00:00:00:00:00:00:13:b0:00
private1-c-dfw 00:50:00:00:0b:00 DR ae2.0
We can see that on spine1 the MAC address was learned locally and spine2 learned the same MAC address from spine1
Verify that the MAC address are advertised as BGP routes
ppaul@sswecc1-dfw> show route table bgp.evpn.0 evpn-mac-address 00:50:00:00:0b:00
bgp.evpn.0: 95 destinations, 95 routes (95 active, 0 holddown, 0 hidden)
Restart Complete
+ = Active Route, - = Last Active, * = Both
2:10.179.1.1:64701::5040::00:50:00:00:0b:00/304 MAC/IP
*[EVPN/170] 00:54:18
Indirect
ppaul@sswecc2-dfw> show route table bgp.evpn.0 evpn-mac-address 00:50:00:00:0b:00
bgp.evpn.0: 95 destinations, 95 routes (95 active, 0 holddown, 0 hidden)
Restart Complete
+ = Active Route, - = Last Active, * = Both
2:10.179.1.1:64701::5040::00:50:00:00:0b:00/304 MAC/IP
*[BGP/170] 00:54:51, localpref 100, from 10.179.1.1
AS path: I, validation-state: unverified
> to 172.16.3.1 via ae0.0
On spine2 let us check that the MAC address is reachable from spine1
ppaul@sswecc2-dfw> show route receive-protocol bgp 10.179.1.1 inet.0: 9 destinations, 9 routes (9 active, 0 holddown, 0 hidden) Restart Complete Tennat1.inet.0: 15 destinations, 18 routes (15 active, 0 holddown, 0 hidden) Tennat2.inet.0: 15 destinations, 19 routes (15 active, 0 holddown, 0 hidden) :vxlan.inet.0: 8 destinations, 8 routes (8 active, 0 holddown, 0 hidden) Restart Complete mpls.0: 2 destinations, 2 routes (2 active, 0 holddown, 0 hidden) Restart Complete inet6.0: 2 destinations, 2 routes (2 active, 0 holddown, 0 hidden) Restart Complete Tennat1.inet6.0: 1 destinations, 1 routes (1 active, 0 holddown, 0 hidden) Tennat2.inet6.0: 1 destinations, 1 routes (1 active, 0 holddown, 0 hidden) bgp.evpn.0: 95 destinations, 95 routes (95 active, 0 holddown, 0 hidden) Restart Complete Prefix Nexthop MED Lclpref AS pat ... 2:10.179.1.1:64701::5040::00:50:00:00:0b:00/304 MAC/IP * 10.179.1.1 100 I
ppaul@sswecc1-dfw> show route forwarding-table destination 00:50:00:00:0b:00 Routing table: default-switch.evpn-vxlan Bridging domain: private1-c-dfw.evpn-vxlan VPLS: Enabled protocols: Bridging, ACKed by all peers, EVPN VXLAN, Destination Type RtRef Next hop Type Index NhRef Netif 00:50:00:00:0b:00/48 user 0 ucst 1805 11 ae2.0
The output above shows that ae2.0 is used to switch traffic for the MAC address 00:50:00:00:0b:00
vSRX configuration
Interface maping EVE-NG/vSRX
On EVE-NG ge-0/0/1 is ge-0/0/0 in the vSRX and ge-0/0/2 is ge-0/0/1 and so on. If you connect on EVE-NG for example ge-0/0/5 when you login to the vSRX the interface that needs to be setup will be ge-0/0/4.
Configure the vSRX's to form a cluster
## on first vsrx set chassis cluster cluster-id 1 node 0 reboot
## On on the second vsrx set chassis cluster cluster-id 1 node 1 reboot
After reboot, login to the first node run the commands below:
set groups node0 system host-name mr1-dfw
set groups node0 interfaces fxp0 unit 0 family inet address 10.193.0.90/24
set groups node1 system host-name mr2-dfw
set groups node1 interfaces fxp0 unit 0 family inet address 10.193.0.91/24
set apply-groups "${node}"
set system host-name mr-dfw
set system management-instance
set routing-instances mgmt_junos routing-options static route 0.0.0.0/0 next-hop 10.193.0.1
set interfaces fab0 fabric-options member-interfaces ge-0/0/0
set interfaces fab1 fabric-options member-interfaces ge-7/0/0
vSRX config
set system login user ppaul uid 2003 set system login user ppaul class super-user set system login user ppaul authentication ssh-rsa "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDST4EbXJc9l/AdrVmOZEPl3sxi6qjGIZyPwkupthSdooFHxPxUIh/a5PC9bMk5go6KvRoChpc4L8XuMRsxLTd6Ro6DsWIZieGHFuO/AL9SRUtmevGiSC2q4ibR7ACosJBUvKPRVK8anYnMSL9YWd7lnmVLnW5mvOM3Alhd8aTNKE3/H9ogDt9UfndEJXmieMTLJzGvx65sw6riqa5hh6iOcw02qb3QQCKLSRJmUJQuToY4oo/ZdLl/prEDKQ0I9DSnOxRYIvZxvUsTzwoXVq9X9dWGkKAAMDw7f2DJfa/4uCNT2dKPydApeN0ea2/69VRL3fmTz47y0CC1RTEd8j1j U18pc " set chassis cluster control-link-recovery set chassis cluster reth-count 5 set chassis cluster redundancy-group 0 node 0 priority 100 set chassis cluster redundancy-group 0 node 1 priority 1 set chassis cluster redundancy-group 1 node 0 priority 200 set chassis cluster redundancy-group 1 node 1 priority 100 set chassis cluster redundancy-group 1 preempt set chassis cluster redundancy-group 1 interface-monitor ge-0/0/4 weight 255 set chassis cluster redundancy-group 1 interface-monitor ge-0/0/5 weight 255 set chassis cluster redundancy-group 1 interface-monitor ge-7/0/4 weight 255 set chassis cluster redundancy-group 1 interface-monitor ge-7/0/5 weight 255 set chassis cluster redundancy-group 2 node 0 priority 200 set chassis cluster redundancy-group 2 node 1 priority 100 set chassis cluster redundancy-group 2 preempt set security policies from-zone Tennant1 to-zone Tennant2 policy Allow_All match source-address any set security policies from-zone Tennant1 to-zone Tennant2 policy Allow_All match destination-address any set security policies from-zone Tennant1 to-zone Tennant2 policy Allow_All match application any set security policies from-zone Tennant1 to-zone Tennant2 policy Allow_All then permit set security policies from-zone Tennant2 to-zone Tennant1 policy Allow_All match source-address any set security policies from-zone Tennant2 to-zone Tennant1 policy Allow_All match destination-address any set security policies from-zone Tennant2 to-zone Tennant1 policy Allow_All match application any set security policies from-zone Tennant2 to-zone Tennant1 policy Allow_All then permit set security zones security-zone Tennant1 host-inbound-traffic system-services ping set security zones security-zone Tennant1 host-inbound-traffic protocols bgp set security zones security-zone Tennant1 host-inbound-traffic protocols bfd set security zones security-zone Tennant1 interfaces reth1.42 set security zones security-zone Tennant2 host-inbound-traffic system-services ping set security zones security-zone Tennant2 host-inbound-traffic protocols bgp set security zones security-zone Tennant2 host-inbound-traffic protocols bfd set security zones security-zone Tennant2 interfaces reth1.46 set interfaces ge-0/0/4 description "To cc1 | xe-0/0/9" set interfaces ge-0/0/4 gigether-options no-auto-negotiation set interfaces ge-0/0/4 gigether-options redundant-parent reth1 set interfaces ge-0/0/5 description "To cc2 | xe-0/0/9" set interfaces ge-0/0/5 gigether-options no-auto-negotiation set interfaces ge-0/0/5 gigether-options redundant-parent reth1 set interfaces ge-7/0/4 description "To cc1 | xe-0/0/8" set interfaces ge-7/0/4 gigether-options no-auto-negotiation set interfaces ge-7/0/4 gigether-options redundant-parent reth1 set interfaces ge-7/0/5 description "To cc2 | xe-0/0/8" set interfaces ge-7/0/5 gigether-options no-auto-negotiation set interfaces ge-7/0/5 gigether-options redundant-parent reth1 set interfaces lo0 unit 0 family inet address 10.179.1.7/32 set interfaces reth1 vlan-tagging set interfaces reth1 redundant-ether-options redundancy-group 1 set interfaces reth1 redundant-ether-options lacp active set interfaces reth1 redundant-ether-options lacp periodic fast set interfaces reth1 unit 42 description "Spine Interconect for Tennant1" set interfaces reth1 unit 42 vlan-id 42 set interfaces reth1 unit 42 family inet address 172.16.4.1/29 set interfaces reth1 unit 46 description "Spine Interconect for Tennant2" set interfaces reth1 unit 46 vlan-id 46 set interfaces reth1 unit 46 family inet address 172.16.5.1/29 set policy-options policy-statement INTERCONNECT-EXPORT term Tenant_Aggregate from protocol static set policy-options policy-statement INTERCONNECT-EXPORT term Tenant_Aggregate from route-filter 10.192.128.0/17 exact set policy-options policy-statement INTERCONNECT-EXPORT term Tenant_Aggregate then accept set policy-options policy-statement INTERCONNECT-EXPORT term loopback from protocol direct set policy-options policy-statement INTERCONNECT-EXPORT term loopback from route-filter 10.179.1.7/32 exact set policy-options policy-statement INTERCONNECT-EXPORT term loopback then accept set policy-options policy-statement INTERCONNECT-EXPORT term Reject_All then reject set policy-options policy-statement INTERCONNECT-IMPORT term Tenant_Routes from route-filter 10.192.128.0/17 longer set policy-options policy-statement INTERCONNECT-IMPORT term Tenant_Routes from route-filter 10.179.0.0/16 longer set policy-options policy-statement INTERCONNECT-IMPORT term Tenant_Routes then accept set policy-options policy-statement INTERCONNECT-IMPORT term DEFAULT then reject set protocols bgp group INTERCONNECT type external set protocols bgp group INTERCONNECT traceoptions file bgp_trace set protocols bgp group INTERCONNECT traceoptions flag state set protocols bgp group INTERCONNECT import INTERCONNECT-IMPORT set protocols bgp group INTERCONNECT family inet unicast set protocols bgp group INTERCONNECT authentication-key "$9$rXzlWxN-wgaUVwQnCuEhVwY24ZUjH" set protocols bgp group INTERCONNECT export INTERCONNECT-EXPORT set protocols bgp group INTERCONNECT local-as 65200 set protocols bgp group INTERCONNECT multipath multiple-as set protocols bgp group INTERCONNECT bfd-liveness-detection minimum-interval 1000 set protocols bgp group INTERCONNECT bfd-liveness-detection multiplier 3 set protocols bgp group INTERCONNECT neighbor 172.16.4.3 peer-as 65113 set protocols bgp group INTERCONNECT neighbor 172.16.4.2 peer-as 65112 set protocols bgp group INTERCONNECT neighbor 172.16.5.2 peer-as 65112 set protocols bgp group INTERCONNECT neighbor 172.16.5.3 peer-as 65113 set routing-options static route 10.192.128.0/17 discard set routing-options static route 172.16.0.0/16 discard
Verification/Testing
Verify BGP on the Vsrx cluster
root@mr-dfw> show bgp summary Threading mode: BGP I/O Default eBGP mode: advertise - accept, receive - accept Groups: 1 Peers: 4 Down peers: 0 Table Tot Paths Act Paths Suppressed History Damp State Pending inet.0 27 23 0 0 0 0 Peer AS InPkt OutPkt OutQ Flaps Last Up/Dwn State|#Active/Received/Accepted/Damped... 172.16.4.2 65112 20 17 0 0 6:38 Establ inet.0: 6/7/7/0 172.16.4.3 65113 16 11 0 0 4:10 Establ inet.0: 6/7/7/0 172.16.5.2 65112 21 17 0 0 6:38 Establ inet.0: 5/6/6/0 172.16.5.3 65113 16 11 0 0 4:00 Establ inet.0: 6/7/7/0
Verify that the VSRX received routes from Tenant1 coming from both spines
{primary:node0}
root@mr-dfw> show route receive-protocol bgp 172.16.4.2
inet.0: 21 destinations, 34 routes (21 active, 0 holddown, 0 hidden)
Prefix Nexthop MED Lclpref AS path
* 10.179.1.3/32 172.16.4.2 65112 I
10.179.1.4/32 172.16.4.2 65112 65100 I
* 10.192.144.0/22 172.16.4.2 65112 I
* 10.192.144.100/32 172.16.4.2 65112 I
* 10.192.160.0/22 172.16.4.2 65112 I
* 10.192.160.100/32 172.16.4.2 65112 I
* 10.192.160.200/32 172.16.4.2 65112 I
mgmt_junos.inet.0: 3 destinations, 3 routes (3 active, 0 holddown, 0 hidden)
inet6.0: 1 destinations, 1 routes (1 active, 0 holddown, 0 hidden)
{primary:node0}
root@mr-dfw> show route receive-protocol bgp 172.16.4.3
inet.0: 21 destinations, 34 routes (21 active, 0 holddown, 0 hidden)
Prefix Nexthop MED Lclpref AS path
10.179.1.3/32 172.16.4.3 65113 65100 I
* 10.179.1.4/32 172.16.4.3 65113 I
10.192.144.0/22 172.16.4.3 65113 I
10.192.144.100/32 172.16.4.3 65113 I
10.192.160.0/22 172.16.4.3 65113 I
10.192.160.100/32 172.16.4.3 65113 I
10.192.160.200/32 172.16.4.3 65113 I
mgmt_junos.inet.0: 3 destinations, 3 routes (3 active, 0 holddown, 0 hidden)
inet6.0: 1 destinations, 1 routes (1 active, 0 holddown, 0 hidden)
Verify that the VSRX received routes from Tenant2 coming from both spines
{primary:node0}
root@mr-dfw> show route receive-protocol bgp 172.16.5.2
inet.0: 21 destinations, 34 routes (21 active, 0 holddown, 0 hidden)
Prefix Nexthop MED Lclpref AS path
* 10.179.1.5/32 172.16.5.2 65112 I
10.179.1.6/32 172.16.5.2 65112 65100 I
* 10.192.176.0/22 172.16.5.2 65112 I
* 10.192.192.0/22 172.16.5.2 65112 I
* 10.192.192.100/32 172.16.5.2 65112 I
* 10.192.192.200/32 172.16.5.2 65112 I
mgmt_junos.inet.0: 3 destinations, 3 routes (3 active, 0 holddown, 0 hidden)
inet6.0: 1 destinations, 1 routes (1 active, 0 holddown, 0 hidden)
{primary:node0}
root@mr-dfw> show route receive-protocol bgp 172.16.5.3
inet.0: 21 destinations, 34 routes (21 active, 0 holddown, 0 hidden)
Prefix Nexthop MED Lclpref AS path
10.179.1.5/32 172.16.5.3 65113 65100 I
* 10.179.1.6/32 172.16.5.3 65113 I
10.192.176.0/22 172.16.5.3 65113 I
* 10.192.176.100/32 172.16.5.3 65113 I
10.192.192.0/22 172.16.5.3 65113 I
10.192.192.100/32 172.16.5.3 65113 I
10.192.192.200/32 172.16.5.3 65113 I
mgmt_junos.inet.0: 3 destinations, 3 routes (3 active, 0 holddown, 0 hidden)
inet6.0: 1 destinations, 1 routes (1 active, 0 holddown, 0 hidden)
Verify the summary routes advertising by the Vsrx cluster to the spines
root@mr-dfw> show route advertising-protocol bgp 172.16.4.2 inet.0: 21 destinations, 34 routes (21 active, 0 holddown, 0 hidden) Prefix Nexthop MED Lclpref AS path * 10.179.1.7/32 Self I * 10.192.128.0/17 Self I
{primary:node0}
root@mr-dfw> show route advertising-protocol bgp 172.16.4.3
inet.0: 21 destinations, 34 routes (21 active, 0 holddown, 0 hidden)
Prefix Nexthop MED Lclpref AS path
* 10.179.1.7/32 Self I
* 10.192.128.0/17 Self I
{primary:node0}
root@mr-dfw> show route advertising-protocol bgp 172.16.5.2
inet.0: 21 destinations, 34 routes (21 active, 0 holddown, 0 hidden)
Prefix Nexthop MED Lclpref AS path
* 10.179.1.7/32 Self I
* 10.192.128.0/17 Self I
{primary:node0}
root@mr-dfw> show route advertising-protocol bgp 172.16.5.3
inet.0: 21 destinations, 34 routes (21 active, 0 holddown, 0 hidden)
Prefix Nexthop MED Lclpref AS path
* 10.179.1.7/32 Self I
* 10.192.128.0/17 Self I
Verify inter-tenant is routed through Vsrx cluster
SRV1 with IP address 10.192.144.100 is part of Tenant1 instance and SRV7 with IP address 10.192.176.200 is part of tenant2 instance both servers are connected to 2 different switches. We are going to ping SRV7 from SRV1 and verify the flow table on the Vsrx cluster.
root@mr-dfw> ... security flow session destination-prefix 10.192.176.200 node0: -------------------------------------------------------------------------- Session ID: 8388, Policy name: Allow_All/6, HA State: Active, Timeout: 20, Session State: Valid In: 10.192.144.100/25694 --> 10.192.176.200/1;icmp, Conn Tag: 0x0, If: reth1.42, Pkts: 1, Bytes: 84, Out: 10.192.176.200/1 --> 10.192.144.100/25694;icmp, Conn Tag: 0x0, If: reth1.46, Pkts: 0, Bytes: 0, Session ID: 8390, Policy name: Allow_All/6, HA State: Active, Timeout: 22, Session State: Valid In: 10.192.144.100/25694 --> 10.192.176.200/2;icmp, Conn Tag: 0x0, If: reth1.42, Pkts: 1, Bytes: 84, Out: 10.192.176.200/2 --> 10.192.144.100/25694;icmp, Conn Tag: 0x0, If: reth1.46, Pkts: 0, Bytes: 0,
As we can see above the in traffic is using reth1-42 witch is Tenant1 interface and the out traffic is using reth1-46 which is tenant2 interface