I wanted to make this tutorial for two reasons. The first most important reason is because it took me five days to resolve a DHCP/PXE boot issue i was having in my network. This will maybe help others that are having the same issue I had or trying to get the same setup that I have. The second reason is I am moving from ESXI 6.7 to Proxmox. I will not be mirgrating my VM's from ESXI to Proxmox but will have a fresh install with the same VM"s I had in my ESXI environment.

If you have any questions about this tutoral plese drop me an email @papaulgigi@gmail.com

Goal

The goal here is not to list all the VM"s in my environment but to setup two(2) Proxmox servers not in a cluster environment but independent servers and make VM's on both servers and on different VLAN's talk to each other. The most important part that took me five(5) days to resolve is, I wanted for VM's on server 1 to be able to get DHCP and PXE boot from a DHCP/PXE VM on server1 and server2 and VM's on server 2 to also be able to get DHCP and PXE boot from DHCP/PXE VM on server1 and server2.

Prerequisites

- Two(2) Proxmox servers (prox1 and prox2 in my case) running Proxmox 6.2

- At least 3 NIC's per server (eno1, eno2, eno3 and eno4 in my case)

- 2 switches. One 1GB switch for management and one 10GB switch to connect both prox1 and prox2

Physical connections

Setup and Configuration

Create Linux Bridges

Just think of Linux Bridge like a physical switch By default, vmbr0 will already exist since Proxmox create vmbr0 during the installation process and my vmbr0 is connect to eno3 ( NIC #3). We are going to create vmbr1 and vmbr2.

vmbr1 will be the switch where all the VM's will be connected to and vmbr2 will be the switch that connects prox1 and prox2.

On prox1

1- login to your prox1 node by typing https://your_ip_addres:8006

2- Click on your prox1 node and click on "network"

3- Click on "Create Linux Bridge"

4- check the box that says "VLAN aware"

5- enter the Bridge ports (in my case eno1)

6- click on ok

Your configuration should look like the image below.

Do step 1 to 6 to create vmbr2 this time change the Bridge ports to "eno2"

On vmbr1 will will have four(4) different networks for each VLAN.

- 10.192.16.0/22  vlan-id = 20
- 10.192.32.0/22  vlan-id = 30
- 10.192.48.0/22  vlan-id = 40
- 10.192.64.0/22  vlan-id = 50

And on vmbr2 we will have 1 network

- 10.192.80.0/22  vlan-id = 99

On prox2

Do step 1 to 6 like you did for prox1.

On prox2 we will have

On vmbr1 will will have four(4) different networks for each VLAN.

- 10.64.0.0/22   vlan-id = 60
- 10.64.16.0/22  vlan-id = 70
- 10.64.32.0/22  vlan-id = 80
- 10.64.48.0/22  vlan-id = 90

And on vmbr2 we will have 1 network

- 10.192.80.0/22  vlan-id = 99

VyOS

Create one VM on prox1 called vrouter1001 and another VM on prox2 called vrouter2001. Follow my other tutorial on "VyOS with ESXI 6.7" at https://www.papaulgigitech.com/wiki/VyOS_with_ESXI_6.7

vrouter1001

eth0 : 10.192.0.52/24

eth1 : 10.192.16.1/22

eth2 : 10.192.32.1/22

eth3 : 10.192.48.1/22

eth4 : 10.192.64.1/22

eth5 : 10.192.80.1/22

vrouter2001

eth0 : 10.192.0.53/24

eth1 : 10.64.0.1/22

eth2 : 10.64.16.1/22

eth3 : 10.64.32.1/22

eth4 : 10.64.48.1/22

eth5 : 10.192.80.2/22

When done, all will look like the image below.

Note: on the Mitrotik switch, the two(2) ports connecting the two(2) servers should be set to truck.

DHCP server

In case you have your DHCP server in VLAN 20 on prox1, all the other VM's on prox2 will also be able to talk to your DHCP server if the configuration on vrouter2001 was done correctly by setting the DHCP relay and DHCP server IP address. See link below

https://www.papaulgigitech.com/wiki/VyOS_with_ESXI_6.7#Setting_DHCP_relay

Multiple DHCP servers

You can also have two(2) DHCP servers. one(1) on prox1 and the other one on prox2. In this case your can configure the DHCP server running on prox1 as primary DHCP server and set up the one running on prox2 as secondary DHCP server.

On vrouter1001 you will have:

service {
    dhcp-relay {
        interface eth1
        interface eth2
        interface eth3
        interface eth4
        relay-options {
            hop-count 10
            max-size 576
            relay-agents-packets discard
        }
        server 10.192.16.5     #primary DHCP server
        server 10.64.0.4       #Secondary DHCP server
}

On vrouter2001 you will have:

service {
    dhcp-relay {
        interface eth1
        interface eth2
        interface eth3
        interface eth4
        relay-options {
            hop-count 10
            max-size 576
            relay-agents-packets discard
        }
        server 10.64.0.4       #primary DHCP server
        server 10.192.16.4      #Secondary DHCP server
}

Proxmox-Vyos-Linux Bridge-DHCP-VLANS

Contents