linux bridge vs ovs Bridge
You are using an out of date browser. It may not display this or other websites correctly.
You should upgrade or use an alternative browser.
vikozo
Well-Known Member
hello
linux bridge vs ovs Bridge
in which situation should be used which of this bridge?
is mixing the Bridge wise?
narrateourale
Active Member
I bet that in 99.9% you will be happy with the regular linux bridge.
Can you tell a bit more what you actually need?
vikozo
Well-Known Member
hvisage
Active Member
REason for Using OpenVSwitch: fancy VLAN support
ie. I have VLAN 666 that is the native-untagged VLAN on the vRack/BAckEnd. So the VMs that needs Interne access I bind to Vlan 666, while the rest communicates on their own VMs.
I have the need to map another interface, to another vlan on another host. that physical interface was originally bound to a linux bridge and I didn’t want to disturb it as yet. I used a veth link between the linux bridge and the open vswitch. the port on the open vSwitch was assigned a vlan that was trunked via the native-untagged above on another vlan to the remote host’s vlan.
*lately* Linux Brdiges have better VLAN 802.1q tagging support, but that wasn’t the case long time ago, with only OpenVSwitch being the «real» alternative for vlans and trunking in the same virtual switch.
I still have some setup I haven’t migrated yet to vlans, where I have multiple Linux bridges to simulate vlans.
spirit
Famous Member
*lately* Linux Brdiges have better VLAN 802.1q tagging support, but that wasn’t the case long time ago, with only OpenVSwitch being the «real» alternative for vlans and trunking in the same virtual switch.
well, vlan support exist since kernel 3.8, so 2013
the only advantage of ovs could be dpdk , but it’s not supported by proxmox currently.
Maybe netflow,sflow support too (but can be done with external daemon with linux bridge too)
you can do vlan, qinq, vxlan, bgp evpn, gre tunnel, ipip tunnel. with linux bridge without any problem.
(Cumulus linux has done a great job to implement all the things this last years, as they use linux bridge for their switch os)
«Are you looking for a French Proxmox training center?
hvisage
Active Member
For somebody that have worked on/with Linux since kernel 0.92 and used bridge since ~2000, 3.8 is quite «new»
spirit
Famous Member
For somebody that have worked on/with Linux since kernel 0.92 and used bridge since ~2000, 3.8 is quite «new»
yes me too But ovs is not old too (~ 2010) if you compare about the vlan feature.
Personally, I still prefer linux bridge than ovs. (if ovs deamon is killed or oom, you don’t have network anymore :/ . and I have see a lot of bug report recently with packet drop and strange stack traces)
«Are you looking for a French Proxmox training center?
kcallis
Active Member
I have been thinking about moving from Linux Bridges to OVS. The reason that I wanted to move to OVS is that I have decided that I wanted to virtualize pfSense and I make use of about 12 VLANs and it seems to easier to work with VLANS under OVS as Linux bridges. Also since I am using PM with a single interface (laptop), I have even more need of VLANs.
spirit
Famous Member
I have been thinking about moving from Linux Bridges to OVS. The reason that I wanted to move to OVS is that I have decided that I wanted to virtualize pfSense and I make use of about 12 VLANs and it seems to easier to work with VLANS under OVS as Linux bridges. Also since I am using PM with a single interface (laptop), I have even more need of VLANs.
«Are you looking for a French Proxmox training center?
kcallis
Active Member
spirit
Famous Member
by default, linux bridge don’t support vlan.
if you enable the option «vlan aware» , in the gui «host->system->network->youbridge options,
the bridge will transport the vlans.(like OVS) (and than you can create vlan interfaces in your pfsense)
«Are you looking for a French Proxmox training center?
kcallis
Active Member
by default, linux bridge don’t support vlan.
if you enable the option «vlan aware» , in the gui «host->system->network->youbridge options,
the bridge will transport the vlans.(like OVS) (and than you can create vlan interfaces in your pfsense)
So would I still have to create entries in /etc/network/interface file? For instance, I am pretty clear that I need to create an interface for pfsense VMBR1 [WAN] VMBR2 [LAN] and VMBR3 [OPT]), so I would create that in /etc/network/interface. Now under pfSense, I create VLAN 5, 10, 20 30 40, etc under the OPT interface., do I need to added entries in the /etc/network/interface or would I be good to go just worrying about creating VLANs only in pfSense.
spirit
Famous Member
you only need 1 bridge, 1 nic in the vm (without any vlan tag in proxmox gui vm nic options), then create vlan interfaces inside pfsense.
«Are you looking for a French Proxmox training center?
hvisage
Active Member
Personally, I still prefer linux bridge than ovs. (if ovs deamon is killed or oom, you don’t have network anymore :/ . and I have see a lot of bug report recently with packet drop and strange stack traces)
If the OOM kicked in, then you are screwed on the Hypervisor in anycase. The OOM is well. let’s say I have a preference for proper OS’s like Solaris and not debate that part.
That said: Which ever rocks your dingy
FOr me, OVS just «worked» way back since before ProxMox moved away from the RedHAt/Centos kernels when they still had OpenVZ (Yet another NIHS issue w.r.t. Linux Kernel devs to move to LXCs) which was I recall a 2.4/2.6 kernel, thus still before 3.8 when I started to use OVS extensively.
I have been thinking about moving from Linux Bridges to OVS. The reason that I wanted to move to OVS is that I have decided that I wanted to virtualize pfSense and I make use of about 12 VLANs and it seems to easier to work with VLANS under OVS as Linux bridges. Also since I am using PM with a single interface (laptop), I have even more need of VLANs.
So, yes the typical would be something like:
create the OVS bridge, «delete» the physical’s config, and attached that physical as an OVSPort to the bridge.
Then you add a OVSintPort and put the IP of the proxmox on that, and attach that to thebridge (I has a bit complexer setup on my «production» servers where I have the physical as a native-untagged with a VLAN tag like 666, then I attach the OVSintPort to that.)
For OPNsense/pfSense:
1) use VirtIO and not e1000 — I’ve had lock ups on those e1000 in FreeBSD ;(
2) disable the hardware acceleration — it breaks UDP and thus DHCP
just when you attach the OPNsense/pfSense interface, attach it as a tagless, then OVS will «default» to trunk capable, and you can create the vlan interfaces inside the opnsense/pfSense interface.
The VMs themselves you add the relevant VLAN tag.
The screenshot of my lab/test server’s network setup inside ProxMox:
kkjensen
New Member
I’m just in the middle of trying to implement OVS after reinstalling my 4 node test cluster. The production cluster will be for a small ISP that has a bunch of vlans. I’m used to working with Cisco switches and was hoping to create a 10G trunk through all the nodes and starting/finishing on one of the big edge switches. The regular bridge mode (I’m probably reading old tutorials and forum posts) seemed to require that every vlan get explicitly defined on a port so when I found out about OVS I thought it made sense to set up the cluster using OVS in each node to form the 10G loop and then the virtual machines can all pick up a connection off that loop once it’s in place with all VLANs allowed to flow over it.
Prior to my reinstall I did get a vlan working on one container but I could see that implementing different vlans for all the different services that would someday be required was going to be a pain IMO.
Now that I have OVS installed on new nodes I’m just seeing a new field «OVS options». I was hoping for some GUI implementation that would let you list the vlans and make one a native vlan if required.
I’ve been looking for an ideal tutorial of some sort using proxmox, OVS and ceph. Any recommendations?
Proxmox Configure Open vSwitch
Proxmox has two kinds of virtual switches that can be utilised. Linux bridges (the default) and Open vSwitch (OVS). Linux Bridges are generally fine unless you want to configure a lot of VLANs and/or do port mirroring. I have a use-case for both, so in this post, I will show you how to configure Proxmox to use Open vSwitch.
The following software was used in this post.
Requried Packages
In order to utilise Open vSwitch you need to install the openvswitch-switch package.
apt install openvswitch-switchNetwork Config
The /etc/network/interfaces file is used to configure the networking for Proxmox nodes. The below is the config I use for my LAB node.
# /etc/network/interfaces auto lo iface lo inet loopback auto eno1 iface eno1 inet manual ovs_type OVSPort ovs_bridge vmbr0 auto enp1s0 iface enp1s0 inet manual ovs_type OVSPort ovs_bridge vmbr1 auto vlan10 iface vlan10 inet static address 172.16.255.12/24 gateway 172.16.255.1 ovs_type OVSIntPort ovs_bridge vmbr0 ovs_options tag=10 auto vmbr0 iface vmbr0 inet manual ovs_type OVSBridge ovs_ports eno1 vlan10 auto vmbr1 iface vmbr1 inet manual ovs_type OVSBridge ovs_ports enp1s0The above config can be summarised as follows.
Physical Interfaces
My Proxmox node has two physical interfaces: eno1 and enp1s0 . Physical interfaces are assigned the OVSPort interface type. The ports are mapped to OVS bridges as follows:
For my setup, management and guest VLANs are trunked to vmbr0 and vmbr1 is used as a port mirror destination.
OVS Bridges
OVS bridges are a virtual switch and are assigned the OVSBridge type.
VLAN Interfaces
VLAN interfaces are used to assign IP addresses to the Proxmox host. I have a single vlan10 interface which is used for management of the host. VLAN interfaces are assigned the OVSIntPort interface type.
Apply Config
When the /etc/network/interfaces file is updated the networking service needs to be restarted for the config to take effect.
systemctl restart networking.serviceSometimes, that is not enough and you may also need to restart the host.
I found that more ofthen than not, the host needs to be restarted after changing from Linux bridges to OVS. To be safe, before attempting anything, migrate the VMs to another host, or shut them down.
Thats it! Now, when you create a VM, you can assign it a VLAN ID and there is nothing to do on the host. The VLAN will be trunked through to the network switch.
Outro
In this post, I showed you how to configure Proxmox to utilise Open vSwitch instead of Linux Bridges. In a future post, I will show you how to enable port mirroring on Open vSwitch bridges.