When I started writing this blog, I didn't intend to write about low-level technical details, because frankly that is not what I'm concerned with in my day job anymore. However, I like to tinker with network labs in my spare time, and I ran into problems getting ethernet multicast protocols like LLDP and LACP to work using Linux bridges. It took me a while to google all the answers, so I thought it might be worthwhile to jot down my notes for whenever I run into this again.
IEEE 802.1D MAC Bridge Filtered MAC Group Addresses
The protocols I'm talking about are using MAC addresses in the range 01-80-C2-00-00-0x, a range that is defined in IEEE standard 802.1D as "MAC Bridge Filtered MAC Group Addresses". This is a range set aside by the IEEE for standard protocols that use link local multicast to communicate with a neighboring device. Frames using these MAC addresses are supposed to be explicitly link local; quoting the IEEE Standard Group MAC Addresses tutorial:
IEEE 802.1D MAC Bridge Filtered MAC Group Addresses: 01-80-C2-00-00-00 to 01-80-C2-00-00-0F; MAC frames that have a destination MAC address within this range are not relayed by MAC bridges conforming to IEEE 802.1D.
|01-80-C2-00-00-00||Spanning Tree (STP/RSPT/MSTP)|
|01-80-C2-00-00-01||Ethernet Flow Control (pause frames)|
|01-80-C2-00-00-02||Link Aggregation Control Protocol (LACP)|
|01-80-C2-00-00-03||802.1X Port-Based Network Access Control|
|01-80-C2-00-00-08||Provider Bridge protocols (STP)|
|01-80-C2-00-00-0D||Provider Bridge protocols (MVRP)|
|01-80-C2-00-00-0E||802.1AB Link Layer Discovery Protocol (LLDP)|
That's nice, so why do I care?
Protocols using these MAC addresses are strictly link local, and any bridge compliant with IEEE 802.1D must filter these frames: either process or drop them. In no circumstance would it be permitted to forward them along to other receivers. This made sense in the early 2000s, when bridges were chunks of iron where you would plug in other chunks of iron. In the age of virtualization that's not quite the case anymore.
In my case I'm using KVM-based host to simulate multiple virtual network devices. I'm interconnecting these devices using the Linux Bridge, and of course I would like to have the bridge behave as much as possible like a wire directly connecting my virtual devices. Unfortunately, the Linux Bridge is neatly written to conform to IEEE 802.1D, and it's filtering all Ethernet frames using destination addresses in the range 01-80-C2-00-00-0x. This means I cannot use my virtual environment to test any of the protocols in the table above. Sad!
Now I do want to test LLDP, LACP, STP, 802.1X whenever I'm building network labs, so I have to figure out a workaround. Obviously I'm not the first one to encounter this, so there are solutions available. Since Linux kernel 2.6 there is a setting that allows you to control which link local frames from the range defined in IEEE 802.1D the bridge should forward, by setting a specific bitmask in /sys/class/net/bridge-iface/bridge/group_fwd_mask. The default value 0 means the Linux bridge does not forward any link local frame. Settings this value to 16384 for example would allow the bridge to forward LLDP frames (01-80-C2-00-00-0E):
echo 16384 > /sys/class/net/br0/bridge/group_fwd_mask
A slight catch is that in the default kernel distribution, bitmask values for the first three MAC addresses (-00, -01 and -02) are restricted, meaning we still cannot use this trick to enable STP and LACP protocols in our labs. To remove this restriction, you need to patch and compile your kernel yourself, like the folks at EVE-NG do. I am lazy, so I just grab the compiled kernel from the EVE-NG repository. Now we can set the group_fwd_mask to any value we like.
Bitmasks for group_fwd_mask?
So what value should we use for this bitmask? The bitmask is a 16-bit number, where the first (least significant) bit represents MAC address 01-80-C2-00-00-00 and the 16th bit (most significant) represents 01-80-C2-00-00-0F. The default value (all bits are 0) does not forward any link local frame. To enable forwarding of frames for a specific MAC address, we need to set the corresponding bit to 1. For example, to allow forwarding of LLDP frames (01-80-C2-00-00-0E) we would need to set the 15th bit 1, and leave the rest at 0:
This means we use the binary number 0100 0000 0000 0000 as bitmask, which translates to decimal number 16384, just like we used in the example earlier.
If we would like to add LACP (01-80-C2-00-00-02) and 802.1X (01-80-C2-00-00-03) to the mix, we would also set the 3rd and 4th bit to 1:
This results in the binary number 0100 0000 0000 1100, which is 16396 in decimal. Using this method, you now have full control over which type of frame to forward or filter.
That's great, but I'm lazy. For my lab I just want to allow as much as possible so I can test whatever protocol I want. So I just set all bit to 1:
- for my patched kernel I can use all bits: 1111 1111 1111 1111 = 65535
- for the unpatched kernel I cannot use the first three bits: 1111 1111 1111 1000 = 65528
In other words: to enable out-of-the-box Linux bridges to forward all IEEE 802.1D MAC Bridge Filtered MAC Group Addresses except the restricted three types, execute this command:
echo 65528 > /sys/class/net/br0/bridge/group_fwd_mask
Final note: it might not be the smartest thing to do in a production environment, especially with a patched kernel flooding STP BPDUs. You're probably best off using this for (virtual) network labs only. Enjoy at your own risk!
About halve of the addresses are still "Reserved for future standardization", so you can still grab one that new layer-2 multicast protocol you're working on. You can apply for an assignment directly through the IEEE website - let me know how that works out. ↩︎
Provider Bridge protocols are identical to regular layer-2 protocols, except that they use a different MAC group address. This makes it possible to differentiate between a customer network's layer-2 protocols when tunneling them through the provider network (using the likes of QinQ and L2PT). To me as a modern enterprise engineer having multitenant nested spanning trees sounds as somebody's worst nightmare, but apparently it made sense two decades ago. ↩︎
Ever wondered why you would find CDP information all over the network when using non-Cisco or unmanaged switches, but never had this problem with LLDP? That's because CDP (01-00-0C-CC-CC-CC) uses a generic multicast MAC address, which if flooded by non-Cisco switches. LLDP (01-80-C2-00-00-0E) uses a Bridge Filtered group MAC address that is dropped (filtered) by any switch worth its salt. ↩︎
Ancient word for "switch". ↩︎
Advise for people running Juniper equipment (for example using Wistar: you can change STP to use 01-80-C2-00-00-08 instead of the restricted group address 01-80-C2-00-00-00 by setting the protocol to use the provider-bridge-group. ↩︎