Andrey V. Elsukov

@ocochardlabbe @_bapt_ Did you some IPsec/crypto tuning? Number of net.isr threads, binding them to CPU cores, net.inet.ipsec.async_crypto, etc.

At @_bapt_ 's request, adding tinc to the #FreeBSD VPNs benches

@encthenet @ed_maste It is much simpler to make working solution, when you trying to solve own issue. So, if you want add ipfw support to this app, just do it :) If you have question, feel free to ask.

@mrhameljr @allanjude @FreeBSDHelp @michaeldexter @Hacker_Horse FreeBSD's if_vxlan(4) does not set DF bit for encapsulated packets as far as I can see from the code.

@allanjude @FreeBSDHelp @michaeldexter @Hacker_Horse @_bu7cher_ While yes it uses UDP, it still requires trickery to get the DF-bit turned off. I'm not sure how VXLANs perform on the Internet. Juniper gear supports it but can't disable the bit on any tunnels. Does FreeBSD support it and bridging performance for a couple of Gbits of traffic?

FreeBSD community, does it's GRE implementation support multithreading, for max throughput? If someone could add "gretap" (Ethernet frames over GRE), bonus points for Mikrotik EoIP compatibility, that would be very much appreciated. CC: @allanjude @michaeldexter @Hacker_Horse

@mrhameljr @allanjude @michaeldexter @Hacker_Horse GRE by self introduces big performance penalty by doing traffic encapsulation, but adding some userlevel application that works via tap(4) makes it much worse.

@mrhameljr @allanjude @michaeldexter @Hacker_Horse If you are talking about tap(4) and some userlevel application that will handle ethernet traffic received from GRE, I doubt that it is possible get big performance.

@mrhameljr @allanjude @michaeldexter @Hacker_Horse GRE uses netisr_dispatch(9) for inbound traffic. So, if you will correctly setup netisr(9), it could work simultaneously for different flows. Anyway our GRE implementation does not support Ethernet over GRE.

@ocochardlabbe @allanjude @mzar_y AFAIR, formerly you used cpuset in the shell script to bound NIC queues to CPUs for Chelsio.

@_bu7cher_ @allanjude @mzar_y I've disabled all unused hardware features on Chelsio (toecaps, rdmacaps, iscsicaps, fcoecaps) and disabled Ethernet flow control on all NIC (if drivers allow it). No other special tuning was applied.

#FreeBSD Commit: " Switch RIB and RADIX_NODE_HEAD lock from rwlock(9) to rmlock(9)", thanks @_bu7cher_

@ocochardlabbe @allanjude @mzar_y Did you make some special tuning except number of queues? e.g. CPU binding, queue length, etc.

@allanjude @mzar_y @_bu7cher_ Is this new bench a good answer ?