Hello,
I'm running RHEL 7 acting as a router and performing throughput tests.
My machine is a 6 physical cores Xeon with Connect-X 3 EN dual-port.
The Spirent generator sends packets from many different flows to one interface and the Linux kernel forwards them to the other interface (unidirectional test).
By default there are 4 RX queues created by the driver for each interafce (largest power of 2 not greater then cpu count), so I changed it to 6 to get one queue per CPU.
I disabled the irqbalance daemon and set my custom IRQ mapping to get the interrupts evenly distributed.
However the load is still distributed among 4 softirqs with 100% load and the remaining 2 are sleeping.
The distribution is perfect according to the /proc/interrupts file.
When I turn hyper-threading ON, to have 12 CPUs and 8 RX queues by default, there are still only 4 softirqs using 100%, which seems strange to me.
The /proc/interrupts file shows even distribution among all cores.
The behaviour is the same with RHEL 7 kernel and upstream vanilla 3.18 kernel.
Any advice how to make more softirqs do the work?