I was tried local private network from Ethernet to IPoIB.
But there is some problem occurred.
I have virtual Untangle VM for DHCP service and control separated private network.
Windows VM works greate on IPoIB vSwitched network like Ethernet vSwitched network.
But Linux VM wasn't.
Exactly speaking Linux VMs between separate ESXi hosts.
Linux VM on same ESXi host works with DHCP service, but separate ESXi host environment wasn't.
(My vUntangle VM on Host01, DHCP client Linux VM on Host02)
Manually configured Linux VM on separate ESXi host works good everytime.
Only problem is interaction with DHCP service.
Is there any suggestions?
IB configuration is here.
4036 QDR switch configured MTU=5 and partition properly in embedded SM.
All ESXi host was configured mtu_4k=1
All HCA is ConnectX-2 with firmware 2.10.0720.