Clik here to view.
How to setup IPoIB correctly?
We have a small network for testing. Its layout is shown below. The orange-colored IP addresses are the current IPoIB setup. They are done with /etc/sysconfig/network-scripts/ifcfg0-ib0 using IP...
View ArticleRe: dma_map_single not able to receive capsules on ARM with mellanox card
Hello, I advise you to contact Mellanox Support (support@mellanox.com) and describe your project, configuration and further elaborate on what you're trying to do with Mellanox NICs over ARM platforms.
View ArticleClik here to view.
Can multiple versions of mlnx-ofed exist in the same IB fabric?
Hi, we have installed compute nodes with RHELS6.3 and MLNX_OFED_LINUX-2.0-2.0.5 using an Infiniband network. We intend to upgrade to RHELS6.7 and mlnx-ofed 3.x.In a rolling upgrade scenario, can we...
View ArticleRe: Windows, TCP/IP - lowest possible latency?
Should I expect any difference between ConnectX 3 Pro and ConnectX 4 EN in this case?
View ArticleI have 2 colocation sites located 6-7km apart. I can rent a 10gb wave between...
Title says about all; We are starting our cloud-hosting company at 2 sites, in the future it will be 3 sites in total. For now however, we start at 2 sites. I have a SX1036 with dual PSU that I am...
View ArticleRe: VMware ESXi 6.0 virtual ib_ipoib interfaces
Hi David, 1) Driver version 2.4.0 is an Ethernet driver only so if you have a VPI card, please make sure the port protocol is configured as 2 which is Ethernet. #/opt/mellanox/bin/mlxconfig -d...
View ArticleRe: ConnectX-3 RoCE fails send WRs when running multiple QPs
I afraid it is tricky to understand without debugging and source code, but most likely there is a problem there. Try to print the values of lkey/rkey, QP numbers, protection domains, other related data...
View ArticleRe: 'State: Initializing' but works
Thank you eddie.notz for your response. Yes.Both port 1 and port 2 status had been 'active' till then. (I did not apply any updates etc in recent and all updates are always manually installed to avoid...
View ArticleRe: Which ESXi driver to use for SRP/iSER over IB (not Eth!)?
7 days later and no reply from Mellanox... Seriously?
View ArticleRe: VMware ESXi 6.0 virtual ib_ipoib interfaces
Hi Sopie we don't want to use it as Ethernet card but as IB over IP. That's the problem that I can't get it to work as a 40GB IBoIP card in ESX 6 anymore, which worked perfectly with ESX 5.5 and the...
View ArticleHow to change speed from FDR to QDR using ibportstate command in centos 6.2
We have Mellanox Infiniband FDR cards and cables supporting it. We want to reduce the speed of the adapters from 56 Gbps to 40 Gbps. We tried using ibportstate <base lid> <port id> of the...
View ArticleRe: How to change speed from FDR to QDR using ibportstate command in centos 6.2
ibportstate <lid > <port> fdr10 0 espeed 30 ibportstate <lid> <port> reset
View ArticleIs it possible to connect 2 ethernet switches (sx1036) located up to 10km...
Hi guys, I have a question that is real important for me; How can I connect my two sites so that I am able to use RDMA between them? I have 2 racks, each located in a different datacenter. Each rack...
View ArticleRe: How to setup IPoIB correctly?
I have been kept digging, and here is what I have observed: I just did a [root@sc2u0n0 ~]# lsinitrd /boot/initramfs-3.10.0-327.28.2.el7.x86_64.img|less[...] drwxr-xr-x 5 root root 0 Aug...
View ArticleRe: VMware ESXi 6.0 virtual ib_ipoib interfaces
Driver 2.4.0 include IPoIB driver and works!Absolutlely! driver 2.4.0 located in ethernet driver section, but why this driver 2.4.0 include IPoIB driver & works?
View ArticleRe: Is it possible to connect 2 ethernet switches (sx1036) located up to 10km...
You can do that using MetroX and InfiniBand connection for 10Km, we don't support RoCE for such distance.This is mainly due to packet loss and buffer size of the switch on each side (depends on the...
View ArticleRe: How to setup IPoIB correctly?
I resolved the problem by moving all IPoIB IPv4 addresses to a different subnet.
View ArticleClik here to view.
Re: Which ESXi driver to use for SRP/iSER over IB (not Eth!)?
Looking at the current driver selection across different OS/cards/medium (IB vs. ETH) it looks like the only space consistently supported by Mellanox is Linux. Indeed, for Linux you have...
View ArticleRe: I have 2 colocation sites located 6-7km apart. I can rent a 10gb wave...
please refer to this post: Is it possible to connect 2 ethernet switches (sx1036) located up to 10km apart without losing RDMA?
View Article