Mixing OFED 1.5.3 and 2.2 in the same network?
Hi all, We have 2 clusters using infiniband, as follows:Computing cluster 1:27 nodesIBM bladecenterCentOS 6.2Each node has 1x MT26428 [ConnectX VPI PCIe 2.0 5GT/s - IB QDR / 10GigE]Firmware version:...
View ArticleClik here to view.
Re: Firmware for Mellanox cards in Intel S2600JFQ
Hi Branko, Thanks for the answer. I think I only got the support contract for the switch from mellanox. The cards were provided as part of the intel system, so I don't think I have specific mellanox...
View ArticlePerformance of 40GbE NICs
Hi,I have two machines, each with 4 40GbE NICs: # lspci | grep -i mellanox02:00.0 Ethernet controller: Mellanox Technologies MT27500 Family [ConnectX-3]07:00.0 Ethernet controller: Mellanox...
View ArticleIPoIB iscsi disconnects. CQE retry exceeded
Hi guys, The new environment we are running is having some issues, the following is the architecture. Compute - XenServer 6.2 (HP DL360 G7s)Storage - OmniOS Comstar TargetVoltaire 4036...
View ArticleRe: IPoIB iscsi disconnects. CQE retry exceeded
If you don't have any limitations on what which driver you can use - try to install latest MLNX OFED (see link below). There have been number of significant improvements between 1.5,x and 2.x OFED...
View ArticleRe: Performance of 40GbE NICs
I'd suggest to check your system settings against recommended settings for BIOS (CPU power management states, PCIe bus), interrupt moderation, NUMA tuning according to guide...
View ArticleClik here to view.
Re: Simultanous TX/RX does not achieve 80Gbps (ConnectX-3 )?
The theoretical max numbers in your case will be limited with following: 1. PCIe limit: PCIe 3.0 is 8 GT/s per lane but effectively it delivers about 1.54% less data due to 128/130 encoding, so...
View ArticleRe: controllers Mellanox MHRH19-XTC, Windows Server 2012R2
Question unchecked because urgently needed to raise the cluster. Again had to return to Windows Server 2008R2.
View ArticleRe: controllers Mellanox MHRH19-XTC, Windows Server 2012R2
But here, too, is not all right, I can not configure the card for maximum performance, lack experience in this business I'm a newbie.Would be very happy if anyone can help, it is unclear what drivers...
View ArticleRe: controllers Mellanox MHRH19-XTC, Windows Server 2012R2
When you run the test MPI Ping Pong: latency getting the following warning:MPI Ping-Pong: LatencyTest Result: SuccessFailed nodes list...
View ArticleRe: IPoIB iscsi disconnects. CQE retry exceeded
Hi Andre, I updated to 2.2, however now my xenserver Kernel panics when loading openibd: Unloading HCA driver: [ OK ] Message from syslogd@ at Mon Jul 7 19:24:45...
View Articleactivate in-band IP access to grid director 4036
Hi, my setup looks like server2 ---seriallink--- 4036(192.168.20.254/24) -----QSFPcable----- server1(192.168.20.1/24) Result: on server1 the switch is...
View ArticleNFS over RDMA on CentOS 6.3 Problem
Trying to setup NFS over RDMA to get best possible performance, but running into a version issue. OS: CentOS release 6.3 (Final)Kernel: 2.6.32-279.el6.x86_64OFED:...
View ArticleRe: IPoIB iscsi disconnects. CQE retry exceeded
Have you checked known issues in the release notes ? I see items 58 and 59 for xen: http://www.mellanox.com/related-docs/prod_software/Mellanox_OFED_Linux_Release_Notes_2_2-1_0_1.pdfPlease share more...
View ArticleRe: IPoIB iscsi disconnects. CQE retry exceeded
The issue appears to be with HP ConnectX cards, we were using 2 of these in our cluster with fw 2.7, we downloaded and burnt 2.8 (sourced from hp site) after carefully checked PSID to match the...
View ArticleRe: IPoIB iscsi disconnects. CQE retry exceeded
I’m assuming you’ve rebooted host after f/w upgrade since mstflint will report new fw without reboot but card will not use it unless you reboot…If you still can access HCA with mstflint please get and...
View ArticleClik here to view.
Re: NFS over RDMA on CentOS 6.3 Problem
You might have a mix of driver module versions on the system. Make sure you have restarted openibd after installing MLNX_OFED_LINUX-1.5.3-4.0.42-rhel6.3-x86_64 - it is likely that you still have kernel...
View Articleubuntu14.04 infiniband set problems
1 first install the driverMLNX_OFED_LINUX-2.2-1.0.1-ubuntu14.04-x86_64.iso2 ibhosts can see Ca : 0x002590ffff17b868 ports 1 "node03 HCA-1"Ca : 0x002590ffff17bd40 ports 1 "node02 HCA-1"Ca :...
View ArticleClik here to view.
Re: 56 GbE over a ConnectX-3 VPI adapter
for back-2-back 56GbE connections between ConnectX-3 and ConnectX-3 Pro, the solution provided in this post is still relevent
View ArticleAm i able to use SFP+ 10Gbase-SR with MCX312A-XCBT adapter?
Greetings.I recently bought the MCX312A-XCBT adapter not knowing which optical module i was going to use. It turned out that the sfp+ 10gbase-SR is actually needed. The question is does this adapter...
View Article