Re: Suggestions for *quiet* Infiniband switch?
Good point. I'll test the cables when I get a chance. The cables I use are GORE IBN5600-5, which are 4X DDR from what I can find. All my servers are running Windows Server 2012 and firmware 2.9.1 on...
View ArticleRe: Suggestions for *quiet* Infiniband switch?
Hrm, I think those are the same cables that I have. I'll check mine as well and can compare. I have the 8m ones I couldn't get even a point-to-point to work with 2.9.1 and ESX 5.1, I had to drop it to...
View ArticleRe: Dual port FDR connectx-3 card
Using ConnectX-3 you can't reach an aggregate BW of 2xFDR.the two ports are consider as an high availability solution of one active and one standby. You can reach the 2xFDR full BW by using Connect-IB.
View ArticleRe: 40GbE optimization and bandwidth testing
4322MB/s using Starwind RAMDisk as a network share!!! As it turns out the Startech PCIE riser I was using in my render nodes was limiting the HCA to PCIe2 x8 which was having a disproportional effect...
View ArticleClik here to view.
Re: Mellanox (old Voltaire) ISR9024D-M recover flash area
Hi inbusiness, can you post here the text you see on your screen? just connect console cable to your 9024D-M, power it up, let it boot until it stops, copy all the text and post here.we'll try to help.
View ArticleRe: Suggestions for *quiet* Infiniband switch?
Well its definitely not the cables since if I direct connect the ports on a MHGH28 card, it will register at 4X DDR speeds. I tried pressing the reset button on the switch but that didn't seem to...
View ArticleRe: Will there be ESXi 5.x driver support for ConnectX adapters?
Thanks again. I'll probably keep poking at it, I think the 4k issue probably has to do with opensm and how it is starting as a service under Windows. Might need to run opensm under a Linux system,...
View ArticleRe: Will there be ESXi 5.x driver support for ConnectX adapters?
Hi wark286,Yes 4K mtu is supported in IPoIB for vSphere 5 as explained in the User Manual.You basically need to configure the SM to enable 4K mtu for IPoIB (check your SM user manual, the configuration...
View ArticleRe: Will there be ESXi 5.x driver support for ConnectX adapters?
I've run through all of the command and configuration recommendations in the user manuals, best I can get is 2044 MTU. Could possibly be opensm not listening to my partitions.conf file as when I run...
View ArticleRe: Will there be ESXi 5.x driver support for ConnectX adapters?
Hi wark286,Indeed, looks like your opensm is not listening to the config file (the test you've done using saquery reflects this fact).If you're running opensm from a host, try to run it manually:>...
View ArticleRe: Will there be ESXi 5.x driver support for ConnectX adapters?
Looks like running it from command line and specifying the file did the trick, hadn't used the -P argument previously. Not sure why it wasn't reading correctly when running as a service in Windows,...
View ArticleRe: Dual port FDR connectx-3 card
What gadis said. If for no other reason then PCIe3 x8 has a theoretical max bandwidth of about 56Gbps.
View ArticleRe: Will there be ESXi 5.x driver support for ConnectX adapters?
Interesting. I had a chance to do some testing and I found that if I connect my 2 nodes direct, I can get an active_mtu of 4096, but if I go through my switch (Voltaire ISR9024D), it drops to...
View ArticleRe: Suggestions for *quiet* Infiniband switch?
So my latest quirk is that if I connect two cards direct, I get an active_mtu of 4096, but if it passes through the switch, it drops to 2048. More digging around.
View ArticleClik here to view.
Re: Will there be ESXi 5.x driver support for ConnectX adapters?
Also noticed something weird when I switched from my ESXi 4.1 test install to a 5.1 install. After getting the drivers installed, it connected to the infiniband network at 4x, however, when I enabled...
View ArticleRe: Dual port FDR connectx-3 card
that's correct.X16 should give you better ability with both ports - the card can. it is what's underneath it that is limiting us. :-)
View ArticleRe: dual port FDR ConnectX-3 card
Redirecting you to this thread - same question: Dual port FDR connectx-3 card
View ArticleRe: MCQH29-XDR firmware for RDMA
I managed to flash my Dell mezzanine cards with firmware version 2.10.720, and I wrote an article about it. RDMA is now enabled, and maximum throughput of an SMB share, measured using IOMeter, improved...
View ArticleRe: Mellanox (old Voltaire) ISR9024D-M recover flash area
Hi yairi! Thank you for your reply. I'll check it this weekend and reply on this thread...:)
View ArticleClik here to view.
Re: sx1036 as IB-to IP gateway
Thanks a lot for the very informative image. A short follow-up question:Is it so that the blocked ports are caused by using split cables? example: port 4 uses a 4*10G split cable so port 1 is...
View Article