Quantcast
Channel: Mellanox Interconnect Community: Message List
Viewing all 6275 articles
Browse latest View live
↧

rx-out-of-buffer

August 29, 2018, 12:04 am
$
0
0

Hi Community,

 

I'd like to understand better a problem we have, which seems to be linked to the fact that DPDK's xstats/ethtool -S shows a lot of "rx-out-of-buffer" packets. I found the performance counter document, but it does not say much about why this could happen, which buffer we're speaking about. I quote "Number of times  receive queue had no software buffers allocated for the adapter's incoming traffic.". As rx_nombufs (DPDK stats) is 0 I guess it does not mean that there is not enough software buffers. Are they some internal MLX buffers? What can be done to prevent that?

 

Thanks,

Tom

↧
Search

Re: Can't ibping Lid or GUID but can ping by ip

August 29, 2018, 7:27 am
$
0
0

Is there any error messages? Does ibtracert work (#ibtracert <src lid> <dst list>?

↧
↧

Re: Can't ibping Lid or GUID but can ping by ip

August 29, 2018, 8:00 am
$
0
0

ibtracert works

 

We actually have connection but we are only able to ibping to the GUID that is binded on OpenSM but can't ibping to the other GUIDs now.

↧

Re: Keeping two versions driver for two kernels

August 29, 2018, 8:10 am
$
0
0

If your question is if it possible to use different MOFED version for different kernel, then it is not . However, you should be able to install Mellanox OFED on one kernel with --disable-kmp option, reboot to other kernel, recompile same Mellanox OFED using mlnx_add_kernel_support and after that install only kernel modules.

↧

Re: Slow File Transfer On 20Gbps IB

August 29, 2018, 8:53 am
$
0
0

According to you, the issue somewhere in OS (I/O, memory allocation, other) and not in the network. 20Gbps on ConnectX-2 will give you maximum theoretical 16 Gbps because of 8/10 encoding, so 15.6 Gbps is pretty close.

I would suggest to use perf to analyze ssh/rsync behaviour, or maybe 'strace -ttt -T' option in order to see how much time it spends in the system calls

↧
↧

Re: Keeping two versions driver for two kernels

August 29, 2018, 11:30 am
$
0
0

I found one bug in the Mellanox drivers. In the RT kernel on the Centos 7.5 system, the ptp4l or phc2sys processes do not want to start automatically as a service (systemctl). There is no problem with drivers built into the kernel.

↧

Re: Windows 2016 Storage Spaces Direct over IPoIB

August 29, 2018, 3:13 pm
$
0
0

Hello Todd -

   Please see: http://www.mellanox.com/related-docs/whitepapers/InfiniBandFAQ_FQ_100.pdf  and let me know if it helps with your questions.

Specifically: 16 & 17

- InfiniBand supports QoS by creating Virtual Lanes (VL). These VLs are separate logical communication links that share a single physical link.

- InfiniBand, by contrast, uses link-level flow control to ensure that packets are not dropped in the fabric

 

Many thanks -

~Steve

↧

Re: Slow File Transfer On 20Gbps IB

August 29, 2018, 7:05 pm
$
0
0

According to you, the issue somewhere in OS (I/O, memory allocation, other) and not in the network. 20Gbps on ConnectX-2 will give you maximum theoretical 16 Gbps because of 8/10 encoding, so 15.6 Gbps is pretty close.

I would suggest to use perf to analyze ssh/rsync behaviour, or maybe 'strace -ttt -T' option in order to see how much time it spends in the system calls

↧
Search

puppet resource command

August 29, 2018, 11:00 pm
$
0
0

I've created a puppet resource for interfaces. Most of the interface names on my switch are lowercase with the exception of Ethernet interfaces, so I munged the interface name to hopefully reduce errors in the manifest; e.g.:

manifest:

  cisco_interface { 'Ethernet1/1': description => 'foo' }

type/cisco_interface.rb:

  newparam(:name) do munge { |value| value.downcase } end

My provider code also downcases the interface names when I collect the list of interfaces with self.instances.

So this works great when I test with the manifest, but not so great with the puppet resource command which only works when I call it with the name already downcased:

switch# puppet resource cisco_interface 'Ethernet1/1' cisco_interface { 'Ethernet1/1': ensure => 'absent', }  switch# puppet resource cisco_interface 'ethernet1/1' cisco_interface { 'ethernet1/1': ensure                       => 'present', description                  => 'foo', }

The puppet resource command name field seems to just be a simple filter so I think I'm stuck, but I thought I'd seen other resource types munging title values like this.

Is it possible to munge the title values in a way that works for both scenarios? If not then I'm not sure whether it would be better to leave it case-sensitive since that is what users will see in the switch config, or to "help" them avoid errors in the manifest.

↧
↧

Re: How to configure host chaining for ConnectX-5 VPI

August 30, 2018, 12:38 am
$
0
0

it still does not work
What drivers are you using?

↧

Factors that determine compatibility of SFPs with new fibre services?

August 30, 2018, 1:02 am
$
0
0
  1. 1.Whilst I understand that product recommendations are off topic can anyone help by explaining what the critical factors are when looking for SFPs that are going to be compatible with a new service?
  2. 2.Is wavelength a defining factor that should be considered/matched or should anything else be used to guide selection?

Sorry I am new to 10G BASE-SR and I can't seem to find a good resource that can confirm if an SFP supported in a Cisco Nexus 5548UP will be compatible with a new service The new service is described as '10 Gigabit Ethernet LAN PHY IEEE 10G BASE-LR10.3125 Gbps +/- 100 ppm 1310nm'
Ultimately I need to understand if a 'cisco sfp-10g-sr' for which the transmitter wavelength spec is described as 850nm is usable.

Thanks for your patient, i plan to take it and any site recommend?

Anyone talk form sfpcables? 10Gb/s SFP+ SR | SFP-10G-SR | J9150A | SFP+ 10GBase-SR - 10Gtek

5% OFF all items there now, i want to save a lot there.

↧

ConnectX-2 on Ubuntu 18.04

August 30, 2018, 6:08 am
$
0
0

Hello-

 

I have a number of older servers with ConnectX-2 VPI IB cards running a very old OS (Ubuntu 10.04). We want to take them to Ubuntu 18.04 instead. As I understand it, Mellanox is only providing Ubuntu 18.04 installers for OFED v 4.x, however support for ConnectX-2 cards was dropped in this version of OFED. So it seems I need to get OFED 3.x installed on Ubuntu 18. Has anyone had any success with that, or could please offer some advice? Or is there any hope an official installer could be provided? I see OFED 3 is supported on Ubuntu 16, but it'd be great if we could use the newer OS for 2 years more longevity. Thanks for any advice!

 

-Lewis

↧

NetworkDirect API: Loading of "NDv2 Provider for Mellanox WinOF-2" requires admin privileges

August 30, 2018, 6:29 am
$
0
0

Hello everybody,

 

I have an issue in my NetworkDirect RDMA application when loading the Mellanox NDv2 provider. It seems as the newer WinOF-2 driver for MLX-5 IB adapters (mlx5nd.dll) requires that the connecting process has adminstrator privileges.

Because when running my application with normal user privileges, I get an error0x80070005 (Access denied), whereas this has never been an issue with the older WinOF driver for MLX-4 adapters (mlx4nd.dll).

Here the failing code sequence from my ndhelper.cpp:

 

static HMODULE      g_hProvider  = NULL;

static IND2Provider g_pIProvider = NULL;

 

static HRESULT LoadProvider( __in WSAPROTOCOL_INFOW* pProtocol )

{

    WCHAR* pPath = ::GetProviderPath( pProtocol ); // %SystemRoot%\System32\mlx5nd.dll

     g_hProvider = ::LoadLibraryW( pPath );

    ::HeapFree( ::GetProcessHeap(), 0, pPath );

   

    DLLGETCLASSOBJECT pfnDllGetClassObject = reinterpret_cast<DLLGETCLASSOBJECT>(

        ::GetProcAddress( g_hProvider, "DllGetClassObject" )

    );

 

    DLLGETCLASSOBJECT pfnDllCanUnloadNow = reinterpret_cast<DLLCANUNLOADNOW>(

        ::GetProcAddress(g_hProvider, "DllCanUnloadNow")

    );

 

    IClassFactory* pClassFactory;

 

    HRESULT hr = pfnDllGetClassObject(

        pProtocol->ProviderId,

        IID_IClassFactory,

        reinterpret_cast<void**>(&pClassFactory)

    );

 

    if (g_pIProvider == NULL) {

        hr = pClassFactory->CreateInstance(

            NULL,

            IID_IND2Provider,

            reinterpret_cast<void**>(&g_pIProvider)

        );

        if (FAILED(hr)) {

            TRACE("ClassFactory->CreateInstance(IID_IND2Provider) failed with error 0x%08X", hr); // Without having admin rights, always ending up here!

            g_pIProvider = NULL;

        }

        pClassFactory->Release();

    }

}

 

Unfortunately, just giving my process admin privileges is not an option for me. So I would appreciate if someone has an idea how to overcome this issue.

Perhaps some tuning of security configuration via dcomcnfg or the like(?)

 

BTW: The Mellanox-provided tools nd_read_bw.exe and nd_write_bw.exe have the same behavior:

  • Those from WinOF driver 5.35 can run WITH or WITHOUT admin privileges.
  • Those from WinOF-2 driver 1.80 can run only WITH admin privileges.

 

 

My system configuration:

  • HPE DL380Gen10
  • MLX-5 EDR Infiniband HCA (HPEIB EDR/EN 100Gb 2P 840QSFP28)
  • Windows Server 2012 R2
  • Mellanox WinOF-2 version 1.80

 

Thanks and Regards

↧
↧

Tagged Ethernet interface (VLAN PFC) fails to activate on Ubuntu 16.04

August 30, 2018, 4:37 pm
$
0
0

Environment:

  • Ubuntu 16.04, Kernel 4.15.0-29-generic
  • MLNX_OFED_LINUX-4.3-1.0.1.0:
  • MCX416A-CCAT

 

Configured PFC with VLAN 100 and Priority 3. While trying to enable the interface, I get:

root@sm16:~# nmcli con up ens1f0.100

Error: Connection activation failed: Failed to find a compatible device for this connection

 

Any pointers?

 

This is how the configuration of the tagged and untagged interfaces look like:

 

root@sm16:~# nmcli c s

NAME        UUID                                  TYPE            DEVICE

ens1f0      e5251358-dffb-4e07-b72f-b9e93ca6eca8  802-3-ethernet  --

ens1f0.100  10c8d003-164a-4a8d-a21f-ff6bb712a090  vlan            --

 

# nmcli con s ens1f0.100 | less

connection.id:                          ens1f0.100

connection.uuid:                        10c8d003-164a-4a8d-a21f-ff6bb712a090

connection.interface-name:              ens1f0

connection.type:                        vlan

connection.autoconnect:                 yes

connection.autoconnect-priority:        0

connection.timestamp:                   0

connection.read-only:                   no

connection.permissions:

connection.zone:                        --

connection.master:                      --

connection.slave-type:                  --

connection.autoconnect-slaves:          -1 (default)

connection.secondaries:

connection.gateway-ping-timeout:        0

connection.metered:                     unknown

connection.lldp:                        -1 (default)

802-3-ethernet.port:                    --

802-3-ethernet.speed:                   0

802-3-ethernet.duplex:                  --

802-3-ethernet.auto-negotiate:          yes

802-3-ethernet.mac-address:             --

802-3-ethernet.cloned-mac-address:      --

802-3-ethernet.mac-address-blacklist:

802-3-ethernet.mtu:                     4200

802-3-ethernet.s390-subchannels:

802-3-ethernet.s390-nettype:            --

802-3-ethernet.s390-options:

802-3-ethernet.wake-on-lan:             1 (default)

802-3-ethernet.wake-on-lan-password:    --

ipv4.method:                            manual

ipv4.dns:

ipv4.dns-search:

ipv4.dns-options:                       (default)

ipv4.dns-priority:                      0

ipv4.addresses:                         10.0.20.75/24

ipv4.gateway:                           --

ipv4.routes:

ipv4.route-metric:                      -1

ipv4.ignore-auto-routes:                no

ipv4.ignore-auto-dns:                   no

ipv4.dhcp-client-id:                    --

ipv4.dhcp-timeout:                      0

ipv4.dhcp-send-hostname:                yes

ipv4.dhcp-hostname:                     --

ipv4.dhcp-fqdn:                         --

ipv4.never-default:                     no

ipv4.never-default:                     no

ipv4.may-fail:                          yes

ipv4.dad-timeout:                       -1 (default)

ipv6.method:                            auto

ipv6.dns:

ipv6.dns-search:

ipv6.dns-options:                       (default)

ipv6.dns-priority:                      0

ipv6.addresses:

ipv6.gateway:                           --

ipv6.routes:

ipv6.route-metric:                      -1

ipv6.ignore-auto-routes:                no

ipv6.ignore-auto-dns:                   no

ipv6.never-default:                     no

ipv6.may-fail:                          yes

ipv6.ip6-privacy:                       -1 (unknown)

ipv6.addr-gen-mode:                     stable-privacy

ipv6.dhcp-send-hostname:                yes

ipv6.dhcp-hostname:                     --

vlan.parent:                            ens1f0

vlan.id:                                100

vlan.flags:                             1 (REORDER_HEADERS)

vlan.ingress-priority-map:

vlan.egress-priority-map:               0:3,1:3,2:3,3:3,4:3,5:3,6:3,7:3

 

# nmcli con s ens1f0| less

connection.id:                          ens1f0

connection.uuid:                        e5251358-dffb-4e07-b72f-b9e93ca6eca8

connection.interface-name:              ens1f0

connection.type:                        802-3-ethernet

connection.autoconnect:                 yes

connection.autoconnect-priority:        0

connection.timestamp:                   0

connection.read-only:                   no

connection.permissions:

connection.zone:                        --

connection.master:                      --

connection.slave-type:                  --

connection.autoconnect-slaves:          -1 (default)

connection.secondaries:

connection.gateway-ping-timeout:        0

connection.metered:                     unknown

connection.lldp:                        -1 (default)

802-3-ethernet.port:                    --

802-3-ethernet.speed:                   0

802-3-ethernet.duplex:                  --

802-3-ethernet.auto-negotiate:          yes

802-3-ethernet.mac-address:             --

802-3-ethernet.cloned-mac-address:      --

802-3-ethernet.mac-address-blacklist:

802-3-ethernet.mtu:                     4200

802-3-ethernet.s390-subchannels:

802-3-ethernet.s390-nettype:            --

802-3-ethernet.s390-options:

802-3-ethernet.wake-on-lan:             1 (default)

802-3-ethernet.wake-on-lan-password:    --

ipv4.method:                            auto

ipv4.dns:

ipv4.dns-search:

ipv4.dns-options:                       (default)

ipv4.dns-priority:                      0

ipv4.addresses:

ipv4.gateway:                           --

ipv4.routes:

ipv4.route-metric:                      -1

ipv4.ignore-auto-routes:                no

ipv4.ignore-auto-dns:                   no

ipv4.dhcp-client-id:                    --

ipv4.dhcp-timeout:                      0

ipv4.dhcp-send-hostname:                yes

ipv4.dhcp-hostname:                     --

ipv4.dhcp-fqdn:                         --

ipv4.never-default:                     no

ipv4.may-fail:                          yes

ipv4.dad-timeout:                       -1 (default)

ipv6.method:                            auto

ipv6.dns:

ipv6.dns-search:

ipv6.dns-options:                       (default)

ipv6.dns-priority:                      0

ipv6.addresses:

ipv6.gateway:                           --

ipv6.routes:

ipv6.route-metric:                      -1

ipv6.ignore-auto-routes:                no

ipv6.ignore-auto-dns:                   no

ipv6.never-default:                     no

ipv6.may-fail:                          yes

ipv6.ip6-privacy:                       -1 (unknown)

ipv6.addr-gen-mode:                     stable-privacy

ipv6.dhcp-send-hostname:                yes

ipv6.dhcp-hostname:                     --

↧

Re: mlx5: ethtool -m not working

August 30, 2018, 4:38 pm
$
0
0

Hi Ryan,

 

I am getting the same end result with RHEL 7.5, Kernel 4.9, ethtool version 4.8 for Connectx-4 & Connectx-3.

I am however able to retrieve the information on a Connectx-5. 

 

You can download the MFT package and use the "mlxcables" utility.

http://www.mellanox.com/page/management_tools

 

Note: see section mst cable add first.

 

Note: Latest FW for that HCA card is 12.23.1020.

 

Sophie.

 

 

 

↧
Search

Re: Soft-RoCE on mininet topology

August 30, 2018, 4:51 pm
$
0
0

Hi Suraj,

Thank you for posting your question on the Mellanox Community.

Based on your information provided (We also noticed your posting on https://mailman.stanford.edu/pipermail/mininet-discuss/2018-August/008031.html), currently we do hot have support for running Soft-RoCE on a Mininet topology. Currently we only provided a method of running from a nic-2-nic.

Also please note that Soft-RoCE is still in BETA

We would recommend to post this question on the Mininet Mailings-list as you already did. Maybe somebody from the mailings-list has a solution for this as they need to implement this into the Mininet Framework

Thanks and regards,
~Mellanox Technical Support

↧

Re: Header Data Split

August 30, 2018, 5:04 pm
$
0
0

Hi Kenneth,

Thank you for posting your question on the Mellanox Community.

We have noticed that you also opened a Mellanox Support case regarding this issue and that we provided you access to the PRM's of the ConnectX-5

If you need anything, please do not hesitate to open a new support case by sending an email to support@mellanox.com

Thanks and regards,
~Mellanox Technical Support

↧
↧

Re: Can't ibping Lid or GUID but can ping by ip

August 29, 2018, 7:27 am
$
0
0

Is there any error messages? Does ibtracert work (#ibtracert <src lid> <dst lid>?

↧

Re: Can't ibping Lid or GUID but can ping by ip

August 31, 2018, 5:56 am
$
0
0

Hi Brian,

When using virtualization, GRH (global routing header) must be present in the packet. For ibping, --dgid <GID> parameter need to be used (see man ibping).

To get GIDs, on the server run 'show_gids' and use the output on the client side

Server

#show_gids

DEV     PORT    INDEX   GID                                     IPv4            VER     DEV

---     ----    -----   ---                                     ------------    ---     ---

mlx5_1  1       0       fe80:0000:0000:0000:248a:0703:009c:01a7                 v1

 

Client

#ibping --dgid  fe80:0000:0000:0000:248a:0703:009c:01a7 18

 

If you like to check RDMA connectivity between VMs, use utilities from perftest package (ib_read_bw, ib_write_bw, etc) with -R parameter.

↧

Re: ASAP2 Live Migration & H/W LAG

September 2, 2018, 8:06 pm
$
0
0

Could you tell me the detail for your application ? why you need such feature ?

↧
Viewing all 6275 articles
Browse latest View live
Search
<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>