Quantcast
Channel: Mellanox Interconnect Community: Message List
Viewing all 6275 articles
Browse latest View live

DPDK-L3FWD PROX Compilation Error :: mlx5_mr.c:39:30: fatal error: infiniband/verbs.h: No such file or directory

$
0
0

Hi,

 

I am trying to compile DPDK PROX L3FWD application for the mlx4 driver and is running into following compilation issues. I have followed the steps from following link as reference.

http://www.mellanox.com/related-docs/prod_software/MLNX_DPDK_Quick_Start_Guide_v2.2_4.2.pdf

 

CC mlx5_ethdev.o

  CC mlx5_mac.o

  CC mlx5_rxq.o

  CC mlx5_vlan.o

  CC mlx5_rxmode.o

  CC mlx5_mr.o

  CC mlx5_rss.o

  CC mlx5_fdir.o

  CC mlx5_stats.o

/usr/src/dpdk/drivers/net/mlx5/mlx5_mr.c:39:30: fatal error: infiniband/verbs.h: No such file or directory

#include <infiniband/verbs.h>

                              ^

compilation terminated.

make[6]: *** [mlx5_mr.o] Error 1

make[6]: *** Waiting for unfinished jobs....

/usr/src/dpdk/mk/internal/rte.compile-pre.mk:126: recipe for target 'mlx5_mr.o' failed

  CC mlx4.o

 

 

I do have the following file and is accessible. There is no other verbs.h in the system.

root@root# ls /usr/include/infiniband/verbs.h

/usr/include/infiniband/verbs.h

root@tbserver15:/usr# find . -name verbs.h

./include/infiniband/verbs.h

root@tbserver15:/usr#

MLNX_OFED_LINUX-3.4-2.0.0.0-ubuntu16.04-x86_64.iso is installed in my system.

 

And firmware version is as below.

root@root:/usr# ibstat

CA 'mlx4_0'

        CA type: MT4103

        Number of ports: 2

        Firmware version: 2.37.7190

        Hardware version: 0

 

I have extracted the DPDK package from www.mellanox.com/downloads/Drivers/MLNX_DPDK_2.2_4.2.tar.gz

 

I am using following Docker file to generate docker image.

 

FROM ubuntu:vivid

 

RUN apt-get update && apt-get install -y --no-install-recommends \

  gcc build-essential make curl git liblua5.2-dev libedit-dev libpcap-dev libncurses5-dev libncursesw5-dev  pkg-config vim \

  && apt-get clean && rm -rf /var/lib/apt/lists/*

 

ENV DPDK_VERSION=2.2.0 \

  RTE_SDK=/usr/src/dpdk \

  RTE_TARGET=x86_64-native-linuxapp-gcc \

  PROX_COMMIT=a52953c4a92ecfcfdd06fc01fce42b57f998cb10 \

  PROX_DIR=/usr/src/PROX

 

RUN curl -sSL  www.mellanox.com/downloads/Drivers/MLNX_DPDK_2.2_4.2.tar.gz | tar -xz; \

  mv MLNX_DPDK_2.2_4.2 ${RTE_SDK}

 

# don't build kernel modules

RUN sed -i s/CONFIG_RTE_EAL_IGB_UIO=y/CONFIG_RTE_EAL_IGB_UIO=n/ ${RTE_SDK}/config/common_linuxapp \

  && sed -i s/CONFIG_RTE_LIBRTE_KNI=y/CONFIG_RTE_LIBRTE_KNI=n/ ${RTE_SDK}/config/common_linuxapp \

  && sed -i s/CONFIG_RTE_LIBRTE_MLX4_PMD=n/CONFIG_RTE_LIBRTE_MLX4_PMD=y/ ${RTE_SDK}/config/common_linuxapp \

  && sed -i s/CONFIG_RTE_KNI_KMOD=y/CONFIG_RTE_KNI_KMOD=n/ ${RTE_SDK}/config/common_linuxapp

 

# don't build unnecessary stuff

RUN sed -i s/CONFIG_RTE_APP_TEST=y/CONFIG_RTE_APP_TEST=n/ ${RTE_SDK}/config/common_linuxapp \

  && sed -i s/CONFIG_RTE_TEST_PMD=y/CONFIG_RTE_TEST_PMD=n/ ${RTE_SDK}/config/common_linuxapp

 

RUN cd ${RTE_SDK} \

  && make install T=${RTE_TARGET} DESTDIR=install -j

 

RUN git config --global http.sslVerify false

 

# clone PROX from the repository

RUN git clone http://github.com/nvf-crucio/PROX.git \

  && cd PROX \

  && git checkout ${PROX_COMMIT} \

  && cd .. \

  && mv PROX ${PROX_DIR}

# build PROX

RUN cd ${PROX_DIR} \

  && make

 

WORKDIR ${PROX_DIR}/build/

 

Can some one let me know how to proceed further?


Re: How to make libvma working in chroot?

Re: MCX313A-BCCT/MCX415A-BCAT SETUP

$
0
0

You have not provided detailed information of the cable & nics (part number, PSID etc), but in general, to be able to connect properly a Mellanox CX-3 adapter to a Mellanox CX-4 adapter over an 40g optical-cable, it  requires:

  1. 1. to ensure that the QSFP 40G AOC cable is supported by both adapters (you have not specified the full details of the cable (part-number etc...)
  2. 2. to ensure compatible NIC fw & WinOF/WinOF-2 versions for each win2012R2 server edge
  3. 3. proprietary port configuration

suggesting to approach to our Mellanox website to get all the information above

http://www.mellanox.com/index.php

In addition, you can open a support-call by mailing our support@mellanox.com, place a detailed description and we'll reply to you accordingly

 

Re: MSX6025F

$
0
0

Hello Thomas,

 

I have done some research on this.

Mellanox is Partners with Hewlett Packard Enterprise, and MSX6025F-2SFS is provided by HPE and MSX6025F-1SFS provided by Mellanox.

Beside that the switches are the same.

Re: DPDK-L3FWD PROX Compilation Error :: mlx5_mr.c:39:30: fatal error: infiniband/verbs.h: No such file or directory

$
0
0

Hi Praveen,

 

If you are trying to compile it for the mlx4 driver, meaning for connectX-3 cards, you do not need the mlx5 packages.

You should set to No all the mlx5 parameters before starting the compilation.

 

Regards,

Viki

 

Re: MSX6025F

$
0
0

Hi Thomas,

 

The different between MSX6025F-1SFS to MSX6025F-2SFS is the number of PSU's coming with the switch.

Re: DPDK-L3FWD PROX Compilation Error :: mlx5_mr.c:39:30: fatal error: infiniband/verbs.h: No such file or directory

$
0
0

Hi Praveen,

If you'll still encounter any issue even after removing the mlx5 parameters, I suggest to open a new case to Mellanox support.

You can open a new support ticket by sending an email to support@mellanox.com

 

Regards,

Viki

SX6710 Keeps Shutting Down

$
0
0

I have an SX6710 in our lab - first exposure to Mellanox - and it keeps going into what I call "Jet Engine Mode" as the fans are spinning so fast it sounds like a plane ready to take off. Even though there is still power and the fans are spinning out of control, all of the ports are off and the web interface is inaccessible as well even though I can still ping the management port that the web is connected to.

 

I'm trying to parse through the logs to see if I can figure out why it is shutting down but being completely unfamiliar, I'm not making too much sense of it. When I look at the log after rebooting (I need to pull the power plugs to reboot) I can clearly see the system reboot message. When I start looking backwards from there, I can clearly find where the fan max threshold is set to 18000 and I can see statsd.ERR messages saying the Health Daemon is already performing a sweep. Searching backwards from there I can find a health.ERR showing Health Daemon  ISR scan failed.

 

Again, being brand new to Mellanox, I'd appreciate any advice or pointer to the direction I should be looking as I'm not even sure I'm in the correct rabbit hole at this point.

 

Thanks!!!


Re: DPDK-L3FWD PROX Compilation Error :: mlx5_mr.c:39:30: fatal error: infiniband/verbs.h: No such file or directory

$
0
0

Hi Viki,

 

Thanks for the support.

 

With the changes suggested by you, I have disabled the mlx5 pmd in the config file and tried to compile the code, but encountered many errors. Please find below a snippet of the same.

 

I have changed the value of CONFIG_RTE_LIBRTE_MLX5_PMD=n  in config/common_linuxapp to ensure that the MLX5 package is not taken while compiling.

 

Can you please guide me to proceed further?

 

/**************************************************************************************************************************************/

 

/usr/src/dpdk/drivers/net/mlx4/mlx4.c:3749:33: error: unused variable 'status' [-Werror=unused-variable]

  enum ibv_exp_query_intf_status status;

                                 ^

/usr/src/dpdk/drivers/net/mlx4/mlx4.c:3748:4: error: variable 'attr' set but not used [-Werror=unused-but-set-variable]

  } attr;

    ^

/usr/src/dpdk/drivers/net/mlx4/mlx4.c:3743:25: error: unused variable 'mod' [-Werror=unused-variable]

  struct ibv_exp_qp_attr mod;

                         ^

/usr/src/dpdk/drivers/net/mlx4/mlx4.c:3734:29: error: unused parameter 'inactive' [-Werror=unused-parameter]

    unsigned int socket, int inactive, const struct rte_eth_rxconf *conf,

                             ^

/usr/src/dpdk/drivers/net/mlx4/mlx4.c: In function 'rxq_setup_qp':

/usr/src/dpdk/drivers/net/mlx4/mlx4.c:3438:1: error: control reaches end of non-void function [-Werror=return-type]

}

^

cc1: all warnings being treated as errors

/usr/src/dpdk/mk/internal/rte.compile-pre.mk:126: recipe for target 'mlx4.o' failed

make[6]: *** [mlx4.o] Error 1

/usr/src/dpdk/mk/rte.subdir.mk:61: recipe for target 'mlx4' failed

make[5]: *** [mlx4] Error 2

/usr/src/dpdk/mk/rte.subdir.mk:61: recipe for target 'net' failed

make[4]: *** [net] Error 2

/usr/src/dpdk/mk/rte.sdkbuild.mk:77: recipe for target 'drivers' failed

make[3]: *** [drivers] Error 2

/usr/src/dpdk/mk/rte.sdkroot.mk:123: recipe for target 'all' failed

make[2]: *** [all] Error 2

/usr/src/dpdk/mk/rte.sdkinstall.mk:84: recipe for target 'pre_install' failed

make[1]: *** [pre_install] Error 2

/usr/src/dpdk/mk/rte.sdkroot.mk:98: recipe for target 'install' failed

 

/**************************************************************************************************************************************/

DPDK-L3FWD PROX Compilation Error with MLX5 params disabled

$
0
0

Hi Team,

 

This has further reference to the thread DPDK-L3FWD PROX Compilation Error :: mlx5_mr.c:39:30: fatal error: infiniband/verbs.h: No such file or directory

 

Based on the resolution provided in the above thread, I have disabled the mlx5 pmd in the config file and tried to compile the code, but encountered many errors. Please find below a snippet of the errors. To make sure that the mlx5 package is not taken, I have changed the value of CONFIG_RTE_LIBRTE_MLX5_PMD=n  in config/common_linuxapp so that its not taken while compiling.

 

Can you please guide me to proceed further?

 

/**************************************************************************************************************************************/

 

/usr/src/dpdk/drivers/net/mlx4/mlx4.c:3749:33: error: unused variable 'status' [-Werror=unused-variable]

  enum ibv_exp_query_intf_status status;

                                 ^

/usr/src/dpdk/drivers/net/mlx4/mlx4.c:3748:4: error: variable 'attr' set but not used [-Werror=unused-but-set-variable]

  } attr;

    ^

/usr/src/dpdk/drivers/net/mlx4/mlx4.c:3743:25: error: unused variable 'mod' [-Werror=unused-variable]

  struct ibv_exp_qp_attr mod;

                         ^

/usr/src/dpdk/drivers/net/mlx4/mlx4.c:3734:29: error: unused parameter 'inactive' [-Werror=unused-parameter]

    unsigned int socket, int inactive, const struct rte_eth_rxconf *conf,

                             ^

/usr/src/dpdk/drivers/net/mlx4/mlx4.c: In function 'rxq_setup_qp':

/usr/src/dpdk/drivers/net/mlx4/mlx4.c:3438:1: error: control reaches end of non-void function [-Werror=return-type]

}

^

cc1: all warnings being treated as errors

/usr/src/dpdk/mk/internal/rte.compile-pre.mk:126: recipe for target 'mlx4.o' failed

make[6]: *** [mlx4.o] Error 1

/usr/src/dpdk/mk/rte.subdir.mk:61: recipe for target 'mlx4' failed

make[5]: *** [mlx4] Error 2

/usr/src/dpdk/mk/rte.subdir.mk:61: recipe for target 'net' failed

make[4]: *** [net] Error 2

/usr/src/dpdk/mk/rte.sdkbuild.mk:77: recipe for target 'drivers' failed

make[3]: *** [drivers] Error 2

/usr/src/dpdk/mk/rte.sdkroot.mk:123: recipe for target 'all' failed

make[2]: *** [all] Error 2

/usr/src/dpdk/mk/rte.sdkinstall.mk:84: recipe for target 'pre_install' failed

make[1]: *** [pre_install] Error 2

/usr/src/dpdk/mk/rte.sdkroot.mk:98: recipe for target 'install' failed

 

/**************************************************************************************************************************************/

 

Thanks

Praveen

Re: MCX313A-BCCT/MCX415A-BCAT SETUP

$
0
0

Hi aviap,

 

Thank you for your response.  I am trying to connect a MC2210310-050 going from MCX313A-BBCT (Driver version: 5.22.12447.0) to MCX415A-BCAT (Driver version: 1.10.13423.0). 

 

I was able to established a link going from MCX313A-BBCT to HPE 5900 (1X40G PORT CONFIGURED) so I know there's nothing wrong with the adapter card.

 

I used the same cable going from the MCX415A-BCAT to the same HPE5900 (1X40G PORT CONFIGURED) and there wasn't any link at all. 

 

I hope this will help.  Thank you in advance.

Re: How to make libvma working in chroot?

$
0
0

no I cant. It looks like chroot need support software (I mean like connectx_config_port, etc). I installed it. And it conflicts with openibd at parent system. also I cant retake control to chrooted openibd.

Re: How to make libvma working in chroot?

$
0
0

I'll test it and reply. tested yet only on 3.4.2/1

Re: SX6710 Keeps Shutting Down

$
0
0

Hi Angelo,

 

If this issue is affecting your production, then please email Mellanox support at support@mellanox.com so the issue can be properly examined.

 

Thanks,

Christeen

ib0: error fetching interface information: Device not found

$
0
0

Hi

 

I have a Infiniband controller: Mellanox Technologies MT27600 [Connect-IB] in HP blades running CentOS6.8

ifconfig ib0 keeps giving the error "fetching interface information: Device not found"

 

I restarted the rdma service with no luck.

#service rdma status

Low level hardware support loaded:

        mlx5_ib mlx5_core

 

Upper layer protocol modules:

        ib_ipoib

 

User space access modules:

        rdma_ucm ib_ucm ib_uverbs ib_umad

 

Connection management modules:

        rdma_cm ib_cm iw_cm

 

Configured IPoIB interfaces: ib0

 

Currently active IPoIB interfaces: none

 

Thank you in advance


Re: SX6710 Keeps Shutting Down

$
0
0

Appreciate that Christeen but as I mentioned, it's in our lab. It's a pain in the butt to reboot all the time and sometimes it runs for 3 or 4 days with no issues and other times we reboot it 4-5 times in one day. I'm pretty sure the answer is somewhere in the log but I am just unfamiliar with them and nothing is jumping out when I search through them for an error.

MLNX OFED error for RHEL 7.3 RT

$
0
0

Hello,

I get an error when I try to install MLNX OFED for RHEL 7.3 Real time kernel.

For normal kernel RHEL 7.3 I can install OFED without any problems. What can be the problem? Btw. I installed kernel-devel and added –add-kernel-support but I got below ERROR after installation:

 

ERROR: Failed executing "MLNX_OFED_SRC-3.4-2.0.0.0/install.pl --tmpdir /tmp/MLNX_OFED_LINUX-3.4-2.0.0.0-3.10.0-514.2.2.rt56.424.el7.x86_64 --kernel-only --kernel 3.10.0-514.2.2.rt56.424.el7.x86_64 --kernel-sources /lib/modules/3.10.0-514.2.2.rt56.424.el7.x86_64/build --builddir /tmp/MLNX_OFED_LINUX-3.4-2.0.0.0-3.10.0-514.2.2.rt56.424.el7.x86_64/mlnx_iso.21610 --disable-kmp --force --build-only --distro rhel7.3"

ERROR: See /tmp/MLNX_OFED_LINUX-3.4-2.0.0.0-3.10.0-514.2.2.rt56.424.el7.x86_64/mlnx_ofed_iso.21610.log

Failed to build MLNX_OFED_LINUX for 3.10.0-514.2.2.rt56.424.el7.x86_64

 

I have Mellanox Connectx-3 VPI

What is the recommended firmware updating tool for Mellanox HCAs and/or NICs?

$
0
0

Happy New Year to all!

 

With the clusters that I manage, I have to take care of many different Mellanox IB and Ethernet switches, HCAs and NICs.  Some examples:

  • MSB7700-ES2F EDR IB 1U switches
  • MCX455A-ECAT ConnectX(r)-4 VPI EDR IB and 100GbE
  • MCB194A-FCAT Connect-IB(r) HCA dual-port FDR 56Gb/s
  • MCX383A-FCNA ConnectX(r)-3 VPI Single-port FDR IB and 40GbE I/O cards
  • MSN2410-CB2F Spectrum 25G/100G 1U switches
  • MCX4121A-ACAT ConnectX(r)-4 Lx EN 25G PCIe 3x8 cards
  • [...]

 

As a rule, I strive to keep their firmware always the most current.  Here is an area where I would like to have Mellanox's input: what's the recommended tool to keep an HCA/NIC's firmware up to date?  As far as I can see, there are four options:

 

  1. If the MLNX_OFED is used (I have), then the mlnxofedinstall Perl script has a few options to update an adapter's firmware
  2. If sufficient inbox IB/RDMA packages are installed, (e.g. the RHEL/CentOS "Infiniband Support" group), thus the mstflint package is available, then it can be used.
  3. The Mellanox also provides the MFT suite.
  4. There is also this Mellanox mlxup.

 

Personally, I have come to the conclusion that mlxup is the simplest and the most flexible.  But am I correct?  Love to hear Mellanox colleagues' comments.

 

Regards,

 

Chin

A few suggestions to Mellanox MLNX_OFED team

$
0
0

Having managed HPC clusters (exclusively servers running RHEL/CentOS) in the national lab environment for quite a few years by now, and with extensive experience with both MLNX_OFED and its inbox counterparts, I would like to make a few suggestions:

 

  1. Although I like the MLNX_OFED's enhancements and the fact that it's a bundle of "everything", but the need to rebuild the RPMs for even minor OS updates is a bore!   As such, on most of the servers that I manage (and my colleagues too), we still use inbox packages for IB-enabled servers.
  2. On servers that I manage, I tend to install minimum number of packagesThis practice is simply logical: the less are installed, the less can go wrong.  The "whole sale install" approach coded in mlxofedinstall thus contradicts this best practice. I must get into the code to "defeat" it. BTW, I do the same to RHEL/CentOS - NEVER do a yum -y groupinstall "Infiniband Support". The majority of the packages are useless to a typical IB-enabled server!  Example: why you need libcxgb3 and libcxgb4 on IB-enabled servers that use only Mellanox IB HCAs?  Is it a good will gesture to Cheliso?
  3. The MLNX_OFED's mlxofedinstall Perl scripts enhances the /etc/security/limits.conf directly, rather than putting a snippet in /etc/security/limits.d subdirectory.  Sure this is minor, but if the kernel developers have decided the best practice is to use a snippets (/etc/sudoers.d, /etc/sysctl.d come to my mind, and many more), why not follow this convention?

 

Regards,

 

Chin

Re: A few suggestions to Mellanox MLNX_OFED team

$
0
0

Hi Chin,

 

We definitely agree that in some cases not all the packages are needed and even redundant, for that we have the option to use a "slim" installation.

In order to use it you can run: "mlnxofedinstall -p", then a conf file called /tmp/ofed-all.conf will be created with all the packages included.

You can choose which packages you need and which not and then run the installation script as following:./mlnxofedinstall -c /tmp/ofed-all.conf and only the packages that you chose will be installed.

This is one option.

You have another option- to install the driver from source, meaning to install only the source RPMs from each release.

This can be done by using the installation script inside the tarball located under the /src directory (MLNX_OFED_SRC).

Here you also have the -p and -c flags for even slimmer installation.

 

Hope this information helps.

 

Regards,

Viki

Viewing all 6275 articles
Browse latest View live


<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>