I afraid it is tricky to understand without debugging and source code, but most likely there is a problem there. Try to print the values of lkey/rkey, QP numbers, protection domains, other related data in every packet you send/receive. Check that a receiver is running first, before sending posting its send request. What is you run only one sender and one receiver, does it work?
Regarding performance, you might check the Mellanox Tuning guide. What is the test you are using, it is part of 'perftest'?
And probably most important question - do you use Mellanox OFED or this is inbox driver? is it Linux?