python-rdma icon indicating copy to clipboard operation
python-rdma copied to clipboard

FDR10 is wrongly displayed as QDR

Open 0blu opened this issue 8 years ago • 5 comments

Hey, unfortunately FDR10 connections are displayed as QDR in ibtool.

$ ibtool ibnetdiscover
[...]
[1]	"S-<GUID>"[5]	# "desc" lid 2 4xQDR
[...]
$ ibnetdiscover
[...]
[1]	"S-<GUID>"[5]	# "desc" lid 2 4xFDR10
[...]

0blu avatar Mar 21 '17 11:03 0blu

FDR10 is a non-standard Mellanox extension, I never looked into how to support it. If you have hardware that can support it, perhaps you could send a patch?

jgunthorpe avatar Mar 21 '17 16:03 jgunthorpe

Thanks for the quick response. Unfortunately i hardly have time to look into the IBA_struct.py code.

If you could make a new branch and add these:

	IB_MLNX_EXT_PORT_STATE_CHG_ENABLE_F,
	IB_MLNX_EXT_PORT_LINK_SPEED_SUPPORTED_F,
	IB_MLNX_EXT_PORT_LINK_SPEED_ENABLED_F,
	IB_MLNX_EXT_PORT_LINK_SPEED_ACTIVE_F,
	IB_MLNX_EXT_PORT_LAST_F,

https://github.com/chu11/libibmad-snapshot/blob/master/include/infiniband/mad.h#L1070-L1074 into IBA_struct.py i could test it and verify the result on my fabric. https://github.com/jgunthorpe/python-rdma/blob/master/rdma/IBA_struct.py

After this we can compare IB_MLNX_EXT_PORT_LINK_SPEED_ACTIVE_F & 0x8(?) if its really FDR10

0blu avatar Mar 22 '17 09:03 0blu

The information in libibmad does not seem to be complete. What is the attribute ID of the vendor structure, what operation is used to read it? The layout in libibmad is also incomplete, lots of holes.

Adding a new struct is easy, if you know what is in it, see for instance:

https://github.com/jgunthorpe/python-rdma/blob/master/codegen/vend_mlx.xml

Can you find a publication where mellanox documents this MAD?

jgunthorpe avatar Mar 22 '17 17:03 jgunthorpe

How did you generate/get the vend_mlx.xml? The file is ~6 Years old, maybe its deprecated.

0blu avatar Mar 23 '17 11:03 0blu

It is hand written from the specification of the wire format. I expect that specific mad is out of date, but something that looks basically like it is what is needed to add the new mellanox vendor mad, I just don't have enough information to do it.

jgunthorpe avatar Mar 23 '17 16:03 jgunthorpe