[gpfsug-discuss] RDMA data from Zimon

Eric Agar agar at us.ibm.com
Thu Feb 15 17:08:08 GMT 2018


Kristy,

I experimented a bit with this some months ago and looked at the ZIMon
source code.  I came to the conclusion that ZIMon is reporting values
obtained from the IB counters (actually, delta values adjusted for time)
and that yes, for port_xmit_data and port_rcv_data, one would need to
multiply the values by 4 to make sense of them.

To obtain a port_xmit_data value, the ZIMon sensor first looks
for /sys/class/infiniband/<ibdev>/ports/<port>/counters_ext/port_xmit_data_64,
 and if that is not found then looks
for /sys/class/infiniband/<ibdev>/ports/<port>/counters/port_xmit_data.
Similarly for other counters/metrics.

Full disclosure: I am not an IB expert nor a ZIMon developer.

I hope this helps.


Eric M. Agar
agar at us.ibm.com




From:	Kristy Kallback-Rose <kkr at lbl.gov>
To:	gpfsug main discussion list <gpfsug-discuss at spectrumscale.org>
Date:	02/14/2018 08:47 PM
Subject:	[gpfsug-discuss] RDMA data from Zimon
Sent by:	gpfsug-discuss-bounces at spectrumscale.org



Hi,

Can one of the IBMers tell me if port_xmit_data and port_rcv_data from
Zimon can be interpreted as RDMA Bytes/sec? Ideally, also how this data is
being collected? I’m looking here:
https://www.ibm.com/support/knowledgecenter/en/STXKQY_5.0.0/com.ibm.spectrum.scale.v5r00.doc/bl1hlp_monnetworksmetrics.htm

But then I also look here: https://community.mellanox.com/docs/DOC-2751

and see "Total number of data octets, divided by 4 (lanes), received on all
VLs. This is 64 bit counter.” So I wasn’t sure if some multiplication by 4
was in order.

Please advise.

Cheers,
Kristy_______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at spectrumscale.org
https://urldefense.proofpoint.com/v2/url?u=http-3A__gpfsug.org_mailman_listinfo_gpfsug-2Ddiscuss&d=DwICAg&c=jf_iaSHvJObTbx-siA1ZOg&r=IbxtjdkPAM2Sbon4Lbbi4w&m=zIRb70L9sx_FvvC9IcWVKLOSOOFnx-hIGfjw0kUN7bw&s=D1g4YTG5WeUiHI3rCPr_kkPxbG9V9E-18UGXBeCvfB8&e=



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20180215/45be24f4/attachment.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: graycol.gif
Type: image/gif
Size: 105 bytes
Desc: not available
URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20180215/45be24f4/attachment.gif>


More information about the gpfsug-discuss mailing list