[gpfsug-discuss] gpfsug-discuss Digest, Vol 71, Issue 49
atmane khiredine
a.khiredine at meteo.dz
Thu Dec 21 14:02:40 GMT 2017
Atmane Khiredine
HPC System Administrator | Office National de la Météorologie
Tél : +213 21 50 73 93 # 303 | Fax : +213 21 50 79 40 | E-mail : a.khiredine at meteo.dz
________________________________________
De : gpfsug-discuss-bounces at spectrumscale.org [gpfsug-discuss-bounces at spectrumscale.org] de la part de gpfsug-discuss-request at spectrumscale.org [gpfsug-discuss-request at spectrumscale.org]
Envoyé : jeudi 21 décembre 2017 13:00
À : gpfsug-discuss at spectrumscale.org
Objet : gpfsug-discuss Digest, Vol 71, Issue 49
Send gpfsug-discuss mailing list submissions to
gpfsug-discuss at spectrumscale.org
To subscribe or unsubscribe via the World Wide Web, visit
http://gpfsug.org/mailman/listinfo/gpfsug-discuss
or, via email, send a message with subject or body 'help' to
gpfsug-discuss-request at spectrumscale.org
You can reach the person managing the list at
gpfsug-discuss-owner at spectrumscale.org
When replying, please edit your Subject line so it is more specific
than "Re: Contents of gpfsug-discuss digest..."
Today's Topics:
1. Re: more than one mlx connectx-4 adapter in same host
(Frank Kraemer)
This Is My Config
mmlsconfig | grep verbsRdma
verbsRdma enable
mmlsconfig | grep verbsPorts
verbsPorts mlx4_0/1 mlx4_0/2 mlx4_1/1 mlx4_1/2
ibstat | grep -E "CA|Port"
CA 'mlx4_0'
CA type: MT4099
Port 1:
Port GUID: 0x0002c903001a1571
Port 2:
Port GUID: 0x0002c903001a1572
CA 'mlx4_1'
CA type: MT4099
Port 1:
Port GUID: 0x0002c903001a15c1
Port 2:
Port GUID: 0x0002c903001a15c2
ethtool ib0 |grep "Link detected"
Link detected: yes
ethtool ib1 |grep "Link detected"
Link detected: yes
ethtool ib2 |grep "Link detected"
Link detected: yes
ethtool ib3 |grep "Link detected"
Link detected: yes
ifconfig ib0 |grep inet
inet 192.168.100.3 netmask 255.255.255.0 broadcast 192.168.100.255
ifconfig ib1 |grep inet
Infiniband hardware address can be incorrect! Please read BUGS section in
ifconfig(8).
ifconfig ib2 |grep inet
Infiniband hardware address can be incorrect! Please read BUGS section in
ifconfig(8).
ifconfig ib3 |grep inet
Infiniband hardware address can be incorrect! Please read BUGS section in
ifconfig(8).
ip addr show ib1
8: ib1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 65520 qdisc mq state UP qlen
1024
link/infiniband
a0:00:03:00:fe:80:00:00:00:00:00:00:00:02:c9:03:00:1a:15:72 brd
00:ff:ff:ff:ff:12:40:1b:ff:ff:00:00:00:00:00:00:ff:ff:ff:ff
9: ib2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 65520 qdisc mq state UP qlen
1024
link/infiniband
a0:00:02:20:fe:80:00:00:00:00:00:00:00:02:c9:03:00:1a:15:c1 brd
00:ff:ff:ff:ff:12:40:1b:ff:ff:00:00:00:00:00:00:ff:ff:ff:ff
10: ib3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 65520 qdisc mq state UP
qlen 1024
link/infiniband
a0:00:03:00:fe:80:00:00:00:00:00:00:00:02:c9:03:00:1a:15:c2 brd
00:ff:ff:ff:ff:12:40:1b:ff:ff:00:00:00:00:00:00:ff:ff:ff:ff
Message: 1
Date: Thu, 21 Dec 2017 08:07:24 +0100
From: "Frank Kraemer" <kraemerf at de.ibm.com>
To: gpfsug-discuss at spectrumscale.org
Subject: Re: [gpfsug-discuss] more than one mlx connectx-4 adapter in
same host
Message-ID:
<OFE99AF1B2.2CCA57A0-ONC12581FD.00272158-C12581FD.00272158 at notes.na.collabserv.com>
Content-Type: text/plain; charset="us-ascii"
David,
> We're trying to get 40 gbe connection between Mellanox switches and Cisco
> switches down at the other end of the machine room
> The BiDi part seems to be the best given about 30m run on multimode.
> However Mellanox support says it's not supported.
> Want to use this to get close to IB speeds for GPFS on nodes that aren't
> on the IB fabric.
> Does anyone have any luck getting 40 or 100 gig at 20-30m when the
> switches are different brands?
maybe that's a good reason to get in contact with the team from Interoptic.
They claim a good expertise for these kind of problems - feedback is good.
http://packetpushers.net/podcast/podcasts/show-360-all-about-optics-interoptic-sponsored/
https://interoptic.com/
Frank Kraemer
IBM Consulting IT Specialist / Client Technical Architect
Am Weiher 24, 65451 Kelsterbach
mailto:kraemerf at de.ibm.com
voice: +49-(0)171-3043699 / +4970342741078
IBM Germany
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://gpfsug.org/pipermail/gpfsug-discuss/attachments/20171221/31aeae52/attachment-0001.html>
------------------------------
_______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at spectrumscale.org
http://gpfsug.org/mailman/listinfo/gpfsug-discuss
End of gpfsug-discuss Digest, Vol 71, Issue 49
**********************************************
More information about the gpfsug-discuss
mailing list