[gpfsug-discuss] gpfsug-discuss Digest, Vol 71, Issue 49

atmane khiredine a.khiredine at meteo.dz
Thu Dec 21 14:02:40 GMT 2017



Atmane Khiredine
HPC System Administrator | Office National de la Météorologie
Tél : +213 21 50 73 93 # 303 | Fax : +213 21 50 79 40 | E-mail : a.khiredine at meteo.dz

________________________________________
De : gpfsug-discuss-bounces at spectrumscale.org [gpfsug-discuss-bounces at spectrumscale.org] de la part de gpfsug-discuss-request at spectrumscale.org [gpfsug-discuss-request at spectrumscale.org]
Envoyé : jeudi 21 décembre 2017 13:00
À : gpfsug-discuss at spectrumscale.org
Objet : gpfsug-discuss Digest, Vol 71, Issue 49

Send gpfsug-discuss mailing list submissions to
        gpfsug-discuss at spectrumscale.org

To subscribe or unsubscribe via the World Wide Web, visit
        http://gpfsug.org/mailman/listinfo/gpfsug-discuss
or, via email, send a message with subject or body 'help' to
        gpfsug-discuss-request at spectrumscale.org

You can reach the person managing the list at
        gpfsug-discuss-owner at spectrumscale.org

When replying, please edit your Subject line so it is more specific
than "Re: Contents of gpfsug-discuss digest..."


Today's Topics:

   1. Re: more than one mlx connectx-4 adapter in same  host
      (Frank Kraemer)

  This Is My Config
    mmlsconfig | grep verbsRdma
    verbsRdma enable
    mmlsconfig | grep verbsPorts
    verbsPorts mlx4_0/1 mlx4_0/2 mlx4_1/1 mlx4_1/2
    ibstat | grep -E "CA|Port"
    CA 'mlx4_0'
             CA type: MT4099
             Port 1:
                     Port GUID: 0x0002c903001a1571
             Port 2:
                     Port GUID: 0x0002c903001a1572
    CA 'mlx4_1'
             CA type: MT4099
             Port 1:
                     Port GUID: 0x0002c903001a15c1
             Port 2:
                     Port GUID: 0x0002c903001a15c2
    ethtool ib0 |grep "Link detected"
    Link detected: yes
    ethtool ib1 |grep "Link detected"
    Link detected: yes
 

    ethtool ib2 |grep "Link detected"
    Link detected: yes
    ethtool ib3 |grep "Link detected"
    Link detected: yes
    ifconfig ib0 |grep inet
    inet 192.168.100.3 netmask 255.255.255.0  broadcast 192.168.100.255
    ifconfig ib1 |grep inet
    Infiniband hardware address can be incorrect! Please read BUGS section in  
    ifconfig(8).
    ifconfig ib2 |grep inet
    Infiniband hardware address can be incorrect! Please read BUGS section in  
    ifconfig(8).
    ifconfig ib3 |grep inet
    Infiniband hardware address can be incorrect! Please read BUGS section in  
    ifconfig(8).
    ip addr show ib1
    8: ib1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 65520 qdisc mq state UP qlen  
    1024
         link/infiniband  
    a0:00:03:00:fe:80:00:00:00:00:00:00:00:02:c9:03:00:1a:15:72 brd  
    00:ff:ff:ff:ff:12:40:1b:ff:ff:00:00:00:00:00:00:ff:ff:ff:ff
    9: ib2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 65520 qdisc mq state UP qlen  
    1024
         link/infiniband  
    a0:00:02:20:fe:80:00:00:00:00:00:00:00:02:c9:03:00:1a:15:c1 brd  
    00:ff:ff:ff:ff:12:40:1b:ff:ff:00:00:00:00:00:00:ff:ff:ff:ff
 
    10: ib3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 65520 qdisc mq state UP  
    qlen 1024
         link/infiniband  
    a0:00:03:00:fe:80:00:00:00:00:00:00:00:02:c9:03:00:1a:15:c2 brd  
    00:ff:ff:ff:ff:12:40:1b:ff:ff:00:00:00:00:00:00:ff:ff:ff:ff

Message: 1
Date: Thu, 21 Dec 2017 08:07:24 +0100
From: "Frank Kraemer" <kraemerf at de.ibm.com>
To: gpfsug-discuss at spectrumscale.org
Subject: Re: [gpfsug-discuss] more than one mlx connectx-4 adapter in
        same    host
Message-ID:
        <OFE99AF1B2.2CCA57A0-ONC12581FD.00272158-C12581FD.00272158 at notes.na.collabserv.com>

Content-Type: text/plain; charset="us-ascii"

David,

> We're trying to get 40 gbe connection between Mellanox switches and Cisco
> switches down at the other end of the machine room
> The BiDi part seems to be the best given about 30m  run on multimode.
> However Mellanox support says it's not supported.
> Want to use this to get close to IB speeds for GPFS on nodes that aren't
> on the IB fabric.
> Does anyone have any luck getting 40 or 100 gig at 20-30m when the
> switches are different brands?

maybe that's a good reason to get in contact with the team from Interoptic.
They claim a good expertise for these kind of problems - feedback is good.
http://packetpushers.net/podcast/podcasts/show-360-all-about-optics-interoptic-sponsored/
https://interoptic.com/

Frank Kraemer
IBM Consulting IT Specialist  / Client Technical Architect
Am Weiher 24, 65451 Kelsterbach
mailto:kraemerf at de.ibm.com
voice: +49-(0)171-3043699 / +4970342741078
IBM Germany
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://gpfsug.org/pipermail/gpfsug-discuss/attachments/20171221/31aeae52/attachment-0001.html>

------------------------------

_______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at spectrumscale.org
http://gpfsug.org/mailman/listinfo/gpfsug-discuss


End of gpfsug-discuss Digest, Vol 71, Issue 49
**********************************************



More information about the gpfsug-discuss mailing list