[gpfsug-discuss] nsdperf crash testing RDMA between Power BE and Intel nodes

Uwe Falke UWEFALKE at de.ibm.com
Wed Oct 25 09:52:29 BST 2017


Hi, Scott, 
thanks, good to hear that it worked for you. 
I can at least confirm that GPFS RDMA itself does work between x86-64 
clients the ESS here, it appears just nsdperf has an issue in my 
particular environment. I'll see what IBM support can do for me as Olaf 
suggested. 
 
Mit freundlichen Grüßen / Kind regards

 
Dr. Uwe Falke
 
IT Specialist
High Performance Computing Services / Integrated Technology Services / 
Data Center Services
-------------------------------------------------------------------------------------------------------------------------------------------
IBM Deutschland
Rathausstr. 7
09111 Chemnitz
Phone: +49 371 6978 2165
Mobile: +49 175 575 2877
E-Mail: uwefalke at de.ibm.com
-------------------------------------------------------------------------------------------------------------------------------------------
IBM Deutschland Business & Technology Services GmbH / Geschäftsführung: 
Thomas Wolter, Sven Schooß
Sitz der Gesellschaft: Ehningen / Registergericht: Amtsgericht Stuttgart, 
HRB 17122 




From:   Scott D <sdenham at gmail.com>
To:     gpfsug main discussion list <gpfsug-discuss at spectrumscale.org>
Date:   10/24/2017 10:35 PM
Subject:        Re: [gpfsug-discuss] nsdperf crash testing RDMA between 
Power BE and Intel nodes
Sent by:        gpfsug-discuss-bounces at spectrumscale.org



I have run nsdperf with RDMA enabled against an ESS ppc-64 server without 
any problems. I don't have access to that system at the moment, and it was 
running a fairly old (4.1.x) version of GPFS, not that that should matter 
for nsdperf unless that source code has changed since 4.1. 

Scott Denham
Staff Engineer
Cray, Inc

On Tue, Oct 24, 2017 at 11:49 AM, Uwe Falke <UWEFALKE at de.ibm.com> wrote:
Hi,
I am about to run nsdperf for testing the IB fabric in a new system
comprising ESS (BE) and Intel-based nodes.
nsdperf crashes reliably when invoking ESS nodes and x86-64 nodes in one
test using RDMA:

client          server          RDMA
x86-64          ppc-64          on              crash
ppc-64          x86-64          on              crash
x86-64          ppc-64          off             success
x86-64          x86-64          on              success
ppc-64          ppc-64          on              success

That implies that the nsdperf RDMA test might struggle with BE vs LE.
However, I learned from a talk given at a GPFS workshop in Germany in 2015
that RDMA works between Power-BE and Intel boxes. Has anyone made similar
or contrary experiences? Is it an nsdperf issue or more general (I have
not yet attempted any GPFS mount)?



Mit freundlichen Grüßen / Kind regards


Dr. Uwe Falke

IT Specialist
High Performance Computing Services / Integrated Technology Services /
Data Center Services
-------------------------------------------------------------------------------------------------------------------------------------------
IBM Deutschland
Rathausstr. 7
09111 Chemnitz
Phone: +49 371 6978 2165
Mobile: +49 175 575 2877
E-Mail: uwefalke at de.ibm.com
-------------------------------------------------------------------------------------------------------------------------------------------
IBM Deutschland Business & Technology Services GmbH / Geschäftsführung:
Thomas Wolter, Sven Schooß
Sitz der Gesellschaft: Ehningen / Registergericht: Amtsgericht Stuttgart,
HRB 17122


_______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at spectrumscale.org
http://gpfsug.org/mailman/listinfo/gpfsug-discuss
_______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at spectrumscale.org
http://gpfsug.org/mailman/listinfo/gpfsug-discuss







More information about the gpfsug-discuss mailing list