[gpfsug-discuss] gpfs rdma expels

[Chris Hunter]

Thanks for the feedback.
Our network is non-uniform, we have three (uniform) rdma networks 
connected by narrow uplinks. Previously we used gpfs on one network, now 
we wish to expand to the other networks.

Previous experience shows we see "PortXmitWait" messages from traffic 
over the narrow uplinks. We find expels happen often from gpfs 
communication over the narrow uplinks.

We acknowledge an inherent weakness with narrow uplinks but for 
practical reasons it would be difficult to resolve.

So the question, is it possible to configure gpfs to be tolerant of 
non-uniform networks with narrow uplinks ?

thanks,
chris hunter

> On 1 Jul 2015, at 16:52, Chris Hunter <chris.hunter at yale.edu> wrote:
>
> Hi UG list,
> We have a large rdma/tcp multi-cluster gpfs filesystem, about 2/3 of
> clients use RDMA. We see a large number of expels of rdma clients but
> less of the tcp clients. Most of the gpfs config is at defaults. We
> are unclear if any of the non-RDMA config items (eg. Idle socket
> timeout) would help our issue. Any suggestions on gpfs config
> parameters we should investigate ?