[gpfsug-discuss] data interface and management infercace.
Salvatore Di Nardo
sdinardo at ebi.ac.uk
Mon Jul 13 14:29:50 BST 2015
Hello Vic.
We are currently draining our gpfs to do all the recabling to add a
management network, but looking what the admin interface does ( man
mmchnode ) it says something different:
--admin-interface={hostname | ip_address}
Specifies the name of the node to be
used by GPFS administration commands when communicating between
nodes. The admin node name must be specified as an IP
address or a hostname that is resolved
by the host command to the desired IP address. If the keyword
DEFAULT is specified, the admin interface for the
node is set to be equal to the daemon
interface for the node.
So, seems used only for commands propagation, hence have nothing to do
with the node-to-node traffic. Infact the other interface description is:
--daemon-interface={hostname | ip_address}
Specifies the host name or IP address
_*to be used by the GPFS daemons for node-to-node
communication*_. The host name or IP address must refer to the
commu-
nication adapter over which the GPFS
daemons communicate. Alias interfaces are not allowed. Use the
original address or a name that is resolved by the
host command to that original address.
The "expired lease" issue and file locking mechanism a( most of our
expells happens when 2 clients try to write in the same file) are
exactly node-to node-comunication, so im wondering what's the point to
separate the "admin network". I want to be sure to plan the right
changes before we do a so massive task. We are talking about adding a
new interface on 700 clients, so the recabling work its not small.
Regards,
Salvatore
On 13/07/15 14:00, Vic Cornell wrote:
> Hi Salavatore,
>
> Does your GSS have the facility for a 1GbE “management” network? If so
> I think that changing the “admin” node names of the cluster members to
> a set of IPs on the management network would give you the split that
> you need.
>
> What about the clients? Can they also connect to a separate admin network?
>
> Remember that if you are using multi-cluster all of the nodes in both
> networks must share the same admin network.
>
> Kind Regards,
>
> Vic
>
>
>> On 13 Jul 2015, at 13:31, Salvatore Di Nardo <sdinardo at ebi.ac.uk
>> <mailto:sdinardo at ebi.ac.uk>> wrote:
>>
>> Anyone?
>>
>> On 10/07/15 11:07, Salvatore Di Nardo wrote:
>>> Hello guys.
>>> Quite a while ago i mentioned that we have a big expel issue on our
>>> gss ( first gen) and white a lot people suggested that the root
>>> cause could be that we use the same interface for all the traffic,
>>> and that we should split the data network from the admin network.
>>> Finally we could plan a downtime and we are migrating the data out
>>> so, i can soon safelly play with the change, but looking what
>>> exactly i should to do i'm a bit puzzled. Our mmlscluster looks like
>>> this:
>>>
>>> GPFS cluster information
>>> ========================
>>> GPFS cluster name: GSS.ebi.ac.uk <http://GSS.ebi.ac.uk>
>>> GPFS cluster id: 17987981184946329605
>>> GPFS UID domain: GSS.ebi.ac.uk <http://GSS.ebi.ac.uk>
>>> Remote shell command: /usr/bin/ssh
>>> Remote file copy command: /usr/bin/scp
>>>
>>> GPFS cluster configuration servers:
>>> -----------------------------------
>>> Primary server: gss01a.ebi.ac.uk <http://gss01a.ebi.ac.uk>
>>> Secondary server: gss02b.ebi.ac.uk <http://gss02b.ebi.ac.uk>
>>>
>>> Node Daemon node name IP address Admin node
>>> name Designation
>>> -----------------------------------------------------------------------
>>> 1 gss01a.ebi.ac.uk <http://gss01a.ebi.ac.uk>
>>> 10.7.28.2 gss01a.ebi.ac.uk <http://gss01a.ebi.ac.uk>
>>> quorum-manager
>>> 2 gss01b.ebi.ac.uk <http://gss01b.ebi.ac.uk>
>>> 10.7.28.3 gss01b.ebi.ac.uk <http://gss01b.ebi.ac.uk>
>>> quorum-manager
>>> 3 gss02a.ebi.ac.uk <http://gss02a.ebi.ac.uk>
>>> 10.7.28.67 gss02a.ebi.ac.uk <http://gss02a.ebi.ac.uk>
>>> quorum-manager
>>> 4 gss02b.ebi.ac.uk <http://gss02b.ebi.ac.uk>
>>> 10.7.28.66 gss02b.ebi.ac.uk <http://gss02b.ebi.ac.uk>
>>> quorum-manager
>>> 5 gss03a.ebi.ac.uk <http://gss03a.ebi.ac.uk>
>>> 10.7.28.34 gss03a.ebi.ac.uk <http://gss03a.ebi.ac.uk>
>>> quorum-manager
>>> 6 gss03b.ebi.ac.uk <http://gss03b.ebi.ac.uk>
>>> 10.7.28.35 gss03b.ebi.ac.uk <http://gss03b.ebi.ac.uk>
>>> quorum-manager
>>>
>>>
>>> It was my understanding that the "admin node" should use a different
>>> interface ( a 1g link copper should be fine), while the daemon node
>>> is where the data was passing , so should point to the bonded 10g
>>> interfaces. but when i read the mmchnode man page i start to be
>>> quite confused. It says:
>>>
>>> --daemon-interface={hostname | ip_address}
>>> Specifies the host name or IP address
>>> _*to be used by the GPFS daemons for node-to-node communication*_.
>>> The host name or IP address must refer to the communication adapter
>>> over which the GPFS daemons communicate.
>>> Alias interfaces are not allowed. Use the
>>> original address or a name that is resolved by the host command to
>>> that original address.
>>>
>>> --admin-interface={hostname | ip_address}
>>> Specifies the name of the node to be used
>>> by GPFS administration commands when communicating between nodes.
>>> The admin node name must be specified as an IP address or a hostname
>>> that is resolved by the host command
>>> tothe desired IP address. If the keyword
>>> DEFAULT is specified, the admin interface for the node is set to be
>>> equal to the daemon interface for the node.
>>>
>>> What exactly means "node-to node-communications" ?
>>> Means DATA or also the "lease renew", and the token communication
>>> between the clients to get/steal the locks to be able to manage
>>> concurrent write to thr same file?
>>> Since we are getting expells ( especially when several clients
>>> contends the same file ) i assumed i have to split this type of
>>> packages from the data stream, but reading the documentation it
>>> looks to me that those internal comunication between nodes use the
>>> daemon-interface wich i suppose are used also for the data. so HOW
>>> exactly i can split them?
>>>
>>>
>>> Thanks in advance,
>>> Salvatore
>>>
>>>
>>>
>>> _______________________________________________
>>> gpfsug-discuss mailing list
>>> gpfsug-discuss atgpfsug.org <http://gpfsug.org>
>>> http://gpfsug.org/mailman/listinfo/gpfsug-discuss
>>
>> _______________________________________________
>> gpfsug-discuss mailing list
>> gpfsug-discuss at gpfsug.org <http://gpfsug.org>
>> http://gpfsug.org/mailman/listinfo/gpfsug-discuss
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20150713/6d7846d8/attachment.htm>
More information about the gpfsug-discuss
mailing list