[gpfsug-discuss] data interface and management infercace.

[Salvatore Di Nardo]

Hello guys.
Quite a while ago i mentioned that we have a big  expel issue on our gss 
( first gen) and white a lot people suggested that the root cause could 
be that we use the same interface for all the traffic, and that we 
should split the data network from the admin network. Finally we could 
plan a downtime and we are migrating the data out so, i can soon safelly 
play with the change, but looking what exactly i should to do i'm a bit 
puzzled. Our mmlscluster looks like this:

            GPFS cluster information
            ========================
               GPFS cluster name: GSS.ebi.ac.uk
               GPFS cluster id: 17987981184946329605
               GPFS UID domain: GSS.ebi.ac.uk
               Remote shell command: /usr/bin/ssh
               Remote file copy command: /usr/bin/scp

            GPFS cluster configuration servers:
            -----------------------------------
               Primary server:    gss01a.ebi.ac.uk
               Secondary server:  gss02b.ebi.ac.uk

              Node  Daemon node name    IP address  Admin node name    
            Designation
            -----------------------------------------------------------------------
                1   gss01a.ebi.ac.uk 10.7.28.2   gss01a.ebi.ac.uk   
            quorum-manager
                2   gss01b.ebi.ac.uk 10.7.28.3   gss01b.ebi.ac.uk   
            quorum-manager
                3   gss02a.ebi.ac.uk 10.7.28.67  gss02a.ebi.ac.uk   
            quorum-manager
                4   gss02b.ebi.ac.uk 10.7.28.66  gss02b.ebi.ac.uk   
            quorum-manager
                5   gss03a.ebi.ac.uk 10.7.28.34  gss03a.ebi.ac.uk   
            quorum-manager
                6   gss03b.ebi.ac.uk 10.7.28.35  gss03b.ebi.ac.uk   
            quorum-manager


It was my understanding that the "admin node" should use a different 
interface ( a 1g link copper should be fine), while the daemon node is 
where the data was passing , so should point to the bonded 10g 
interfaces.  but when i read the mmchnode man page i start to be quite 
confused. It says:

--daemon-interface={hostname | ip_address}
                          Specifies  the  host  name or IP address _*to 
be used by the GPFS daemons for node-to-node communication*_.  The host 
name or IP address must refer to the communication adapter over which 
the GPFS daemons communicate.
                          Alias interfaces are not allowed. Use the 
original address or a name that is resolved by the host command to that 
original address.

                    --admin-interface={hostname | ip_address}
                          Specifies the name of the node to be used by 
GPFS administration commands when communicating between nodes. The admin 
node name must be specified as an IP address or a hostname that is 
resolved by the  host command
                          tothe desired IP address.  If the keyword 
DEFAULT is specified, the admin interface for the node is set to be 
equal to the daemon interface for the node.

What exactly means "node-to node-communications" ?
Means DATA or also the "lease renew", and the token communication 
between the clients to get/steal the locks to be able to manage 
concurrent write to thr same file?
Since we are getting expells ( especially when several clients contends 
the same file ) i assumed i have to split this type of packages from the 
data stream, but reading the documentation it looks to me that those 
internal comunication between nodes use the daemon-interface wich i 
suppose are used also for the data. so HOW exactly i can split them?
_**_

Thanks in advance,
Salvatore

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20150710/0f00d571/attachment.htm>