[gpfsug-discuss] Specifying nodes in commands

Fri Nov 10 17:15:55 GMT 2017

Hi Frederick,

The ILM active policy (set by mmchpolicy) has an external list rule, the command for the external list runs the mmapplypolicy command. /gpfs1/s3upload/policies/migration.policy has external pool & a migration rule in it. The handler script for the external pool writes the hostname of the server running it out to a file, so that's how I'm trapping which server is running the policy, and that mmapplypolicy is being run.

Hope that explains things, if not let me know and I'll have another try :)

Regards,

Peter

-----Original Message-----
From: gpfsug-discuss-bounces at spectrumscale.org [mailto:gpfsug-discuss-bounces at spectrumscale.org] On Behalf Of gpfsug-discuss-request at spectrumscale.org
Sent: 10 November 2017 16:43
To: gpfsug-discuss at spectrumscale.org
Subject: gpfsug-discuss Digest, Vol 70, Issue 32

Send gpfsug-discuss mailing list submissions to
	gpfsug-discuss at spectrumscale.org

To subscribe or unsubscribe via the World Wide Web, visit
	http://gpfsug.org/mailman/listinfo/gpfsug-discuss
or, via email, send a message with subject or body 'help' to
	gpfsug-discuss-request at spectrumscale.org

You can reach the person managing the list at
	gpfsug-discuss-owner at spectrumscale.org

When replying, please edit your Subject line so it is more specific than "Re: Contents of gpfsug-discuss digest..."

Today's Topics:

   1. Re: Spectrum Scale with NVMe (John Hearns)
   2. Specifying nodes in commands (Chase, Peter)
   3. Re: Specifying nodes in commands (Frederick Stock)
   4. Re: Specifying nodes in commands (Marc A Kaplan)

----------------------------------------------------------------------

Message: 1
Date: Fri, 10 Nov 2017 13:25:40 +0000
From: John Hearns <john.hearns at asml.com>
To: gpfsug main discussion list <gpfsug-discuss at spectrumscale.org>
Subject: Re: [gpfsug-discuss] Spectrum Scale with NVMe
Message-ID:
	<HE1PR02MB1450CE9F60F877D4C010FD8C88540 at HE1PR02MB1450.eurprd02.prod.outlook.com>

Content-Type: text/plain; charset="us-ascii"

Chad,
Thankyou for the reply.
Indded I had that issue - I only noticed because I looked at the utisation of the NSDs and a set of them were not being filled with data...
A set which were coincidentally all connected to the same server  (me whistles innocently....)

-----Original Message-----
From: gpfsug-discuss-bounces at spectrumscale.org [mailto:gpfsug-discuss-bounces at spectrumscale.org] On Behalf Of Chad Kerner
Sent: Tuesday, November 07, 2017 2:05 PM
To: gpfsug main discussion list <gpfsug-discuss at spectrumscale.org>
Subject: Re: [gpfsug-discuss] Spectrum Scale with NVMe

Hey John,

     Once you get /var/mmfs/etc/nsddevices set up, it is all straight forward.  We have seen times on reboot where the devices were not ready before gpfs started and the file system started with those disks in an offline state.  But, that was just a timing issue with the startup.

Chad
--
Chad Kerner, Senior Storage Engineer
Storage Enabling Technologies
National Center for Supercomputing Applications University of Illinois, Urbana-Champaign

On 11/7/17, John Hearns <john.hearns at asml.com> wrote:
> I am looking for anyone with experience of using Spectrum Scale with 
> nvme devices.
>
> I could use an offline brain dump...
>
>
> The specific issue I have is with the nsd device discovery and the naming.
>
> Before anyone replies, I am gettign excellent support from IBM and 
> have been directed to the correct documentation.
>
> I am just looking for any wrinkles or tips that anyone has.
>
>
> Thanks
>
> -- The information contained in this communication and any attachments 
> is confidential and may be privileged, and is for the sole use of the 
> intended recipient(s). Any unauthorized review, use, disclosure or 
> distribution is prohibited. Unless explicitly stated otherwise in the 
> body of this communication or the attachment thereto (if any), the 
> information is provided on an AS-IS basis without any express or 
> implied warranties or liabilities. To the extent you are relying on 
> this information, you are doing so at your own risk. If you are not 
> the intended recipient, please notify the sender immediately by 
> replying to this message and destroy all copies of this message and 
> any attachments. Neither the sender nor the company/group of companies 
> he or she represents shall be liable for the proper and complete 
> transmission of the information contained in this communication, or for any delay in its receipt.
>

--
--
Chad Kerner, Senior Storage Engineer
Storage Enabling Technologies
National Center for Supercomputing Applications University of Illinois, Urbana-Champaign _______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at spectrumscale.org
https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fgpfsug.org%2Fmailman%2Flistinfo%2Fgpfsug-discuss&data=01%7C01%7Cjohn.hearns%40asml.com%7Ce3875dc1def842e88ee308d525e01e80%7Caf73baa8f5944eb2a39d93e96cad61fc%7C1&sdata=pwti5NtVf7c4SClTUc1PWNz5YW4QHWjM5%2F%2BGLdYHoqQ%3D&reserved=0
-- The information contained in this communication and any attachments is confidential and may be privileged, and is for the sole use of the intended recipient(s). Any unauthorized review, use, disclosure or distribution is prohibited. Unless explicitly stated otherwise in the body of this communication or the attachment thereto (if any), the information is provided on an AS-IS basis without any express or implied warranties or liabilities. To the extent you are relying on this information, you are doing so at your own risk. If you are not the intended recipient, please notify the sender immediately by replying to this message and destroy all copies of this message and any attachments. Neither the sender nor the company/group of companies he or she represents shall be liable for the proper and complete transmission of the information contained in this communication, or for any delay in its receipt.

------------------------------

Message: 2
Date: Fri, 10 Nov 2017 16:18:36 +0000
From: "Chase, Peter" <peter.chase at metoffice.gov.uk>
To: "'gpfsug-discuss at spectrumscale.org'"
	<gpfsug-discuss at spectrumscale.org>
Subject: [gpfsug-discuss] Specifying nodes in commands
Message-ID:
	<F4C2AA4AB5D84B4A8046E4CEDD08253E43804862 at EXXCMPD1DAG1.cmpd1.metoffice.gov.uk>

Content-Type: text/plain; charset="iso-8859-1"

Hello all,

I'm running a script triggered from an ILM external list rule.

The script has the following command in, and it isn't work as I'd expect:
/usr/lpp/mmfs/bin/mmapplypolicy /gpfs1/aws -N cloudNode -P /gpfs1/s3upload/policies/migration.policy --scope fileset

I'd expect the mmapplypolicy command to run the policy on all the nodes in the cloudNode class, but it doesn't, it runs on the node that triggered the script.

However, the following command does work as I'd expect:
/usr/lpp/mmfs/bin/mmdsh -N cloudNode /usr/lpp/mmfs/bin/mmapplypolicy /gpfs1/aws -P /gpfs1/s3upload/policies/migration.policy --scope fileset

Can any one shed any light on this? Have I just misconstrued how mmapplypolicy works?

Regards,

Peter Chase
GPCS Team
Met Office? FitzRoy Road? Exeter? Devon? EX1 3PB? United Kingdom
Tel: +44 (0)1392 886921
Email: peter.chase at metoffice.gov.uk Website: www.metoffice.gov.uk 

------------------------------

Message: 3
Date: Fri, 10 Nov 2017 11:41:19 -0500
From: "Frederick Stock" <stockf at us.ibm.com>
To: gpfsug main discussion list <gpfsug-discuss at spectrumscale.org>
Subject: Re: [gpfsug-discuss] Specifying nodes in commands
Message-ID:
	<OF70BA2EF9.4C554F8A-ON852581D4.005B8BC3-852581D4.005BAC01 at notes.na.collabserv.com>

Content-Type: text/plain; charset="iso-8859-1"

How do you determine if mmapplypolicy is running on a node?  Normally mmapplypolicy as a process runs on a single node but its helper processes, policy-help or something similar, run on all the nodes which are referenced by the -N option.

Fred
__________________________________________________
Fred Stock | IBM Pittsburgh Lab | 720-430-8821 stockf at us.ibm.com

From:   "Chase, Peter" <peter.chase at metoffice.gov.uk>
To:     "'gpfsug-discuss at spectrumscale.org'" 
<gpfsug-discuss at spectrumscale.org>
Date:   11/10/2017 11:18 AM
Subject:        [gpfsug-discuss] Specifying nodes in commands
Sent by:        gpfsug-discuss-bounces at spectrumscale.org

Hello all,

I'm running a script triggered from an ILM external list rule.

The script has the following command in, and it isn't work as I'd expect:
/usr/lpp/mmfs/bin/mmapplypolicy /gpfs1/aws -N cloudNode -P /gpfs1/s3upload/policies/migration.policy --scope fileset

I'd expect the mmapplypolicy command to run the policy on all the nodes in the cloudNode class, but it doesn't, it runs on the node that triggered the script.

However, the following command does work as I'd expect:
/usr/lpp/mmfs/bin/mmdsh -N cloudNode /usr/lpp/mmfs/bin/mmapplypolicy /gpfs1/aws -P /gpfs1/s3upload/policies/migration.policy --scope fileset

Can any one shed any light on this? Have I just misconstrued how mmapplypolicy works?

Regards,

Peter Chase
GPCS Team
Met Office?  FitzRoy Road?  Exeter?  Devon?  EX1 3PB?  United Kingdom
Tel: +44 (0)1392 886921
Email: peter.chase at metoffice.gov.uk Website: www.metoffice.gov.uk 

_______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at spectrumscale.org
https://urldefense.proofpoint.com/v2/url?u=http-3A__gpfsug.org_mailman_listinfo_gpfsug-2Ddiscuss&d=DwIFAw&c=jf_iaSHvJObTbx-siA1ZOg&r=p_1XEUyoJ7-VJxF_w8h9gJh8_Wj0Pey73LCLLoxodpw&m=spXDnba2A_tVauiszV7sXhSkn6GeEljABN4lUEB4f8s&s=1Hd1SNkXtfLRcirmeRfg1JuAERuhbyiVqsLEdYlhFsM&e=

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://gpfsug.org/pipermail/gpfsug-discuss/attachments/20171110/3b1dc733/attachment-0001.html>

------------------------------

Message: 4
Date: Fri, 10 Nov 2017 11:42:28 -0500
From: "Marc A Kaplan" <makaplan at us.ibm.com>
To: gpfsug main discussion list <gpfsug-discuss at spectrumscale.org>
Subject: Re: [gpfsug-discuss] Specifying nodes in commands
Message-ID:
	<OF95AE7CEB.8278F465-ON852581D4.005A9E64-852581D4.005BC739 at notes.na.collabserv.com>

Content-Type: text/plain; charset="iso-8859-1"

mmapplypolicy ... -N nodeClass ... 
  will use the nodes in nodeClass as helper nodes to get its work done.

mmdsh -N nodeClass command ... 
  will run the SAME command on each of the nodes -- probably not what you want to do with mmapplypolicy.

To see more about what mmapplypolicy is doing use  options -d 1 (debug
info) 

If you are using -N because you have a lot of files to process, you should also use -g /some-gpfs-temp-directory (see doc)

If you are running a small test case, it may happen that you don't see the helper nodes doing anything, because there's not enough time and work to get them going...

For test purposes you can coax the helper nodes into action with:

options -B 1  -m 1 so that each helper node only does one file at a time. 

From:   "Chase, Peter" <peter.chase at metoffice.gov.uk>
To:     "'gpfsug-discuss at spectrumscale.org'" 
<gpfsug-discuss at spectrumscale.org>
Date:   11/10/2017 11:18 AM
Subject:        [gpfsug-discuss] Specifying nodes in commands
Sent by:        gpfsug-discuss-bounces at spectrumscale.org

Hello all,

I'm running a script triggered from an ILM external list rule.

The script has the following command in, and it isn't work as I'd expect:
/usr/lpp/mmfs/bin/mmapplypolicy /gpfs1/aws -N cloudNode -P /gpfs1/s3upload/policies/migration.policy --scope fileset

I'd expect the mmapplypolicy command to run the policy on all the nodes in the cloudNode class, but it doesn't, it runs on the node that triggered the script.

However, the following command does work as I'd expect:
/usr/lpp/mmfs/bin/mmdsh -N cloudNode /usr/lpp/mmfs/bin/mmapplypolicy /gpfs1/aws -P /gpfs1/s3upload/policies/migration.policy --scope fileset

Can any one shed any light on this? Have I just misconstrued how mmapplypolicy works?

Regards,

Peter Chase
GPCS Team
Met Office?  FitzRoy Road?  Exeter?  Devon?  EX1 3PB?  United Kingdom
Tel: +44 (0)1392 886921
Email: peter.chase at metoffice.gov.uk Website: www.metoffice.gov.uk 

_______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at spectrumscale.org
https://urldefense.proofpoint.com/v2/url?u=http-3A__gpfsug.org_mailman_listinfo_gpfsug-2Ddiscuss&d=DwIFAw&c=jf_iaSHvJObTbx-siA1ZOg&r=cvpnBBH0j41aQy0RPiG2xRL_M8mTc1izuQD3_PmtjZ8&m=WjhVVKkS23BlFGP2KHmkndM0AZ4yB2aC81UUHv8iIZs&s=-dPme1SlhBAqo45xVmtvVWNeAjumd7JrtEksW1U8o5w&e=

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://gpfsug.org/pipermail/gpfsug-discuss/attachments/20171110/725a922c/attachment.html>

------------------------------

_______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at spectrumscale.org
http://gpfsug.org/mailman/listinfo/gpfsug-discuss

End of gpfsug-discuss Digest, Vol 70, Issue 32
**********************************************