[gpfsug-discuss] Control of where mmcrsnapshot runs?

[Bryan Banister]

The cgroups are something we moved onto, which has helped a lot with GPFS Clients responding to necessary GPFS commands demanding a low latency response (e.g. mmcrsnapshots, byte range locks, quota reporting, etc).

Good luck!
-Bryan

From: gpfsug-discuss-bounces at spectrumscale.org [mailto:gpfsug-discuss-bounces at spectrumscale.org] On Behalf Of Keith Ball
Sent: Thursday, March 29, 2018 11:15 AM
To: gpfsug-discuss at spectrumscale.org
Subject: Re: [gpfsug-discuss] Control of where mmcrsnapshot runs?

Note: External Email
________________________________
You're right, Brian, the key load will be on the filesystem manager in any case, and as you say, all nodes nodes must quiesce - it's not really an issue of where to run the command, like it would be for mmfsck, etc.
GPFS version is 3.5.0.26. We'll investigate upgrade to later version that accommodates combined operations.
I will also look into the cgroups approach - is this a documented thing, or just something that people have tinkered with/hand rolled?
Thanks,
  Keith

On Wed, Mar 28, 2018 at 7:00 AM, <gpfsug-discuss-<mailto:gpfsug-discuss-request at spectrumscale.org>

What version of GPFS are you running Keith?

All nodes mounting the file system must briefly quiesce I/O operations during the snapshot create operations, hence the ?Quiescing all file system operations.? message in the output.  So don?t really see a way to specify a specific set of nodes for these operations.  They have made updates in newer releases of GPFS to combine operations (e.g. create and delete snapshots at the same time) which IBM says ?system performance is increased by batching operations and reducing overhead.?.

Trying to isolate GPFS resources (e.g. cgroups) on the clients (e.g. CPU and memory resources dedicated to GPFS/SSH/kernel/networking/etc) can help them respond more quickly to quiesce I/O requests.

HTH,
-Bryan

From: gpfsug-discuss-bounces at spectrumscale.org<mailto:gpfsug-discuss-bounces at spectrumscale.org> [mailto:gpfsug-discuss-bounces at spectrumscale.org<mailto:gpfsug-discuss-bounces at spectrumscale.org>] On Behalf Of Keith Ball
Sent: Tuesday, March 27, 2018 5:26 PM
To: gpfsug-discuss at spectrumscale.org<mailto:gpfsug-discuss at spectrumscale.org>
Subject: [gpfsug-discuss] Control of where mmcrsnapshot runs?

Note: External Email
________________________________
Hi All,
Two questions on snapshots:
1.) I note that neither "mmcrsnapshot" nor "mmdelsnapshot" appear to have an "-N" option as "PIT" commands typically do. Is there any way to control where threads for snapshot creation/deletion run? (I assume the filesystem manager will always be involved regardless).

2.) When mmdelsnapshot hangs or times out, the error messages tend to appear on client nodes, and not necessarily the node where mmdelsnapshot is run from, not the FS manager. Besides telling all users "don't use any I/O" when runnign these commands, are there ways that folks have found to avoid hangs and timeouts of mmdelsnapshot?
FWIW our filesystem manager is probably overextended (replication factor 2 on data+MD, 30 daily snapshots kept, a number of client clusters served, plus the FS manager is also an NSD server).

Many Thanks,
  Keith
RedLine Performance Solutions LLC

________________________________


________________________________

Note: This email is for the confidential use of the named addressee(s) only and may contain proprietary, confidential or privileged information. If you are not the intended recipient, you are hereby notified that any review, dissemination or copying of this email is strictly prohibited, and to please notify the sender immediately and destroy this email and any attachments. Email transmission cannot be guaranteed to be secure or error-free. The Company, therefore, does not make any guarantees as to the completeness or accuracy of this email or any attachments. This email is for informational purposes only and does not constitute a recommendation, offer, request or solicitation of any kind to buy, sell, subscribe, redeem or perform any type of transaction of a financial product.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20180329/d62ecd78/attachment.htm>