[gpfsug-discuss] *New* IBM Spectrum Protect Whitepaper "Petascale Data Protection" (Dominic Mueller-Wicke)
Dominic Mueller-Wicke01
dominic.mueller at de.ibm.com
Wed Aug 31 06:52:38 BST 2016
Thanks for reading the paper. I agree that the restore of a large number of
files is a challenge today. The restore is the focus area for future
enhancements for the integration between IBM Spectrum Scale and IBM
Spectrum Protect. If something will be available that helps to improve the
restore capabilities the paper will be updated with this information.
Greetings, Dominic.
From: gpfsug-discuss-request at spectrumscale.org
To: gpfsug-discuss at spectrumscale.org
Date: 31.08.2016 01:25
Subject: gpfsug-discuss Digest, Vol 55, Issue 55
Sent by: gpfsug-discuss-bounces at spectrumscale.org
Send gpfsug-discuss mailing list submissions to
gpfsug-discuss at spectrumscale.org
To subscribe or unsubscribe via the World Wide Web, visit
http://gpfsug.org/mailman/listinfo/gpfsug-discuss
or, via email, send a message with subject or body 'help' to
gpfsug-discuss-request at spectrumscale.org
You can reach the person managing the list at
gpfsug-discuss-owner at spectrumscale.org
When replying, please edit your Subject line so it is more specific
than "Re: Contents of gpfsug-discuss digest..."
Today's Topics:
1. Maximum value for data replication?
(Simon Thompson (Research Computing - IT Services))
2. greetings (Kevin D Johnson)
3. GPFS 3.5.0 on RHEL 6.8 (Lukas Hejtmanek)
4. Re: GPFS 3.5.0 on RHEL 6.8 (Kevin D Johnson)
5. Re: GPFS 3.5.0 on RHEL 6.8 (mark.bergman at uphs.upenn.edu)
6. Re: *New* IBM Spectrum Protect Whitepaper "Petascale Data
Protection" (Lukas Hejtmanek)
7. Re: *New* IBM Spectrum Protect Whitepaper "Petascale Data
Protection" (Sven Oehme)
----- Message from "Simon Thompson (Research Computing - IT Services)"
<S.J.Thompson at bham.ac.uk> on Tue, 30 Aug 2016 19:09:05 +0000 -----
To: "gpfsug-discuss at spectrumscale.org"
<gpfsug-discuss at spectrumscale.org>
Subject: [gpfsug-discuss] Maximum value for data replication?
Is there a maximum value for data replication in Spectrum Scale?
I have a number of nsd servers which have local storage and Id like each
node to have a full copy of all the data in the file-system, say this value
is 4, can I set replication to 4 for data and metadata and have each server
have a full copy?
These are protocol nodes and multi cluster mount another file system (yes I
know not supported) and the cesroot is in the remote file system. On
several occasions where GPFS has wibbled a bit, this has caused issues with
ces locks, so I was thinking of moving the cesroot to a local filesysyem
which is replicated on the local ssds in the protocol nodes. I.e. Its a
generally quiet file system as its only ces cluster config.
I assume if I stop protocols, rsync the data and then change to the new ces
root, I should be able to get this working?
Thanks
Simon
----- Message from "Kevin D Johnson" <kevindjo at us.ibm.com> on Tue, 30 Aug
2016 19:43:39 +0000 -----
To: gpfsug-discuss at spectrumscale.org
Subject: [gpfsug-discuss] greetings
I'm in Lab Services at IBM - just joining and happy to help any way I can.
Kevin D. Johnson, MBA, MAFM
Spectrum Computing, Senior Managing Consultant
IBM Certified Deployment Professional - Spectrum Scale V4.1.1
IBM Certified Deployment Professional - Cloud Object Storage V3.8
720.349.6199 - kevindjo at us.ibm.com
----- Message from Lukas Hejtmanek <xhejtman at ics.muni.cz> on Tue, 30 Aug
2016 22:39:18 +0200 -----
To: gpfsug-discuss at spectrumscale.org
Subject: [gpfsug-discuss] GPFS 3.5.0 on
RHEL 6.8
Hello,
does it work for anyone? As of kernel 2.6.32-642, GPFS 3.5.0 (including the
latest patch 32) does start but does not mount and file system. The
internal
mount cmd gets stucked.
--
Lukáš Hejtmánek
----- Message from "Kevin D Johnson" <kevindjo at us.ibm.com> on Tue, 30 Aug
2016 20:51:39 +0000 -----
To: gpfsug-discuss at spectrumscale.org
Subject: Re: [gpfsug-discuss] GPFS 3.5.0 on
RHEL 6.8
RHEL 6.8/2.6.32-642 requires 4.1.1.8 or 4.2.1. You can either go to 6.7
for GPFS 3.5 or bump it up to 7.0/7.1.
See Table 13, here:
http://www.ibm.com/support/knowledgecenter/en/STXKQY/gpfsclustersfaq.html?view=kc#linuxq
Kevin D. Johnson, MBA, MAFM
Spectrum Computing, Senior Managing Consultant
IBM Certified Deployment Professional - Spectrum Scale V4.1.1
IBM Certified Deployment Professional - Cloud Object Storage V3.8
720.349.6199 - kevindjo at us.ibm.com
----- Original message -----
From: Lukas Hejtmanek <xhejtman at ics.muni.cz>
Sent by: gpfsug-discuss-bounces at spectrumscale.org
To: gpfsug-discuss at spectrumscale.org
Cc:
Subject: [gpfsug-discuss] GPFS 3.5.0 on RHEL 6.8
Date: Tue, Aug 30, 2016 4:39 PM
Hello,
does it work for anyone? As of kernel 2.6.32-642, GPFS 3.5.0 (including
the
latest patch 32) does start but does not mount and file system. The
internal
mount cmd gets stucked.
--
Lukáš Hejtmánek
_______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at spectrumscale.org
http://gpfsug.org/mailman/listinfo/gpfsug-discuss
----- Message from mark.bergman at uphs.upenn.edu on Tue, 30 Aug 2016 17:07:21
-0400 -----
To: gpfsug main discussion list
<gpfsug-discuss at spectrumscale.org>
Subject: Re: [gpfsug-discuss] GPFS 3.5.0 on RHEL 6.8
In the message dated: Tue, 30 Aug 2016 22:39:18 +0200,
The pithy ruminations from Lukas Hejtmanek on
<[gpfsug-discuss] GPFS 3.5.0 on RHEL 6.8> were:
=> Hello,
GPFS 3.5.0.[23..3-0] work for me under [CentOS|ScientificLinux] 6.8,
but at kernel 2.6.32-573 and lower.
I've found kernel bugs in blk_cloned_rq_check_limits() in later kernel
revs that caused multipath errors, resulting in GPFS being unable to
find all NSDs and mount the filesystem.
I am not updating to a newer kernel until I'm certain this is resolved.
I opened a bug with CentOS:
https://bugs.centos.org/view.php?id=10997
and began an extended discussion with the (RH & SUSE) developers of that
chunk of kernel code. I don't know if an upstream bug has been opened
by RH, but see:
https://patchwork.kernel.org/patch/9140337/
=>
=> does it work for anyone? As of kernel 2.6.32-642, GPFS 3.5.0 (including
the
=> latest patch 32) does start but does not mount and file system. The
internal
=> mount cmd gets stucked.
=>
=> --
=> Lukáš Hejtmánek
--
Mark Bergman voice: 215-746-4061
mark.bergman at uphs.upenn.edu fax: 215-614-0266
http://www.cbica.upenn.edu/
IT Technical Director, Center for Biomedical Image Computing and Analytics
Department of Radiology University of Pennsylvania
PGP Key: http://www.cbica.upenn.edu/sbia/bergman
----- Message from Lukas Hejtmanek <xhejtman at ics.muni.cz> on Wed, 31 Aug
2016 00:02:50 +0200 -----
To: gpfsug main discussion list <gpfsug-discuss at spectrumscale.org>
Subject: Re: [gpfsug-discuss] *New* IBM Spectrum Protect Whitepaper "Petascale
Data Protection"
Hello,
On Mon, Aug 29, 2016 at 09:20:46AM +0200, Frank Kraemer wrote:
> Find the paper here:
>
>
https://www.ibm.com/developerworks/community/wikis/home?lang=en#!/wiki/Tivoli%20Storage%20Manager/page/Petascale%20Data%20Protection
thank you for the paper, I appreciate it.
However, I wonder whether it could be extended a little. As it has the
title
Petascale Data Protection, I think that in Peta scale, you have to deal
with
millions (well rather hundreds of millions) of files you store in and this
is
something where TSM does not scale well.
Could you give some hints:
On the backup site:
mmbackup takes ages for:
a) scan (try to scan 500M files even in parallel)
b) backup - what if 10 % of files get changed - backup process can be
blocked
several days as mmbackup cannot run in several instances on the same file
system, so you have to wait until one run of mmbackup finishes. How long
could
it take at petascale?
On the restore site:
how can I restore e.g. 40 millions of file efficiently? dsmc restore
'/path/*'
runs into serious troubles after say 20M files (maybe wrong internal
structures used), however, scanning 1000 more files takes several minutes
resulting the dsmc restore never reaches that 40M files.
using filelists the situation is even worse. I run dsmc restore -filelist
with a filelist consisting of 2.4M files. Running for *two* days without
restoring even a single file. dsmc is consuming 100 % CPU.
So any hints addressing these issues with really large number of files
would
be even more appreciated.
--
Lukáš Hejtmánek
----- Message from Sven Oehme <oehmes at gmail.com> on Tue, 30 Aug 2016
16:24:59 -0700 -----
To: gpfsug main discussion list <gpfsug-discuss at spectrumscale.org>
Subject: Re: [gpfsug-discuss] *New* IBM Spectrum Protect Whitepaper "Petascale
Data Protection"
so lets start with some simple questions.
when you say mmbackup takes ages, what version of gpfs code are you
running ?
how do you execute the mmbackup command ? exact parameters would be
useful .
what HW are you using for the metadata disks ?
how much capacity (df -h) and how many inodes (df -i) do you have in the
filesystem you try to backup ?
sven
On Tue, Aug 30, 2016 at 3:02 PM, Lukas Hejtmanek <xhejtman at ics.muni.cz>
wrote:
Hello,
On Mon, Aug 29, 2016 at 09:20:46AM +0200, Frank Kraemer wrote:
> Find the paper here:
>
>
https://www.ibm.com/developerworks/community/wikis/home?lang=en#!/wiki/Tivoli%20Storage%20Manager/page/Petascale%20Data%20Protection
thank you for the paper, I appreciate it.
However, I wonder whether it could be extended a little. As it has
the title
Petascale Data Protection, I think that in Peta scale, you have to
deal with
millions (well rather hundreds of millions) of files you store in and
this is
something where TSM does not scale well.
Could you give some hints:
On the backup site:
mmbackup takes ages for:
a) scan (try to scan 500M files even in parallel)
b) backup - what if 10 % of files get changed - backup process can be
blocked
several days as mmbackup cannot run in several instances on the same
file
system, so you have to wait until one run of mmbackup finishes. How
long could
it take at petascale?
On the restore site:
how can I restore e.g. 40 millions of file efficiently? dsmc restore
'/path/*'
runs into serious troubles after say 20M files (maybe wrong internal
structures used), however, scanning 1000 more files takes several
minutes
resulting the dsmc restore never reaches that 40M files.
using filelists the situation is even worse. I run dsmc restore
-filelist
with a filelist consisting of 2.4M files. Running for *two* days
without
restoring even a single file. dsmc is consuming 100 % CPU.
So any hints addressing these issues with really large number of
files would
be even more appreciated.
--
Lukáš Hejtmánek
_______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at spectrumscale.org
http://gpfsug.org/mailman/listinfo/gpfsug-discuss
_______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at spectrumscale.org
http://gpfsug.org/mailman/listinfo/gpfsug-discuss
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20160831/f3574c25/attachment.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: graycol.gif
Type: image/gif
Size: 105 bytes
Desc: not available
URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20160831/f3574c25/attachment.gif>
More information about the gpfsug-discuss
mailing list