[gpfsug-discuss] Online data migration tool
Sven Oehme
oehmes at gmail.com
Thu Dec 21 16:38:27 GMT 2017
Daniel,
while this might be easier to think about it, its not true :-)
lets just use an example. a disk drive can do 100 io's per second with
128kb random writes and 80 iops with 256kb writes . now lets do the math
with a 8+2p setup for each of the 2 cases. this means you can do 100 times
1mb writes (8*128k) or 80 times 2 mb writes so 100 MB/sec or 160 MB/sec
with the exact same drives. given you can fit 2 times as many subblocks
into the 2mb block you would gain 60% of speed by just going to this larger
size. so if you now go to a 16MB blocksize and you have just 50 iops @ 2MB
each you can write ~800 MB/sec with the exact same setup and same size
small writes, that's a factor of 8 .
so i/o size AND number of subblocks matter.
Sven
On Thu, Dec 21, 2017 at 12:22 PM Daniel Kidger <daniel.kidger at uk.ibm.com>
wrote:
> My suggestion is that it is better to not think of the performance coming
> from having more than 32 sub-blocks but instead that the performance comes
> from smaller sub-blocks. The fact that there are now more of them in say a
> 4MB blocksize filesytem is just a side effect.
>
> Daniel
> [image: /spectrum_storage-banne]
>
>
> [image: Spectrum Scale Logo]
>
>
> *Dr Daniel Kidger*
> IBM Technical Sales Specialist
> Software Defined Solution Sales
>
> + <+%2044-7818%20522%20266> 44-(0)7818 522 266 <+%2044-7818%20522%20266>
> daniel.kidger at uk.ibm.com
>
> On 19 Dec 2017, at 21:32, Aaron Knister <aaron.s.knister at nasa.gov> wrote:
>
> Thanks, Sven. Understood!
>
> On 12/19/17 3:20 PM, Sven Oehme wrote:
>
> Hi,
>
>
> the zero padding was never promoted into a GA stream, it was an
>
> experiment to proof we are on the right track when we eliminate the
>
> overhead from client to NSD Server, but also showed that alone is not
>
> good enough. the work for the client is the same compared to the >32
>
> subblocks, but the NSD Server has more work as it can't pack as many
>
> subblocks and therefore files into larger blocks, so you need to do more
>
> writes to store the same number of files.
>
> thats why there is the additional substantial improvement when we then
>
> went to >32 subblocks.
>
>
> sven
>
>
> On Mon, Dec 18, 2017 at 9:13 PM Knister, Aaron S. (GSFC-606.2)[COMPUTER
>
> SCIENCE CORP] <aaron.s.knister at nasa.gov
>
> <mailto:aaron.s.knister at nasa.gov <aaron.s.knister at nasa.gov>>> wrote:
>
>
> Thanks Sven! That makes sense to me and is what I thought was the
>
> case which is why I was confused when I saw the reply to the thread
>
> that said the >32 subblocks code had no performance impact.
>
>
> A couple more question for you— in your presentation there’s a
>
> benchmark that shows the file create performance without the zero
>
> padding. Since you mention this is done for security reasons was
>
> that feature ever promoted to a GA Scale release? I’m also wondering
>
> if you could explain the performance difference between the no zero
>
> padding code and the > 32 subblock code since given your the example
>
> of 32K files and 16MB block size I figure both cases ought to write
>
> the same amount to disk.
>
>
> Thanks!
>
>
> -Aaron
>
>
>
>
>
>
> On December 15, 2017 at 18:07:23 EST, Sven Oehme <oehmes at gmail.com
>
> <mailto:oehmes at gmail.com <oehmes at gmail.com>>> wrote:
>
> i thought i answered that already, but maybe i just thought about
>
> answering it and then forgot about it :-D
>
>
> so yes more than 32 subblocks per block significant increase the
>
> performance of filesystems with small files, for the sake of the
>
> argument let's say 32k in a large block filesystem again for sake
>
> of argument say 16MB.
>
>
> you probably ask why ?
>
>
> if you create a file and write 32k into it in a pre 5.0.0 Version
>
> 16 MB filesystem your client actually doesn't write 32k to the NSD
>
> Server, it writes 512k, because thats the subblock size and we
>
> need to write the full subblock (for security reasons). so first
>
> you waste significant memory on the client to cache that zero
>
> padding, you waste network bandwidth and you waste NSD Server
>
> cache because you store it there too. this means you overrun the
>
> cache more quickly, means you start doing read/modify writes
>
> earlier on all your nice large raid tracks... i guess you get the
>
> story by now.
>
>
> in fact, if you have a good raid code that can drive really a lot
>
> of bandwidth out of individual drives like a GNR system you get
>
> more performance for small file writes as larger your blocksize
>
> is, because we can 'pack' more files into larger i/os and
>
> therefore turn a small file create workload into a bandwidth
>
> workload, essentially exactly what we did and i demonstrated in
>
> the CORAL presentation .
>
>
> hope that makes this crystal clear now .
>
>
> sven
>
>
>
>
> On Fri, Dec 15, 2017 at 10:47 PM Aaron Knister
>
> <aaron.s.knister at nasa.gov <mailto:aaron.s.knister at nasa.gov
> <aaron.s.knister at nasa.gov>>> wrote:
>
>
> Thanks, Alex. I'm all too familiar with the trade offs between
>
> large
>
> blocks and small files and we do use pretty robust SSD storage
>
> for our
>
> metadata. We support a wide range of workloads and we have
>
> some folks
>
> with many small (<1M) files and other folks with many large
>
> (>256MB) files.
>
>
> My point in this thread is that IBM has said over and over
>
> again in
>
> presentations that there is a significant performance gain
>
> with the >32
>
> subblocks code on filesystems with large block sizes (although
>
> to your
>
> point I'm not clear on exactly what large means since I didn't
>
> define
>
> large in this context). Therefore given that the >32 subblock
>
> code gives
>
> a significant performance gain one could reasonably assume
>
> that having a
>
> filesystem with >32 subblocks is required to see this gain
>
> (rather than
>
> just running the >32 subblocks code on an fs w/o > 32 subblocks).
>
>
> This lead me to ask about a migration tool because in my mind
>
> if there's
>
> a performance gain from having >32 subblocks on the FS I'd
>
> like that
>
> feature and having to manually copy 10's of PB to new hardware
>
> to get
>
> this performance boost is unacceptable. However, IBM can't
>
> seem to make
>
> up their mind about whether or not the >32 subblocks code
>
> *actually*
>
> provides a performance increase. This seems like a pretty
>
> straightforward question.
>
>
> -Aaron
>
>
> On 12/15/17 3:48 PM, Alex Chekholko wrote:
>
> Hey Aaron,
>
>
> Can you define your sizes for "large blocks" and "small
>
> files"? If you
>
> dial one up and the other down, your performance will be
>
> worse. And in
>
> any case it's a pathological corner case so it shouldn't
>
> matter much for
>
> your workflow, unless you've designed your system with the
>
> wrong values.
>
>
> For example, for bioinformatics workloads, I prefer to use 256KB
>
> filesystem block size, and I'd consider 4MB+ to be "large
>
> block size",
>
> which would make the filesystem obviously unsuitable for
>
> processing
>
> millions of 8KB files.
>
>
> You can make a histogram of file sizes in your existing
>
> filesystems and
>
> then make your subblock size (1/32 of block size) on the
>
> smaller end of
>
> that. Also definitely use the "small file in inode"
>
> feature and put
>
> your metadata on SSD.
>
>
> Regards,
>
> Alex
>
>
> On Fri, Dec 15, 2017 at 11:49 AM, Aaron Knister
>
> <aaron.s.knister at nasa.gov <mailto:aaron.s.knister at nasa.gov
> <aaron.s.knister at nasa.gov>>
>
> <mailto:aaron.s.knister at nasa.gov <aaron.s.knister at nasa.gov>
>
> <mailto:aaron.s.knister at nasa.gov <aaron.s.knister at nasa.gov>>>>
> wrote:
>
>
> Thanks, Bill.
>
>
> I still don't feel like I've got an clear answer from
>
> IBM and frankly
>
> the core issue of a lack of migration tool was totally
>
> dodged.
>
>
> Again in Sven's presentation from SSUG @ SC17
>
>
>
> (
> https://urldefense.proofpoint.com/v2/url?u=http-3A__files.gpfsug.org_presentations_2017_SC17_SC17-2DUG-2DCORAL-5FV3.pdf&d=DwIGaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=HlQDuUjgJx4p54QzcXd0_zTwf4Cr2t3NINalNhLTA2E&m=ROiUtPAdbQ6DF9wWYS4MIUax_Xetm1p9qXbKzt6ZVf4&s=EdlC_gbmU-xxT7HcFq8IYttHSMts8BdrbqDSCqnt-_g&e=
>
> <
> https://urldefense.proofpoint.com/v2/url?u=http-3A__files.gpfsug.org_presentations_2017_SC17_SC17-2DUG-2DCORAL-5FV3.pdf&d=DwIGaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=HlQDuUjgJx4p54QzcXd0_zTwf4Cr2t3NINalNhLTA2E&m=ROiUtPAdbQ6DF9wWYS4MIUax_Xetm1p9qXbKzt6ZVf4&s=EdlC_gbmU-xxT7HcFq8IYttHSMts8BdrbqDSCqnt-_g&e=
> >)
>
> he mentions "It has a significant performance penalty
>
> for small files in
>
> large block size filesystems" and the demonstrates that
>
> with several
>
> mdtest runs (which show the effect with and without the >32
>
> subblocks code):
>
>
>
> 4.2.1 base code - SUMMARY: (of 3 iterations)
>
> File creation : Mean = 2237.644
>
>
> zero-end-of-file-padding (4.2.2 + ifdef for zero
>
> padding): SUMMARY: (of
>
> 3 iterations)
>
> File creation : Mean = 12866.842
>
>
> more sub blocks per block (4.2.2 + morethan32subblock code):
>
> File creation : Mean = 40316.721
>
>
> Can someone (ideally Sven) give me a straight answer as
>
> to whether or
>
> not the > 32 subblock code actually makes a performance
>
> difference for
>
> small files in large block filesystems? And if not, help
>
> me understand
>
> why his slides and provided benchmark data have
>
> consistently indicated
>
> it does?
>
>
> -Aaron
>
>
> On 12/1/17 11:44 AM, Bill Hartner wrote:
>
> > ESS GL4 4u106 w/ 10 TB drives - same HW Sven reported
>
> some of the
>
> > results @ user group meeting.
>
> >
>
> > -Bill
>
> >
>
> > Bill Hartner
>
> > IBM Systems
>
> > Scalable I/O Development
>
> > Austin, Texas
>
> > bhartner at us.ibm.com <mailto:bhartner at us.ibm.com
> <bhartner at us.ibm.com>>
>
> <mailto:bhartner at us.ibm.com <bhartner at us.ibm.com> <
> mailto:bhartner at us.ibm.com <bhartner at us.ibm.com>>>
>
> > home office 512-784-0980 <(512)%20784-0980> <tel:(512)%20784-0980>
>
> <tel:512-784-0980 <512-784-0980> <tel:(512)%20784-0980>>
>
> >
>
> >
>
> > Inactive hide details for Jan-Frode Myklebust
>
> ---12/01/2017 06:53:44
>
> > AM---Bill, could you say something about what the
>
> metadataJan-Frode
>
> > Myklebust ---12/01/2017 06:53:44 AM---Bill, could you
>
> say something
>
> > about what the metadata-storage here was?
>
> ESS/NL-SAS/3way replication?
>
> >
>
> > From: Jan-Frode Myklebust <janfrode at tanso.net
>
> <mailto:janfrode at tanso.net <janfrode at tanso.net>> <
> mailto:janfrode at tanso.net <janfrode at tanso.net>
>
> <mailto:janfrode at tanso.net <janfrode at tanso.net>>>>
>
> > To: gpfsug main discussion list
>
> <gpfsug-discuss at spectrumscale.org
>
> <mailto:gpfsug-discuss at spectrumscale.org
> <gpfsug-discuss at spectrumscale.org>>
>
> <mailto:gpfsug-discuss at spectrumscale.org
> <gpfsug-discuss at spectrumscale.org>
>
> <mailto:gpfsug-discuss at spectrumscale.org
> <gpfsug-discuss at spectrumscale.org>>>>
>
> > Date: 12/01/2017 06:53 AM
>
> > Subject: Re: [gpfsug-discuss] Online data migration tool
>
> > Sent by: gpfsug-discuss-bounces at spectrumscale.org
>
> <mailto:gpfsug-discuss-bounces at spectrumscale.org
> <gpfsug-discuss-bounces at spectrumscale.org>>
>
> <mailto:gpfsug-discuss-bounces at spectrumscale.org
> <gpfsug-discuss-bounces at spectrumscale.org>
>
> <mailto:gpfsug-discuss-bounces at spectrumscale.org
> <gpfsug-discuss-bounces at spectrumscale.org>>>
>
> >
>
> >
>
>
>
>
> ------------------------------------------------------------------------
>
> >
>
> >
>
> >
>
> > Bill, could you say something about what the
>
> metadata-storage here was?
>
> > ESS/NL-SAS/3way replication?
>
> >
>
> > I just asked about this in the internal slack channel
>
> #scale-help today..
>
> >
>
> >
>
> >
>
> > -jf
>
> >
>
> > fre. 1. des. 2017 kl. 13:44 skrev Bill Hartner
>
> <_bhartner at us.ibm.com_
>
> > <mailto:bhartner at us.ibm.com <bhartner at us.ibm.com>
>
> <mailto:bhartner at us.ibm.com <bhartner at us.ibm.com>> <
> mailto:bhartner at us.ibm.com <bhartner at us.ibm.com>
>
> <mailto:bhartner at us.ibm.com <bhartner at us.ibm.com>>>>>:
>
> >
>
> > > "It has a significant performance penalty for
>
> small files in
>
> large
>
> > > block size filesystems"
>
> >
>
> > Aaron,
>
> >
>
> > Below are mdtest results for a test we ran for
>
> CORAL - file
>
> size was
>
> > 32k.
>
> >
>
> > We have not gone back and ran the test on a file
>
> system formatted
>
> > without > 32 subblocks. We'll do that at some point...
>
> >
>
> > -Bill
>
> >
>
> > -- started at 10/28/2017 17:51:38 --
>
> >
>
> > mdtest-1.9.3 was launched with 228 total task(s)
>
> on 12 node(s)
>
> > Command line used: /tmp/mdtest-binary-dir/mdtest -d
>
> > /ibm/fs2-16m-10/mdtest-60000 -i 3 -n 294912 -w
>
> 32768 -C -F -r
>
> -p 360
>
> > -u -y
>
> > Path: /ibm/fs2-16m-10
>
> > FS: 128.1 TiB Used FS: 0.3% Inodes: 476.8 Mi Used
>
> Inodes: 0.0%
>
> >
>
> > 228 tasks, 67239936 files
>
> >
>
> > SUMMARY: (of 3 iterations)
>
> > Operation Max Min Mean Std Dev
>
> > --------- --- --- ---- -------
>
> > File creation : 51953.498 50558.517 51423.221 616.643
>
> > File stat : 0.000 0.000 0.000 0.000
>
> > File read : 0.000 0.000 0.000 0.000
>
> > File removal : 96746.376 92149.535 94658.774 1900.187
>
> > Tree creation : 1.588 0.070 0.599 0.700
>
> > Tree removal : 0.213 0.034 0.097 0.082
>
> >
>
> > -- finished at 10/28/2017 19:51:54 --
>
> >
>
> > Bill Hartner
>
> > IBM Systems
>
> > Scalable I/O Development
>
> > Austin, Texas_
>
> > __bhartner at us.ibm.com_ <mailto:bhartner at us.ibm.com
> <bhartner at us.ibm.com>
>
> <mailto:bhartner at us.ibm.com <bhartner at us.ibm.com>>
>
> <mailto:bhartner at us.ibm.com <bhartner at us.ibm.com> <
> mailto:bhartner at us.ibm.com <bhartner at us.ibm.com>>>>
>
> > home office 512-784-0980 <(512)%20784-0980>
> <tel:(512)%20784-0980>
>
> <tel:512-784-0980 <512-784-0980> <tel:(512)%20784-0980>>
>
> >
>
> > _
>
> > __gpfsug-discuss-bounces at spectrumscale.org_
>
> > <mailto:gpfsug-discuss-bounces at spectrumscale.org
> <gpfsug-discuss-bounces at spectrumscale.org>
>
> <mailto:gpfsug-discuss-bounces at spectrumscale.org
> <gpfsug-discuss-bounces at spectrumscale.org>>
>
> <mailto:gpfsug-discuss-bounces at spectrumscale.org
> <gpfsug-discuss-bounces at spectrumscale.org>
>
> <mailto:gpfsug-discuss-bounces at spectrumscale.org
> <gpfsug-discuss-bounces at spectrumscale.org>>>> wrote on
>
> > 11/29/2017 04:41:48 PM:
>
> >
>
> > > From: Aaron Knister <_aaron.knister at gmail.com_
>
> > <mailto:aaron.knister at gmail.com <aaron.knister at gmail.com>
>
> <mailto:aaron.knister at gmail.com <aaron.knister at gmail.com>>
>
> <mailto:aaron.knister at gmail.com <aaron.knister at gmail.com>
>
> <mailto:aaron.knister at gmail.com <aaron.knister at gmail.com>>>>>
>
> >
>
> >
>
> > > To: gpfsug main discussion list
>
> > <_gpfsug-discuss at spectrumscale.org_
>
> > <mailto:gpfsug-discuss at spectrumscale.org
> <gpfsug-discuss at spectrumscale.org>
>
> <mailto:gpfsug-discuss at spectrumscale.org
> <gpfsug-discuss at spectrumscale.org>>
>
> <mailto:gpfsug-discuss at spectrumscale.org
> <gpfsug-discuss at spectrumscale.org>
>
> <mailto:gpfsug-discuss at spectrumscale.org
> <gpfsug-discuss at spectrumscale.org>>>>>
>
> >
>
> > > Date: 11/29/2017 04:42 PM
>
> >
>
> >
>
> > > Subject: Re: [gpfsug-discuss] Online data
>
> migration tool
>
> > > Sent by: _gpfsug-discuss-bounces at spectrumscale.org_
>
> > <mailto:gpfsug-discuss-bounces at spectrumscale.org
> <gpfsug-discuss-bounces at spectrumscale.org>
>
> <mailto:gpfsug-discuss-bounces at spectrumscale.org
> <gpfsug-discuss-bounces at spectrumscale.org>>
>
> <mailto:gpfsug-discuss-bounces at spectrumscale.org
> <gpfsug-discuss-bounces at spectrumscale.org>
>
> <mailto:gpfsug-discuss-bounces at spectrumscale.org
> <gpfsug-discuss-bounces at spectrumscale.org>>>>
>
> >
>
> > >
>
> >
>
> > > Thanks, Nikhil. Most of that was consistent with
>
> my understnading,
>
> > > however I was under the impression that the >32
>
> subblocks code is
>
> > > required to achieve the touted 50k file
>
> creates/second that Sven has
>
> > > talked about a bunch of times:
>
> > >
>
> > >
>
> >
>
>
>
> _
> https://urldefense.proofpoint.com/v2/url?u=http-3A__files.gpfsug.org_presentations_2017_Manchester_08-5FResearch-5FTopics.pdf-5F&d=DwIGaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=HlQDuUjgJx4p54QzcXd0_zTwf4Cr2t3NINalNhLTA2E&m=ROiUtPAdbQ6DF9wWYS4MIUax_Xetm1p9qXbKzt6ZVf4&s=V_Pb-mxqz3Ji9fHRp9Ic9_ztzMsHk1bSzTmhbgGkRKU&e=
>
>
>
> <
> https://urldefense.proofpoint.com/v2/url?u=http-3A__files.gpfsug.org_presentations_2017_Manchester_08-5FResearch-5FTopics.pdf-5F&d=DwIGaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=HlQDuUjgJx4p54QzcXd0_zTwf4Cr2t3NINalNhLTA2E&m=ROiUtPAdbQ6DF9wWYS4MIUax_Xetm1p9qXbKzt6ZVf4&s=V_Pb-mxqz3Ji9fHRp9Ic9_ztzMsHk1bSzTmhbgGkRKU&e=
> >
>
> >
>
>
>
> <
> https://urldefense.proofpoint.com/v2/url?u=http-3A__files.gpfsug.org_presentations_2017_Manchester_08-5FResearch-5FTopics.pdf&d=DwMFaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=Ew59QH6nxuyx6oTs7a8AYX7kKG3gaWUGDGo5ZZr3wQ4&m=KLv9eH4GG8WlXC5ENj_jXnzCpm60QSNAADfp6s94oa4&s=UGLr4Z6sa2yWvKL99g7SuQdgwxnoZwhVmDuIbYsLqYY&e=
>
>
>
> <
> https://urldefense.proofpoint.com/v2/url?u=http-3A__files.gpfsug.org_presentations_2017_Manchester_08-5FResearch-5FTopics.pdf&d=DwMFaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=Ew59QH6nxuyx6oTs7a8AYX7kKG3gaWUGDGo5ZZr3wQ4&m=KLv9eH4GG8WlXC5ENj_jXnzCpm60QSNAADfp6s94oa4&s=UGLr4Z6sa2yWvKL99g7SuQdgwxnoZwhVmDuIbYsLqYY&e=
> >>
>
> > >
>
> >
>
>
>
> _
> https://urldefense.proofpoint.com/v2/url?u=http-3A__files.gpfsug.org_presentations_2017_Ehningen_31-5F-2D-5FSSUG17DE-5F-2D-5F&d=DwIGaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=HlQDuUjgJx4p54QzcXd0_zTwf4Cr2t3NINalNhLTA2E&m=ROiUtPAdbQ6DF9wWYS4MIUax_Xetm1p9qXbKzt6ZVf4&s=61HBHh68SJXjnUv1Lyqjzmg_Vl24EG5cZ-0Z3WgLX3A&e=
>
>
>
> <
> https://urldefense.proofpoint.com/v2/url?u=http-3A__files.gpfsug.org_presentations_2017_Ehningen_31-5F-2D-5FSSUG17DE-5F-2D-5F&d=DwIGaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=HlQDuUjgJx4p54QzcXd0_zTwf4Cr2t3NINalNhLTA2E&m=ROiUtPAdbQ6DF9wWYS4MIUax_Xetm1p9qXbKzt6ZVf4&s=61HBHh68SJXjnUv1Lyqjzmg_Vl24EG5cZ-0Z3WgLX3A&e=
> >
>
>
>
> <
> https://urldefense.proofpoint.com/v2/url?u=http-3A__files.gpfsug.org_presentations_2017_Ehningen_31-5F-2D-5FSSUG17DE-5F-2D&d=DwMFaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=Ew59QH6nxuyx6oTs7a8AYX7kKG3gaWUGDGo5ZZr3wQ4&m=KLv9eH4GG8WlXC5ENj_jXnzCpm60QSNAADfp6s94oa4&s=Il2rMx4AtGwjVRzX89kobZ0W25vW8TGm0KJevLd7KQ8&e=
>
>
>
> <
> https://urldefense.proofpoint.com/v2/url?u=http-3A__files.gpfsug.org_presentations_2017_Ehningen_31-5F-2D-5FSSUG17DE-5F-2D&d=DwMFaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=Ew59QH6nxuyx6oTs7a8AYX7kKG3gaWUGDGo5ZZr3wQ4&m=KLv9eH4GG8WlXC5ENj_jXnzCpm60QSNAADfp6s94oa4&s=Il2rMx4AtGwjVRzX89kobZ0W25vW8TGm0KJevLd7KQ8&e=
> >>
>
> > > _Sven_Oehme_-_News_from_Research.pdf
>
> > >
>
> _
> https://urldefense.proofpoint.com/v2/url?u=http-3A__files.gpfsug.org_presentations_2016_SC16_12-5F-2D-5F&d=DwIGaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=HlQDuUjgJx4p54QzcXd0_zTwf4Cr2t3NINalNhLTA2E&m=ROiUtPAdbQ6DF9wWYS4MIUax_Xetm1p9qXbKzt6ZVf4&s=fDAdLyWu9yx3_uj0z_N3IQ98yjXF7q5hDrg7ZYZYtRE&e=
>
> <
> https://urldefense.proofpoint.com/v2/url?u=http-3A__files.gpfsug.org_presentations_2016_SC16_12-5F-2D-5F&d=DwIGaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=HlQDuUjgJx4p54QzcXd0_zTwf4Cr2t3NINalNhLTA2E&m=ROiUtPAdbQ6DF9wWYS4MIUax_Xetm1p9qXbKzt6ZVf4&s=fDAdLyWu9yx3_uj0z_N3IQ98yjXF7q5hDrg7ZYZYtRE&e=
> >
>
> >
>
>
>
> <
> https://urldefense.proofpoint.com/v2/url?u=http-3A__files.gpfsug.org_presentations_2016_SC16_12-5F-2D&d=DwMFaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=Ew59QH6nxuyx6oTs7a8AYX7kKG3gaWUGDGo5ZZr3wQ4&m=KLv9eH4GG8WlXC5ENj_jXnzCpm60QSNAADfp6s94oa4&s=u_qcvB--uvtByHp9H471EowagobMpPLXYT_FFzMkQiw&e=
>
>
>
> <
> https://urldefense.proofpoint.com/v2/url?u=http-3A__files.gpfsug.org_presentations_2016_SC16_12-5F-2D&d=DwMFaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=Ew59QH6nxuyx6oTs7a8AYX7kKG3gaWUGDGo5ZZr3wQ4&m=KLv9eH4GG8WlXC5ENj_jXnzCpm60QSNAADfp6s94oa4&s=u_qcvB--uvtByHp9H471EowagobMpPLXYT_FFzMkQiw&e=
> >>
>
> > >
>
> _Sven_Oehme_Dean_Hildebrand_-_News_from_IBM_Research.pdf
>
> >
>
> >
>
> > > from those presentations regarding 32 subblocks:
>
> > >
>
> > > "It has a significant performance penalty for
>
> small files in large
>
> > > block size filesystems"
>
> >
>
> > > although I'm not clear on the specific
>
> definition of "large". Many
>
> > > filesystems I encounter only have a 1M block
>
> size so it may not
>
> > > matter there, although that same presentation
>
> clearly shows the
>
> > > benefit of larger block sizes which is yet
>
> *another* thing for which
>
> > > a migration tool would be helpful.
>
> >
>
> > > -Aaron
>
> > >
>
> > > On Wed, Nov 29, 2017 at 2:08 PM, Nikhil Khandelwal
>
> > <_nikhilk at us.ibm.com_ <mailto:nikhilk at us.ibm.com
> <nikhilk at us.ibm.com>
>
> <mailto:nikhilk at us.ibm.com <nikhilk at us.ibm.com>>
>
> <mailto:nikhilk at us.ibm.com <nikhilk at us.ibm.com>
>
> <mailto:nikhilk at us.ibm.com <nikhilk at us.ibm.com>>>>> wrote:
>
> >
>
> > > Hi,
>
> > >
>
> > > I would like to clarify migration path to 5.0.0
>
> from 4.X.X
>
> clusters.
>
> > > For all Spectrum Scale clusters that are
>
> currently at 4.X.X,
>
> it is
>
> > > possible to migrate to 5.0.0 with no offline
>
> data migration
>
> and no
>
> > > need to move data. Once these clusters are at
>
> 5.0.0, they will
>
> > > benefit from the performance improvements, new
>
> features (such as
>
> > > file audit logging), and various enhancements
>
> that are
>
> included in
>
> > 5.0.0.
>
> > >
>
> > > That being said, there is one enhancement that
>
> will not be
>
> applied
>
> > > to these clusters, and that is the increased
>
> number of
>
> sub-blocks
>
> > > per block for small file allocation. This means
>
> that for file
>
> > > systems with a large block size and a lot of
>
> small files, the
>
> > > overall space utilization will be the same it
>
> currently is
>
> in 4.X.X.
>
> > > Since file systems created at 4.X.X and earlier
>
> used a block
>
> size
>
> > > that kept this allocation in mind, there should
>
> be very little
>
> > > impact on existing file systems.
>
> > >
>
> > > Outside of that one particular function, the
>
> remainder of the
>
> > > performance improvements, metadata improvements,
>
> updated
>
> > > compatibility, new functionality, and all of the
>
> other
>
> enhancements
>
> > > will be immediately available to you once you
>
> complete the
>
> upgrade
>
> > > to 5.0.0 -- with no need to reformat, move data,
>
> or take
>
> your data
>
> > offline.
>
> > >
>
> > > I hope that clarifies things a little and makes
>
> the upgrade path
>
> > > more accessible.
>
> > >
>
> > > Please let me know if there are any other
>
> questions or concerns.
>
> > >
>
> > > Thank you,
>
> > > Nikhil Khandelwal
>
> > > Spectrum Scale Development
>
> > > Client Adoption
>
> > >
>
> > > _______________________________________________
>
> > > gpfsug-discuss mailing list
>
> > > gpfsug-discuss at _spectrumscale.org_
>
> >
>
>
>
> <
> https://urldefense.proofpoint.com/v2/url?u=http-3A__spectrumscale.org&d=DwMFaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=Ew59QH6nxuyx6oTs7a8AYX7kKG3gaWUGDGo5ZZr3wQ4&m=KLv9eH4GG8WlXC5ENj_jXnzCpm60QSNAADfp6s94oa4&s=Q-P8kRqnjsWB7ePz6YtA3U0xguo7-lVWKmb_zyZPndE&e=
>
>
>
> <
> https://urldefense.proofpoint.com/v2/url?u=http-3A__spectrumscale.org&d=DwMFaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=Ew59QH6nxuyx6oTs7a8AYX7kKG3gaWUGDGo5ZZr3wQ4&m=KLv9eH4GG8WlXC5ENj_jXnzCpm60QSNAADfp6s94oa4&s=Q-P8kRqnjsWB7ePz6YtA3U0xguo7-lVWKmb_zyZPndE&e=
> >>
>
> > > _
> https://urldefense.proofpoint.com/v2/url?u=http-3A__gpfsug.org_mailman_listinfo_gpfsug-2Ddiscuss-5F&d=DwIGaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=HlQDuUjgJx4p54QzcXd0_zTwf4Cr2t3NINalNhLTA2E&m=ROiUtPAdbQ6DF9wWYS4MIUax_Xetm1p9qXbKzt6ZVf4&s=uD-N75Y8hXNsZ7FmnqLA4D6P8WsMrRGMIM9-Oy2vIgE&e=
>
> <
> https://urldefense.proofpoint.com/v2/url?u=http-3A__gpfsug.org_mailman_listinfo_gpfsug-2Ddiscuss-5F&d=DwIGaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=HlQDuUjgJx4p54QzcXd0_zTwf4Cr2t3NINalNhLTA2E&m=ROiUtPAdbQ6DF9wWYS4MIUax_Xetm1p9qXbKzt6ZVf4&s=uD-N75Y8hXNsZ7FmnqLA4D6P8WsMrRGMIM9-Oy2vIgE&e=
> >
>
> >
>
>
>
> <
> https://urldefense.proofpoint.com/v2/url?u=http-3A__gpfsug.org_mailman_listinfo_gpfsug-2Ddiscuss&d=DwMFaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=Ew59QH6nxuyx6oTs7a8AYX7kKG3gaWUGDGo5ZZr3wQ4&m=KLv9eH4GG8WlXC5ENj_jXnzCpm60QSNAADfp6s94oa4&s=WolSBY_TPJVJVPj5WEZ6JAbDZQK3j7oqn8u_Y5xORkE&e=
>
>
>
> <
> https://urldefense.proofpoint.com/v2/url?u=http-3A__gpfsug.org_mailman_listinfo_gpfsug-2Ddiscuss&d=DwMFaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=Ew59QH6nxuyx6oTs7a8AYX7kKG3gaWUGDGo5ZZr3wQ4&m=KLv9eH4GG8WlXC5ENj_jXnzCpm60QSNAADfp6s94oa4&s=WolSBY_TPJVJVPj5WEZ6JAbDZQK3j7oqn8u_Y5xORkE&e=
> >>
>
> >
>
> > > _______________________________________________
>
> > > gpfsug-discuss mailing list
>
> > > gpfsug-discuss at _spectrumscale.org_
>
> >
>
>
>
> <
> https://urldefense.proofpoint.com/v2/url?u=http-3A__spectrumscale.org&d=DwMFaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=Ew59QH6nxuyx6oTs7a8AYX7kKG3gaWUGDGo5ZZr3wQ4&m=KLv9eH4GG8WlXC5ENj_jXnzCpm60QSNAADfp6s94oa4&s=Q-P8kRqnjsWB7ePz6YtA3U0xguo7-lVWKmb_zyZPndE&e=
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20171221/6513c936/attachment.htm>
More information about the gpfsug-discuss
mailing list