[gpfsug-discuss] Online data migration tool

Fri Dec 1 12:44:38 GMT 2017

> "It has a significant performance penalty for small files in large
> block size filesystems"

Aaron,

Below are mdtest results for a test we ran for CORAL - file size was 32k.

We have not gone back and ran the test on a file system formatted without >
32 subblocks.  We'll do that at some point...

-Bill

-- started at 10/28/2017 17:51:38 --

mdtest-1.9.3 was launched with 228 total task(s) on 12 node(s)
Command line used: /tmp/mdtest-binary-dir/mdtest
-d /ibm/fs2-16m-10/mdtest-60000 -i 3 -n 294912 -w 32768 -C -F -r -p 360 -u
-y
Path: /ibm/fs2-16m-10
FS: 128.1 TiB   Used FS: 0.3%   Inodes: 476.8 Mi   Used Inodes: 0.0%

228 tasks, 67239936 files

SUMMARY: (of 3 iterations)
   Operation                      Max            Min           Mean
Std Dev
   ---------                      ---            ---           ----
-------
   File creation     :      51953.498      50558.517      51423.221
616.643
   File stat         :          0.000          0.000          0.000
0.000
   File read         :          0.000          0.000          0.000
0.000
   File removal      :      96746.376      92149.535      94658.774
1900.187
   Tree creation     :          1.588          0.070          0.599
0.700
   Tree removal      :          0.213          0.034          0.097
0.082

-- finished at 10/28/2017 19:51:54 --

Bill Hartner
IBM Systems
Scalable I/O Development
Austin, Texas
bhartner at us.ibm.com
home office 512-784-0980

gpfsug-discuss-bounces at spectrumscale.org wrote on 11/29/2017 04:41:48 PM:

> From: Aaron Knister <aaron.knister at gmail.com>
> To: gpfsug main discussion list <gpfsug-discuss at spectrumscale.org>
> Date: 11/29/2017 04:42 PM
> Subject: Re: [gpfsug-discuss] Online data migration tool
> Sent by: gpfsug-discuss-bounces at spectrumscale.org
>
> Thanks, Nikhil. Most of that was consistent with my understnading,
> however I was under the impression that the >32 subblocks code is
> required to achieve the touted 50k file creates/second that Sven has
> talked about a bunch of times:
>
>
http://files.gpfsug.org/presentations/2017/Manchester/08_Research_Topics.pdf

> http://files.gpfsug.org/presentations/2017/Ehningen/31_-_SSUG17DE_-
> _Sven_Oehme_-_News_from_Research.pdf
> http://files.gpfsug.org/presentations/2016/SC16/12_-
> _Sven_Oehme_Dean_Hildebrand_-_News_from_IBM_Research.pdf

> from those presentations regarding 32 subblocks:
>
> "It has a significant performance penalty for small files in large
> block size filesystems"

> although I'm not clear on the specific definition of "large". Many
> filesystems I encounter only have a 1M block size so it may not
> matter there, although that same presentation clearly shows the
> benefit of larger block sizes which is yet *another* thing for which
> a migration tool would be helpful.

> -Aaron
>
> On Wed, Nov 29, 2017 at 2:08 PM, Nikhil Khandelwal <nikhilk at us.ibm.com>
wrote:
> Hi,
>
> I would like to clarify migration path to 5.0.0 from 4.X.X clusters.
> For all Spectrum Scale clusters that are currently at 4.X.X, it is
> possible to migrate to 5.0.0 with no offline data migration and no
> need to move data. Once these clusters are at 5.0.0, they will
> benefit from the performance improvements, new features (such as
> file audit logging), and various enhancements that are included in 5.0.0.
>
> That being said, there is one enhancement that will not be applied
> to these clusters, and that is the increased number of sub-blocks
> per block for small file allocation. This means that for file
> systems with a large block size and a lot of small files, the
> overall space utilization will be the same it currently is in 4.X.X.
> Since file systems created at 4.X.X and earlier used a block size
> that kept this allocation in mind, there should be very little
> impact on existing file systems.
>
> Outside of that one particular function, the remainder of the
> performance improvements, metadata improvements, updated
> compatibility, new functionality, and all of the other enhancements
> will be immediately available to you once you complete the upgrade
> to 5.0.0 -- with no need to reformat, move data, or take your data
offline.
>
> I hope that clarifies things a little and makes the upgrade path
> more accessible.
>
> Please let me know if there are any other questions or concerns.
>
> Thank you,
> Nikhil Khandelwal
> Spectrum Scale Development
> Client Adoption
>
> _______________________________________________
> gpfsug-discuss mailing list
> gpfsug-discuss at spectrumscale.org
> http://gpfsug.org/mailman/listinfo/gpfsug-discuss

> _______________________________________________
> gpfsug-discuss mailing list
> gpfsug-discuss at spectrumscale.org
> https://urldefense.proofpoint.com/v2/url?
>
u=http-3A__gpfsug.org_mailman_listinfo_gpfsug-2Ddiscuss&d=DwICAg&c=jf_iaSHvJObTbx-

>
siA1ZOg&r=Ew59QH6nxuyx6oTs7a8AYX7kKG3gaWUGDGo5ZZr3wQ4&m=DHoqgBeMFgcM0LpXEI0VCYvvb8ollct5aSYUDln2t68&s=iOxGm-853L_W0XkB3jGsGzCTVlSYUvANOTSewcR_Ue8&e=

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20171201/18d266d7/attachment.htm>