[gpfsug-discuss] Online data migration tool

[Sven Oehme]

thats not how GPFS aehm Scale works :-)
each client has pre-allocated inodes in memory and creating files is a
matter of spooling records. yes, eventually you need to destage this to the
disk, but that happens only every few seconds and given this i/os are
usually very colocated so good storage cache technology can reduce i/os to
physical media significant.

to proof the point look at this numbers :
-- started at 10/17/2017 14:29:13 --

mdtest-1.9.3 was launched with 110 total task(s) on 11 node(s)
Command line used: /ghome/oehmes/mpi/bin/mdtest-pcmpi9131-existingdir -d
/ibm/fs2-16m-09/shared/mdtest-ec -i 1 -n 10000 -F -w 0 -Z -p 8 -N 11 -u
Path: /ibm/fs2-16m-09/shared
FS: 128.1 TiB   Used FS: 0.2%   Inodes: 476.8 Mi   Used Inodes: 0.0%

110 tasks, 1100000 files

SUMMARY: (of 1 iterations)
   Operation                      Max            Min           Mean
Std Dev
   ---------                      ---            ---           ----
-------
   File creation     :     444221.343     444221.343     444221.343
  0.000
   File stat         :    6704498.841    6704498.841    6704498.841
  0.000
   File read         :    3859105.596    3859105.596    3859105.596
  0.000
   File removal      :     409336.606     409336.606     409336.606
  0.000
   Tree creation     :          5.344          5.344          5.344
  0.000
   Tree removal      :          1.145          1.145          1.145
  0.000

-- finished at 10/17/2017 14:29:27 --

this is a run against a 16mb blocksize filesystem with only spinning disks
(just one GL6 ESS) , not a single SSD and as you can see , this system on
11 nodes produces 444k creates / second something far above and beyond of
what drives can do.

and yes i know this stuff is all very complicated and not easy to explain
:-)

sven


On Thu, Dec 21, 2017 at 8:35 PM <valdis.kletnieks at vt.edu> wrote:

> On Thu, 21 Dec 2017 16:38:27 +0000, Sven Oehme said:
>
> > size. so if you now go to a 16MB blocksize and you have just 50 iops @
> 2MB
> > each you can write ~800 MB/sec with the exact same setup and same size
> > small writes, that's a factor of 8 .
>
> That's assuming your metadata storage is able to handle
> open/read/write/close
> on enough small files per second to push 800MB/sec.  If you're talking
> 128K subblocks,
> you're going to need some 6,400 small files per second to fill that pipe...
> _______________________________________________
> gpfsug-discuss mailing list
> gpfsug-discuss at spectrumscale.org
> http://gpfsug.org/mailman/listinfo/gpfsug-discuss
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20171222/a66aca84/attachment.htm>