[gpfsug-discuss] Migrating billions of files? mmfind ... mmxcp

Simon Thompson S.J.Thompson at bham.ac.uk
Wed Mar 6 15:07:09 GMT 2019


Last time this was mentioned, it doesn't do ACLs?

Simon
________________________________________
From: gpfsug-discuss-bounces at spectrumscale.org [gpfsug-discuss-bounces at spectrumscale.org] on behalf of makaplan at us.ibm.com [makaplan at us.ibm.com]
Sent: 06 March 2019 15:01
To: gpfsug main discussion list
Cc: gpfsug-discuss-bounces at spectrumscale.org
Subject: Re: [gpfsug-discuss] Migrating billions of files? mmfind ... mmxcp

mmxcp may be in samples/ilm  if not, perhaps we can put it on an approved file sharing service ...


   + mmxcp script, for use with mmfind ... -xargs mmxcp ...
      Which makes parallelized file copy relatively easy and super fast!

Usage: /gh/bin/mmxcp -t target -p strip_count source_pathname1 source_pathname2 ...

 Run "cp" in a  mmfind ... -xarg ... pipeline, e.g.

  mmfind -polFlags '-N all -g /gpfs/tmp' /gpfs/source -gpfsWeight DIRECTORY_HASH -xargs mmxcp -t /target -p 2

 Options:
  -t target_path : Copy files to this path.
  -p strip_count : Remove this many directory names from the pathnames of the source files.
  -a  : pass -a to cp
  -v  : pass -v to cp


[Marc A Kaplan]
-------------- next part --------------
A non-text attachment was scrubbed...
Name: ATT00001.gif
Type: image/gif
Size: 21994 bytes
Desc: ATT00001.gif
URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20190306/6d5db23c/attachment.gif>


More information about the gpfsug-discuss mailing list