[gpfsug-discuss] Migrating billions of files? mmfind ... mmxcp
Simon Thompson
S.J.Thompson at bham.ac.uk
Wed Mar 6 15:07:09 GMT 2019
Last time this was mentioned, it doesn't do ACLs?
Simon
________________________________________
From: gpfsug-discuss-bounces at spectrumscale.org [gpfsug-discuss-bounces at spectrumscale.org] on behalf of makaplan at us.ibm.com [makaplan at us.ibm.com]
Sent: 06 March 2019 15:01
To: gpfsug main discussion list
Cc: gpfsug-discuss-bounces at spectrumscale.org
Subject: Re: [gpfsug-discuss] Migrating billions of files? mmfind ... mmxcp
mmxcp may be in samples/ilm if not, perhaps we can put it on an approved file sharing service ...
+ mmxcp script, for use with mmfind ... -xargs mmxcp ...
Which makes parallelized file copy relatively easy and super fast!
Usage: /gh/bin/mmxcp -t target -p strip_count source_pathname1 source_pathname2 ...
Run "cp" in a mmfind ... -xarg ... pipeline, e.g.
mmfind -polFlags '-N all -g /gpfs/tmp' /gpfs/source -gpfsWeight DIRECTORY_HASH -xargs mmxcp -t /target -p 2
Options:
-t target_path : Copy files to this path.
-p strip_count : Remove this many directory names from the pathnames of the source files.
-a : pass -a to cp
-v : pass -v to cp
[Marc A Kaplan]
-------------- next part --------------
A non-text attachment was scrubbed...
Name: ATT00001.gif
Type: image/gif
Size: 21994 bytes
Desc: ATT00001.gif
URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20190306/6d5db23c/attachment.gif>
More information about the gpfsug-discuss
mailing list