[gpfsug-discuss] Combine different rules - tip: use mmfind & co; FOR FILESET; FILESET_NAME
Marc A Kaplan
makaplan at us.ibm.com
Wed Nov 1 14:43:27 GMT 2017
Thanks Jonathan B for your comments and tips on experience using
mmapplypolicy and policy rules. Good to see that some of the features we
put into the product are actually useful.
For those not quite as familiar, and have come somewhat later to the game,
like Matthias K - I have a few remarks and tips that may be helpful:
You can think of and use mmapplypolicy as a fast, parallelized version of
the classic `find ... | xargs ... ` pipeline.
In fact we've added some "sample" scripts with options that make this
easy:
samples/ilm/mmfind : "understands" the classic find search arguments as
well as all the mmapplypolicy options and the recent versions
also support an -xargs option so you can write the classic pipepline as
one command: mmfind ... -xargs ...
There are debug/diagnostic options so you can see the underlying GPFS
commands and policy rules that are generated, so if mmfind doesn't do
exactly what you were hoping, you can capture the commands and rules that
it does do and tweak/hack those.
Two of the most crucial and tricky parts of mmfind are available as
separate scripts that can be used separately:
tr_findToPol.pl : convert classic options to policy rules.
mmxargs : 100% correctly deal with the problem of whitespace and/or
"special" characters in the pathnames output as file lists by
mmapplypolicy. This is somewhat tricky.
EVEN IF you've already worked out your own policy rules and use policy
RULE ... EXTERNAL ... EXEC 'myscript'
you may want to use mmxargs or "lift" some of the code there-in -- because
it is very likely your 'myscript' is not handling the problem of special
characters correctly.
FILESETs vs POOLs - yes these are "orthogonal" concepts in GPFS (Spectrum
Scale!) BUT some customer/admins may choose to direct GPFS to assign to
POOL based on FILESET using policy rules clauses like:
FOR FILESET('a_fs', 'b_fs') /* handy to restrict a rule to one or a
few filesets */
WHERE ... AND (FILESET_NAME LIKE 'xyz_%') AND ... /* restrict to
filesets whose name matches a pattern */
-- marc of GPFS
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20171101/38dea8f1/attachment-0002.htm>
More information about the gpfsug-discuss
mailing list