[gpfsug-discuss] policy ilm features?

[Edward Wahl]

Thanks John, this was the path I was HOPING to go down as I do similar things
already, but there appears to be no extended attribute in ILM for what I want.
Data block replication flag exists in the ILM, but not MetaData, or balance.

Yet these states ARE reported by mmlsattr, so there must be a flag somewhere.


bad MD replication & balance example:

 mmlsattr -L /fs/scratch/sysp/ed/180days.pol 
file name:            /fs/scratch/sysp/ed/180days.pol
metadata replication: 1 max 2
data replication:     1 max 2
flags:                illreplicated,unbalanced
Encrypted:            yes

File next to it for comparison. note proper MD replication and balance.

 mmlsattr -L  /fs/scratch/sysp/ed/120days.pol 
file name:            /fs/scratch/sysp/ed/120days.pol
metadata replication: 2 max 2
data replication:     1 max 2
flags:                
Encrypted:            yes

misc_attributes flags from a policy run showing no difference in status:
FJAEu -- /fs/scratch/sysp/ed/180days.pol
FJAEu -- /fs/scratch/sysp/ed/120days.pol


File system has MD replication enabled, but not Data, so ALL files show "J" ilm
flag

mmlsfs scratch -m
flag                value                    description
------------------- ------------------------ -----------------------------------
 -m                 2                        Default number of metadata replicas
mmlsfs scratch -r
flag                value                    description
------------------- ------------------------ -----------------------------------
 -r                 1                        Default number of data replicas


I poked around a little trying to find out if perhaps using GetXattr would
work and show me what I wanted, it does not. All I sem to be able to get is the
File Encryption Key.


I was hoping perhaps someone had found a cheaper way for this to work rather
than hundreds of millions of 'mmlsattr' execs.  :-(

On the plus side, I've only run across a few of these and all appear to be
from before we did the MD replication and re-striping.  On the minus, I have NO
idea where they are, and they appears to be on both of our filesystems.  So
several hundred million files to check. 

Ed


On Mon, 22 Jan 2018 08:29:42 +0000
John Hearns <john.hearns at asml.com> wrote:

> Ed,
> This is not a perfect answer. You need to look at policies for this. I have
> been doing something similar recently.
> 
> Something like:
> 
> RULE 'list_file' EXTERNAL LIST 'all-files' EXEC
> '/var/mmfs/etc/mmpolicyExec-list' RULE 'listall' list 'all-files'
> SHOW( varchar(kb_allocated) || '  ' || varchar(file_size) || ' ' ||
> varchar(misc_attributes) || ' ' || name || ' ' || fileset_name  ) WHERE
> REGEX(misc_attributes,'[J]')
> 
> 
> So this policy shows the kbytes allocates, file size, the miscellaneous
> attributes, name and fileset name For all files with  miscellaneous
> attributes of 'J'   which means 'Some data blocks might be ill replicated'
> 
> 
> 
> 
> -----Original Message-----
> From: gpfsug-discuss-bounces at spectrumscale.org
> [mailto:gpfsug-discuss-bounces at spectrumscale.org] On Behalf Of Edward Wahl
> Sent: Friday, January 19, 2018 10:38 PM To: gpfsug-discuss at spectrumscale.org
> Subject: [gpfsug-discuss] policy ilm features?
> 
> 
> This one has been on my list a long time so I figured I'd ask here first
> before I open an apar or request an enhancement (most likely).
> 
>  Is there a way using the policy engine to determine the following?
> 
> -metadata replication total/current
> -unbalanced file
> 
> Looking to catch things like this that stand out on my filesystem without
> having to run several hundred million 'mmlsattr's.
> 
> metadata replication: 1 max 2
> flags:                unbalanced
> 
> Ed
> 
> 
> 
> --
> 
> Ed Wahl
> Ohio Supercomputer Center
> 614-292-9302
> _______________________________________________
> gpfsug-discuss mailing list
> gpfsug-discuss at spectrumscale.org
> https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fgpfsug.org%2Fmailman%2Flistinfo%2Fgpfsug-discuss&data=01%7C01%7Cjohn.hearns%40asml.com%7C056e34c5a8df4d8f10fd08d55f91e73c%7Caf73baa8f5944eb2a39d93e96cad61fc%7C1&sdata=dnt7vV4TCd68l7fSJnY35eyNM%2B8pNrZElImSZeZbit8%3D&reserved=0
> -- The information contained in this communication and any attachments is
> confidential and may be privileged, and is for the sole use of the intended
> recipient(s). Any unauthorized review, use, disclosure or distribution is
> prohibited. Unless explicitly stated otherwise in the body of this
> communication or the attachment thereto (if any), the information is provided
> on an AS-IS basis without any express or implied warranties or liabilities.
> To the extent you are relying on this information, you are doing so at your
> own risk. If you are not the intended recipient, please notify the sender
> immediately by replying to this message and destroy all copies of this
> message and any attachments. Neither the sender nor the company/group of
> companies he or she represents shall be liable for the proper and complete
> transmission of the information contained in this communication, or for any
> delay in its receipt. _______________________________________________
> gpfsug-discuss mailing list gpfsug-discuss at spectrumscale.org
> http://gpfsug.org/mailman/listinfo/gpfsug-discuss



-- 

Ed Wahl
Ohio Supercomputer Center
614-292-9302