[gpfsug-discuss] inode update delay? / mmapplypolicy

Marc A Kaplan makaplan at us.ibm.com
Sun Jul 24 16:11:06 BST 2016


mmapplypolicy uses the inodescan API which to gain overall speed, bypasses 
various buffers, caches, locks, ... and just reads inodes "directly" from 
disk.

So the "view" of inodescan is somewhat "behind" the overall state of the 
live filesystem as viewed from the usual Posix APIs, such as stat(2).
(Not to worry, all metadata updates are logged, so in event of a power 
loss or OS crash, GPFS recovers a consistent state from its log files...)

This is at least mentioned in the docs.

`mmfsctl suspend-write; mmfsctl resume;`   is the only practical way I 
know to guarantee a forced a flush of all "dirty" buffers to disk -- any 
metadata updates before the suspend will for sure
become visible to an inodescan after the resume.  (Classic `sync` is not 
quite the same...)

But think about this --- scanning a "live" file system is always somewhat 
iffy-dodgy and the result is smeared over the time of the scan  -- if 
there are any concurrent changes
during the scan your results are imprecise.

An alternative is to use `mmcrsnapshot` and scan the snapshot. 




From:   Aaron Knister <aaron.s.knister at nasa.gov>
To:     <gpfsug-discuss at spectrumscale.org>
Date:   07/23/2016 12:46 AM
Subject:        [gpfsug-discuss] inode update delay?
Sent by:        gpfsug-discuss-bounces at spectrumscale.org



I've noticed that there can be a several minute delay between the time 
changes to an inode occur and when those changes are reflected in the 
results of an inode scan. I've been working on code that checks ia_xperm 
to determine if a given file has extended acl entries and noticed in 
testing it that the acl flag wasn't getting set immediately after giving 
a file an acl. Here's what I mean:

# cd /gpfsm/dnb32

# date; setfacl -b  acltest*
Sat Jul 23 00:24:57 EDT 2016

# date; /usr/lpp/mmfs/samples/util/tsinode /gpfsm/dnb32 | egrep acl | wc 
-l
Sat Jul 23 00:24:59 EDT 2016
5

# date; /usr/lpp/mmfs/samples/util/tsinode /gpfsm/dnb32 | egrep acl | wc 
-l
Sat Jul 23 00:25:10 EDT 2016
5

# date; /usr/lpp/mmfs/samples/util/tsinode /gpfsm/dnb32 | egrep acl | wc 
-l
Sat Jul 23 00:25:21 EDT 2016
0

I'm a little confused about what's going on here-- is there some kind of 
write-behind for inode updates? Is there a way I can cause the cluster 
to quiesce and flush all pending inode updates (an mmfsctl suspend and 
resume seem to have this effect but I was looking for something a little 
less user-visible)? If I access the directory containing the files from 
another node via the VFS mount then the update appears immediately in 
the inode scan. A mere inode scan from another node w/o touching the 
filesystem mount doesn't necessarily seem to trigger this behavior.

Thanks!

-Aaron

-- 
Aaron Knister
NASA Center for Climate Simulation (Code 606.2)
Goddard Space Flight Center
(301) 286-2776
_______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at spectrumscale.org
http://gpfsug.org/mailman/listinfo/gpfsug-discuss




-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20160724/d4ef8616/attachment.htm>


More information about the gpfsug-discuss mailing list