[gpfsug-discuss] inode update delay? / mmapplypolicy

[Aaron Knister]

Thanks again, Marc. You're quite right about the results being smeared 
over time on a live filesystem even if the inodescan didn't lag behind 
slightly. The use case here is a mass uid number migration. File 
ownership is easy because I can be guaranteed after a certain point in 
time that no new files under the user's old uid number can be created. 
However, in part because of inheritance I'm not so lucky when it comes 
to ACLs. I almost need to do 2 passes when looking at the ACLs but even 
that's not guaranteed to catch everything.

Using a snapshot is an interesting idea to give me a stable point in 
time snapshot to determine if I got everything.

-Aaron

On 7/24/16 11:11 AM, Marc A Kaplan wrote:
> mmapplypolicy uses the inodescan API which to gain overall speed,
> bypasses various buffers, caches, locks, ... and just reads inodes
> "directly" from disk.
>
> So the "view" of inodescan is somewhat "behind" the overall state of the
> live filesystem as viewed from the usual Posix APIs, such as stat(2).
> (Not to worry, all metadata updates are logged, so in event of a power
> loss or OS crash, GPFS recovers a consistent state from its log files...)
>
> This is at least mentioned in the docs.
>
> `mmfsctl suspend-write; mmfsctl resume;`   is the only practical way I
> know to guarantee a forced a flush of all "dirty" buffers to disk -- any
> metadata updates before the suspend will for sure
> become visible to an inodescan after the resume.  (Classic `sync` is not
> quite the same...)
>
> But think about this --- scanning a "live" file system is always
> somewhat iffy-dodgy and the result is smeared over the time of the scan
>  -- if there are any concurrent changes
> during the scan your results are imprecise.
>
> An alternative is to use `mmcrsnapshot` and scan the snapshot.
>
>
>
>
> From:        Aaron Knister <aaron.s.knister at nasa.gov>
> To:        <gpfsug-discuss at spectrumscale.org>
> Date:        07/23/2016 12:46 AM
> Subject:        [gpfsug-discuss] inode update delay?
> Sent by:        gpfsug-discuss-bounces at spectrumscale.org
> ------------------------------------------------------------------------
>
>
>
> I've noticed that there can be a several minute delay between the time
> changes to an inode occur and when those changes are reflected in the
> results of an inode scan. I've been working on code that checks ia_xperm
> to determine if a given file has extended acl entries and noticed in
> testing it that the acl flag wasn't getting set immediately after giving
> a file an acl. Here's what I mean:
>
> # cd /gpfsm/dnb32
>
> # date; setfacl -b  acltest*
> Sat Jul 23 00:24:57 EDT 2016
>
> # date; /usr/lpp/mmfs/samples/util/tsinode /gpfsm/dnb32 | egrep acl | wc -l
> Sat Jul 23 00:24:59 EDT 2016
> 5
>
> # date; /usr/lpp/mmfs/samples/util/tsinode /gpfsm/dnb32 | egrep acl | wc -l
> Sat Jul 23 00:25:10 EDT 2016
> 5
>
> # date; /usr/lpp/mmfs/samples/util/tsinode /gpfsm/dnb32 | egrep acl | wc -l
> Sat Jul 23 00:25:21 EDT 2016
> 0
>
> I'm a little confused about what's going on here-- is there some kind of
> write-behind for inode updates? Is there a way I can cause the cluster
> to quiesce and flush all pending inode updates (an mmfsctl suspend and
> resume seem to have this effect but I was looking for something a little
> less user-visible)? If I access the directory containing the files from
> another node via the VFS mount then the update appears immediately in
> the inode scan. A mere inode scan from another node w/o touching the
> filesystem mount doesn't necessarily seem to trigger this behavior.
>
> Thanks!
>
> -Aaron
>
> --
> Aaron Knister
> NASA Center for Climate Simulation (Code 606.2)
> Goddard Space Flight Center
> (301) 286-2776
> _______________________________________________
> gpfsug-discuss mailing list
> gpfsug-discuss at spectrumscale.org
> http://gpfsug.org/mailman/listinfo/gpfsug-discuss
>
>
>
>
>
> _______________________________________________
> gpfsug-discuss mailing list
> gpfsug-discuss at spectrumscale.org
> http://gpfsug.org/mailman/listinfo/gpfsug-discuss
>

-- 
Aaron Knister
NASA Center for Climate Simulation (Code 606.2)
Goddard Space Flight Center
(301) 286-2776