[gpfsug-discuss] Fragmented Inode Space and Performance of Policy Scans

[Luke Raimbach]

Hi Experts,

I'm worrying about creating independent filesets which start to get in the way of one another:

Let's pretend I create independent "fileset1" with a new inode space and preallocate say 1 million inodes.

I start pushing files in to fileset1. I also start pushing files in to the root fileset.

When fileset1 fills up, and because I've not specified a maximum number of inodes at creation time, presumably the file system manager gets on the case and preallocates more inodes as needed for fileset1. But while fileset1 has been filling up, I've also been filling up the root fileset (pretend I put 5 million files in there by the time fileset1 filled up).

Now what happens? I'm guessing the FS manager starts chopping out sections of the root fileset beyond the 6 million region and marking them as independent and belonging to fileset1, in much the same way as it extends the allocated inode space when the file system starts running low. I hope this happens anyway. Let's say it does.

So, now what about performance when running policy engine scans? I'm wanting to speed things up by restricting the policy scan to fileset1 (because I know that's where I put my special files). So I write the rules to select files and give them to mmapplypolicy. Does performance go down the drain as my once neatly arranged inode space is fragmented by not having enough inodes preallocated to begin with?

Imagine fileset1 and the root fileset filling continuously at a similar rate and constantly "overtaking" one another in the inode space... does the performance advantage of independence start to go away rapidly?

Cheers,
Luke.

--

Luke Raimbach
IT Manager
Oxford e-Research Centre
7 Keble Road,
Oxford,
OX1 3QG

+44(0)1865 610639