[gpfsug-discuss] Couple of questions related to storage pools and mmapplypolicy
Jan-Frode Myklebust
janfrode at tanso.net
Tue Dec 18 09:13:05 GMT 2018
I don't think it's going to figure this out automatically between the two
rules.. I believe you will need to do something like the below (untested,
and definitely not perfect!!) rebalancing:
define(
weight_junkedness,
(CASE
/* Create 3 classes of files */
/* ReallyOldStuff */
WHEN ((access_age > 365) AND (KB_ALLOCATED > 3584)) THEN
1000
/* OldStuff */
WHEN ((access_age > 90) AND (KB_ALLOCATED > 3584)) THEN 100
/* everything else */
ELSE 0
END)
)
RULE 'defineTiers' GROUP POOL 'TIERS'
IS 'data' LIMIT(98)
THEN 'capacity' LIMIT(98)
THEN 'oc'
RULE 'Rebalance' MIGRATE FROM POOL 'TIERS' TO POOL 'TIERS'
WEIGHT(weight_junkedness)
Based on /usr/lpp/mmfs/samples/ilm/mmpolicy-fileheat.sample.
-jf
On Tue, Dec 18, 2018 at 9:10 AM Buterbaugh, Kevin L <
Kevin.Buterbaugh at vanderbilt.edu> wrote:
> Hi All,
>
> As those of you who suffered thru my talk at SC18 already know, we’re
> really short on space on one of our GPFS filesystems as the output of mmdf
> piped to grep pool shows:
>
> Disks in storage pool: system (Maximum disk size allowed is 24 TB)
> (pool total) 4.318T 1.078T ( 25%)
> 79.47G ( 2%)
> Disks in storage pool: data (Maximum disk size allowed is 262 TB)
> (pool total) 494.7T 38.15T ( 8%)
> 4.136T ( 1%)
> Disks in storage pool: capacity (Maximum disk size allowed is 519 TB)
> (pool total) 640.2T 14.56T ( 2%)
> 716.4G ( 0%)
>
> The system pool is metadata only. The data pool is the default pool. The
> capacity pool is where files with an atime (yes, atime) > 90 days get
> migrated. The capacity pool is comprised of NSDs that are 8+2P RAID 6 LUNs
> of 8 TB drives, so roughly 58.2 TB usable space per NSD.
>
> We have the new storage we purchased, but that’s still being tested and
> held in reserve for after the first of the year when we create a new GPFS 5
> formatted filesystem and start migrating everything to the new filesystem.
>
> In the meantime, we have also purchased a 60-bay JBOD and 30 x 12 TB
> drives and will be hooking it up to one of our existing storage arrays on
> Wednesday. My plan is to create another 3 8+2P RAID 6 LUNs and present
> those to GPFS as NSDs. They will be about 88 TB usable space each (because
> … beginning rant … a 12 TB drive is < 11 TB is size … and don’t get me
> started on so-called “4K” TV’s … end rant).
>
> A very wise man who used to work at IBM but now hangs out with people in
> red polos (<grin>) once told me that it’s OK to mix NSDs of slightly
> different sizes in the same pool, but you don’t want to put NSDs of vastly
> different sizes in the same pool because the smaller ones will fill first
> and then the larger ones will have to take all the I/O. I consider 58 TB
> and 88 TB to be pretty significantly different and am therefore planning on
> creating yet another pool called “oc” (over capacity if a user asks, old
> crap internally!) and migrating files with an atime greater than, say, 1
> year to that pool. But since ALL of the files in the capacity pool haven’t
> even been looked at in at least 90 days already, does it really matter?
> I.e. should I just add the NSDs to the capacity pool and be done with it?
>
> If it’s a good idea to create another pool, then I have a question about
> mmapplypolicy and migrations. I believe I understand how things work, but
> after spending over an hour looking at the documentation I cannot find
> anything that explicitly confirms my understanding … so if I have another
> pool called oc that’s ~264 TB in size and I write a policy file that looks
> like:
>
> define(access_age,(DAYS(CURRENT_TIMESTAMP) - DAYS(ACCESS_TIME)))
>
> RULE 'ReallyOldStuff'
> MIGRATE FROM POOL 'capacity'
> TO POOL 'oc'
> LIMIT(98)
> SIZE(KB_ALLOCATED/NLINK)
> WHERE ((access_age > 365) AND (KB_ALLOCATED > 3584))
>
> RULE 'OldStuff'
> MIGRATE FROM POOL 'data'
> TO POOL 'capacity'
> LIMIT(98)
> SIZE(KB_ALLOCATED/NLINK)
> WHERE ((access_age > 90) AND (KB_ALLOCATED > 3584))
>
> Keeping in mind that my capacity pool is already 98% full, is
> mmapplypolicy smart enough to calculate how much space it’s going to free
> up in the capacity pool by the “ReallyOldStuff” rule and therefore be able
> to potentially also move a ton of stuff from the data pool to the capacity
> pool via the 2nd rule with just one invocation of mmapplypolicy? That’s
> what I expect that it will do. I’m hoping I don’t have to run the
> mmapplypolicy twice … the first to move stuff from capacity to oc and then
> a second time for it to realize, oh, I’ve got a much of space free in the
> capacity pool now.
>
> Thanks in advance...
>
> Kevin
>
> P.S. In case you’re scratching your head over the fact that we have files
> that people haven’t even looked at for months and months (more than a year
> in some cases) sitting out there … we sell quota in 1 TB increments … once
> they’ve bought the quota, it’s theirs. As long as they’re paying us the
> monthly fee if they want to keep files relating to research they did during
> the George Bush Presidency out there … and I mean Bush 41, not Bush 43
> ….then that’s their choice. We do not purge files.
>
> —
> Kevin Buterbaugh - Senior System Administrator
> Vanderbilt University - Advanced Computing Center for Research and
> Education
> Kevin.Buterbaugh at vanderbilt.edu - (615)875-9633
>
>
>
> _______________________________________________
> gpfsug-discuss mailing list
> gpfsug-discuss at spectrumscale.org
> http://gpfsug.org/mailman/listinfo/gpfsug-discuss
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20181218/18ceda13/attachment.htm>
More information about the gpfsug-discuss
mailing list