[gpfsug-discuss] RAID config for SSD's used for data

[Alex Chekholko]

On 04/19/2017 12:53 PM, Buterbaugh, Kevin L wrote:
>
> So you’re considering the purchase of a dual-controller FC storage array
> with 12 or so 1.8 TB SSD’s in it, with the idea being that that storage
> would be in its’ own storage pool and that pool would be the default
> location for I/O for your main filesystem … at least for smaller files.
>  You intend to use mmapplypolicy nightly to move data to / from this
> pool and the spinning disk pools.

We did this and failed in interesting (but in retrospect obvious) ways. 
You will want to ensure that your users cannot fill your write target 
pool within a day.  The faster the storage, the more likely that is to 
happen.  Or else your users will get ENOSPC. You will want to ensure 
that your pools can handle the additional I/O from the migration in 
aggregate with all the user I/O.  Or else your users will see worse 
performance from the fast pool than the slow pool while the migration is 
running.  You will want to make sure that the write throughput of your 
slow pool is faster than the read throughput of your fast pool.

In our case, the fast pool was undersized in capacity, and oversized in 
terms of performance.  And overall the filesystem was oversubscribed 
(~100 10GbE clients, 8 x 10GbE NSD servers) So the fast pool would fill 
very quickly.  Then I would switch the placement policy to the big slow 
pool and performance would drop dramatically, and then if I ran a 
migration it would either (depending on parameters) take up all the I/O 
to the slow pool (leaving none for the users), or else take forever 
(weeks) because the user I/O was maxing out the slow pool.

Things should better today with QoS stuff, but your relative pool 
capacities (in our case it was like 1% fast, 99% slow) and your relative 
pool performance (in our case, slow pool had fewer IOPS than fast pool) 
are still going to matter a lot.

-- 
Alex Chekholko chekh at stanford.edu