[gpfsug-discuss] GPFS and Flash/SSD Storage tiered storage
Jonathan Buzzard
jonathan.buzzard at strath.ac.uk
Sat Feb 24 12:01:08 GMT 2018
On 23/02/18 01:27, valleru at cbio.mskcc.org wrote:
> Thanks, I will try the file heat feature but i am really not sure, if it
> would work - since the code can access cold files too, and not
> necessarily files recently accessed/hot files.
>
> With respect to LROC. Let me explain as below:
>
> The use case is that -
> The code initially reads headers (small region of data) from thousands
> of files as the first step. For example about 30,000 of them with each
> about 300MB to 500MB in size.
> After the first step, with the help of those headers - it mmaps/seeks
> across various regions of a set of files in parallel.
> Since its all small IOs and it was really slow at reading from GPFS over
> the network directly from disks - Our idea was to use AFM which i
> believe fetches all file data into flash/ssds, once the initial few
> blocks of the files are read.
> But again - AFM seems to not solve the problem, so i want to know if
> LROC behaves in the same way as AFM, where all of the file data is
> prefetched in full block size utilizing all the worker threads - if few
> blocks of the file is read initially.
>
Imagine a single GPFS file system, metadata in SSD, a fast data pool and
a slow data pool (fast and slow being really good names to avoid the 8
character nonsense).
Now if your fast data pool is appropriately sized then your slow data
pool will normally be doing diddly squat. We are talking under 10 I/O's
per second. Frankly under 5 I/O's per second is more like it from my
experience.
If your slow pool is 8-10PB in size, then it has thousands of spindles
in it, and should be able to absorb the start of the job without
breaking sweat. For numbers a 7.2K RPM disk can do around 120 random
I/O's per second, so using RAID6 and 8TB disks that's 130 LUN's so
around 15,000 random I/O's per second spare overhead, more if it's not
random. It should take all of around 1-2s to read in those headers.
Therefore unless these jobs only run for a few seconds or you have
dozens of them starting every minute it should not be an issue.
Finally if GPFS is taking ages to read the files over the network, then
it sounds like your network needs an upgrade or GPFS needs tuning which
may or may not require a larger fast storage pool.
JAB.
--
Jonathan A. Buzzard Tel: +44141-5483420
HPC System Administrator, ARCHIE-WeSt.
University of Strathclyde, John Anderson Building, Glasgow. G4 0NG
More information about the gpfsug-discuss
mailing list