[gpfsug-discuss] File_heat for GPFS File Systems

[Andreas Landhäußer]

On Tue, 4 Oct 2016, Daniel Kidger <daniel.kidger at uk.ibm.com> wrote:

Daniel,

ahh this might also help in my HPC Environment.

We are already using AFM, just on HDD, for distributing HPC Data for usage 
on remote scientific sites.

Using the AFM for a temp space seems to be a good idea, will the 
individual file heat be increased by using ILM policy and  AFM?

Using a large GPFS Filesystem for scientific results, it seems to be very 
unlikely all users starting using their cold files at the same point of 
time, so filesystem space should be sufficient.

Thanks for the hint

 	Andreas

T-Systems SfR GmbH

> If you need functionality like you describe, then as well as using tiering
> with the ILM Policy Engine, also consider using AFM to cache onto say SSDs. 
> That way you would have dynamic caching: as soon as a file gets accessed, it
> goes into the cache and further I/O will be do the SSD copy so faster until
> such time as the file ages and newer files replace it in the AFM cache. The
> cost of this is perhaps a little more complexity and also a 'hot' file
> consumes space on both SSD and disk.
>  
> Daniel
> /spectrum_storage-banne
>                                v
>  
> Spectrum Scale Logo
>  
>  
> Dr Daniel Kidger
> IBM Technical Sales Specialist
> Software Defined Solution Sales
> 
> +44-(0)7818 522 266
> daniel.kidger at uk.ibm.com
>  
>  
>  
>       ----- Original message -----
>       From: Eric Horst <erich at uw.edu>
>       Sent by: gpfsug-discuss-bounces at spectrumscale.org
>       To: gpfsug main discussion list
>       <gpfsug-discuss at spectrumscale.org>
>       Cc:
>       Subject: Re: [gpfsug-discuss] File_heat for GPFS File Systems
>       Date: Tue, Sep 27, 2016 9:56 PM
>         >>
>       >> if a file gets hot again, there is no rule for putting the
>       file back
>       >> into a faster storage device?
>       >
>       >
>       > The file will get moved when you run the policy again.  You
>       can run the
>       > policy as often as you like.
>
>       I think its worth stating clearly that if a file is in the
>       Thrifty
>       slow pool and a user opens and reads/writes the file there is
>       nothing
>       that moves this file to a different tier. A policy run is the
>       only
>       action that relocates files. So if you apply the policy daily
>       and over
>       the course of the day users access many cold files, the
>       performance
>       accessing those cold files may not be ideal until the next day
>       when
>       they are repacked by heat. A file is not automatically moved to
>       the
>       fast tier on access read or write. I mention this because this
>       aspect
>       of tiering was not immediately clear from the docs when I was a
>       neophyte GPFS admin and I had to learn by observation. It is
>       easy for
>       one to make an assumption that it is a more dynamic tiering
>       system
>       than it is.
>
>       -Eric
>
>       --
>       Eric Horst
>       University of Washington
>       _______________________________________________
>       gpfsug-discuss mailing list
>       gpfsug-discuss at spectrumscale.org
>       http://gpfsug.org/mailman/listinfo/gpfsug-discuss
>        
> 
>  
> Unless stated otherwise above:
> IBM United Kingdom Limited - Registered in England and Wales with number
> 741598.
> Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
> 
> 
>

-- 
Andreas Landhäußer				+49 151 12133027 (mobile)
alandhae at gmx.de