[gpfsug-discuss] GPFS best practises : end user standpoint

Jonathan Buzzard jonathan.buzzard at strath.ac.uk
Tue Jan 16 17:25:47 GMT 2018


On Tue, 2018-01-16 at 16:35 +0000, Buterbaugh, Kevin L wrote:

[SNIP]

> 
> We’re in Tennessee, so not only do we not speak English, we barely
> speak American … y’all will just have to understand, bless your
> hearts!  ;-). 
> 
> But seriously, like most Universities, we have a ton of users for
> whom English is not their “primary” language, so dealing with
> “interesting” filenames is pretty hard to avoid.  And users’ problems
> are our problems whether or not they’re our problem.
> 

User comes with problem, you investigate find problem is due to "wacky"
characters point them to the mandatory training documentation, tell
them they need to rename their files to something sane and take no
further action. Sure English is not their primary language but *they*
have chosen to study in an English speaking country so best to actually
embrace that.

I do get it, many of our users are not native English speakers as well.
Yes it's a tough policy but on the other hand pandering to them does
them no favours either.

[SNIP]

> If you’ve got (bio)medical users using your cluster I don’t see how
> you avoid this … they’re using commercial apps that do this kind of
> stupid stuff (10’s of thousands of files in a directory and the full
> path to each file is longer than the contents of the files
> themselves!).

Well then they have justified the use; aka it's not their fault so you
up the quota for them. Though they could use different less brain dead
software. The idea is to force a bump in the road so the users are
aware that what they are doing is considered bad practice. Most users
have no idea that putting a million files in a directory is not
sensible and worse that trying to access them using a GUI file manager
is positively brain dead.

[SNIP]

> OK, so here’s my main question … you’re right that SSD’s are the
> answer … but how do you charge them more?  SSDs are move expensive
> than hard disks, and enterprise SSDs are stupid expensive … and users
> barely want to pay hard drive prices for their storage.  If you’ve
> got the magic answer to how to charge them enough to pay for SSDs I’m
> sure I’m not the only one who’d love to hear how you do it?!?!
> 

Give every user a one million file number quota. Need to store more
than one million files, then you are going to have to pay $X per extra
million files. Either they cough up the money to continue using their
brain dead software or they switch to less stupid software. If they
complain you just say that enterprise SSD's are stupidly expensive and
you are using that space up at an above average rate and so have to pay
the costs.

I am quite sure someone storing 1PB has to pay more than someone
storing 1TB, so why should someone storing 20 million files not have to
pay more than someone storing 100k files? The only difference is people
are used to paying more to store extra bytes and not used to paying
more for more files, but that is because most sane people don't store
millions and millions of files necessitating the purchase of large
amounts of expensive enterprise SSD's.


JAB.

-- 
Jonathan A. Buzzard                         Tel: +44141-5483420
HPC System Administrator, ARCHIE-WeSt.
University of Strathclyde, John Anderson Building, Glasgow. G4 0NG




More information about the gpfsug-discuss mailing list