[gpfsug-discuss] GPFS Independent Fileset Limit vs Quotas?
Peinkofer, Stephan
Stephan.Peinkofer at lrz.de
Mon Aug 13 08:26:00 BST 2018
Dear Marc,
OK, so let’s give it a try:
[root at datdsst100 pr74qo]# mmlsfileset dsstestfs01
Filesets in file system 'dsstestfs01':
Name Status Path
root Linked /dss/dsstestfs01
...
quota_test_independent Linked /dss/dsstestfs01/quota_test_independent
quota_test_dependent Linked /dss/dsstestfs01/quota_test_independent/quota_test_dependent
[root at datdsst100 pr74qo]# mmsetquota dsstestfs01:quota_test_independent --user a2822bp --block 1G:1G --files 10:10
[root at datdsst100 pr74qo]# mmsetquota dsstestfs01:quota_test_dependent --user a2822bp --block 10G:10G --files 100:100
[root at datdsst100 pr74qo]# mmrepquota -u -v dsstestfs01:quota_test_independent
*** Report for USR quotas on dsstestfs01
Block Limits | File Limits
Name fileset type KB quota limit in_doubt grace | files quota limit in_doubt grace entryType
a2822bp quota_test_independent USR 0 1048576 1048576 0 none | 0 10 10 0 none e
root quota_test_independent USR 0 0 0 0 none | 1 0 0 0 none i
[root at datdsst100 pr74qo]# mmrepquota -u -v dsstestfs01:quota_test_dependent
*** Report for USR quotas on dsstestfs01
Block Limits | File Limits
Name fileset type KB quota limit in_doubt grace | files quota limit in_doubt grace entryType
a2822bp quota_test_dependent USR 0 10485760 10485760 0 none | 0 100 100 0 none e
root quota_test_dependent USR 0 0 0 0 none | 1 0 0 0 none i
Looks good …
[root at datdsst100 pr74qo]# cd /dss/dsstestfs01/quota_test_independent/quota_test_dependent/
[root at datdsst100 quota_test_dependent]# for foo in `seq 1 99`; do touch file${foo}; chown a2822bp:pr28fa file${foo}; done
[root at datdsst100 quota_test_dependent]# mmrepquota -u -v dsstestfs01:quota_test_dependent
*** Report for USR quotas on dsstestfs01
Block Limits | File Limits
Name fileset type KB quota limit in_doubt grace | files quota limit in_doubt grace entryType
a2822bp quota_test_dependent USR 0 10485760 10485760 0 none | 99 100 100 0 none e
root quota_test_dependent USR 0 0 0 0 none | 1 0 0 0 none i
[root at datdsst100 quota_test_dependent]# mmrepquota -u -v dsstestfs01:quota_test_independent
*** Report for USR quotas on dsstestfs01
Block Limits | File Limits
Name fileset type KB quota limit in_doubt grace | files quota limit in_doubt grace entryType
a2822bp quota_test_independent USR 0 1048576 1048576 0 none | 0 10 10 0 none e
root quota_test_independent USR 0 0 0 0 none | 1 0 0 0 none i
So it seems that per fileset per user quota is really not depending on independence. But what is the documentation then meaning with:
>>> User group and user quotas can be tracked at the file system level or per independent fileset.
???
However, there still remains the problem with mmbackup and mmapplypolicy …
And if you look at some of the RFEs, like the one from DESY, they want even more than 10k independent filesets …
Best Regards,
Stephan Peinkofer
--
Stephan Peinkofer
Dipl. Inf. (FH), M. Sc. (TUM)
Leibniz Supercomputing Centre
Data and Storage Division
Boltzmannstraße 1, 85748 Garching b. München
Tel: +49(0)89 35831-8715 Fax: +49(0)89 35831-9700
URL: http://www.lrz.de
On 12. Aug 2018, at 15:05, Marc A Kaplan <makaplan at us.ibm.com<mailto:makaplan at us.ibm.com>> wrote:
That's interesting, I confess I never read that piece of documentation.
What's also interesting, is that if you look at this doc for quotas:
https://www.ibm.com/support/knowledgecenter/en/STXKQY_4.2.3/com.ibm.spectrum.scale.v4r23.doc/bl1adm_change_quota_anynum_users_onproject_basis_acrs_protocols.htm
The word independent appears only once in a "Note": It is recommended to create an independent fileset for the project.
AND if you look at the mmchfs or mmchcr command you see:
--perfileset-quota
Sets the scope of user and group quota limit checks to the individual fileset level, rather than to the entire file system.
With no mention of "dependent" nor "independent"...
From: "Peinkofer, Stephan" <Stephan.Peinkofer at lrz.de<mailto:Stephan.Peinkofer at lrz.de>>
To: gpfsug main discussion list <gpfsug-discuss at spectrumscale.org<mailto:gpfsug-discuss at spectrumscale.org>>
Date: 08/11/2018 03:03 AM
Subject: Re: [gpfsug-discuss] GPFS Independent Fileset Limit
Sent by: gpfsug-discuss-bounces at spectrumscale.org<mailto:gpfsug-discuss-bounces at spectrumscale.org>
________________________________
Dear Marc,
so at least your documentation says:
https://www.ibm.com/support/knowledgecenter/en/STXKQY_5.0.1/com.ibm.spectrum.scale.v5r01.doc/bl1hlp_filesfilesets.htm
>>> User group and user quotas can be tracked at the file system level or per independent fileset.
But obviously as a customer I don't know if that "Really" depends on independence.
Currently about 70% of our filesets in the Data Science Storage systems get backed up to ISP. But that number may change over time as it depends on the requirements of our projects. For them it is just selecting "Protect this DSS Container by ISP" in a Web form an our portal then automatically does all the provisioning of the ISP Node to one of our ISP servers, rolling out the new dsm config files to the backup workers and so on.
Best Regards,
Stephan Peinkofer
________________________________
From: gpfsug-discuss-bounces at spectrumscale.org<mailto:gpfsug-discuss-bounces at spectrumscale.org> <gpfsug-discuss-bounces at spectrumscale.org<mailto:gpfsug-discuss-bounces at spectrumscale.org>> on behalf of Marc A Kaplan <makaplan at us.ibm.com<mailto:makaplan at us.ibm.com>>
Sent: Friday, August 10, 2018 7:15 PM
To: gpfsug main discussion list
Subject: Re: [gpfsug-discuss] GPFS Independent Fileset Limit
I know quota stuff was cooked into GPFS before we even had "independent filesets"...
So which particular quota features or commands or options now depend on "independence"?! Really?
Yes, independent fileset performance for mmapplypolicy and mmbackup scales with the inodespace sizes. But I'm curious to know how many of those indy filesets are mmback-ed-up.
Appreciate your elaborations, 'cause even though I've worked on some of this code, I don't know how/when/if customers push which limits.
---------------------
Dear Marc,
well the primary reasons for us are:
- Per fileset quota (this seems to work also for dependent filesets as far as I know)
- Per user per fileset quota (this seems only to work for independent filesets)
- The dedicated inode space to speedup mmpolicy runs which only have to be applied to a specific subpart of the file system
- Scaling mmbackup by backing up different filesets to different TSM Servers economically
We have currently more than 1000 projects on our HPC machines and several different existing and planned file systems (use cases):
_______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at spectrumscale.org<http://spectrumscale.org>
http://gpfsug.org/mailman/listinfo/gpfsug-discuss
_______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at spectrumscale.org<http://spectrumscale.org>
http://gpfsug.org/mailman/listinfo/gpfsug-discuss
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20180813/3571a2bf/attachment.htm>
More information about the gpfsug-discuss
mailing list