[gpfsug-discuss] Metadata space usage NFS4 vs POSIX ACL

[Peter Childs]

On Sat, 2019-04-06 at 23:50 +0200, Michal Zacek wrote:

Hello,


we decided to convert NFS4 acl to POSIX (we need share same data

between  SMB, NFS and GPFS clients), so I created script to convert

NFS4 to posix ACL. It is very simple, first I do "chmod -R 770 DIR" and

then "setfacl -R ..... DIR".  I was surprised that conversion to posix

acl has taken more then 2TB of metadata space.There is about one hundred
million files at GPFS filesystem. Is this expected behavior?


Thanks,

Michal


Example of NFS4 acl:


#NFSv4 ACL

#owner:root

#group:root

special:owner@:rwx-:allow

  (X)READ/LIST (X)WRITE/CREATE (X)APPEND/MKDIR (X)SYNCHRONIZE

(X)READ_ACL  (X)READ_ATTR  (X)READ_NAMED

  (-)DELETE    (X)DELETE_CHILD (-)CHOWN        (X)EXEC/SEARCH

(-)WRITE_ACL (X)WRITE_ATTR (X)WRITE_NAMED


special:group@:----:allow

  (-)READ/LIST (-)WRITE/CREATE (-)APPEND/MKDIR (X)SYNCHRONIZE

(X)READ_ACL  (X)READ_ATTR  (X)READ_NAMED

  (-)DELETE    (-)DELETE_CHILD (-)CHOWN        (-)EXEC/SEARCH

(-)WRITE_ACL (X)WRITE_ATTR (-)WRITE_NAMED


special:everyone@:----:allow

  (-)READ/LIST (-)WRITE/CREATE (-)APPEND/MKDIR (X)SYNCHRONIZE

(X)READ_ACL  (X)READ_ATTR  (X)READ_NAMED

  (-)DELETE    (-)DELETE_CHILD (-)CHOWN        (-)EXEC/SEARCH

(-)WRITE_ACL (X)WRITE_ATTR (-)WRITE_NAMED


group:ag_cud_96_lab:rwx-:allow:FileInherit:DirInherit

  (X)READ/LIST (X)WRITE/CREATE (X)APPEND/MKDIR (X)SYNCHRONIZE

(X)READ_ACL  (X)READ_ATTR  (X)READ_NAMED

  (-)DELETE    (X)DELETE_CHILD (-)CHOWN        (X)EXEC/SEARCH

(-)WRITE_ACL (X)WRITE_ATTR (X)WRITE_NAMED


group:ag_cud_96_lab_ro:r-x-:allow:FileInherit:DirInherit

  (X)READ/LIST (-)WRITE/CREATE (-)APPEND/MKDIR (X)SYNCHRONIZE

(X)READ_ACL  (X)READ_ATTR  (X)READ_NAMED

  (-)DELETE    (-)DELETE_CHILD (-)CHOWN        (X)EXEC/SEARCH

(-)WRITE_ACL (X)WRITE_ATTR (-)WRITE_NAMED



converted to posix acl:


# owner: root

# group: root

user::rwx

group::rwx

mask::rwx

other::---

default:user::rwx

default:group::rwx

default:mask::rwx

default:other::---

group:ag_cud_96_lab:rwx

default:group:ag_cud_96_lab:rwx

group:ag_cud_96_lab_ro:r-x

default:group:ag_cud_96_lab_ro:r-x




_______________________________________________

gpfsug-discuss mailing list

gpfsug-discuss at spectrumscale.org

https://eur01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fgpfsug.org%2Fmailman%2Flistinfo%2Fgpfsug-discuss&data=02%7C01%7Cp.childs%40qmul.ac.uk%7Ce1059833f7ed448b027608d6bad9ffec%7C569df091b01340e386eebd9cb9e25814%7C0%7C1%7C636901842833614488&sdata=ROQ3LKmLZ06pI%2FTfdKZ9oPJx5a2xCUINqBnlIfEKF2Q%3D&reserved=0



I've been trying to get my head round acls, with the plan to implement Cluster Export Services SMB rather than roll your own SMB.

I'm not sure that plan is going to work Michal, although it might if your not using the Cluster Export Services version of SMB.

Put simply if your running Cluster export services SMB you need to set ACLs in Spectrum Scale to "nfs4" we currently have it set to "all" and it won't let you export the shares until you change it, currently I'm still testing, and have had to write a change to go the other way.

If you using linux kernel nfs4 that uses posix, however CES nfs uses ganasha which uses nfs4 acl correctly.

It gets slightly more annoying as nfs4-setfacl does not work with Spectrum Scale and you have to use mmputacl which has no recursive flag, I even found a ibm article from a few years ago saying the best way to set acls is to use find, and a temporary file..... The other workaround they suggest is to update acls from windows or nfs to get the right.

One thing I think may happen if you do as you've suggested is that you will break any acls under Samba badly. I think the other reason that command is taking up more space than expected is that your giving files acls that never had them to start with.

I would love someone to say that I'm wrong, as changing our acl setting is going to be a pain. as while we don't make a lot of use of them we make enough that having to use nfs4 acls all the time is going to be a pain.


--

Peter Childs
ITS Research Storage
Queen Mary, University of London

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20190507/faaa8bb2/attachment.htm>