[gpfsug-discuss] Copying ACLs from outside sources

[Jonathan Buzzard]

On Wed, 2013-10-02 at 20:44 +0000, Fosburgh,Jonathan wrote: 
> Hello, I'm new to this list, as well as GPFS.  
> 
> 
> We have new (installed over the last 6 weeks) GPFS cluster that we are
> currently trying to get into service.  The NSDs are running on Red Hat
> (6 NSDs), the storage is DCS3700 directly attached via SAS.  We are
> running GPFS 3.5.0.12.
> 
> Some of the data that will be placed on this cluster will be existing
> data coming from other sources, one of which is our NAS (Isilon,
> supporting both NFS and CIFS).  We have tried a few different ways of
> copying the data.  First we tried to rsync directly between an NSD and
> one of the Isilon nodes.  Unfortunately, it appears the rsync doesn't
> work with the ACL format used on the Isilon.  We have also tried using
> various methods of copying data via CIFS, but we continue to have no
> luck getting ACLs to copy.  We know that, for instance, copying data
> via CIFS into the Isilon using robocopy will preserve the ACLs, and we
> were hoping that would work for getting data onto GPFS as well.
> However, the ACLs are being stripped and replaced with whatever ACL is
> present on GPFS.  Does anyone have experience doing something like
> that who can give me some pointers on what my configuration should
> look like? I assume there are samba and/or GPFS parameters that need
> to be tweaked in order to make this work, especially since this
> functionality would have to be present in SONAS.
> 

I guess the first thing to do is confirm that you have functioning
CIFS's ACL's in your Samba GPFS file server. Specifically you would need
to have NFSv4 ACL's enabled in the file system and the gpfs VFS module
loaded in Samba. Note having mixed ACL modes did not work very well for
me, so best to turn off Posix ACL's and only do NFSv4 ACL's.

These are the samba options I use in combination with GPFS to make the
server as windows like as possible.

# general options
vfs objects = fileid gpfs
        unix extensions = no
        mangled names = no
        case sensitive = no

# store DOS attributes in extended attributes (vfs_gpfs then stores them
in the file system)
ea support = yes
store dos attributes = yes
map readonly = no
map archive = no
map system = no

# the ctdb clustering and GPFS stuff
clustering = yes
ctdbd socket = /tmp/ctdb.socket
        fileid : algorithm = fsname
        gpfs : sharemodes = yes
        gpfs : winattr = yes
        force unknown acl user = yes
        nfs4 : mode = special
        nfs4 : chown = no
        nfs4 : acedup = merge

The best way to make sure it is working is point a Windows client at the
file server and manipulate the ACL's by hand.

Note that if you are going for that full make my Samba/GPFS file server
look as close as possible to a pucker MS Windows server, you might want
to consider setting the following GPFS options

     cifsBypassShareLocksOnRename
     cifsBypassTraversalChecking
     allowWriteWithDeleteChild

All fairly self explanatory, and make GPFS follow Windows schematics
more closely, though they are "undocumented". There are a couple of
other options as allowSambaCaseInsensitiveLookup and
syncSambaMetadataOps but I have not determined exactly what they do...

There is also there is an undocumented option for ACL's on mmchfs 

      mmchfs test -k samba

Even shows up in the output of mmlsfs. Not entirely sure what samba
ACL's are mind you...

Note that IBM have alluded at various times that there these "magic"
options exist and are used in SONAS etc.

Anyway getting back to ACL's robocopy will move them over, but in my
experience copying from a Windows file server the results where not what
I wanted. I ended up with extra ACL's for administrators and bizzare
file ownership issues. Some of that might be because the files had
originally been on a Netware file server which died, and where hastily
moved to a Windows server. I gave up in the end and manually unpicked
what the ACL's where doing for the migration to the GPFS file server.
However I only had 20GB of data...

You should also find that xcopy will move them over as well.

A possible alternative if you have access to an AIX box would be to
NFSv4 mount the Isilon on the AIX box and mount GPFS as well and just
use cp.


JAB.

-- 
Jonathan A. Buzzard                 Email: jonathan (at) buzzard.me.uk
Fife, United Kingdom.