[gpfsug-discuss] Copying ACLs between filesystems

Orlando Richards orlando.richards at ed.ac.uk
Mon Oct 1 10:41:55 BST 2012 

On 01/10/12 10:33, Jez Tucker wrote:
> Nice.  Can you put it on the github?
>
> https://github.com/gpfsug/gpfsug-tools
>

Sure - the whole rsync source, or just the patch?

Also - I've discovered that it won't identify differences between ACLs 
to trigger a copy. When it does copy, it copies the ACLs though.

For background - we're thinking of using this for building an 
asynchronously-synced DR filesystem, which is an HSM filesystem (as 
opposed to the production filesystem, which is disk only). Thus we'll be 
using GPFS to find candidate files (based on CTIME and/or MTIME) and 
passing the list to rsync to sync over. So we could pass the list to 
"touch" to touch the destination files before syncing to them, to force 
rsync to do a copy.

--
Orlando

>
>> -----Original Message-----
>> From: gpfsug-discuss-bounces at gpfsug.org [mailto:gpfsug-discuss-
>> bounces at gpfsug.org] On Behalf Of Orlando Richards
>> Sent: 01 October 2012 10:20
>> To: gpfsug-discuss at gpfsug.org
>> Subject: Re: [gpfsug-discuss] Copying ACLs between filesystems
>>
>> Whoop! Got it!
>>
>> Attached is the complete patch against the rsync-3.0.9 release bundle:
>>    http://www.samba.org/ftp/rsync/src/rsync-3.0.9.tar.gz
>>
>> The patch was taken with:
>> diff -rupN rsync-3.0.9/ rsync-3.0.9-patched/  > gpfs_support_rsync-
>> 3.0.9.patch
>>
>> To build once patched - do:
>> ./configure --enable-acl-support --enable-xattr-support --enable-gpfs-
>> support
>>
>> (make sure that libacl1-dev, or equivalent, is available).
>>
>> Seems to work as desired.
>>
>> In the end, I removed all the "if DEBUG_GTE" statements from the gpfs.c
>> code to get it working (!).
>>
>> --
>> Orlando
>>
>> On 28/09/12 15:46, Jez Tucker wrote:
>>> Well, since it's Friday and I've just been tracing gpfs_get_winattr issues all
>> the way back from SAMBAs vfs_gpfs.h to gpfs.so, might as well have a punt.
>>> Grabbing rsync latest git.
>>>
>>>> -----Original Message-----
>>>> From: gpfsug-discuss-bounces at gpfsug.org [mailto:gpfsug-discuss-
>>>> bounces at gpfsug.org] On Behalf Of Orlando Richards
>>>> Sent: 28 September 2012 15:11
>>>> To: gpfsug-discuss at gpfsug.org
>>>> Subject: Re: [gpfsug-discuss] Copying ACLs between filesystems
>>>>
>>>> On 28/09/12 12:30, Jonathan Buzzard wrote:
>>>>> On Fri, 2012-09-28 at 12:06 +0100, Orlando Richards wrote:
>>>>>> On 28/09/12 11:28, Barry Evans wrote:
>>>>>>> Just tried rsync -A (preserve acl's) on this end, no joy there either.
>>>>>>> two filesystems, same machine.
>>>>>>>
>>>>>>> sounds very, very evil but have you tried robocopy from one share
>>>>>>> to another? Not ideal.
>>>>>>
>>>>>> I've considered it - but not yet desperate enough to try it :)
>>>>>>
>>>>>
>>>>> When I have tried doing this from Windows file servers the results
>>>>> where not nice as Windows has ACL's for things like administrators
>>>>> etc. that robocopy attempted to copy over. It was also painfully slow.
>>>>> I found xcopy was a lot faster, though as my source was a Windows
>>>>> server the resultant ACL's were still borked.
>>>>>
>>>>>> Still looking for something sane.
>>>>>>
>>>>>
>>>>> Would AFM not do the trick?
>>>>>
>>>>>> Good to see I'm not just missing something blatantly obvious though!
>>>>>>
>>>>>
>>>>> What's your coding like? Depending how often you are going to be
>>>>> attempting this I would fix the IBM patches for rsync so they work
>>>>> on a modern version of rsync. Patch does not look to complicated to me.
>>>>>
>>>>
>>>> Beyond my abilities it seems. If anyone fancies having a friday
>>>> afternoon coding challenge, here's the problem:
>>>>
>>>> I'm trying to apply this patch:
>>>>
>>>> http://lists.samba.org/archive/rsync/attachments/20081021/015484b1/rs
>>>> ync
>>>> _gpfs_attrs-git.obj
>>>>
>>>> to rsync - let's say version 3.0.6:
>>>>
>>>> http://rsync.samba.org/ftp/rsync/src/rsync-3.0.6.tar.gz
>>>>
>>>> So I do:
>>>> mkdir -p src/rsync
>>>> cd src/rsync
>>>> wget http://rsync.samba.org/ftp/rsync/src/rsync-3.0.6.tar.gz
>>>> wget
>>>> http://lists.samba.org/archive/rsync/attachments/20081021/015484b1/rs
>>>> ync
>>>> _gpfs_attrs-git.obj
>>>> tar -zxf rsync-3.0.6.tar.gz
>>>> cd rsync-3.0.6
>>>> patch -p1 -F3 -i ../rsync_gpfs_attrs-git.diff
>>>>
>>>> ./configure --prefix=/tmp
>>>> make
>>>>
>>>>
>>>> Error message is:
>>>>
>>>> $ make gcc -std=gnu99 -I. -I. -g -O2 -DHAVE_CONFIG_H -Wall -W  -c
>>>> gpfs.c -o gpfs.o In file included from rsync.h:920:0,
>>>>                     from gpfs.c:20:
>>>> proto.h:111:28: warning: 'struct rsync_gpfs_attr' declared inside
>>>> parameter list [enabled by default]
>>>> proto.h:111:28: warning: its scope is only this definition or
>>>> declaration, which is probably not what you want [enabled by default]
>>>> gpfs.c:130:6: error: conflicting types for 'gpfs_free_attr'
>>>> proto.h:111:6: note: previous declaration of 'gpfs_free_attr' was
>>>> here
>>>> gpfs.c: In function 'gpfs_free_sxp':
>>>> gpfs.c:142:10: error: 'stat_x' has no member named 'gpfs_attr'
>>>> gpfs.c:144:20: error: 'stat_x' has no member named 'gpfs_attr'
>>>> gpfs.c:145:10: error: 'stat_x' has no member named 'gpfs_attr'
>>>> gpfs.c:146:5: error: 'stat_x' has no member named 'gpfs_attr'
>>>> gpfs.c: In function 'gpfs_free_list':
>>>> gpfs.c:162:3: warning: implicit declaration of function 'DEBUG_GTE'
>>>> [-Wimplicit-function-declaration]
>>>> gpfs.c:162:17: error: 'ACL' undeclared (first use in this function)
>>>> gpfs.c:162:17: note: each undeclared identifier is reported only once
>>>> for each function it appears in
>>>> gpfs.c: In function 'gpfs_receive_attr_int':
>>>> gpfs.c:212:16: error: 'ACL' undeclared (first use in this function)
>>>> gpfs.c: In function 'gpfs_find_attr':
>>>> gpfs.c:344:20: error: 'ACL' undeclared (first use in this function)
>>>> gpfs.c: In function 'gpfs_cache_attr':
>>>> gpfs.c:382:33: error: 'stat_x' has no member named 'gpfs_attr'
>>>> gpfs.c: In function 'gpfs_send_attr':
>>>> gpfs.c:398:33: error: 'stat_x' has no member named 'gpfs_attr'
>>>> gpfs.c:403:17: error: 'ACL' undeclared (first use in this function)
>>>> gpfs.c: In function 'gpfs_get_attr':
>>>> gpfs.c:519:5: error: 'stat_x' has no member named 'gpfs_attr'
>>>> gpfs.c:522:16: error: 'ACL' undeclared (first use in this function)
>>>> gpfs.c: In function 'gpfs_set_attr':
>>>> gpfs.c:538:16: error: 'ACL' undeclared (first use in this function)
>>>> gpfs.c: In function 'gpfs_attr_get_changed':
>>>> gpfs.c:587:10: error: 'stat_x' has no member named 'gpfs_attr'
>>>> gpfs.c:595:37: error: 'stat_x' has no member named 'gpfs_attr'
>>>> gpfs.c:597:21: error: 'stat_x' has no member named 'gpfs_attr'
>>>> gpfs.c:598:7: error: 'stat_x' has no member named 'gpfs_attr'
>>>> gpfs.c:601:16: error: 'ACL' undeclared (first use in this function)
>>>> make: *** [gpfs.o] Error 1
>>>>
>>>>
>>>>
>>>>
>>>> --
>>>>                --
>>>>       Dr Orlando Richards
>>>>      Information Services
>>>> IT Infrastructure Division
>>>>           Unix Section
>>>>        Tel: 0131 650 4994
>>>>
>>>> The University of Edinburgh is a charitable body, registered in
>>>> Scotland, with registration number SC005336.
>>>> _______________________________________________
>>>> gpfsug-discuss mailing list
>>>> gpfsug-discuss at gpfsug.org
>>>> http://gpfsug.org/mailman/listinfo/gpfsug-discuss
>>>
>>>
>>> _______________________________________________
>>> gpfsug-discuss mailing list
>>> gpfsug-discuss at gpfsug.org
>>> http://gpfsug.org/mailman/listinfo/gpfsug-discuss
>>>
>>
>>
>> --
>>               --
>>      Dr Orlando Richards
>>     Information Services
>> IT Infrastructure Division
>>          Unix Section
>>       Tel: 0131 650 4994
>>
>> The University of Edinburgh is a charitable body, registered in Scotland, with
>> registration number SC005336.
>
> _______________________________________________
> gpfsug-discuss mailing list
> gpfsug-discuss at gpfsug.org
> http://gpfsug.org/mailman/listinfo/gpfsug-discuss
>


-- 
             --
    Dr Orlando Richards
   Information Services
IT Infrastructure Division
        Unix Section
     Tel: 0131 650 4994

The University of Edinburgh is a charitable body, registered in 
Scotland, with registration number SC005336.

More information about the gpfsug-discuss mailing list