[gpfsug-discuss] GPFS and both Samba and NFS

[Sven Oehme]

Orlando, 

give that there are so many emails/interest on this topic in recent month, 
let me share some personal expertise on this :-)

any stock Samba or CTDB version you will find any distro is not sufficient 
and it doesn't matter which you choose (SLES, RHEL or any form of debian 
and any version of all of them). 
the reason is that Samba doesn't have the GPFS header and library files 
included in its source, and at compile time it dynamically 
enables/disables all GPFS related things based on the availability of the 
GPFS packages . as non of the distros build machines have GPFS installed 
all this packages end up with  binaries in their rpms which don't have the 
required code enabled to properly support GPFS and non of the vfs modules 
get build either. 

the only way to get something working (don't get confused with officially 
Supported) is to recompile the CTDB src packages AND the Samba src 
packages on a node that has GPFS already installed. also the inclusion of 
CTDB into Samba will not address this, its just a more convenient 
packaging. 

Only if the build happens on such a node things like the vfs modules for 
GPFS are build and included in the package. 

said all this the binaries alone are only part of the Solution, after you 
have the correct packages, you need to properly configuration the system 
and setting all the right options (on GPFS as well as on CTDB and 
smbd.conf), which unfortunate are very System configuration specific, as 
otherwise you still can end up with data corruption if not set right.

also some people in the past have used a single instance of Samba to 
export shares over CIFS as they believe its a safe alternative to a more 
complex CTDB setup. also here a word of caution, even if you have a single 
instance of Samba running on top of GPFS you are exposed to potential data 
corruption if you don't use the proper Samba version (explained above) and 
the proper configuration, you can skip CTDB For that, but you still 
require a proper compiled version of Samba with GPFS code installed on the 
build machine. 

to be very clear the problem is not GPFS, its that Samba does locking AND 
caching on top of the Filesystem without GPFS knowledge if you don't use 
the right code/config to 'tell' GPFS about it, so GPFS can not ensure data 
consistency, not even on the same physical node for data thats shared over 
CIFS. 

there are unfortunate no shortcuts. 

i also have to point out that if you recompile the packages and configure 
everything correctly this is most likely to work, but you won't get 
official support for the CIFS part of this setup from IBM.

This email is not an official Statement/Response of IBM, see it as 
personal 'AS-IS' Information sharing.

Sven



From:   Orlando Richards <orlando.richards at ed.ac.uk>
To:     gpfsug-discuss at gpfsug.org
Date:   12/13/2013 07:35 AM
Subject:        Re: [gpfsug-discuss] GPFS and both Samba and NFS
Sent by:        gpfsug-discuss-bounces at gpfsug.org



On 13/12/13 15:31, Jonathan Buzzard wrote:
> On Fri, 2013-12-13 at 15:15 +0000, Orlando Richards wrote:
>
> [SNIP]
>
>> Hi Lindsay,
>>
>> We rebuild ctdb from the (git) source (in the 1.2.40 branch currently),
>> after running into performance problems with the sernet bundled version
>> (1.0.114). It's easy to build:
>
> Interestingly the RHEL7 beta is shipping ctdb 2.1 in combination with
> Samba 4.1
>
> http://ftp.redhat.com/pub/redhat/rhel/beta/7/x86_64/os/Packages/
>
> JAB.
>
The samba team are currently working to bring ctdb into the main samba 
source tree - so hopefully this will become a moot point soon!


-- 
             --
    Dr Orlando Richards
   Information Services
IT Infrastructure Division
        Unix Section
     Tel: 0131 650 4994
   skype: orlando.richards

The University of Edinburgh is a charitable body, registered in 
Scotland, with registration number SC005336.
_______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at gpfsug.org
http://gpfsug.org/mailman/listinfo/gpfsug-discuss


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20131213/57d2bc60/attachment.htm>