[gpfsug-discuss] GPFS and both Samba and NFS

[Orlando Richards]

On 12/12/13 18:14, Lindsay Todd wrote:
> Hello,
>
> Since this is my first note to the group, I'll introduce myself first.
>   I am Lindsay Todd, a Systems Programmer at Rensselaer Polytechnic
> Institute's Center for Computational Innovations, where I run a 1.2PiB
> GPFS cluster serving a Blue Gene/Q and a variety of Opteron and Intel
> clients, run an IBM Watson, and serve as an adjunct faculty.  I also do
> some freelance consulting, including GPFS, for several customers.
>
> One of my customers is needing to serve GPFS storage through both NFS
> and Samba; they have GPFS 3.5 running on RHEL5 (not RHEL6) servers.  I
> did not set this up for them, but was called to help fix it.  Currently
> they export NFS using cNFS; I think we have that straightened out
> server-side now.  Also they run Samba on several of the servers; I'm
> sure the group will not be surprised to hear they experience file
> corruption and other strange problems.
>
> I've been pushing them to use Samba-CTDB, and it looks like it will
> happen.  Except, I've never used this myself.  So this raises a couple
> questions:
>
> 1) It looks like RHEL5 bundles in an old version of CTDB. Should that be
> used, or would we be better with a build from the Enterprise Samba site,
> or even a build from source?
>

Hi Lindsay,

We rebuild ctdb from the (git) source (in the 1.2.40 branch currently), 
after running into performance problems with the sernet bundled version 
(1.0.114). It's easy to build:

git clone git://git.samba.org/ctdb.git ctdb.git
cd ctdb.git
git branch -r
git checkout -b "my_build" origin/1.2.40
cd packaging/RPM/
./makerpms.sh
yum install /root/rpmbuild/RPMS/x86_64/ctdb*.rpm

I then take the Sernet src rpm and rebuild it, using ctdb.h from the 
above rather than the 1.0.114 version they use. This is possibly not 
required, but I thought it best to be sure that the differing headers 
wouldn't cause any problems. I remain, as ever, very grateful to Sernet 
for providing these!


> 2) Given that CTDB can also run NFS, what are people who need both
> finding works best: run both cNFS + Samba-CTDB, or let CTDB run both?
>   It seems to me that if I let CTDB run both, I only need a single
> floating IP address for each server, while if I also use cNFS, I will
> want a floating address for both NFS and Samba, on each server.
>

We let CTDB run both, but we didn't come to that decision by comparing 
the merits of both options. I think Bristol (Bob Cregan is cc'd, I'm not 
sure he's on this list) run cNFS and CTDB side by side. As you say - 
you'd at least require different IP addresses to do that.


> Thanks for the help!

Best of luck :)

>
> R. Lindsay Todd, PhD
>
>
>
> _______________________________________________
> gpfsug-discuss mailing list
> gpfsug-discuss at gpfsug.org
> http://gpfsug.org/mailman/listinfo/gpfsug-discuss
>


-- 
             --
    Dr Orlando Richards
   Information Services
IT Infrastructure Division
        Unix Section
     Tel: 0131 650 4994
   skype: orlando.richards

The University of Edinburgh is a charitable body, registered in 
Scotland, with registration number SC005336.