[gpfsug-discuss] Odd behavior - GPSF failed to start after initial node add
Edward Wahl
ewahl at osc.edu
Mon Jun 5 20:56:55 BST 2017
On Mon, 5 Jun 2017 15:54:31 -0400
Edward Wahl <ewahl at osc.edu> wrote:
> Just a thought, as we noticed the EXACT opposite of this, and what I think is
> new behavior in either mmmount or .. Does the file system exist in
> your /etc/fstab (or AIX equiv) yet?
Apologies, I meant mmsdrfsdef, not mmfsfuncs.
Ed
>
> Ed
>
> On Mon, 5 Jun 2017 15:54:09 +0000
> "Oesterlin, Robert" <Robert.Oesterlin at nuance.com> wrote:
>
> > Our node build process re-adds a node to the cluster and then does a
> > “service gpfs start”, but GPFS doesn’t start. >From the build log:
> >
> > + ssh -o StrictHostKeyChecking=no nrg1-gpfs01.nrg1.us.grid.nuance.com
> > '/usr/local/sbin/addnode.sh cnq-r02r09u27.nrg1.us.grid.nuance.com'
> > + rc=0
> > + chkconfig gpfs on
> > + service gpfs start
> >
> > The “service gpfs start” command hangs and never seems to return.
> >
> > If I look at the process tree:
> >
> > [root at cnq-r02r09u27 ~]# ps ax | egrep "mm|gpfs"
> > 11715 ? S 0:00 /bin/bash ./nrgX_gpfs_post
> > 12191 ? Ssl 0:00 /usr/lpp/mmfs/bin/mmsdrserv 1191 10
> > 10 /var/adm/ras/mmsdrserv.log 128 yes no 12208 ? S
> > 0:00 /usr/lpp/mmfs/bin/mmksh /usr/lpp/mmfs/bin/mmccrmonitor 15 12271 ?
> > S 0:00 /bin/sh /sbin/service gpfs start 12276 ? S
> > 0:00 /bin/sh /etc/init.d/gpfs start 12278 ? S
> > 0:00 /usr/lpp/mmfs/bin/mmksh /usr/lpp/mmfs/bin/mmautoload reboot
> > 12292 ? S
> > 0:00 /usr/lpp/mmfs/bin/mmksh /usr/lpp/mmfs/bin/mmautoload reboot
> > 12293 ? S 0:00 /bin/grep -lw /var/mmfs/gen/nodeFiles/*.num
> > 12294 ? S 0:00 /bin/sed -e s%/var/mmfs/gen/nodeFiles/....%% -e
> > s/\.num$// 21639 ? S
> > 0:00 /usr/lpp/mmfs/bin/mmksh /usr/lpp/mmfs/bin/mmccrmonitor 15
> >
> > This is GPFS 4.2.2-1
> >
> > This seems to occur only on the initial startup after build - if I try to
> > start GPFS again, it works just fine - any ideas on what it’s sitting here
> > waiting? Nothing in mmfslog (does not exist)
> >
> > Bob Oesterlin
> > Sr Principal Storage Engineer, Nuance
> > 507-269-0413
> >
> >
>
>
>
--
Ed Wahl
Ohio Supercomputer Center
614-292-9302
More information about the gpfsug-discuss
mailing list