[gpfsug-discuss] mount: /dev/gpfs01 already mounted or /gpfs01 busy

[Simon Thompson (Research Computing - IT Services)]

Process running which is either from gpfs or using it.

We used to see it when we ran torque from gpfs and forgot to stop it before shutting down gpfs.

Or if you have dsmrecalld (for example) running on the node.

Simon
________________________________________
From: gpfsug-discuss-bounces at spectrumscale.org [gpfsug-discuss-bounces at spectrumscale.org] on behalf of Tejas Rao [raot at bnl.gov]
Sent: 29 June 2016 16:58
To: gpfsug main discussion list
Subject: Re: [gpfsug-discuss] mount: /dev/gpfs01 already mounted or /gpfs01 busy

"umount -l" returns with "not mounted" message. See below.


[root@ ~]# umount -l /gpfs01
umount: /gpfs01: not mounted

[root@ ~]# umount -l /dev/gpfs01
umount: /dev/gpfs01: not mounted
[root@ ~]#


[root@ ~]# /usr/lpp/mmfs/bin/mmmount gpfs01
Wed Jun 29 11:56:58 EDT 2016: mmmount: Mounting file systems ...
mount: /dev/gpfs01 already mounted or /gpfs01 busy
mmmount: Command failed. Examine previous error messages to determine cause.

Thanks.

On 6/29/2016 11:46, Buterbaugh, Kevin L wrote:
Hi Tejas,

One thing that I have had occasional success with is a lazy unmount - i.e. “unmount -l /gpfs01”.  Doesn’t help more of the time than it does help in my experience, but it’s one thing I try before bouncing a node.  HTHAL…

Kevin

On Jun 29, 2016, at 10:43 AM, Tejas Rao <<mailto:raot at bnl.gov>raot at bnl.gov<mailto:raot at bnl.gov>> wrote:

Hi all,

I cannot get the filesystem mounted again on few nodes.

[root@ ~]# /usr/lpp/mmfs/bin/mmmount gpfs01
Wed Jun 29 11:29:50 EDT 2016: mmmount: Mounting file systems ...
mount: /dev/gpfs01 already mounted or /gpfs01 busy
mmmount: Command failed. Examine previous error messages to determine cause.



Things I have tried and checked.



1. Checked /proc/mounts and /etc/mtab and lsof /dev/gpfs01 and both /dev/gpfs01 and /gpfs01 are not seen anywhere.

2. Tried mmshutdown/mmstartup and it fails to unload mmfs modules. See below.

3. mmumount gpfs01 -f


A reboot of the client would probably fix it but as the client is used in production, this is not easy to do.


Are there any tricks/hacks to remount the filesystem again without rebooting the client.


[root@~]# /usr/lpp/mmfs/bin/mmshutdown
Wed Jun 29 11:10:52 EDT 2016: mmshutdown: Starting force unmount of GPFS file systems
Wed Jun 29 11:10:57 EDT 2016: mmshutdown: Shutting down GPFS daemons
Shutting down!
'shutdown' command about to kill process 11431
Unloading modules from /lib/modules/2.6.32-504.8.1.el6.x86_64/extra
mmfsenv: Module mmfslinux is still in use.
Unmount all GPFS file systems and issue the command:
 mmfsadm cleanup
Unloading module mmfs26
ERROR: Module mmfs26 is in use
mmfsenv: Error unloading module mmfs26.
Unloading module mmfslinux
ERROR: Module mmfslinux is in use by mmfs26
mmfsenv: Error unloading module mmfslinux.
Wed Jun 29 11:11:03 EDT 2016: mmshutdown: Finished
[root@~]#

Thanks.
_______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at spectrumscale.org<http://spectrumscale.org>
http://gpfsug.org/mailman/listinfo/gpfsug-discuss

—
Kevin Buterbaugh - Senior System Administrator
Vanderbilt University - Advanced Computing Center for Research and Education
Kevin.Buterbaugh at vanderbilt.edu<mailto:Kevin.Buterbaugh at vanderbilt.edu> - (615)875-9633






_______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at spectrumscale.org
http://gpfsug.org/mailman/listinfo/gpfsug-discuss