[gpfsug-discuss] Question about NSD "Devtype" setting, nsddevices file
Bryan Banister
bbanister at jumptrading.com
Thu Jan 18 16:59:15 GMT 2018
Yes, just change the /var/mmfs/etc/nsddevices file so that it sets the Devtype to the “correct” setting, for example:
if [[ $osName = Linux ]]
then
: # Add function to discover disks in the Linux environment.
ls -l /dev/mpath/ 2>/dev/null | awk '{print "mpath/"$9 " generic"}'
ls -l /dev/mapper/ 2>/dev/null | awk '{print "mapper/"$9 " dmm"}'
ls -1 /dev/vd* 2>/dev/null | awk -F '/' '{print ""$3 " generic"}'
fi
My question is what is the correct setting?
And what does GPFS do differently with each setting?
Thanks,
-Bryan
From: gpfsug-discuss-bounces at spectrumscale.org [mailto:gpfsug-discuss-bounces at spectrumscale.org] On Behalf Of Jeffrey R. Lang
Sent: Thursday, January 18, 2018 9:20 AM
To: gpfsug main discussion list <gpfsug-discuss at spectrumscale.org>
Subject: Re: [gpfsug-discuss] Question about NSD "Devtype" setting, nsddevices file
Note: External Email
________________________________
So is there a way to change it if it’s set incorrectly?
jeff
From: gpfsug-discuss-bounces at spectrumscale.org<mailto:gpfsug-discuss-bounces at spectrumscale.org> [mailto:gpfsug-discuss-bounces at spectrumscale.org] On Behalf Of Jim Doherty
Sent: Wednesday, January 17, 2018 6:28 PM
To: gpfsug main discussion list <gpfsug-discuss at spectrumscale.org<mailto:gpfsug-discuss at spectrumscale.org>>
Subject: Re: [gpfsug-discuss] Question about NSD "Devtype" setting, nsddevices file
Run a mmlsnsd -X I suspect you will see that GPFS is using one of the /dev/sd* "generic" paths to the LUN, not the /dev/mapper/ path. In our case the device is setup as dmm
[root at service5 ~]# mmlsnsd -X
Disk name NSD volume ID Device Devtype Node name Remarks
---------------------------------------------------------------------------------------------------
volume1 0972B6CD587CD8E0 /dev/dm-0 dmm service5.pok.stglabs.ibm.com server node
volume1 0972B6CD587CD8E0 /dev/dm-0 dmm service6.pok.stglabs.ibm.com server node
volume2 0972B6CE587CD8E4 /dev/dm-4 dmm service5.pok.stglabs.ibm.com server node
volume2 0972B6CE587CD8E4 /dev/dm-3 dmm service6.pok.stglabs.ibm.com server node
volume3 0972B6CD587CD8E7 /dev/dm-1 dmm service5.pok.stglabs.ibm.com server node
volume3 0972B6CD587CD8E7 /dev/dm-2 dmm service6.pok.stglabs.ibm.com server node
volume4 0972B6CE587CF625 /dev/dm-3 dmm service5.pok.stglabs.ibm.com server node
volume4 0972B6CE587CF625 /dev/dm-4 dmm service6.pok.stglabs.ibm.com server node
[root at service5 ~]# grep volume1 /var/mmfs/gen/mmsdrfs | grep SG_DISK
%%home%%:60_SG_DISKS:gpfs5:1:volume1:0:5001:dataAndMetadata:0972B6CD587CD8E0:nsd:service5.pok.stglabs.ibm.com,service6.pok.stglabs.ibm.com::other::dmm:user:::quorumDisk:ready::system:service5.pok.stglabs.ibm.com,service6.pok.stglabs.ibm.com:::::
[root at service5 ~]#
If you run an tspreparedisk -s it will show you all of the paths.
[root at service5 ~]# tspreparedisk -s | grep 0972B6CD587CD8E0
0972B6CD587CD8E0 /dev/sda generic
0972B6CD587CD8E0 /dev/sdk generic
0972B6CD587CD8E0 /dev/sdu generic
0972B6CD587CD8E0 /dev/sdah generic
0972B6CD587CD8E0 /dev/dm-0 dmm
[root at service5 ~]#
Jim
Jim
On Wednesday, January 17, 2018, 5:12:10 PM EST, Bryan Banister <bbanister at jumptrading.com<mailto:bbanister at jumptrading.com>> wrote:
Hi all,
We are reviewing some of our configurations and were not sure what to make of the NSD Device Types that GPFS uses and what, if anything, do they change about how GPFS accesses/recovers/manages/etc the underlying storage based on this setting.
The documentation doesn’t say much about it other than to consult the /usr/lpp/mmfs/bin/mmdevdiscover command (no man page), which has this section:
# Known disk types currently are:
#
# powerdisk - EMC power path disk
# vpath - IBM virtual path disk
# dmm - Device-Mapper Multipath (DMM)
# dlmfdrv - Hitachi dlm
# hdisk - AIX hard disk
# lv - AIX logical volume. Historical usage only.
# Not allowed as a new device to mmcrnsd.
# gpt - GPFS partition on Windows disk
# generic - Device having no unique failover or multipathing
# characteristic (predominantly Linux devices).
# dasd - DASD device (for Linux on z Systems)
We have our storage under Linux Device-Mapper Multipath control (two device paths to all storage, active/passive) and are accessible under /dev/mapper, but the NSD types are current set to ‘generic’ not ‘dmm’. This is configured in the /var/mmfs/etc/nsddevices file:
if [[ $osName = Linux ]]
then
: # Add function to discover disks in the Linux environment.
ls -l /dev/mpath/ 2>/dev/null | awk '{print "mpath/"$9 " generic"}'
ls -l /dev/mapper/ 2>/dev/null | awk '{print "mapper/"$9 " generic"}'
ls -1 /dev/vd* 2>/dev/null | awk -F '/' '{print ""$3 " generic"}'
fi
Can somebody from IBM explain what the correct setting should be and what differences GPFS does with ‘generic’ vs. ‘dmm’ vs. others?
Thanks in advance!
-Bryan
________________________________
Note: This email is for the confidential use of the named addressee(s) only and may contain proprietary, confidential or privileged information. If you are not the intended recipient, you are hereby notified that any review, dissemination or copying of this email is strictly prohibited, and to please notify the sender immediately and destroy this email and any attachments. Email transmission cannot be guaranteed to be secure or error-free. The Company, therefore, does not make any guarantees as to the completeness or accuracy of this email or any attachments. This email is for informational purposes only and does not constitute a recommendation, offer, request or solicitation of any kind to buy, sell, subscribe, redeem or perform any type of transaction of a financial product.
_______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at spectrumscale.org
http://gpfsug.org/mailman/listinfo/gpfsug-discuss
________________________________
Note: This email is for the confidential use of the named addressee(s) only and may contain proprietary, confidential or privileged information. If you are not the intended recipient, you are hereby notified that any review, dissemination or copying of this email is strictly prohibited, and to please notify the sender immediately and destroy this email and any attachments. Email transmission cannot be guaranteed to be secure or error-free. The Company, therefore, does not make any guarantees as to the completeness or accuracy of this email or any attachments. This email is for informational purposes only and does not constitute a recommendation, offer, request or solicitation of any kind to buy, sell, subscribe, redeem or perform any type of transaction of a financial product.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20180118/5f430cb8/attachment.htm>
More information about the gpfsug-discuss
mailing list