[gpfsug-discuss] GPFS on ZFS?
Allen, Benjamin S.
bsallen at alcf.anl.gov
Mon Jun 13 17:33:54 BST 2016
Jaime,
See https://www.ibm.com/support/knowledgecenter/STXKQY_4.2.0/com.ibm.spectrum.scale.v4r2.adm.doc/bl1adm_nsddevices.htm. An example I have for add /dev/nvme* devices:
* GPFS doesn't know how that /dev/nvme* are valid block devices, use a user exit script to let it know about it
cp /usr/lpp/mmfs/samples/nsddevices.sample /var/mmfs/etc/nsddevices
* Edit /var/mmfs/etc/nsddevices, and add to linux section:
if [[ $osName = Linux ]]
then
: # Add function to discover disks in the Linux environment.
for dev in $( cat /proc/partitions | grep nvme | awk '{print $4}' )
do
echo $dev generic
done
fi
* Copy edited nsddevices to the rest of the nodes at the same path
for host in n01 n02 n03 n04; do
scp /var/mmfs/etc/nsddevices ${host}:/var/mmfs/etc/nsddevices
done
Ben
> On Jun 13, 2016, at 11:26 AM, Jaime Pinto <pinto at scinet.utoronto.ca> wrote:
>
> Hi Chris
>
> As I understand, GPFS likes to 'see' the block devices, even on a hardware raid solution such at DDN's.
>
> How is that accomplished when you use ZFS for software raid?
> On page 4 I see this info, and I'm trying to interpret it:
>
> General Configuration
> ...
> * zvols
> * nsddevices
> - echo "zdX generic"
>
>
> Thanks
> Jaime
>
> Quoting "Hoffman, Christopher P" <cphoffma at lanl.gov>:
>
>> Hi Jaime,
>>
>> What in particular would you like explained more? I'd be more than happy to discuss things further.
>>
>> Chris
>> ________________________________________
>> From: gpfsug-discuss-bounces at spectrumscale.org [gpfsug-discuss-bounces at spectrumscale.org] on behalf of Jaime Pinto [pinto at scinet.utoronto.ca]
>> Sent: Monday, June 13, 2016 10:11
>> To: gpfsug main discussion list
>> Subject: Re: [gpfsug-discuss] GPFS on ZFS?
>>
>> I just came across this presentation on "GPFS with underlying ZFS
>> block devices", by Christopher Hoffman, Los Alamos National Lab,
>> although some of the
>> implementation remains obscure.
>>
>> http://files.gpfsug.org/presentations/2016/anl-june/LANL_GPFS_ZFS.pdf
>>
>> It would be great to have more details, in particular the possibility
>> of straight use of GPFS on ZFS, instead of the 'archive' use case as
>> described on the presentation.
>>
>> Thanks
>> Jaime
>>
>>
>>
>>
>> Quoting "Jaime Pinto" <pinto at scinet.utoronto.ca>:
>>
>>> Since we can not get GNR outside ESS/GSS appliances, is anybody using
>>> ZFS for software raid on commodity storage?
>>>
>>> Thanks
>>> Jaime
>>>
>>>
>>
>>
>>
>>
>> ************************************
>> TELL US ABOUT YOUR SUCCESS STORIES
>> http://www.scinethpc.ca/testimonials
>> ************************************
>> ---
>> Jaime Pinto
>> SciNet HPC Consortium - Compute/Calcul Canada
>> www.scinet.utoronto.ca - www.computecanada.org
>> University of Toronto
>> 256 McCaul Street, Room 235
>> Toronto, ON, M5T1W5
>> P: 416-978-2755
>> C: 416-505-1477
>>
>> ----------------------------------------------------------------
>> This message was sent using IMP at SciNet Consortium, University of Toronto.
>>
>> _______________________________________________
>> gpfsug-discuss mailing list
>> gpfsug-discuss at spectrumscale.org
>> http://gpfsug.org/mailman/listinfo/gpfsug-discuss
>> _______________________________________________
>> gpfsug-discuss mailing list
>> gpfsug-discuss at spectrumscale.org
>> http://gpfsug.org/mailman/listinfo/gpfsug-discuss
>>
>
>
>
>
>
>
> ************************************
> TELL US ABOUT YOUR SUCCESS STORIES
> http://www.scinethpc.ca/testimonials
> ************************************
> ---
> Jaime Pinto
> SciNet HPC Consortium - Compute/Calcul Canada
> www.scinet.utoronto.ca - www.computecanada.org
> University of Toronto
> 256 McCaul Street, Room 235
> Toronto, ON, M5T1W5
> P: 416-978-2755
> C: 416-505-1477
>
> ----------------------------------------------------------------
> This message was sent using IMP at SciNet Consortium, University of Toronto.
>
> _______________________________________________
> gpfsug-discuss mailing list
> gpfsug-discuss at spectrumscale.org
> http://gpfsug.org/mailman/listinfo/gpfsug-discuss
More information about the gpfsug-discuss
mailing list