[gpfsug-discuss] preventing HSM tape recall storms
Frederick Stock
stockf at us.ibm.com
Mon Jul 9 19:57:38 BST 2018
Another option is to request Apple to support the OFFLINE flag in the SMB
protocol. The more Mac customers making such a request (I have asked
others to do likewise) might convince Apple to add this checking to their
SMB client.
Fred
__________________________________________________
Fred Stock | IBM Pittsburgh Lab | 720-430-8821
stockf at us.ibm.com
From: "Christof Schmitt" <christof.schmitt at us.ibm.com>
To: gpfsug-discuss at spectrumscale.org
Date: 07/09/2018 02:53 PM
Subject: Re: [gpfsug-discuss] preventing HSM tape recall storms
Sent by: gpfsug-discuss-bounces at spectrumscale.org
> we had left out "gpfs" from the
> vfs objects =
> line in smb.conf
>
> so setting
> vfs objects = gpfs (etc)
> gpfs:hsm = yes
> gpfs:recalls = yes (not "no" as I had originally, and is implied by the
manual)
Thank you for the update. gpfs:recalls=yes is the default, allowing
recalls of files. If you set that to 'no', Samba will deny access to
"OFFLINE" files in GPFS through SMB.
> and setting the offline flag on the file by migrating it, so that
> # mmlsattr -L filename.jpg
> ...
> Misc attributes: ARCHIVE OFFLINE
>
> now Explorer on Windows 7 and 10 do not recall the file while viewing
the folder with "Large icons"
>
> and a standard icon with an X is displayed.
>
> But after the file is then opened and recalled, the icon displays the
thumbnail image and the OFFLINE flag is lost.
Yes, that is working as intended. While the file is only in the
"external pool" (e.g. HSM tape), the OFFLINE flag is reported. Once
you read/write data, that triggers a recall to the disk pool and the
flag is cleared.
> Also as you observed, Finder on MacOSX 10.13 ignores the file's offline
flag,
>
> so we still risk a recall storm caused by them.
The question here would be how to handle the Mac clients. You could
configured two SMB shares on the same path: One with gpfs:recalls=yes
and tell the Windows users to access that share; the other one with
gpfs:recalls=no and tell the Mac users to use that share. That would
avoid the recall storms, but runs the risk of Mac users connecting to
the wrong share and avoiding this workaround...
Regards,
Christof Schmitt || IBM || Spectrum Scale Development || Tucson, AZ
christof.schmitt at us.ibm.com || +1-520-799-2469 (T/L: 321-2469)
----- Original message -----
From: Cameron Dunn <Cameron.Dunn at bristol.ac.uk>
Sent by: gpfsug-discuss-bounces at spectrumscale.org
To: "gpfsug-discuss at spectrumscale.org" <gpfsug-discuss at spectrumscale.org>
Cc:
Subject: Re: [gpfsug-discuss] preventing HSM tape recall storms
Date: Sat, Jul 7, 2018 2:30 PM
Thanks Christof,
we had left out "gpfs" from the
vfs objects =
line in smb.conf
so setting
vfs objects = gpfs (etc)
gpfs:hsm = yes
gpfs:recalls = yes (not "no" as I had originally, and is implied by the
manual)
and setting the offline flag on the file by migrating it, so that
# mmlsattr -L filename.jpg
...
Misc attributes: ARCHIVE OFFLINE
now Explorer on Windows 7 and 10 do not recall the file while viewing the
folder with "Large icons"
and a standard icon with an X is displayed.
But after the file is then opened and recalled, the icon displays the
thumbnail image and the OFFLINE flag is lost.
Also as you observed, Finder on MacOSX 10.13 ignores the file's offline
flag,
so we still risk a recall storm caused by them.
All the best,
Cameron
From: gpfsug-discuss-bounces at spectrumscale.org
<gpfsug-discuss-bounces at spectrumscale.org> on behalf of Christof Schmitt
<christof.schmitt at us.ibm.com>
Sent: 03 July 2018 20:37:08
To: gpfsug-discuss at spectrumscale.org
Cc: gpfsug-discuss at spectrumscale.org
Subject: Re: [gpfsug-discuss] preventing HSM tape recall storms
> HSM over LTFS-EE runs the risk of a recall storm if files which have
been migrated to tape
> are then shared by Samba to Macs and PCs.
> MacOS Finder and Windows Explorer will want to display all the thumbnail
images of a
> folder's contents, which will recall lots of files from tape.
SMB clients can query file information, including the OFFLINE
flag. With Spectrum Scale and the "gpfs" module loaded in Samba that
is mapped from the the OFFLINE flag that is visible in "mmlsattr
-L". In those systems, the SMB client can determine that a file is
offline.
In our experience this is handled correctly in Windows Explorer; when
an "offline" file is encountered, no preview is generated from the
file data. The Finder on Mac clients does not seem to honor the
OFFLINE flag, thus the main problems are typically recall storms
caused by Mac clients.
> According to the Samba documentation this is preventable by setting the
following
> ----------------------------------------------
> https://www.samba.org/samba/docs/current/man-html/vfs_gpfs.8.html
>
> gpfs:recalls = [ yes | no ]
> When this option is set to no, an attempt to open an offline file
> will be rejected with access denied.
> This helps preventing recall storms triggered by careless applications
like Finder and Explorer.
>
> yes(default) - Open files that are offline. This will recall the files
from HSM.
> no - Reject access to offline files with access denied. This will
prevent recalls of files from HSM.
> Using this setting also requires gpfs:hsm to be set to yes.
>
> gpfs:hsm = [ yes | no ]
> Enable/Disable announcing if this FS has HSM enabled.
> no(default) - Do not announce HSM.
> yes - Announce HSM.
> --------------------------------------------------
>
> However we could not get this to work.
>
> On Centos7/Samba4.5, smb.conf contained
> gpfs:hsm = yes
> gpfs:recalls = no
> (also tried setting gpfs:offline = yes, though this is not documented)
These options apply to the "gpfs" module in Samba. The Samba version
you are using needs to be built with GPFS support and the "gpfs"
module needs to be loaded through the "vfs objects" configuration. As
Centos7/Samba4.5 is mentioned, would guess that the CentOS provided
Samba version is used, which is probably not compiled with GPFS
support.
>From IBM we would recommend to use CES for protocol services, which
also provides Samba for SMB. The Samba provided through CES is
configured so that the gpfs:recalls option can be used:
https://www.ibm.com/support/knowledgecenter/en/STXKQY_4.2.3/com.ibm.spectrum.scale.v4r23.doc/bl1adm_mmsmb.htm
gpfs:recalls
If the value is set as yes files that have been migrated from disk
will be recalled on access. By default, this is enabled. If
recalls = no files will not be recalled on access and the client
will receive ACCESS_DENIED message.
> We made a share containing image files that were then migrated to tape
by LTFS-EE,
> to see if these flags were respected by OS X Finder or Windows Explorer.
>
> Neither Mac OS X (using SMB3) or Windows 7 (using SMB2) respected the
settings,
> so that when browsing the stubs in the share, the files were recalled
from tape
> and the thumbnails displayed.
>
> Has anyone seen these flags working as they are supposed to ?
Yes, they are working, as we use them in our Samba build. Debugging
this would require looking at the Samba configuration and possibly
collecting a trace. If my above assumption was wrong and this problem
occurs with the CES Samba (gpfs.smb), please open a PMR for debugging
this issue. If this is not the CES Samba, please contact the provider
of the Samba package for additional support.
Regards,
Christof
Christof Schmitt || IBM || Spectrum Scale Development || Tucson, AZ
christof.schmitt at us.ibm.com || +1-520-799-2469 (T/L: 321-2469)
----- Original message -----
From: Cameron Dunn <Cameron.Dunn at bristol.ac.uk>
Sent by: gpfsug-discuss-bounces at spectrumscale.org
To: "gpfsug-discuss at spectrumscale.org" <gpfsug-discuss at spectrumscale.org>
Cc:
Subject: [gpfsug-discuss] preventing HSM tape recall storms
Date: Tue, Jul 3, 2018 6:22 AM
HSM over LTFS-EE runs the risk of a recall storm if files which have been
migrated to tape
are then shared by Samba to Macs and PCs.
MacOS Finder and Windows Explorer will want to display all the thumbnail
images of a
folder's contents, which will recall lots of files from tape.
According to the Samba documentation this is preventable by setting the
following
----------------------------------------------
https://www.samba.org/samba/docs/current/man-html/vfs_gpfs.8.html
gpfs:recalls = [ yes | no ]
When this option is set to no, an attempt to open an offline file
will be rejected with access denied.
This helps preventing recall storms triggered by careless applications
like Finder and Explorer.
yes(default) - Open files that are offline. This will recall the files
from HSM.
no - Reject access to offline files with access denied. This will prevent
recalls of files from HSM.
Using this setting also requires gpfs:hsm to be set to yes.
gpfs:hsm = [ yes | no ]
Enable/Disable announcing if this FS has HSM enabled.
no(default) - Do not announce HSM.
yes - Announce HSM.
--------------------------------------------------
However we could not get this to work.
On Centos7/Samba4.5, smb.conf contained
gpfs:hsm = yes
gpfs:recalls = no
(also tried setting gpfs:offline = yes, though this is not documented)
We made a share containing image files that were then migrated to tape by
LTFS-EE,
to see if these flags were respected by OS X Finder or Windows Explorer.
Neither Mac OS X (using SMB3) or Windows 7 (using SMB2) respected the
settings,
so that when browsing the stubs in the share, the files were recalled from
tape
and the thumbnails displayed.
Has anyone seen these flags working as they are supposed to ?
Many thanks for any ideas,
Cameron
Cameron Dunn
Advanced Computing Systems Administrator
Advanced Computing Research Centre
University of Bristol
_______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at spectrumscale.org
http://gpfsug.org/mailman/listinfo/gpfsug-discuss
_______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at spectrumscale.org
http://gpfsug.org/mailman/listinfo/gpfsug-discuss
_______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at spectrumscale.org
http://gpfsug.org/mailman/listinfo/gpfsug-discuss
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20180709/404a1f8c/attachment.htm>
More information about the gpfsug-discuss
mailing list