[gpfsug-discuss] wanted...gpfs policy that places larger files onto a pool based on size

[Chris Scott]

Hi Brian

This is exactly what I do with a SSD tier on top of 10K and 7.2K tiers.

HAWC is another recent option that might address Eric's requirement but
needs further consideration of the read requirements you want from the
small files.

Cheers
Chris

On 31 October 2016 at 17:23, Brian Marshall <mimarsh2 at vt.edu> wrote:

> When creating a "fast tier" storage pool in a filesystem is the normal
> case to create a placement policy that places all files in the fast tier
> and migrates out old and large files?
>
>
> Brian Marshall
>
> On Mon, Oct 31, 2016 at 1:20 PM, Jez Tucker <jez.tucker at gpfsug.org> wrote:
>
>> Hey Bryan
>>
>>   There was a previous RFE for path placement from the UG, but Yuri told
>> me this was not techically possible as an inode has no knowledge about the
>> parent dentry.  (IIRC).    You can see this in effect in the C API.  It is
>> possible to work this out at kernel level, but it's so costly that it
>> becomes non-viable at scale / performance.
>>
>> IBMers please chip in and expand if you will.
>>
>> Jez
>>
>>
>> On 31/10/16 17:09, Bryan Banister wrote:
>>
>> The File Placement Policy that you are trying to set cannot use the size
>> of the file to determine the placement of the file in a GPFS Storage Pool.
>> This is because GPFS has no idea what the file size will be when the file
>> is open()’d for writing.
>>
>>
>>
>> Hope that helps!
>>
>> -Bryan
>>
>>
>>
>> PS. I really wish that we could use a path for specifying data placement
>> in a GPFS Pool, and not just the file name, owner, etc.  I’ll submit a RFE
>> for this.
>>
>>
>>
>> *From:* gpfsug-discuss-bounces at spectrumscale.org [
>> mailto:gpfsug-discuss-bounces at spectrumscale.org
>> <gpfsug-discuss-bounces at spectrumscale.org>] *On Behalf Of *J. Eric
>> Wonderley
>> *Sent:* Monday, October 31, 2016 11:53 AM
>> *To:* gpfsug main discussion list
>> *Subject:* [gpfsug-discuss] wanted...gpfs policy that places larger
>> files onto a pool based on size
>>
>>
>>
>> I wanted to do something like this...
>>
>>
>> [root at cl001 ~]# cat /opt/gpfs/home.ply
>> /*Failsafe migration of old small files back to spinning media
>> pool(fc_8T) */
>> RULE 'theshold' MIGRATE FROM POOL 'system' THRESHOLD(90,70)
>> WEIGHT(ACCESS_TIME) TO POOL 'fc_8T'
>> /*Write files larger than 16MB to pool called "fc_8T" */
>> RULE 'bigfiles' SET POOL 'fc_8T' WHERE FILE_SIZE>16777216
>> /*Move anything else to system pool */
>> RULE 'default' SET POOL 'system'
>>
>> Apparently there is no happiness using FILE_SIZE in a placement policy:
>> [root at cl001 ~]# mmchpolicy home /opt/gpfs/home.ply
>> Error while validating policy `home.ply': rc=22:
>> PCSQLERR: 'FILE_SIZE' is an unsupported or unknown attribute or variable
>> name in this context.
>> PCSQLCTX: at line 4 of 6: RULE 'bigfiles' SET POOL 'fc_8T' WHERE
>> {{{FILE_SIZE}}}>16777216
>> runRemoteCommand_v2: cl002.cl.arc.internal: tschpolicy /dev/home
>> /var/mmfs/tmp/tspolicyFile.mmchpolicy.113372 -t home.ply   failed.
>> mmchpolicy: Command failed. Examine previous error messages to determine
>> cause.
>>
>> Can anyone suggest a way to accomplish this using policy?
>>
>> ------------------------------
>>
>> Note: This email is for the confidential use of the named addressee(s)
>> only and may contain proprietary, confidential or privileged information.
>> If you are not the intended recipient, you are hereby notified that any
>> review, dissemination or copying of this email is strictly prohibited, and
>> to please notify the sender immediately and destroy this email and any
>> attachments. Email transmission cannot be guaranteed to be secure or
>> error-free. The Company, therefore, does not make any guarantees as to the
>> completeness or accuracy of this email or any attachments. This email is
>> for informational purposes only and does not constitute a recommendation,
>> offer, request or solicitation of any kind to buy, sell, subscribe, redeem
>> or perform any type of transaction of a financial product.
>>
>>
>> _______________________________________________
>> gpfsug-discuss mailing list
>> gpfsug-discuss at spectrumscale.orghttp://gpfsug.org/mailman/listinfo/gpfsug-discuss
>>
>>
>> _______________________________________________
>> gpfsug-discuss mailing list
>> gpfsug-discuss at spectrumscale.org
>> http://gpfsug.org/mailman/listinfo/gpfsug-discuss
>>
>>
>
> _______________________________________________
> gpfsug-discuss mailing list
> gpfsug-discuss at spectrumscale.org
> http://gpfsug.org/mailman/listinfo/gpfsug-discuss
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20161031/2f8a0428/attachment.htm>