[gpfsug-discuss] file layout API + file fragmentation

Aaron Knister aaron.s.knister at nasa.gov
Sun Nov 5 23:39:07 GMT 2017


Thanks Marc, that helps. I can't easily use tsdbfs for what I'm working
on since it needs to be run as unprivileged users.

Perhaps I'm not asking the right question. I'm wondering how the file
layout api behaves if a given "block"-aligned offset in a file is made
up of sub-blocks/fragments that are not all on the same NSD. The
assumption based on how I've seen the API used so far is that all
sub-blocks within a block at a given offset within a file are all on the
same NSD.

-Aaron

On 11/5/17 6:01 PM, Marc A Kaplan wrote:
> I googled GPFS_FCNTL_GET_DATABLKDISKIDX
> 
> and found this discussion:
> 
>  https://www.ibm.com/developerworks/community/forums/html/topic?id=db48b190-4f2f-4e24-a035-25d3e2b06b2d&ps=50
> 
> In general, GPFS files ARE deliberately "fragmented" but we don't say
> that - we say they are "striped" over many disks -- and that is
> generally a good thing for parallel performance.
> 
> Also, in GPFS, if the last would-be block of a file is less than a
> block, then it is stored in a "fragment" of a block.  
> So you see we use "fragment" to mean something different than other file
> systems you may know.
> 
> --marc
> 
> 
> 
> From:        Aaron Knister <aaron.s.knister at nasa.gov>
> To:        gpfsug main discussion list <gpfsug-discuss at spectrumscale.org>
> Date:        11/04/2017 12:22 PM
> Subject:        [gpfsug-discuss] file layout API + file fragmentation
> Sent by:        gpfsug-discuss-bounces at spectrumscale.org
> ------------------------------------------------------------------------
> 
> 
> 
> I've got a question about the file layout API and how it reacts in the
> case of fragmented files.
> 
> I'm using the GPFS_FCNTL_GET_DATABLKDISKIDX structure and have some code
> based on tsGetDataBlk.C. I'm basing the block size based off of what's
> returned by filemapOut.blockSize but that only seems to return a value >
> 0 when filemapIn.startOffset is 0.
> 
> In a case where a file were to be made up of a significant number of
> non-contiguous fragments (which... would be awful in of itself) how
> would this be reported by the file layout API? Does the interface
> technically just report the disk location information of the first block
> of the $blockSize range and assume that it's contiguous?
> 
> Thanks!
> 
> -Aaron
> 
> -- 
> Aaron Knister
> NASA Center for Climate Simulation (Code 606.2)
> Goddard Space Flight Center
> (301) 286-2776
> _______________________________________________
> gpfsug-discuss mailing list
> gpfsug-discuss at spectrumscale.org
> https://urldefense.proofpoint.com/v2/url?u=http-3A__gpfsug.org_mailman_listinfo_gpfsug-2Ddiscuss&d=DwICAg&c=jf_iaSHvJObTbx-siA1ZOg&r=cvpnBBH0j41aQy0RPiG2xRL_M8mTc1izuQD3_PmtjZ8&m=wnR7m6d4urZ_8dM4mkHQjMbFD9xJEeesmJyzt1osCnM&s=-dgGO6O5i1EqWj-8MmzjxJ1Iz2I5gT1aRmtyP44Cvdg&e=
> 
> 
> 
> 
> 
> 
> _______________________________________________
> gpfsug-discuss mailing list
> gpfsug-discuss at spectrumscale.org
> http://gpfsug.org/mailman/listinfo/gpfsug-discuss
> 

-- 
Aaron Knister
NASA Center for Climate Simulation (Code 606.2)
Goddard Space Flight Center
(301) 286-2776



More information about the gpfsug-discuss mailing list