[gpfsug-discuss] mmfsd write behavior

[Aaron Knister]

Thanks Sven. I didn't think GPFS itself was caching anything on that 
layer, but it's my understanding that O_DIRECT isn't sufficient to force 
I/O to be flushed (e.g. the device itself might have a volatile caching 
layer). Take someone using ZFS zvol's as NSDs. I can write() all day log 
to that zvol (even with O_DIRECT) but there is absolutely no guarantee 
those writes have been committed to stable storage and aren't just 
sitting in RAM until an fsync() occurs (or some other bio function that 
causes a flush). I also don't believe writing to a SATA drive with 
O_DIRECT will force cache flushes of the drive's writeback cache.. 
although I just tested that one and it seems to actually trigger a scsi 
cache sync. Interesting.

-Aaron

On 9/7/17 10:55 PM, Sven Oehme wrote:
> I am not sure what exactly you are looking for but all blockdevices are 
> opened with O_DIRECT , we never cache anything on this layer .
> 
> 
> On Thu, Sep 7, 2017, 7:11 PM Aaron Knister <aaron.s.knister at nasa.gov 
> <mailto:aaron.s.knister at nasa.gov>> wrote:
> 
>     Hi Everyone,
> 
>     This is something that's come up in the past and has recently resurfaced
>     with a project I've been working on, and that is-- it seems to me as
>     though mmfsd never attempts to flush the cache of the block devices its
>     writing to (looking at blktrace output seems to confirm this). Is this
>     actually the case? I've looked at the gpl headers for linux and I don't
>     see any sign of blkdev_fsync, blkdev_issue_flush, WRITE_FLUSH, or
>     REQ_FLUSH. I'm sure there's other ways to trigger this behavior that
>     GPFS may very well be using that I've missed. That's why I'm asking :)
> 
>     I figure with FPO being pushed as an HDFS replacement using commodity
>     drives this feature has *got* to be in the code somewhere.
> 
>     -Aaron
> 
>     --
>     Aaron Knister
>     NASA Center for Climate Simulation (Code 606.2)
>     Goddard Space Flight Center
>     (301) 286-2776
>     _______________________________________________
>     gpfsug-discuss mailing list
>     gpfsug-discuss at spectrumscale.org <http://spectrumscale.org>
>     http://gpfsug.org/mailman/listinfo/gpfsug-discuss
> 
> 
> 
> _______________________________________________
> gpfsug-discuss mailing list
> gpfsug-discuss at spectrumscale.org
> http://gpfsug.org/mailman/listinfo/gpfsug-discuss
> 

-- 
Aaron Knister
NASA Center for Climate Simulation (Code 606.2)
Goddard Space Flight Center
(301) 286-2776