[gpfsug-discuss] mmfsd write behavior
Aaron Knister
aaron.s.knister at nasa.gov
Fri Sep 8 04:05:42 BST 2017
Thanks Sven. I didn't think GPFS itself was caching anything on that
layer, but it's my understanding that O_DIRECT isn't sufficient to force
I/O to be flushed (e.g. the device itself might have a volatile caching
layer). Take someone using ZFS zvol's as NSDs. I can write() all day log
to that zvol (even with O_DIRECT) but there is absolutely no guarantee
those writes have been committed to stable storage and aren't just
sitting in RAM until an fsync() occurs (or some other bio function that
causes a flush). I also don't believe writing to a SATA drive with
O_DIRECT will force cache flushes of the drive's writeback cache..
although I just tested that one and it seems to actually trigger a scsi
cache sync. Interesting.
-Aaron
On 9/7/17 10:55 PM, Sven Oehme wrote:
> I am not sure what exactly you are looking for but all blockdevices are
> opened with O_DIRECT , we never cache anything on this layer .
>
>
> On Thu, Sep 7, 2017, 7:11 PM Aaron Knister <aaron.s.knister at nasa.gov
> <mailto:aaron.s.knister at nasa.gov>> wrote:
>
> Hi Everyone,
>
> This is something that's come up in the past and has recently resurfaced
> with a project I've been working on, and that is-- it seems to me as
> though mmfsd never attempts to flush the cache of the block devices its
> writing to (looking at blktrace output seems to confirm this). Is this
> actually the case? I've looked at the gpl headers for linux and I don't
> see any sign of blkdev_fsync, blkdev_issue_flush, WRITE_FLUSH, or
> REQ_FLUSH. I'm sure there's other ways to trigger this behavior that
> GPFS may very well be using that I've missed. That's why I'm asking :)
>
> I figure with FPO being pushed as an HDFS replacement using commodity
> drives this feature has *got* to be in the code somewhere.
>
> -Aaron
>
> --
> Aaron Knister
> NASA Center for Climate Simulation (Code 606.2)
> Goddard Space Flight Center
> (301) 286-2776
> _______________________________________________
> gpfsug-discuss mailing list
> gpfsug-discuss at spectrumscale.org <http://spectrumscale.org>
> http://gpfsug.org/mailman/listinfo/gpfsug-discuss
>
>
>
> _______________________________________________
> gpfsug-discuss mailing list
> gpfsug-discuss at spectrumscale.org
> http://gpfsug.org/mailman/listinfo/gpfsug-discuss
>
--
Aaron Knister
NASA Center for Climate Simulation (Code 606.2)
Goddard Space Flight Center
(301) 286-2776
More information about the gpfsug-discuss
mailing list