[gpfsug-discuss] read replica fastest tuning for short distance

[Zachary Giles]

Hello all,
( Making good use of the mailing list recently.. :) )

I have two datacenters that are fairly close to each other (about
0.5ms away by-the-wire) and have a fairly small pipe between them (
single 40Gbit ).
There is a stretched filesystem between the datacenters, two failure
groups, and replicas=2 on all data and metadata.
I'm trying to ensure that clients on each side only read their local
replica instead of filling the pipe with reads from the other side.

While readreplica=local would make sense, text suggests that it mostly
checks to see if you're in the same subnet to check for local reads.
This won't work for me since there are many many subnets on each side.

The newer option of readreplica=fastest looks like a good idea, except
that the latency of the connection between the datacenters is so small
compared to the disk latency that reads often come from the wrong
side.
I've tried tuning fastestPolicyCmpThreshold down to 5 and
fastestPolicyMinDiffPercent down to 10, but I still see reads from
both sides.

Does anyone have any pointers for tuning read replica using fastest on
close-by multidatacenter installs to help ensure reads are only from
one side?
Any numbers that have been shown to work?
I haven't been able to find a way to inspect the GPFS read latencies
that it is using to make the decision. I looked in the dumps, but
don't seem to see anything. Anyone know if it's possible and where
they are?

Thanks
-Zach


-- 
Zach Giles
zgiles at gmail.com