[gpfsug-discuss] Odd behavior with cat followed by grep.
John Hanks
griznog at gmail.com
Wed Feb 14 14:20:32 GMT 2018
Hi Luis,
GPFS is 4.2.3 (gpfs.base-4.2.3-6.x86_64), All servers (8 in front of a DDN
SFA12K) are RHEL 7.3 (stock DDN setup). All 47 clients are CentOS 7.4.
GPFS mount:
# mount | grep gpfs
gsfs0 on /srv/gsfs0 type gpfs (rw,relatime)
NFS mount:
mount | grep $HOME
10.210.15.57:/srv/gsfs0/home/griznog on /home/griznog type nfs
(rw,relatime,vers=3,rsize=1048576,wsize=1048576,namlen=255,soft,proto=tcp,timeo=600,retrans=2,sec=sys,mountaddr=10.210.15.57,mountvers=3,mountport=20048,mountproto=tcp,local_lock=none,addr=10.210.15.57)
Example script:
#!/bin/bash
cat pt.txt pt.txt pt.txt pt.txt pt.txt pt.txt pt.txt pt.txt pt.txt >
/srv/gsfs0/projects/pipetest.tmp.txt
grep L1 /srv/gsfs0/projects/pipetest.tmp.txt | wc -l
cat pt.txt pt.txt pt.txt pt.txt pt.txt pt.txt pt.txt pt.txt pt.txt >
$HOME/pipetest.tmp.txt
grep L1 $HOME/pipetest.tmp.txt | wc -l
Example output:
# ./pipetest.sh
1
1836
# ls -aln /srv/gsfs0/projects/pipetest.tmp.txt $HOME/pipetest.tmp.txt
-rw-r--r-- 1 39073 3953 530721 Feb 14 06:10 /home/griznog/pipetest.tmp.txt
-rw-r--r-- 1 39073 3001 530721 Feb 14 06:10
/srv/gsfs0/projects/pipetest.tmp.txt
We can "fix" the user case that exposed this by not using a temp file or
inserting a sleep, but I'd still like to know why GPFS is behaving this way
and make it stop.
mmlsconfig below.
Thanks,
jbh
mmlsconfig
Configuration data for cluster SCG-GS.scg-gs0:
----------------------------------------------
clusterName SCG-GS.scg-gs0
clusterId 8456032987852400706
dmapiFileHandleSize 32
maxblocksize 4096K
cnfsSharedRoot /srv/gsfs0/GS-NFS
cnfsMountdPort 597
socketMaxListenConnections 1024
fileHeatPeriodMinutes 1440
fileHeatLossPercent 1
pingPeriod 5
minMissedPingTimeout 30
afmHashVersion 1
minReleaseLevel 4.2.0.1
[scg-gs0,scg-gs1,scg-gs2,scg-gs3,scg-gs4,scg-gs5,scg-gs6,scg-gs7]
nsdbufspace 70
[common]
healthCheckInterval 20
maxStatCache 512
maxFilesToCache 50000
nsdMinWorkerThreads 512
nsdMaxWorkerThreads 1024
deadlockDetectionThreshold 0
deadlockOverloadThreshold 0
prefetchThreads 288
worker1Threads 320
maxMBpS 2000
[scg-gs0,scg-gs1,scg-gs2,scg-gs3,scg-gs4,scg-gs5,scg-gs6,scg-gs7]
maxMBpS 24000
[common]
atimeDeferredSeconds 300
pitWorkerThreadsPerNode 2
cipherList AUTHONLY
pagepool 1G
[scg-gs0,scg-gs1,scg-gs2,scg-gs3,scg-gs4,scg-gs5,scg-gs6,scg-gs7]
pagepool 8G
[common]
cnfsNFSDprocs 256
nfsPrefetchStrategy 1
autoload yes
adminMode central
File systems in cluster SCG-GS.scg-gs0:
---------------------------------------
/dev/gsfs0
On Tue, Feb 13, 2018 at 10:53 PM, Luis Bolinches <luis.bolinches at fi.ibm.com>
wrote:
> Sorry
>
> With cat
>
> [root at specscale01 IBM_REPO]# cp test a
> [root at specscale01 IBM_REPO]# cat a a a a > test && grep ATAG test | wc -l
> && sleep 4 && grep ATAG test | wc -l
> 0
> 0
> --
> Ystävällisin terveisin / Kind regards / Saludos cordiales / Salutations
> Luis Bolinches
> Consultant IT Specialist
> Mobile Phone: +358503112585 <+358%2050%203112585>
> https://www.youracclaim.com/user/luis-bolinches
>
> "If you always give you will always have" -- Anonymous
>
>
>
> From: Luis Bolinches <luis.bolinches at fi.ibm.com>
> To: gpfsug main discussion list <gpfsug-discuss at spectrumscale.org>
> Date: 14/02/2018 08:49
> Subject: Re: [gpfsug-discuss] Odd behavior with cat followed by
> grep.
> Sent by: gpfsug-discuss-bounces at spectrumscale.org
> ------------------------------
>
>
>
> Hi
>
> This seems to be setup specific
>
> Care to explain a bit more of the setup. Number of nodes GPFS versions,
> number of FS, Networking, running from admin node, server / client, number
> of NSD, separated meta and data, etc?
>
> I got interested and run a quick test on a gpfs far from powerful cluster
> of 3 nodes on KVM
>
> [root at specscale01 IBM_REPO]# echo "a a a a a a a a a a" > test && grep
> ATAG test | wc -l && sleep 4 && grep ATAG test | wc -l
> 0
> 0
> [root at specscale01 IBM_REPO]#
>
>
> --
> Ystävällisin terveisin / Kind regards / Saludos cordiales / Salutations
> Luis Bolinches
> Consultant IT Specialist
> Mobile Phone: +358503112585 <+358%2050%203112585>
> *https://www.youracclaim.com/user/luis-bolinches*
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__www.youracclaim.com_user_luis-2Dbolinches&d=DwMFAw&c=jf_iaSHvJObTbx-siA1ZOg&r=1mZ896psa5caYzBeaugTlc7TtRejJp3uvKYxas3S7Xc&m=HrR-mBJ82ubcbtBin7NGVl2VenLj726Fcah6-3XFvDs&s=hGQKSxR54bjW2i9KVDRpflHxrMcJhY269FZkQGWHtLI&e=>
>
> "If you always give you will always have" -- Anonymous
>
>
>
> From: John Hanks <griznog at gmail.com>
> To: gpfsug-discuss <gpfsug-discuss at spectrumscale.org>
> Date: 14/02/2018 07:33
> Subject: [gpfsug-discuss] Odd behavior with cat followed by grep.
> Sent by: gpfsug-discuss-bounces at spectrumscale.org
> ------------------------------
>
>
>
> Hi,
>
> We have a GPFS filesystem mounted on CentOS 7.4 as type gpfs, pretty
> straightforward run of the mill stuff. But are seeing this odd behavior. If
> I do this in a shell script, given a file called "a"
>
> cat a a a a a a a a a a > /path/to/gpfs/mount/test
> grep ATAG /path/to/gpfs/mount/test | wc -l
> sleep 4
> grep ATAG /path/to/gpfs/mount/test | wc -l
>
> The first grep | wc -l returns 1, because grep outputs "Binary file
> /path/to/gpfs/mount/test matches"
>
> The second grep | wc -l returns the correct count of ATAG in the file.
>
> Why does it take 4 seconds (3 isn't enough) for that file to be properly
> recognized as a text file and/or why is it seen as a binary file in the
> first place since a is a plain text file?
>
> Note that I have the same filesystem mounted via NFS and over an NFS mount
> it works as expected.
>
> Any illumination is appreciated,
>
> jbh_______________________________________________
> gpfsug-discuss mailing list
> gpfsug-discuss at spectrumscale.org
>
> *https://urldefense.proofpoint.com/v2/url?u=http-3A__gpfsug.org_mailman_listinfo_gpfsug-2Ddiscuss&d=DwICAg&c=jf_iaSHvJObTbx-siA1ZOg&r=1mZ896psa5caYzBeaugTlc7TtRejJp3uvKYxas3S7Xc&m=ut35qIIMxjZMX3obFJ2xtUMng4MtGtKz4YHxpkgQbak&s=cNt66GjRD6rVhq7nGcvT76l-0_u2C3UTz9SfwzHf1xw&e=*
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__gpfsug.org_mailman_listinfo_gpfsug-2Ddiscuss&d=DwICAg&c=jf_iaSHvJObTbx-siA1ZOg&r=1mZ896psa5caYzBeaugTlc7TtRejJp3uvKYxas3S7Xc&m=ut35qIIMxjZMX3obFJ2xtUMng4MtGtKz4YHxpkgQbak&s=cNt66GjRD6rVhq7nGcvT76l-0_u2C3UTz9SfwzHf1xw&e=>
>
>
>
>
> Ellei edellä ole toisin mainittu: / Unless stated otherwise above:
> Oy IBM Finland Ab
> PL 265, 00101 Helsinki, Finland
> Business ID, Y-tunnus: 0195876-3
> Registered in Finland_______________________________________________
> gpfsug-discuss mailing list
> gpfsug-discuss at spectrumscale.org
> https://urldefense.proofpoint.com/v2/url?u=http-3A__gpfsug.
> org_mailman_listinfo_gpfsug-2Ddiscuss&d=DwICAg&c=jf_iaSHvJObTbx-siA1ZOg&r=
> 1mZ896psa5caYzBeaugTlc7TtRejJp3uvKYxas3S7Xc&m=HrR-
> mBJ82ubcbtBin7NGVl2VenLj726Fcah6-3XFvDs&s=d5YiAyXz4el9bF0zjGL9gVjnTfbX4z
> -qelZodxRqlz0&e=
>
>
>
>
>
> Ellei edellä ole toisin mainittu: / Unless stated otherwise above:
> Oy IBM Finland Ab
> PL 265, 00101 Helsinki, Finland
> Business ID, Y-tunnus: 0195876-3
> Registered in Finland
>
> _______________________________________________
> gpfsug-discuss mailing list
> gpfsug-discuss at spectrumscale.org
> http://gpfsug.org/mailman/listinfo/gpfsug-discuss
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20180214/fe02f685/attachment.htm>
More information about the gpfsug-discuss
mailing list