[gpfsug-discuss] mmfsd recording High CPU usage

[Jim Doherty]

 At a guess with no data ....   if the application is opening more files than can fit in the maxFilesToCache (MFTC) objects  GPFS will expand the MFTC to support the open files,  but it will also scan to try and free any unused objects.    If you can identify the user job that is causing this  you could monitor a system more closely. 

Jim

    On Wednesday, November 21, 2018, 2:10:45 AM EST, Saula, Oluwasijibomi <oluwasijibomi.saula at ndsu.edu> wrote:  
 
  <!--#yiv4325073500 P {margin-top:0;margin-bottom:0;}-->
Hello Scalers,




First, let me say Happy Thanksgiving to those of us in the US and to those beyond, well, it's a still happy day seeing we're still above ground!😊




Now, what I have to discuss isn't anything extreme so don't skip the turkey for this, but lately, on a few of our compute GPFS client nodes, we've been noticing high CPU usage by the mmfsd process and are wondering why. Here's a sample:






[~]# top -b -n 1 | grep mmfs

   PID USER     PR  NI   VIRT    RES   SHR S  %CPU %MEM     TIME+ COMMAND










231898 root       0 -20 14.508g 4.272g  70168 S 93.8  6.8 69503:41 mmfsd

 4161 root       0 -20 121876   9412   1492 S   0.0 0.0   0:00.22 runmmfs




Obviously, this behavior was likely triggered by a not-so-convenient user job that in most cases is long finished by the time we investigate. Nevertheless, does anyone have an idea why this might be happening? Any thoughts on preventive steps even?




This is GPFS v4.2.3 on Redhat 7.4, btw...





Thanks, Siji SaulaHPC System AdministratorCenter for Computationally Assisted Science & TechnologyNORTH DAKOTA STATE UNIVERSITY 
Research 2 Building – Room 220BDept 4100, PO Box 6050  / Fargo, ND 58108-6050p:701.231.7749www.ccast.ndsu.edu | www.ndsu.edu


_______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at spectrumscale.org
http://gpfsug.org/mailman/listinfo/gpfsug-discuss
  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20181121/16de3172/attachment.htm>