[gpfsug-discuss] mmbackup logging issue

Edward Wahl ewahl at osc.edu
Fri Mar 3 18:50:55 GMT 2017


Comments inline

On Fri, 3 Mar 2017 12:59:48 +0000
Jez Tucker <jtucker at pixitmedia.com> wrote:

> Hi Ash,
> 
>    This is the primary reason for using snapshots for mmbackup ( -S 
> snapname ) and making sure that only mmbackup sends data to backup 
> rather than an oob dsmc:
> 
> Tue Jan 17 18:07:31 2017 mmbackup:Audit files /cs/mmbackup.audit.gpfs* 
> contain 0 failed paths but there were 10012 failures.
> Cannot reconcile shadow database.
> Unable to compensate for all TSM errors in new shadow database.
>    Preserving previous shadow database.
>    Run next mmbackup with -q to synchronize shadow database.  exit 12 
> <------------ ick!
> 
> Other obvious causes are very, very, odd filenames / paths.

Yes, GPFS allows much more lenient filenames than TSM does.  You can attempt to
cut this back a bit with 'WILDCARDSARELITERAL yes' and 'QUOTESARELITERAL yes'
in your dsm.opt  (and I recommend SKIPACLUPDATECHECK yes and UPDATECTIME
yes )

But this still won't catch everything.  We still tend to find foreign character
sets, control sequences that look like people trying to exit emacs, etc. in
file names.  Valid for the Filesystem, not so much for TSM.

Ed

> 
> Jez
> 
> 
> On 28/02/17 16:10, Ashish Thandavan wrote:
> > Dear all,
> >
> > We have a small GPFS cluster and a separate server running TSM and one 
> > of the three NSD servers backs up our GPFS filesystem to the TSM 
> > server using mmbackup. After a recent upgrade from v3.5 to 4.1.1, 
> > we've noticed that mmbackup no longer logs stuff like it used to :
> >
> > ...
> > Thu Jan 19 05:45:41 2017 mmbackup:Backing up files: 0 backed up, 
> > 870532 expired, 2 failed.
> > Thu Jan 19 06:15:41 2017 mmbackup:Backing up files: 0 backed up, 
> > 870532 expired, 3 failed.
> > Thu Jan 19 06:45:41 2017 mmbackup:Backing up files: 0 backed up, 
> > 870532 expired, 3 failed.
> > ...
> >
> >
> > instead of
> >
> > ...
> > Sat Dec  3 12:01:00 2016 mmbackup:Backing up files: 105030 backed up, 
> > 635456 expired, 30 failed.
> > Sat Dec  3 12:31:00 2016 mmbackup:Backing up files: 205934 backed up, 
> > 635456 expired, 57 failed.
> > Sat Dec  3 13:01:00 2016 mmbackup:Backing up files: 321702 backed up, 
> > 635456 expired, 169 failed.
> > ...
> >
> > like it used to pre-upgrade.
> >
> > I am therefore unable to see how far long it has got, and indeed if it 
> > completed successfully, as this is what it logs at the end of a job :
> >
> > ...
> > Tue Jan 17 18:07:31 2017 mmbackup:Completed policy backup run with 0 
> > policy errors, 10012 files failed, 0 severe errors, returning rc=9.
> > Tue Jan 17 18:07:31 2017 mmbackup:Policy for backup returned 9 Highest 
> > TSM error 12
> > mmbackup: TSM Summary Information:
> >     Total number of objects inspected:     20617273
> >     Total number of objects backed up:     0
> >     Total number of objects updated:     0
> >     Total number of objects rebound:     0
> >     Total number of objects deleted:     0
> >     Total number of objects expired:     1
> >     Total number of objects failed:     10012
> >     Total number of objects encrypted:     0
> >     Total number of bytes inspected:     3821624716861
> >     Total number of bytes transferred:     3712040943672
> > Tue Jan 17 18:07:31 2017 mmbackup:Audit files /cs/mmbackup.audit.gpfs* 
> > contain 0 failed paths but there were 10012 failures.
> > Cannot reconcile shadow database.
> > Unable to compensate for all TSM errors in new shadow database.
> >   Preserving previous shadow database.
> >   Run next mmbackup with -q to synchronize shadow database.  exit 12
> >
> > If it helps, the mmbackup job is kicked off with the following options :
> >  /usr/lpp/mmfs/bin/mmbackup gpfs -n 8 -t full -B 20000 -L 1 
> > --tsm-servers gpfs_weekly_stanza -N glossop1a | /usr/bin/tee 
> > /var/log/mmbackup/gpfs_weekly/backup_log.`date +%Y%m%d_%H_%M`
> >
> > (The excerpts above are from the backup_log.<datestamp> file.)
> >
> > Our NSD servers are running GPFS 4.1.1-11, TSM is at 7.1.1.100 and the 
> > File system version is 12.06 (3.4.0.3). Has anyone else seen this 
> > behaviour with mmbackup and if so, found a fix?
> >
> > Thanks,
> >
> > Regards,
> > Ash
> >
> 
> 



-- 

Ed Wahl
Ohio Supercomputer Center
614-292-9302



More information about the gpfsug-discuss mailing list