[gpfsug-discuss] Online data migration tool

Buterbaugh, Kevin L Kevin.Buterbaugh at Vanderbilt.Edu
Wed Nov 29 16:51:51 GMT 2017


Hi All,

Well, actually a year ago we started the process of doing pretty much what Richard describes below … the exception being that we rsync’d data over to the new filesystem group by group.  It was no fun but it worked.  And now GPFS (and it will always be GPFS … it will never be Spectrum Scale) version 5 is coming and there are compelling reasons to want to do the same thing over again … despite the pain.

Having said all that, I think it would be interesting to have someone from IBM give an explanation of why Apple can migrate millions of devices to a new filesystem with 99.999999% of the users never even knowing they did it … but IBM can’t provide a way to migrate to a new filesystem “in place.”

And to be fair to IBM, they do ship AIX with root having a password and Apple doesn’t, so we all have our strengths and weaknesses!  ;-)

Kevin
—
Kevin Buterbaugh - Senior System Administrator
Vanderbilt University - Advanced Computing Center for Research and Education
Kevin.Buterbaugh at vanderbilt.edu<mailto:Kevin.Buterbaugh at vanderbilt.edu> - (615)875-9633

On Nov 29, 2017, at 10:39 AM, Sobey, Richard A <r.sobey at imperial.ac.uk<mailto:r.sobey at imperial.ac.uk>> wrote:

Could we utilise free capacity in the existing filesystem and empty NSDs, create a new FS and AFM migrate data in stages? Terribly long winded and frought with danger and peril... do not pass go... ah, answered my own question.

😊

Richard

-----Original Message-----
From: gpfsug-discuss-bounces at spectrumscale.org<mailto:gpfsug-discuss-bounces at spectrumscale.org> [mailto:gpfsug-discuss-bounces at spectrumscale.org] On Behalf Of Jonathan Buzzard
Sent: 29 November 2017 16:35
To: gpfsug main discussion list <gpfsug-discuss at spectrumscale.org<mailto:gpfsug-discuss at spectrumscale.org>>
Subject: Re: [gpfsug-discuss] Online data migration tool

On Wed, 2017-11-29 at 11:00 -0500, Yugendra Guvvala wrote:
Hi,

I am trying to understand the technical challenges to migrate to GPFS
5.0 from GPFS 4.3. We currently run GPFS 4.3 and i was all exited to
see 5.0 release and hear about some promising features available. But
not sure about complexity involved to migrate.


Oh that's simple. You copy all your data somewhere else (good luck if you happen to have a few hundred TB or maybe a PB or more) then reformat your files system with the new disk format then restore all your data to your shiny new file system.

Over the years there have been a number of these "reformats" to get all the new shiny features, which is the cause of the grumbles because it is not funny and most people don't have the disk space to just hold another copy of the data, and even if they did it is extremely disruptive.

JAB.

--
Jonathan A. Buzzard                         Tel: +44141-5483420
HPC System Administrator, ARCHIE-WeSt.
University of Strathclyde, John Anderson Building, Glasgow. G4 0NG

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20171129/ea366ea4/attachment.htm>


More information about the gpfsug-discuss mailing list