Back in early 2006 md/raid5 gained the ability to increase the number of devices in a RAID5, thus making more space available. As you can imagine, this is a slow process as every block of data (except possibly those in the first stripe) needs to be relocated. i.e they need to be read from one place and written to another. md/raid5 allows this reshaping to happen while the array is live. It temporarily blocks access to a few stripes at a time while those stripes a rearranged. So instead of the whole array being unavailable for several hours, little bits are unavailable for a fraction of a second each.
Then in early 2007 we gained the same functionality for RAID6. This was no more complex than RAID5, it just involved a little more code and testing.
Now, in mid 2009, we have most of the rest of the reshaping options that had been planned. These include changing the stripe size, changing the layout (i.e. where the parity blocks get stored) and reducing the number of devices.
Changing the layout provides valuable functionality as it is an important part of converting a RAID5 to a RAID6.
# How Level Changing Works
# Complexities of re-striping data
# Reducing the number of devices
# Currently supported reshapes
# Future Work
# Comments...
How Level Changing Works
If we think of "RAID5" as a little more generic than the standard definition, and allow it to be any layout which stripes data plus 1 parity block across a number of devices, then we can think of RAID4 as just a special case of RAID5. Then we can imagine a conversion from RAID0 to RAID5 as taking two steps. The first converts to RAID5 using the RAID4 layout with the parity disk as the last disk. This clearly doesn't require any data to be relocated so the change can be instant. It creates a degraded RAID5 in a RAID4 layout so it is not complete, but it is clearly a step in the right direction.I'm sure you can see what comes next. After converting the RAID0 to a degraded RAID5 with an unusual layout we would use the new change-the-layout functionality to convert to a real RAID5.
It is a very similar process that can now be used to convert a RAID5 to a RAID6. We first change the RAID5 to RAID6 with a non-standard layout that has the parity blocks distributed as normal, but the Q blocks all on the last device (a new device). So this is RAID6 using the RAID6 driver, but with a non-RAID6 layout. So we "simply" change the layout and the job is done.
A RAID6 can be converted to a RAID5 by the reverse process. First we change the layout to a layout that is almost RAID5 but with an extra Q disk. Then we convert to real RAID5 by forgetting about the Q disk.
Complexities of re-striping data
In all of this the messiest part is ensuring that the data survives a crash or other system shutdown. With the first reshape which just allowed increasing the number of devices, this was quite easy. For most of the time there is a gap in the devices between where data in the old layout in being read, and where data in the new layout is being written. This gap allows us to have two copies of that data. If we disable writes to a small section while it is being reshaped, then after a crash we know that the old layout still has good data, and simply re-layout the last few stripes from where-ever we recorded that we were up to.
This doesn't work for the first few stripes as they require writing the new layout over the old layout. So after a crash the old layout is probably corrupted and the new layout may be incomplete. So mdadm takes care to make a backup of those first few stripes and when it assembles an array that was still in the early phase of a reshape it first restores from the backup.
For a reshape that does not change the number of devices, such as changing chunksize or layout, every write will be over-writing the old layout of that same data so after a crash there will definitely be a range of blocks that we cannot know whether they are in the old layout or the new layout or a bit of both. So we need to always have a backup of the range of blocks that are currently being reshaped.
This is the most complex part of the new functionality in mdadm 3.1 (which is not released yet but can be found in the devel-3.1 branch for git://neil.brown.name/mdadm). mdadm monitors the reshape, setting an upper bound of how far it can progress at any time and making sure the area that it is allow to rearrange has writes disabled and has been backed-up.
This means that all the data is copied twice, once to the backup and once to the new layout on the array. This clearly means that such a reshape will go very slowly. But that is the price we have to pay for safety. It is like insurance. You might hate having to pay it, but you would hate it much more if you didn't and found that you needed it.
One way to avoid the extra cost of doing the backup is to increase the number of devices at the same time. e.g. if you had a 4-drive RAID5 you could convert it to a 6-drive RAID6 with a command like mdadm --grow /dev/md0 --level=6 --raid-disk=6 then it will combine the increase in the number of devices with the change to the layout and will copy every block only once (except for the backup it needs to do for the first few stripes).
Reducing the number of devices
The other change worth discussing is the ability to reduce the number of devices. This can be useful to reverse an increase in number of devices that was not intentional, to shrink an array to gain back a spare device, or as part of converting an array from several smaller devices to fewer large devices (thus saving power for example).
While increasing the number of devices or reshaping while leaving the number of devices the same always proceeds from the start of the devices towards the end, reducing the number of devices proceeds from the end of the devices to the beginning. This means that the critical section when a backup is needed happens right at the end of the reshape process, so mdadm runs in the background watching the array and does the backup just when it is needed.
Naturally decreasing the number of devices reduced the amount of available space in the array and the very first write-out to the new arrangement will destroy data that was at the end of the original array. Data that hopefully isn't wanted any more. However people sometimes make mistake and as reducing the number of devices is immediately destructive of data mdadm requires a little more care. In particular, before reducing the number of devices in a RAID5 or RAID6 you must first reduce the size of the array using the new --array-size= option to mdadm --grow. This truncates the array in a non-destructive way. You can check if the data that you care about is still accessible and then when you are sure, use mdadm --grow --raid-disks= to reduce the number of devices.
If you have replaced all the devices with larger devices, you can avoid the need to reduce the size of the array by increasing the component size at the same time as reducing the number of devices. e.g. on a 4-disk RAID5, mdadm --grow --size max --raid-disk 3 ... or at least you should be able to. The current mdadm pre-release don't get that right but hopefully it will before mdadm-3.1 is really released.
Currently supported reshapes
A bit of a summary of what is actually supported seems in order here.RAID1
A RAID1 can change the number of devices or change the size of individual devices. A 2 drive RAID1 can be converted to a 2 drive RAID5.RAID4
A RAID4 can change the number of devices or the size of individual devices. It cannot be converted to RAID5 yet (though that should be trivial to implement).RAID5
A RAID5 can change the number of devices, the size of the individual devices, the chunk size and the layout. A 2 drive RAID5 can be converted to RAID1, and a 3 or more drive RAID5 can be converted to RAID6.RAID6
A RAID6 can change the number of devices, the size of the individual devices, the chunk size and the layout. And RAID6 can be converted to RAID5 by first changing the layout to be similar to RAID5, then changing the level.LINEAR
A lINEAR array can have a device added to it which will simply increase its size.RAID0 and RAID10
These arrays cannot be reshaped at all at present.Future Work
While the work describe here greatly increases the options for reshaping an array there is still more to do (apart from just fixing the bugs in the current code).While conversion from RAID0 to RAID5 was used as an example above, it isn't actually implemented yet. md/RAID0 is broader than the regular definition of RAID0 in that the devices can be different sizes. If they are, then the array cannot be converted to RAID5 at all, so a general conversion is not possible.
It would still be good to support a simple conversion if the RAID0 does just use the same amount of space from each device. It would also be nice to teach RAID5 to handle arrays with devices of different sizes. There are some complications there as you could have a hot spare that can replace some devices but not all. There would also be the need to store the active sizes of all devices in the metadata - something RAID0 doesn't need as it doesn't need to be able to cope with missing devices.
If we did have support for variable drive sizes in a RAID5, then we could implement extending a RAID0 by converting it to a degraded RAID5, perform the conversion there using common code, then convert back to RAID0
It would also be nice to add some reshape options to RAID10. Currently you cannot change a RAID10 at all. The first step here would be enumerating exactly what conversions make sense with the various possible layouts (near/far/offset) and then finding how to implement them most easily. But that is a job for another days.
Comments...
Re: Converting RAID5 to RAID6 and other shape changing in md/raid (19 August 2009, 12:52 UTC) Wow. Impressive work. Thank you for all of your hard and careful work on this.
Minor nit, you have a typo of sunheading rather than subheading in the Raid5 supported operations section.
-ben
Re: Converting RAID5 to RAID6 and other shape changing in md/raid (19 August 2009, 13:31 UTC) Re: Converting RAID5 to RAID6 and other shape changing in md/raid (20 August 2009, 01:02 UTC) I have a question about this new feature.
mdadm --grow --size max --raid-disk 3
Here is what I am doing. I currently have a 6 Drive Raid 5 consisting of 5x1TB, and a 1TB LVM (2x320GB, and 1x500GB (OS included in extra space)). I just recently bought 3 more 1TB drives and was hoping to build a new Raid 5 using 256k chunks instead of 64k chunks. I wanted to gradually remove and shrink the original array freeing up drives and then move the drives over to the new array.
I am getting the error "cannot reduce number of data disks yet", and I am using version 2.6.7.1 in Ubuntu 9.04
Is this possible at all in any version of mdadm, or am I going to have to get creative?
Re: Converting RAID5 to RAID6 and other shape changing in md/raid (20 August 2009, 06:56 UTC) Yes it is possible with the new code. However it isn't quite released yet (Though you can pull it from
my git tree). You will need mdadm-3.1 and linux-2.6.31.
If you just want to change the chunk size, the new mdadm will do that for you
If you want to reduce the number of devices in an array you can. You will have to
shrink the filesystem first, then shrink the array with
and then change the number of devices.
Re: Converting RAID5 to RAID6 and other shape changing in md/raid (20 August 2009, 14:09 UTC) Thanks, I managed to find a way to backup all my data on my old array so I can kill it and just add the drives to the new array.
Hopefully next time I need to shrink it will be released and stable, awesome to see the quick feedback.
I trust mdadm software raid way more than any hardware raid.Allows me to build a relatively cheap system and give me all the protection I need. And it's worked awesome switching between 4 different motherboards now, always detects everything no problem, awesome.
And it's nice to be able to do everything live.
Keep it up, Thanks.
Re: Converting RAID5 to RAID6 and other shape changing in md/raid (21 August 2009, 17:35 UTC) Following your instructions, i upgraded to mdadm, but this command looks not to work
mdadm --grow /dev/md0 --level=6 --raid-disk=6
madam: /dev/md0: Cannot reshape array without increasing size (yet).
I wanted to convert a 5-disk RAID5 to 6-disk RAID6.
If i use this command below, there is another error:
mdadm --grow /dev/md0 --level=6 --raid-disk=6 --size=max
madam: can change at most one of size, raiddisks, bitmap, and layout.
Could you give some instructions? Thanks.
Re: Converting RAID5 to RAID6 and other shape changing in md/raid (22 September 2009, 22:08 UTC) Hi,
I'm considering reshaping an almost full (99%) 7-disk RAID5 array to a 9-disk RAID6 array, but the fact that mdadm-3.1 (which I understand is the only one capable of doing that) hasn't been released (even in -rc) yet has me a little worried...
Do you have any ETA for a release? Do you consider the code stable enough already? I know I should backup before the reshape, but it's not technically (and financially ;-P) feasible; that's why I use el-cheapo software RAID on el-cheapo consumer grade hardware... :)
Thanks for your hard work
Re: Converting RAID5 to RAID6 and other shape changing in md/raid (23 September 2009, 15:37 UTC) No, I don't really have an ETA for a release. Maybe next week, almost certainly before the
end of October. The only thing that I need to do before a release it to write a bunch of
test cases so I can increase my confidence in the code.
I think you should be safe running the code from the 'master' branch of my git tree.
However if anything does go wrong, I suggest that you don't try to fix it, but rather
stop touching the array and email all the details to linux-raid@vger.kernel.org (feel
free to Cc me explicitly). I will work with you to find out exactly what happened and
to fix it. But I really don't expect that to be needed.
Whether it works or not, I would really appreciate an email to linux-raid@vger.kernel.org detailing your experience.
Thanks.
Re: Converting RAID5 to RAID6 and other shape changing in md/raid (07 October 2009, 02:52 UTC) I was wondering how to reduce the number of devices on a RAID-5 without having to buy extra devices for the move... and you just implemented it!
On Linux 2.6.31.2 I performed a test with a RAID-5 of 4 LVM volumes, hosting an ext3 filesystem. After shrinking the filesystem, I reduce the array size with mdadm from the branch devel-3.1 then I reduce the number of devices to 3 and grow the filesystem. It works perfectly.
Thank you and congratulations for all your work.
Re: Converting RAID5 to RAID6 and other shape changing in md/raid (18 October 2009, 20:11 UTC) Dear Neil,
I love this mdadm-stuff, as it adding a huge benefit for linux (and of course for myself). You are talking about future-capabilities, which I would love to see: "If we did have support for variable drive sizes in a RAID5"
My server has actually a raid6 of 6 active 500GB-drives and 1 spare. I would like to add another 1.5TB-drive, but I do not like to split this new drive into 3 * 500GB, as then failing this single 1.5TB drive would render my raid unusable. I have seen ideas of striping e.g. the existing 500GB-disks to a 1-TB-disk, so I could assemble my raid6 with a new 1 *TB native and e.g. 3 * striped 500GB (=3 * 1-TB-disk). However this would result into a be degraded raid6 as well, unless I add 3 *1TB to this raid.
So having the capability of mixing different sizes in a raid6 would be genious. But I guess there would be limitations, as of the size for the spare and I am not sure what would happen if the largest-drive would fail.
However, even with todays flexibility, this mdadm is an outstanding useful tool! congrats!
christian
Re: Converting RAID5 to RAID6 and other shape changing in md/raid (18 October 2009, 23:19 UTC) christian: You can stack the MD levels... So make the 6 500G disks into two 3-disk RAID-0s. Then you have 3 1.5T sized devices (the two RAID-0s and the 1.5T partition) and can build your final RAID-5 on top of that.
Ugly but doable.
-ben
Re: Converting RAID5 to RAID6 and other shape changing in md/raid (19 October 2009, 04:57 UTC) thats what I meant with my (afterwards suboptimal) statement: 3 * striped 500GB. So many thanks, as you suggested what I already thought. I feel now safer going down this road of stacking md-levels. Thanks alot for your useful help
christian
Re: Converting RAID5 to RAID6 and other shape changing in md/raid (21 October 2009, 14:02 UTC) git branch --track devel-3.1 origin/devel-3.1
git checkout -f devel-3.1
make
sudo make install
mdadm - v3.0.2 - 25th September 2009
Re: Converting RAID5 to RAID6 and other shape changing in md/raid (25 December 2009, 00:12 UTC) Firstly -I am impressed with the new RAID6 and reshape features added recently - respect!
After 1-2 days I noted the rate of progress dropped very significantly, and "/proc/mdstat" now reports anything from 5000kbs to 200kbs, mostly around 500kbs. Everything was running OK but just miserably slow. After 4 days its now 71.2% complete.
I have just shutdown and restarted the machine and onreboot it has restarted the reshape and is currently running at a "proper" speed again at between 8000 and 10000 kbs (8-10MB/s). However, it may be after a few hours the rate drops back to 500kbs again?
Is this normal? What have I misconfigured?
Re: Converting RAID5 to RAID6 and other shape changing in md/raid (04 January 2010, 05:04 UTC) Loving the software, have just kicked off a raid 5 to 6 conversion from 6 to 7 drives. It is running at 2384K/sec averagely dunno if that is normal? Used my mirrored raid 1 drive as the backup.
Good job!
(PS it'd be really, really handy if your posts/messages had a date on them?)
Re: Converting RAID5 to RAID6 and other shape changing in md/raid (04 January 2010, 08:44 UTC) Yes, when you do an in-place raid conversion (array size doesn't change) it will always be very slow as all data has to be written twice and there isn't much opportunity for streaming.
And thanks for the suggestion, there should be dates everywhere now.
Re: Converting RAID5 to RAID6 and other shape changing in md/raid (06 January 2010, 07:25 UTC) I was wondering if I could pick your brain and get opinions on my upgrade plan. I am currently running a 4x500GB RAID5 array that is about 80% full now. I've decided to upgrade to 5x2TB drives in a RAID6 configuration for extra redundancy. One thing that is a bit of a problem is that my machine can only support 5 drives, so I can't just attach the new drives and copy the data. I've come up with two options to do this upgrade:
1. Add one 2TB drive as a stand-alone, copy all data to the 2TB drive, replace the 4 old drives, create 4x2TB RAID6 array, copy data over from stand-alone drive, and finally expand the array to include the fifth drive.
2. Add 2TB drive, make current RAID5 array a RAID6 array including that drive, then swap out the 4 500GB drives one at a time as if they had each died.
I'm leading towards option 1 as I think that would be quicker, the longest part being the expanding of the array from 4 drives to 5 at the end, but the copying to and from the 2TB drive will probably just take several hours each way. It also seems fairly safe because I'll have the 4 500GB unaltered if anything should go wrong.
Thanks for any input, and thanks for all your hard work on this project.
Keith
Re: Converting RAID5 to RAID6 and other shape changing in md/raid (07 January 2010, 06:34 UTC) Option 1 is definitely better if you are happy with the extended down-time that it requires. It will very likely be faster as the RAID5 -> RAID6 conversion is very slow.
As you say, you will have a complete, valid copy of the data on the smaller drives which is good insurance.
So go with option 1.
NeilBrown
Re: Converting RAID5 to RAID6 and other shape changing in md/raid (08 January 2010, 04:33 UTC) Keith
Re: Converting RAID5 to RAID6 and other shape changing in md/raid (09 January 2010, 17:43 UTC) Neil was wondering if I could get your advice. I'm currently running at 4x1TB Raid5 and I would like to change that to a 6x1TB Raid6. I guess confused on the process. So essentially i can just use the mdadm --add /dev/mdX /dev/sdX and add the 2 drives making it 6x1TB. Then mdadm --grow /dev/md0 --level=6 --raid-disk=6 will correctly reshape my raid5 to a raid6 then i'd just have to xfs_growfs /dev/mdX afterwards? Just wondering if my thought process is correct here. Thanks in advance this is some great progress.
Re: Converting RAID5 to RAID6 and other shape changing in md/raid (10 January 2010, 05:11 UTC) Hi all, I'm converting a 7 disk RAID-5 array to an 8 disk RAID-6 array and wanted to share. Unlike the previous poster, I am not increasing the size of the RAID device, I am simply increasing redundancy.
First of all I made sure I was running mdadm 3.1 or above, which my distro (gentoo) did not have by default. So I installed mdadm 3.1.1, which I believe is current as of the time of this writing.
To add the 8th disk to my existing RAID-5 array, I ran:
mdadm --add /dev/md3 /dev/sdh3
this added /dev/sdh3 as a hot spare. Then, to convert this to a RAID-6 array, I ran:
mdadm --grow /dev/md3 --level=6 --raid-devices=8 --backup-file=/nfs/media/tmp/md3.backup
Notice the argument is "--raid-devices", not "--raid-disk" as in Neil's post.
I had tried to run the --grow command without the --backup-file argument, as Neil's post seems to say that a backup file is not necessary when a hot spare is present. But mdadm wasn't having it, it told me:
mdadm level of /dev/md3 changed to raid6
mdadm: /dev/md3: Cannot grow - need backup-file
mdadm: aborting level change
With the --backup-file argument everything seems to be working fine. Here's the relevant part of my /proc/mdstat:
md3 : active raid6 sdh3[7] sdg3[6] sdf3[5] sde3[4] sda3[0] sdb3[2] sdc3[3] sdd3[1]
120052224 blocks super 0.91 level 6, 256k chunk, algorithm 18 [8/7] [uuuuuuu_]
[=>...................] reshape = 6.3% (1269760/20008704) finish=151.2min speed=2064K/sec
My next step is to convert my 4 terabyte /dev/md5 to a RAID-6 array. Neil wasn't kidding when he said the reshape is a slow process... at the rate that /dev/md3 is converting, I estimate that it will take 4.5 days to convert my /dev/md5 to RAID-6.
Re: Converting RAID5 to RAID6 and other shape changing in md/raid (12 January 2010, 23:49 UTC) mdadm 3.1.1
kernel 2.6.32.10
/dev/md1 is a 3 disk raid5 64k chunk size.
/mnt/tmp is a spare hard drive.
Re: Converting RAID5 to RAID6 and other shape changing in md/raid (25 January 2010, 09:47 UTC) Re: Converting RAID5 to RAID6 and other shape changing in md/raid (12 February 2010, 22:16 UTC) I have just started a conversion of a 6 disk 2TB RAID5 to a 8 disk 2TB RAID6 on my system.
The performance during conversion was terrible - 1MB/second. This got me suspicious, as I know the array could perform much better than that.
Issuing "echo 200000 > /sys/block/<mddevice>/md/sync_speed_min" fixed it, as it now reshapes at 20MB/second.
This ofcourse blocks everything else, but issuing an "echo 10000" gives the a fair tradeoff, as the "when idle" system obviously seems to have a problem.
So try this, instead of rebooting.
Comment (22 February 2010, 12:03 UTC) first of all I want to thank Neil and all other developers of mdadm for their great work. I have been using mdadm for several years now and my arrays have survived two disk-faults flawlessly - from the notification-email to rebuilding the array everything went as it should.
I have a question regarding my raid6-array, consisting of 7 1TB-disks providing a single lvm-physical volume. I plan to expand that array with a 1.5TB-disk and want to make sure that I got everything written here and on the mailing list right:
- If I grow that array after adding the new 1.5TB-disk to it, only 1TB of the new disk will be used and the array-size will increase by 1TB. (Do I have to issue some special commands when growing or will mdadm automatically just use 1TB of the 1.5TB-disk?)
- If I resize the lvm-physical volume the array provides, the 1TB will become usable by lvm.
Is that correct?
Then I got another question: To grow a raid6-array, I need mdadm past version 3.x. Currently, the array is on a debian-server running a year or so without reboot, therefore it runs on mdadm-2.6.x. I recently updated the mdadm-package to the current version 3.1.1 using aptitude.
Is a reboot necessary for the grow-feature to become available or does it suffice to just stop and start the array (or is neither of these actions necessary)?
Thank you all for your help!
rman
Re: Converting RAID5 to RAID6 and other shape changing in md/raid (22 February 2010, 11:03 UTC) - yes, only 1TB of the new device will be used. md doesn't need to be told any special for this to happen.
- similarly, lvm will only be able to see 1TB of extra data.
Providing your kernel is v2.6.21 or more recent you should not need to reboot. If your kernel is older than
that, you will need a new kernel.
Comment (23 February 2010, 03:34 UTC) md11 : active raid6 sdh[9](S) sdc[0] sdi[8] sdf[7] sde[4] sdk[3] sdj[2] sdd[1]
4883811840 blocks super 1.2 level 6, 512k chunk, algorithm 2 [7/7] [uuuuuuu]
If i now issue 'mdadm /dev/md11 --grow --raid-devices=8 --backup-file=/root/backup_md11.bak' I get
mdadm: this change will reduce the size of the array.
use --grow --array-size first to truncate array.
e.g. mdadm --grow /dev/md11 --array-size 1565606912
Why would adding a disk REDUCE the size? Am I missing something here? If I add the '--size=max' switch to the command line, i.e.
'mdadm --grow /dev/md11 --raid-devices=8 --level=6 --backup-file=/root/backup_md11.bak --size=max', I get
mdadm: cannot change component size at the same time as other changes.
Change size first, then check data is intact before making other changes.
I really don't understand this as the new disk is exactly the same model as the other ones. 'hdparm -I' on the new disk gives me a device size which is identical to the other disks. Has anyone a clue what is going on? Has it something to do with the non-standard chunk-size?
Thanks for helping,
rman
Re: Converting RAID5 to RAID6 and other shape changing in md/raid (23 February 2010, 07:56 UTC) You have found a bug! It has been fixed for a while, but I haven't released an updated version yet.
The two patches to fix it are:
http://neil.brown.name/git?p=mdadm;a=commitdiff;h=2ed4f75388f99968be58097941a9704f6e42d701
and
http://neil.brown.name/git?p=mdadm;a=commitdiff;h=f98841b3852ceb7fce56a6f818236a4af9b5a00a
If you have 'git' installed, you might do best to:
git clone git://neil.brown.name/mdadm mdadm
cd mdadm
make install
and try that.
Re: Converting RAID5 to RAID6 and other shape changing in md/raid (06 March 2010, 08:41 UTC) has changing the chunk size been fixed?
i have a 3 disk raid5 that has 64k chunks and i want to go to 256k chunks.
trying to do this crashed the array and forced a resync
Re: Changing chunk size (06 March 2010, 09:41 UTC) As far as I know, changing chunksize works OK.
There might be some issues when going from a smaller to a larger chunksize if the size of the components is not a multiple of the larger chunk size - mdadm doesn't seem to check that properly at the moment, but the failure mode is to do nothing, not to crash the array.
If you have specific details - kernel log error messages, "mdadm --detail" details of the array, version of kernel and mdadm, and anything else that might help me reproduce your problem, please post them to linux-raid@vger.kernel.org (you don't have to subscribe to post).
I just successfully reshaped a small scratch 3-drive-raid5 from 64K chunks to 256K chunks using mdadm-3.1.1 and linux-2.6.32.
Re: Converting RAID5 to RAID6 and other shape changing in md/raid (14 March 2010, 01:15 UTC) Neil,
I'm trying to do a procedure you describe in the examples, converting a 4 disk raid5 to a 6 disk raid6 volume.
I have added the 2 new disks to the raid5 array (sd[ef]4):
md0 : active raid5 sdf4[4](S) sde4[5](S) sdd1[1] sdb1[3] sdc1[2] sda1[0]
937705728 blocks level 5, 128k chunk, algorithm 2 [4/4] [uuuu]
bitmap: 1/150 pages [4kb], 1024KB chunk
However, when I run the grow command it aborts:
# mdadm --grow /dev/md0 --level=6 --raid-devices=6 --backup-file=/root/raid-backup
mdadm level of /dev/md0 changed to raid6
mdadm: Need to backup 1536K of critical section..
mdadm: Cannot set device shape for /dev/md0
mdadm: aborting level change
I'm using the following versions of mdadm and Gentoo kernel:
mdadm - v3.1.1 - 19th November 2009
2.6.31-hardened-r11
Dave
Re: Converting RAID5 to RAID6 and other shape changing in md/raid (14 March 2010, 07:38 UTC) Hi Dave,
The reason it doesn't work for you is that you have a write-intent bitmap. It is not currently
possible to reshape an array with a bitmap as resizing the bitmap is not yet implemented.
You need to remove the bitmap (mdadm -G /dev/md0 --bitmap none), then reshape and wait for it
to complete, then re-add the bitmap (mdadm -G /dev/md0 --bitmap internal).
In think mdadm 3.1.2 (only just released) gives a more useful error message in this circumstance.
Re: Converting RAID5 to RAID6 and other shape changing in md/raid (14 March 2010, 12:54 UTC) You rule Neil! Thanks for the quick response. I'm almost half done the reshape. Looks like it's going to end up taking about 10 hours total. Seriously, mdadm is magic. I'm glad I didn't waste money on a hardware controller. I feel a lot safer using linux raid.
Thanks again!
Query about integrating non-hotswap 'spare' drives into a hot-swap array (18 March 2010, 09:46 UTC) I'm planning out a 5-20 drive RAID array (to be grown as needed by adding more drives) and have been considering ways to integrate a 'spare' drive in one or more of the non-hotswap bays available in the chassis. Would it be reasonable to do the following?
Theory behind this is that it allows the rebuild to begin immediately, without requiring to wait for the rebuild to finish before the failed drive can be hot-swapped. Once the hot-swap drive is replaced and the rebuild finished + mirrored back in, the 'cold swap' drive can be pulled back out of the appropriate RAID-1.
Or is there a more efficient way to implement this idea of allowing for cold-swap versus hot-swap status for drives in a RAID array? Or would the RAID-1 coming back up in this manner cause problems with the RAID-5/RAID-6 wrapped around it?
Re: Converting RAID5 to RAID6 and other shape changing in md/raid (28 March 2010, 11:31 UTC) Hi,
I use mdadm 3.1.1 with ubuntu 10.04 2.6.32-17 kernel I want to shrink a 6 drive raid 5 to a 5 drive raid 5
via: $ sudo mdadm --grow /dev/md0 --raid-devices=5 --backup-file /mdadmbackup
cmdline shows me following output:
mdadm: Need to backup 1280K of critical section..
$
and nothing happen even under cat /proc/mdstat shows me just the normal array and nothing more no drive interaction
What is happen?
Will there be a algorithm which create Raid - Sys without Write-hole like zfs? If yes when?
Thanks TT
Re: Thanks for all the amazing work (23 April 2010, 17:17 UTC) Hi
As a contributor to several open source projects myself, I know how rarely you hear "thanks". So:
Thanks very, very much for all you've done on md/mdadm. It's a brilliantly reliable, solid and easy to use system that I strongly prefer to fakeraid/hostraid setups and almost always favor over true hardware RAID as well. It's a sysadmin's dream.
About the only time I ever use anything else is when I'm not using a Linux server or I need write-back caching, in which case hardware RAID is currently a necessity to get persistent write cache.
I was recently astonished at the ease with which mdadm reshaped and grew an existing RAID 6 array from 8 to 10 disks. One command and it's merrily chugging away without the rest of the system even noticing. Thanks to the status-watching cron job it even emailed me when it was done two days later!
I think people take the md subsystem for granted, and that's a shame as it's one of the best things around on Linux systems.
(soapyfrogs.blogspot.com has contact info).
Comment (05 May 2010, 14:09 UTC) We strongly prefer mdadm to every other RAID system, software and hardware, and have chosen it hands-down, every time for years.
mdadm is of aspiration quality. Thank you for creating this, touching millions if not billions of lives, truly improving humanity, and making something deserving of utmost pride and admiration.
Best regards from outside of Washington, DC,
5 May 2010
Michael Brenden
RAID10 reshaping (04 August 2010, 13:48 UTC) Hi, here are some reshaping options involving RAID10 that I would really like to have, and my understanding of how they would work:
- 2-drive RAID1 to 2-drive near-RAID10 and back; should be trivial
- 2-drive RAID0 to 4-drive near-RAID10 degraded to 2 drives, and back; should be trivial too
- changing the number of devices, especially 2->4; that's more difficult, but it's already done for RAID5 and 6 and I think it's not very different
- changing the RAID10 layout, probably not harder than the above
- changing the size of devices for near layout; relatively easy I guess
aditsu
Re: Converting RAID5 to RAID6 and other shape changing in md/raid (12 May 2010, 16:02 UTC) Just wanted to say thanks for making such a great product! I just converted a 3-drive RAID-5 array to a 4-drive RAID-6 array, with exactly one simple command. The array is the root of my Linux system, and it didn't even hiccup as the conversion happened.
Really nice work.
Thanks,
Mason
Re: Converting RAID5 to RAID6 and other shape changing in md/raid (18 May 2010, 12:01 UTC) I'm in a position where I have a 6 drive (1 failed, 5 active and working with the array in a degraded state) RAID6 array that I'd like to shrink/grow to a 5 drive RAID5 array.
The logic is that I've had a bad run with 1.5tb sized drives, so I'm more itnerested in just making do with the 5 drives / 6tb worth of useable storage I have now - and limp along until I can put together a new array from 2tb drives to take it's place. It also doesn't help that my current array is reiserfs, and I'd really feel a lot more comfortable with ext or xfs.
So I admit I'm having problems with the directions so far, insofar as shrink/growing the array from 6x RAID6 to 5x RAID5.
Suggestions or assistance?
Re: Converting RAID5 to RAID6 and other shape changing in md/raid (18 May 2010, 12:12 UTC) It is easier to answer these sort of questions when posted to linux-raid@vger.kernel.org.
However you just need mdadm-3.1.1 or newer, linux kernel 2.6.32 or newer and then
mdadm --grow /dev/mdXX --level raid5 --backup /some/file/not/on/the/array
It will take a while rearranging all the data, but when it is done you should have a RAID5, and while it is going you should still have access to your data.
Re: Converting RAID5 to RAID6 and other shape changing in md/raid (01 June 2010, 02:55 UTC) Fantastic utility. I recently had to re-shape (twice), re-size and even re-layout an existing and poorly thought out 5 1TB Disk RAID4 array. As I couldn't use my running kernel or installed mdadm (too low a version for both) I boot to System Rescue CD 1.5.4, which had the requisite kernel version and upper mdadm version, http://www.sysresccd.org/ and was able to do everything. It took 48 hours for each operation (3 separate operations), but in the end I had a left symmetric, 6 disk RAID6 array. The production OS was able to assemble the device and mount the volume without incident!!
Thank you for your hard work and attention to detail.
Christian
Re: Converting RAID5 to RAID6 and other shape changing in md/raid (11 June 2010, 02:08 UTC) Excellent utility. I am using it now for my home network share. Here is the problem I am have (or maybe not having). Last night, I added 4 drives to my 4 drive RAID5 array. At the same time, I changed to RAID6. The command I used to do this was (if I remember right):
mdadm --grow /dev/md0 --level=6 --raid-devices=8 --backup-file=/home/tbmorris/RAID.bak
When I did it, I do not remember seeing any errors. I then ran:
mdadm --detail /dev/md0
What I saw did not make me happy. Drives /dev/sd[bcdeg] were active sync, while /dev/sdf said spare rebuiling and /dev/sd[hi] said fauly spare rebuilding. Now while I was cursing that I just dumped 1.2Tb of movies and files, my wife was still sitting on the couch watching those very same movies that I thought I had destroyed. This morning when I checked up on the array, it said the same. My question is.... Is this normal? Is this merely what it looks like when you add 4 drives and change to RAID6 at the same time?
Thanks,
Terry
Re: Converting RAID5 to RAID6 and other shape changing in md/raid (11 June 2010, 14:25 UTC) Hi Terry,
it is hard to be sure without seeing concrete details (e.g. mdadm --detail output, or cat /proc/mdstat) but I would guess that do of the new drives suffered some sort of error while new data was being written to them. So your RAID6 now is doubly-degraded. i.e. the data is safe, but another disk failure will cause loss of data.
You should wait until the reshape completes (if it hasn't already) then add two known-good drives. If you think the too failed drives are actually good, you should test them well before adding them to the array. Maybe look for error messages in the kernel logs to see what so of error occurred on them.
Re: Converting RAID5 to RAID6 and other shape changing in md/raid (11 June 2010, 16:33 UTC) Here are the stats:
/dev/md0:
Version : 0.91
Creation Time : Mon May 30 03:29:47 2005
Raid Level : raid6
Array Size : 1462862592 (1395.09 GiB 1497.97 GB)
Used Dev Size : 487620864 (465.03 GiB 499.32 GB)
Raid Devices : 8
Total Devices : 8
Preferred Minor : 0
Persistence : Superblock is persistent
Update Time : Thu Jun 10 19:15:18 2010
State : clean, FAILED
Active Devices : 5
Working Devices : 6
Failed Devices : 2
Spare Devices : 1
Layout : left-symmetric-6
Chunk Size : 64K
Delta Devices : 3, (5->8)
New Layout : left-symmetric
UUID : e2ebff64:c0d9f90e:4bd6122a:78a10948
Events : 0.34221
Number Major Minor RaidDevice State
0 8 83 0 active sync /dev/sdf3
1 8 99 1 active sync /dev/sdg3
2 8 115 2 active sync /dev/sdh3
3 8 131 3 active sync /dev/sdi3
4 8 64 4 spare rebuilding /dev/sde
5 8 48 5 active sync /dev/sdd
8 8 32 6 faulty spare rebuilding /dev/sdc
9 8 16 7 faulty spare rebuilding /dev/sdb
and /proc/mdstat
Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10]
md0 : active raid6 sde[4] sdd[5] sdc[8](F) sdb[9](F) sdg3[1] sdi3[3] sdf3[0] sdh3[2]
1462862592 blocks super 0.91 level 6, 64k chunk, algorithm 18 [8/5] [UUUU_U__]
To me it means nothing. Seems like it says it's good, but it's bad. Kind of like a pint of ice cream.
Re: Converting RAID5 to RAID6 and other shape changing in md/raid (11 June 2010, 16:43 UTC) Hmmm... that looks rather messed up - something is definitely wrong.
The fact that the reshape has stopped in the middle looks bad, but maybe not too bad.
I need to see the output of "mdadm -E" on all 8 devices.
Rather than try to put it in a comment (Which will mess up the formatting) can you post it to me at neilb at suse.de.
Thanks.
Re: Converting RAID5 to RAID6 and other shape changing in md/raid (26 July 2010, 12:24 UTC) # mdadm /dev/md2 --grow --size=max --raid-devices=4
mdadm: cannot change component size at the same time as other changes.
Change size first, then check data is intact before making other changes.
Linux 2.6.34 x86_64
mdadm - v3.1.2 - 10th March 2010
Is it possible to do what I'm trying to do? If so, what do I need to do?
Re: Converting RAID5 to RAID6 and other shape changing in md/raid (26 July 2010, 13:00 UTC) Yes it is possible, but you have to go through a slightly round-about route. I have made a note to look at fixing it for a future release.
You need to first
mdadm --grow /dev/md2 --size=max
and then try
mdadm --grow /dev/md2 --raid-devices=4
That won't work but will tell you to use "--grow --array-size=" to reduce the array size. It will tell you
what size is needed. So
mdadm --grow /dev/md0 --array-size=whatever
mdadm --grow /dev/md2 --raid-devices=4
That should do what you want.
Re: Converting RAID5 to RAID6 and other shape changing in md/raid (26 July 2010, 13:10 UTC) Unfortunately, that sequence did not work:
# mdadm --grow /dev/md2 --size=max
mdadm: component size of /dev/md2 has been set to 976759808K
IMO, going from 1T to 2T should result in a component size double that value. There are 4x 2T devices in the array now and 2x 1T devices. Isn't max the max common to all devices, and thus 1T?
The 2T devices are partitioned as follows:
# fdisk -l /dev/sdb
Disk /dev/sdb: 2000.4 GB, 2000398934016 bytes
255 heads, 63 sectors/track, 243201 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x2051f33f
Device Boot Start End Blocks Id System
/dev/sdb1 1 243201 1953512001 fd Linux raid autodetect
# mdadm --grow /dev/md2 --raid-devices=4
mdadm: this change will reduce the size of the array.
use --grow --array-size first to truncate array.
e.g. mdadm --grow /dev/md2 --array-size 2930279424
This is telling me to shrink the array to 3T which I cannot do since the array is nearly full (5T of data).
Is it necessary to fail the 1T devices first? If so, I can only reduce by 1 then. Yes?
Re: Converting RAID5 to RAID6 and other shape changing in md/raid (26 July 2010, 13:40 UTC) 976759808K == 976759808 * 1024 == 1000202043392 == 1TB
So it did what you expected.
But that is a small matter.
I see your real problem though.. If you have 4 2TB devices and 2 1TB devices, md will only acknowledge
1TB of each device and you cannot grow to use more of fewer devices.
If you had 1 more 2TB device that you could swap for a 1TB device, then you could degraded the array so
that it all lives on 2TB devices, then you could reshape. But I wouldn't recommend that - you should never
discard your redundancy by choice - the risk is too high.
If you still have the 4 1TB devices that you replace, then you can use RAID0 to combine them into 2TB devices,
and so make the whole array consist of 2TB devices (4 real, 2 RAID0). Then you should be able to reshape nicely.
It would be good to make md/mdadm handle this better - it isn't entirely straight-forward though. I'll have to give it some thought.
Re: Converting RAID5 to RAID6 and other shape changing in md/raid (31 July 2010, 18:15 UTC) I'm using ubuntu and don't have the latest mdadm. Ubuntu uses v2.6.7.1
I want to reshape a RAID5 (3x 1TiB) and change the chunk size. I have read that increasing chunk size can greatly improve performance on a large filesystem with large files. Most (90%+) of my files are over 1GiB. Would there be any perceivable improvement?
Assuming there would be an improvement, what are the risks involved with installing a newer version of mdadm while it's running? Could I make a livecd with the latest version, reshape from that, then continue using 2.6.7.1 once it's reshaped??
I plan to add another drive in the near future, so I'll be reshaping then and may change chunk size at that point and kill 2 birds... Again I need to fully understand the implications of upgrading mdadm vs putting it on a live cd.
Re: Converting RAID5 to RAID6 and other shape changing in md/raid (02 August 2010, 10:15 UTC) Performance is very dependent on hardware and load characteristics. A larger chunk size does seem to improve performance for sequential reads, but I cannot promise it will in your circumstances. If you are going to add a device, then changing the chunksize at the same time is little extra cost so you may as well.
You don't need to install a new mdadm to make use of it. Just compile it somewhere and run ./mdadm --args to
run it. You can leave the installed on unchanged. If you have to shutdown or crash while the reshape is happening, then the installed mdadm won't be able to restart the array, so if the array holds '/', then you hit problems. But if the array is separate, and the mdadm you compiled isn't on the the array, then it is easy to restart the array with the new mdadm.
You do however need a new kernel - 2.6.32 at least. If your install has an old mdadm it is unlikely to have a new kernel. So the need for a new kernel might be enough to justify a livecd approach.
- NeilBrown
Re: Converting RAID5 to RAID6 and other shape changing in md/raid (02 August 2010, 23:51 UTC) I've searched https://raid.wiki.kernel.org/ but couldn't find an answer to this problem.
I have a question about converting RAID1 to RAID5. At the moment I have 2*500GB as RAID1 (so 500GB usable), and I want to change it to 3*500GB as RAID5 (so 1000GB usable). I see two ways of achieving this:
b- create a degraded 3 partition RAID5 (ie. 2 partitions present, and one missing) using the "failed" one and the new one
c- copy the data from the remaining RAID1 partition
d- delete the RAID1 array
e- add the remaining partition to the RAID5 and let it resync
b- grow the RAID5 by adding the new partition
Way 1 would obviously lead to data loss if a drive dies. Does way 2 suffer the same problem? If not I'll obviously take that route. But if way2 also suffers the problem, which route would you recommend?
Thanks very much!
Steffen
Re: Converting RAID5 to RAID6 and other shape changing in md/raid (03 August 2010, 08:46 UTC) If you just convert straight to RAID5 with
mdadm /dev/md0 -a /dev/newdev
mdadm --grow /dev/md0 --level=5 --raid-devices=3
then a problem with 'newdev' will mean that you lose redundancy but do not lose your data (until another device failed). A problem with one of the original devices would be harder to recovery from... I'm not sure off hand exactly what would happen, it might survive but as I haven't tested it I cannot be sure.
If you want to be extra safe, I would do the following, assuming the old drives are /dev/A and /dev/B and the new drive is /dev/C, and the array is /dev/md0.
mdadm -S /dev/md0
mdadm -C /dev/md0 -l5 -n2 /dev/A missing -x1 /dev/C
wait for resync
mdadm -G /dev/md0 -n 3
wait for reshape to complete.
Now you have all your data still safe on /dev/A, and assuming nothing has gone wrong, all your data is also
safe of /dev/md0 - though the array is degraded. Further you have run a resync and a reshape on /dev/B and /dev/C so you should have some confidence in them.
Now
mdadm /dev/md0 -a /dev/A
This will recover /dev/A which should be safe because we believe /dev/B and /dev/C are healthy having exercised them heavily. If /dev/A has a problem, it will fall out of the array and your data will still be safe.
Though on reflection, I suspect the first option would probably handle errors well.... maybe I should add that to my testing.
Re: Converting RAID5 to RAID6 and other shape changing in md/raid (03 August 2010, 13:46 UTC) Neil,
Thanks so much for the advice. I decided to go ahead and reshape for chunk size for the experience, so when i've got a raid-full of important data i know what to do about adding a drive. My computer locked up on me around 6% but i was able to recover using:
/[pathto]/mdadm -S /md0
/[pathto]/mdadm --assemble /md0 /dev/sdb /dev/sdc /dev/sdd --backup-file [path_to_backup_file]
Ubuntu tried to reassemble on boot, but failed. After manually assembling and mounting everything works as normal.
Another thing that hung me up... your guide said to use --chunk-size but the parameter is actually --chunk.
Other than those few hiccups it's going smoothly. I'm thoroughly impressed that i can still use md0 as it rebuilds. I'm able to play music and video from the raid while encoding a video to the raid with no perceptible slowdown. The only way i know it's slowing down is from the readout from /proc/mdstat.
Last thing (for now)... is there any reason to keep the packaged version from ubuntu if i've got the latest from source?
Thanks again!
justinmteal
Re: Converting RAID5 to RAID6 and other shape changing in md/raid (03 August 2010, 14:26 UTC) Locking up is a bit of a worry, but if it was locked-up in a way the "mdadm -S" still worked it cannot have been locked too hard. I wonder what the cause was..
While I try to keep new version of mdadm mostly backwards-compatible with old versions that isn't always possible. It is possible - though maybe not likely - that Ubuntu init.d scripts depend on some behaviour of mdadm that has changed between the release that they ship and the current release. So it is always good to be cautious.
It should be fairly safe to simply install the mdadm that you have build and see what breaks. If nothing: you are happy. If something does break, you should still be able to "apt-get install --reinstall mdadm" (or whatever the command it) to get the ubuntu mdadm back in place.
NeilBrown
Re: Converting RAID5 to RAID6 and other shape changing in md/raid (03 August 2010, 23:11 UTC) I am trying to shrink a raid 5 and reduce the nr of disks.
I am using mdadm v3.1.2 and kernel 2.6.32.12.
I have successfully shrunk the the raid using " mdadm --grow --array-size=2600000000 /dev/md0" and the file system seems ok after this.
I try to run "mdadm --grow --raid-disks=4 /dev/md0 /mdadm.backupfile" to reduce the nr of disks but I get "mdadm: --add cannot be used with other geometry changes in --grow mode" as a response. I have done this before a couple of times, but dont recall getting this message. I dont understand it, can someone tell me what is meant and what I can do about it?
This output might be useful:
[root@cerberos/]# mdadm --detail /dev/md0
/dev/md0:
Version : 0.90
Creation Time : Tue Sep 8 17:28:29 2009
Raid Level : raid5
Array Size : 2600000000 (2479.55 GiB 2662.40 GB)
Used Dev Size : 976762496 (931.51 GiB 1000.20 GB)
Raid Devices : 6
Total Devices : 6
Preferred Minor : 0
Persistence : Superblock is persistent
Update Time : Tue Aug 3 14:57:20 2010
State : clean
Active Devices : 6
Working Devices : 6
Failed Devices : 0
Spare Devices : 0
Layout : left-symmetric
Chunk Size : 128K
UUID : 7983e71a:77a9a523:89cd91be:8d911c05
Events : 0.793229
Number Major Minor RaidDevice State
0 8 0 0 active sync /dev/sda
1 8 48 1 active sync /dev/sdd
2 8 16 2 active sync /dev/sdb
3 8 32 3 active sync /dev/sdc
4 8 80 4 active sync /dev/sdf
5 8 96 5 active sync /dev/sdg
Thanks in advanced (and a big THANKS for the great mdadm that lets me do everything I can think about to my arrays without ever causing me problems).
Re: Converting RAID5 to RAID6 and other shape changing in md/raid (04 August 2010, 05:14 UTC) Thanks for you quick reply, I'll try it now. After that I got a 4th partition to add to 3*1TB RAID5 and once that's all done a badblocks check of 2 big drives... so when all this is finished I'll post it into the wiki I indicated in my earlier post.
Once again, thank you and everyone else who worked on this so much for your work!
Steffen
Re: Converting RAID5 to RAID6 and other shape changing in md/raid (04 August 2010, 09:31 UTC) I think you have a simple error in your usage.
You used
mdadm --grow --raid-disks=4 /dev/md0 /mdadm.backupfile
but should use
mdadm --grow --raid-disks=4 /dev/md0 --backup-file /mdadm.backupfile
I've fixed the sunheading - thanks.
mdadm --grow /dev/mdX --chunk-size=128
mdadm --grow --array-size
Neil,
Dear Ben,
So I guess I'm the newbie here, because I can't seem to get mdadm v3.1 installed...
git clone git://neil.brown.name/mdadm
All seems to work just fine, until mdadm --version:
Even doing ./mdadm --version within the devel-3.1 branch shows 3.0.2
Am I just confused, or did I do something wrong.
I am currently reshaping a RAID6 array from 6x1.5TB discs to 8x1.5TB discs. Simply - I have added 2 identical discs to an existing (almost empty) 6disc x RAID6 array. The operation started a 4 days ago and reporting running at 7000kb/s, and was consuming 5-6% CPU (load average 1.4-1.6 from memory). This was slower than I expected. (No raid controller, using 5x onboard SATA ports and 2x 2-port SIL SATA PCE-Express cards, Quad Core AMD 3GHz, 4GB RAM etc)
Neil,
Neil, thanks for the input. I'll go that route and I don't think the downtime should be too bad as the array is primarily media storage. I won't be able to write to it for a while, but I'll probably be able to stream from it without much trouble.
sudo mdadm --grow /dev/md1 --chunk=128 --backup-file=/mnt/tmp/backup
locks up the computer and forces a resync.
#####
mdX : active raid5 sdd1[8](S) sdb1[7](S) sdf8[0] sdl8[4] sdk2[5] sdc1[6] sdj6[3] sdi8[1]
. Y blocks super 1.1 level 5, 128k chunk, algorithm 2 [6/6] [uuuuuu]
# mdadm --grow /dev/mdX --level=6 --raid-devices=8 --backup-file=/root/mdX.backupfile
mdX : active raid6 sdd1[8] sdb1[7] sdf8[0] sdl8[4] sdk2[5] sdc1[6] sdj6[3] sdi8[1]
. Y blocks super 1.1 level 6, 128k chunk, algorithm 18 [8/9] [uuuuuu_u]
. [>....................] reshape = 0.0% (33920/484971520) finish=952.6min speed=8480K/sec
Nooo mdadm 3.1.1 I wanted an 8 device raid-6 Why do you show 9?
-- update
What is it showing me now??? blocks super 1.1 level 6, 128k chunk, algorithm 2 [8/10] [uuuuuuuu]
mdadm --detail /dev/mdX
/dev/mdX:
. Version : 1.01
. Raid Level : raid6
. Array Size : 6z ()
. Used Dev Size : z ()
. Raid Devices : 8
. Total Devices : 8
. Persistence : Superblock is persistent
. Update Time : Sun Jan 24 14:40:32 2010
. State : clean
.Active Devices : 8
Working Devices : 8
.Failed Devices : 0
. Spare Devices : 0
. Layout : left-symmetric
. Chunk Size : 128K
. Number Major Minor RaidDevice State
. 0 8 . 0 active sync /dev/sdf
. 1 8 . 1 active sync /dev/sdi
. 3 8 . 2 active sync /dev/sdj
. 6 8 . 3 active sync /dev/sdc
. 5 8 . 4 active sync /dev/sdk
. 4 8 . 5 active sync /dev/sdl
. 8 8 . 6 active sync /dev/sdd
. 7 8 . 7 active sync /dev/sdb
... Did it actually do what I want but just show me the wrong result with kernel 2.6.32-gentoo-r2
Hey there,
Thank you very much for your answers, everything as I expected. I decided to add another 1TB disk instead of a 1.5TB disk as I didn't consider that the 1.5TB disk would be much slower than all other preexisting 1TB disk.
So here my situation again: I want to add another 1TB-disk to an existing raid6-array consisting of seven 1TB-disks. Unfortunately mdadm gives me some strange errors.
At first I added the new disk (sdh) to the existing array (md11), which went fine. So now I have a 7-disk array with a single spare:
We use mdadm every day, across hundreds of servers, on mission critical work...some of it life-or-death stuff...meaning, if it dies we don't live.
Thanks for the great work!
I would like to reshape a 6x1T raid5 array to a 4x2T raid5 array. From the instructions on this page, I tried:
Thank you for the lightning fast response!
Hi Neil, thanks for this wonderful piece of software! Thanks to you I have survived many a hard drive death :)
1.
a- fail one of the RAID1 partitions
2.
a- convert the RAID1 to a 2 drive RAID5
Downtime is not a concern, and I'm not bothered about speed, but my concern is - what if a drive dies during the exhausting resync (with way 1) or the probably even more exhausting grow (with way 2)?