Version 0.97 (2004-06-03) Lucas Albers -- admin At cs DOT montana dot edu and Roger Chrisman
Home of most recent version: http://alioth.debian.org/projects/rootraiddoc
Thanks to: Alvin Olga, Era Eriksson, Yazz D. Atlas, James Bromberger, Timothy F Nagy, and alioth.debian.org

WARNING: No warranty of any kind. Proceed at your own risk. A typo, especially in lilo.conf, can leave your system unbootable. Back-up data and make a boot floppy before starting this procedure.

Summary

We begin with Debian installed on the Primary Master drive, hda (step 1). We need RAID support in our Kernel (step 2). We add another disk as Secondary Master, hdc, set it up for RAID (step 3), and copy Debian to it (step 4). Now we can reboot to the RAID device (step 5) and declare hda part of the RAID and it automatically syncs with hdc to complete our RAID 1 device (step 6).

If all goes well

• You do not need a rescue disk or to boot off anything except the hard drive.
• You can do this operation completely remotely.
• And you will not lose any data.

Use this HowTo at your own risk. We are not responsible for what happens!

First things first

• Backup your data.
• Create a boot floppy.

Whenever you change your partitions, you need to reboot! (If you know what you are doing, ignore this advice.)

I assume you will mess up a step so wherever possible, we include verification.

I use 'mdadm' because it is easier than 'raidtools' or 'raidtools2'.

We now have grub and lilo directions, grub directions are still in beta form.
Read the grub directions, and comment on them.

Procedure

1. Install Debian

Do a fresh install the normal way on your first drive, hda (the Primary Master drive in your computer). Or, if you already have a running Debian system that you want to use on hda; skip ahead to step 2. If you need Debian installation instructions, see:

Debian Installation HowTo » http://www.debian.org/releases/stable/installmanual

Sarge Debian Installation HowTo » http://d-i.alioth.debian.org/manual/

2. Upgrade to a RAID savvy Kernel

2.1 Compile and install a RAID savvy Kernel.

RAID must be compiled into the Kernel, not added as a module, for you to boot from the RAID device (unless you use a RAID savvy initrd kernel or boot from a non-RAID boot drive. (I now cover initrd methods!). You need RAID 1 but I usually include RAID 5, too. For step by step Kernel compile and install instructions, see:

Creating custom Kernels with Debian's kernel-package system » http://newbiedoc.sourceforge.net/system/kernel-pkg.html

2.2 Verify your RAID savvy Kernel.

cat /proc/mdstat

(You should see the RAID "personalities" your Kernel supports.)

Personalities : [linear] [raid0] [raid1] [raid5] read_ahead 1024 sectors md4 : active raid5 hdh4[3] hdg4[2] hdf4[1] hde4[0] 356958720 blocks level 5, 64k chunk, algorithm 2 [4/4] [UUUU] unused devices:

YOU MUST VERIFY you have raid support via /proc/mdstat. This is the most important item to verify before going any farther. So the kernel has to support it or you have to load the modules in initrd.

(This will show you if raid is compiled into kernel, or detected as a module from initrd.) /etc/modules will not list RAID if Kernel has RAID compiled in instead of loaded as modules.
Use lsmod to list currently loaded modules, this will show raid modules loaded.

reiserfs
raid1
ext2
ide-disk
raid5
ext3

cat /etc/modules

(IF YOU SEE ANY RAID LISTED IN /etc/modules, then you probably have your Kernel loading RAID via modules. That will prevent you from booting from your RAID device, unless you use initrd. To boot from your RAID device, unless you use a RAID savvy initrd, you need RAID compiled into Kernel, not added as a module.)

2.3 Install 'mdadm':

apt-get install mdadm

ls /proc/ide

3. Setup RAID 1

Setup RAID 1 and declare disk-one of your RAID to be 'missing' and disk-two of your RAID to be 'hdc'.

3.1 Create RAID (fd) partition on hdc

Warning: ALWAYS give the partition when editing with cfdisk. By default cfdisk will select the first disk in the system. I accidentally wiped the wrong partition with cfdisk, once.

Do A or B, either way will work:

A. Create partitions on new disk.

cfdisk /dev/hdc

or

B. copy existing partitions to new disk with sfdisk.

sfdisk -d /dev/hda | sfdisk /dev/hdc

3.2 Create correct partition type signatures on new partition.

cfdisk /dev/hdc

• Select Type, then hit enter, then type 'fd' (this means RAID type partition).
• Select Write
• Select Quit.

reboot

(To verify that everything is working ok.)

3.3 Create RAID device

that has two members and one of the members does not exist yet. md0 is the RAID partition we are creating, /dev/hdc1 is the initial partition. We will be adding /dev/hda1 back into the /dev/md0

RAID set after we boot into /dev/md0.

mdadm --create /dev/md0 --level=1 --raid-disks=2 missing /dev/hdc1

If this gives errors then you need to zero the super block, see useful mdadm commands.

3.4 Format RAID device

You can use reiserfs or ext3 for this, both work, I use reiserfs for larger devices. Go with what you trust.

mkfs.ext3 /dev/md0

or

mkfs -t reiserfs /dev/md0

4. Copy your Debian system

Copy your Debian system from hda to /dev/md0 ('missing' + 'hdc'). Then, check to make sure that the new RAID device is still setup right and can be mounted correctly. We do this with an entry in hda's /etc/fstab and a reboot. Note that by editing hda's /etc/fstab after the copy, instead of before, we leave the copy on md0 unaltered and only are editing hda's /etc/fstab.

NB: THIS IS A BRANCH IN OUR SYSTEM CONFIGURATION (eg temporary!), but it will overwritten later by the md0 version of /etc/fstab by the sync in step 6.

4.1 Create a mount point.

mkdir /mnt/md0

4.2 Mount your RAID device.

mount /dev/md0 /mnt/md0

4.3 Copy your Debian system to RAID device.

cp -axu / /mnt/md0

You don't need the -u switch; it just tells cp not to copy the files again if they exist. If you are running the command a second time it will run faster with the -u switch.

4.4 Edit /etc/fstab so that you mount your new RAID partition on boot up.

This verifies that you have the correct partition signatures on the partition and that your partition is correct. Sample Line in /etc/fstab:

/dev/md0 /mnt/md0 ext3 defaults 0 0

Then

reboot

And see if the RAID partition comes up.

mount

Should show /dev/md0 mounted on /mnt/md0.

5. Reboot to RAID device

For step 5 reboot, we will tell Lilo that

• as before, /boot and MBR are still on hda,
  
• and now we want root (/) to mount on md0.
  

We will, as before, be using hda's MBR (Master Boot Record is the first 512 bytes on a disk and is what the BIOS reads first in determining how to boot up a system) and hda's /boot dir (the kernel-image and some other stuff live here), but instead of mounting root (/) from hda, we will mount md0's root (/) (the root of our RAID device, currently running off of only hdc because we declared the first disk 'missing').

5.1 Configure Lilo to boot to the RAID device

(Later we will configure Lilo to write the boot sector to the RAID boot device also, so we can still boot even if either disk fails.)

Add a stanza labeled 'RAID' to /etc/lilo.conf on hda1 so that we can boot with /dev/md0, our RAID device, as root (/):

#the same boot drive as before.
boot=/dev/hda
image=/vmlinuz
label=RAID
read-only
#our new root partition.
root=/dev/md0

That makes an entry labeled 'RAID' specific to the RAID device, so you can still boot to /dev/hda if /dev/md0 does not work.

#sample working lilo.conf for raid.
#hda1,hdc1 are boot, hda2,hdc2 are swap
#hda3,hdc3 are the partition used by array
#root partition is /dev/md3 on / type reiserfs (rw)
#I named the raid volumes the same as the partition numbers
#this is the final lilo.conf file of a system completely finished,
#and booted into raid.


lba32
boot=/dev/md1
root=/dev/hda3
install=/boot/boot-menu.b
map=/boot/map
prompt
delay=50
timeout=50
vga=normal
raid-extra-boot=/dev/hda,/dev/hdd
default=RAID
image=/boot/vmlinuz-RAID
label=RAID
read-only
root=/dev/md3
alias=1
image=/vmlinuz
label=Linux
read-only
alias=2

image=/vmlinuz.old
label=LinuxOLD
read-only
optional

5.2 Test our new lilo.conf

lilo -t -v

(With a RAID installation, always run lilo -t first just to have Lilo tell you what it is about to do; use the -v flag, too, for verbose output.)

5.3 Run Lilo

Configure a one time Lilo boot via the -R flag and with a reboot with Kernel panic

The -R <boot-parameters-here> tells Lilo to only use the specified image for the next boot. So once you reboot it will revert to your old Kernel.

From 'man lilo':
-R command line
This option sets the default command for the boot loader the next time it executes. The boot loader will then erase this line: this is a once-only command. It is typically used in reboot scripts, just before calling `shutdown -r'. Used without any arguments, it will cancel a lock-ed or fallback command line.

Before you can do the 'lilo -v -R RAID' command, you must first do a 'lilo' command to update the Lilo boot record with the contents of your new lilo.conf. Otherwise Lilo does not know what you mean by 'RAID' and you just get a 'Fatal: No image "RAID" is defined' error message when you do 'lilo -v -R RAID'. So,

lilo
lilo -v -R RAID

5.4 Edit /mnt/md0/etc/fstab and reboot

to have /dev/md0 mount as root (/), when Lilo boots from our RAID device, /dev/md0.

Previous root (/) in fstab was:

/dev/hda1 / reiserfs defaults 0 0

Edit it to:

/dev/md0 / ext3 defaults 0 0

Note: edit /mnt/md0/etc/fstab, not /etc/fstab, because at the moment we are booted with hda1 as root (/) but we want to change the /etc/fstab that we currently have mounted on /mnt/md0/etc/fstab, our RAID device.

Reboot to check if system boots our RAID device, /dev/md0, as root (/). If it does not, just reboot again and you will come up with your previous boot partition courtesy of the -R flag in step 5.3 above.

reboot

Verify /dev/md0 is mounted as root (/)

mount

should show:

/dev/md0 on / type reiserfs (rw)
proc on /proc type proc (rw)
devpts on /dev/pts type devpts (rw,gid=5,mode=620)

'type reiserfs' is just my example; you will see whatever your file system type is.

Now we are booted into the new RAID device -- md0 as root (/). Our RAID device only has one disk in it at the moment because we earlier declared the other disk as 'missing'. That was because we needed that other disk, hda, to install Debian on or because it was our pre-existing Debian system.

6. Reformat hda as 'fd' and declare it as disk-one of your RAID

For step 6 reboots, we tell Lilo that

• as in step 5 above, our root (/) is now on md0.
• and now, /boot is also on md0,
• and MBR is on both hda and hdc.

Here we not only use md0's root (/) as in step 5, but also md0's /boot (it contains an identical kernel-image to the one on hda because we copied it here from hda in step 4, but we will be overwriting everything on hda in step 6 and can't continue relying on the stuff on hda) and MBR from either hda or hdc, whichever the BIOS can find (they will be identical MBRs and the BIOS will still find hda's MBR but in case the hda disk were to fail down the road we would want the BIOS to look on hdc as a fail over so that it could still boot up the system).

6.1 Change the signature on /dev/hda to software RAID

cfdisk /dev/hda

• Select "/dev/hda1"
• Then select "[Type]"
• Then hit "enter".
• Then type "FD".
• We are setting partition to "Software RAID"
• Should already be set.
• Then Select "Boot" if not set, so that you can boot off the device.
• All the boot partitions that are members of your bootable RAID device (hda1 and hdc1) should have the bootable flag set. If one is not set, set it here now
• Then select "Write" and enter 'yes'.
• Then select "Quite".

My two hard disks are from different manufacturers and as it happens, while both are roughly 40G, they have different architectures in terms of sectors and precise size. So cfdisk was unable to make the partitions precisely the same size and I had hda1 29,997.60MB and hdc1 30,000MB. This didn't work when I get to the 'mdadm --add /dev/md0 /dev/hda1' step. I got a, "failed: no space left on device!" error. So I ran cfdisk again and made hda1 slightly larger than hdc1, since I could not make them both exactly the same size. Now hda1 is 30,005.83MB and the 'mdadm -add /dev/md0 /dev/hda1' step works :-). (The remaining 10,000MB on each disk I am using for other purposes, including a md1 of 1,000MB composed of hda2 and hdc2.)

6.2 Add the first-disk to our existing RAID device

And watch the booted RAID system automatically mirror itself onto the new drive. We are currently booted from MBR and /boot device on /dev/hdc1, with /dev/md0 as root (/).

mdadm --add /dev/md0 /dev/hda1

Note: We are adding /dev/hda1 into our existing RAID device. See if it is syncing.

cat /proc/mdstat

should show that it is syncing.

6.3 Write new /etc/lilo.conf settings

these are from when we are booted onto RAID.

boot=/dev/md0
root=/dev/md0
#this writes the boot signatures to either disk.
raid-extra-boot=/dev/hda,/dev/hdc
image=/vmlinuz
label=RAID
read-only

YOU NEED THE raid-extra-boot to have it write the boot loader to all the disks.

YOU ARE OVERWRITING THE BOOT LOADER ON BOTH /dev/hda and /dev/hdc.

You can keep your old boot option to boot /dev/hda so you can boot RAID and /dev/hda.

But remember you don't want to boot into a RAID device in non RAID as it will hurt the synchronization. If you make changes on one disk and not the other.

6.4 Run Lilo with -R option and reboot

(we are currently booted into RAID)

lilo -t -v

lilo -R RAID

The -R option tells Lilo it to use the new Lilo setting only for the next reboot, and then revert back to previous setting.

reboot

and check

cat /proc/mdstat

and check

mount

to be sure all is as expected.

6.5 Now run Lilo normally (without -R) and reboot

See what Lilo will do.

lilo -t -v

If it looks okay, do it:

lilo

reboot

and check

cat /proc/mdstat

and check

mount

as a final system check.

Done.

Part II. RAID using initrd and grub

I used the following procedure with stock Debian 2.6.5, which has an initrd with all the modules ready to boot into RAID. The procedure also covers using grub as the boot loader. I built this from a bare install of Sarge using the new installer with grub as the boot loader, but most of this document is distro independent. My file system throughout is ext3 and it shouldn't take too much to use reiserfs.

These steps reference back to the procedure sections outlined above and indicate where things differ due to initrd or grub, so you will have to read/do/be familiar with the above steps. Also, make sure you currently use grub as your boot loader, if you are using LILO, install grub and make sure it works before proceeding!

Section - 2. Upgrade to a RAID savvy kernel

When using initrd the kernel does not need to have the RAID compiled in, they will be loaded as modules. Make sure the kernel loads the RAID modules.

Edit /etc/modules and add

md
raid1

Section - 3. Setup RAID 1

Section - 4. Copy your Debian system

Section - 5. Reboot to RAID device

Instead of section 5 using LILO, grub is used as the boot loader, and initrd used to load the kernel. A new kernel entry in the grub menu is created that refers to an initrd that is created which will start the md [raid] device. The original kernel entry will remain and can be reverted to if something goes wrong until RAID is running. This will still use grub loaded installed on the /dev/hda MBR.

5.1 Build a new RAID initrd

A) Make sure the initrd has the modules it needs, by editing /etc/mkinitrd/modules. Add the following [you can see what modules are available by mounting the initrd and looking in the lib/modules - see section 8.]:

md
raid1

B) Update the initrd so that the root device loaded is the raid device, not probed. Edit the /etc/mkinitrd/mkinitrd.conf, and update the ROOT line
ROOT=/dev/md0

C) Create the new initrd and a link to it.

mkinitrd -o /boot/initrd.img-2.6.5-raid

5.2 Update the grub boot menuorrectly. We do this with an entry in hda's /etc/fstab and a reboot. Note that by editing hda's /etc/fstab after the copy, instead of before, we leave the copy on md0 unaltered and only are editing hda's /etc/fstab.

NB: THIS IS A BRANCH IN OUR SYSTEM CONFIGURATION (eg temporary!), but it will overwritten later by the md0 version of /etc/fstab by the sync in step 6.

4.1 Create a mount point.

mkdir /mnt/md0

4.2 Mount your RAID device.

mount /dev/md0 /mnt/md0

4.3 Copy your Debian system to RAID device.

cp -axu / /mnt/md0

You don't need the -u switch; it just tells cp not to copy the files again if they exist. If you are running the command a second time it will run faster with the -u switch.

4.4 Edit /etc/fstab so that you mount your new RAID partition on boot up.

This verifies that you have the correct partition signatures on the partition and that your partition is correct. Sample Line in /etc/fstab:

/dev/md0 /mnt/md0 ext3 defaults 0 0

Then

reboot

And see if the RAID partition comes up.

mount

Should show /dev/md0 mounted on /mnt/md0.

5. Reboot to RAID device

For step 5 reboot, we will tell Lilo that

• as before, /boot and MBR are still on hda,
  
• and now we want root (/) to mount on md0.
  

We will, as before, be using hda's MBR (Master Boot Record is the first 512 bytes on a disk and is what the BIOS reads first in determining how to boot up a system) and hda's /boot dir (the kernel-image and some other stuff live here), but instead of mounting root (/) from hda, we will mount md0's root (/) (the root of our RAID device, currently running off of only hdc because we declared the first disk 'missing').

5.1 Configure Lilo to boot to the RAID device

(Later we will configure Lilo to write the boot sector to the RAID boot device also, so we can still boot even if either disk fails.)

Add a stanza labeled 'RAID' to /etc/lilo.conf on hda1 so that we can boot with /dev/md0, our RAID device, as root (/):

#the same boot drive as before.
boot=/dev/hda
image=/vmlinuz
label=RAID
read-only
#our new root partition.
root=/dev/md0

That makes an entry labeled 'RAID' specific to the RAID device, so you can still boot to /dev/hda if /dev/md0 does not work.

#sample working lilo.conf for raid.
#hda1,hdc1 are boot, hda2,hdc2 are swap
#hda3,hdc3 are the partition used by array
#root partition is /dev/md3 on / type reiserfs (rw)
#I named the raid volumes the same as the partition numbers
#this is the final lilo.conf file of a system completely finished,
#and booted into raid.


lba32
boot=/dev/md1
root=/dev/hda3
install=/boot/boot-menu.b
map=/boot/map
prompt
delay=50
timeout=50
vga=normal
raid-extra-boot=/dev/hda,/dev/hdd
default=RAID
image=/boot/vmlinuz-RAID
label=RAID
read-only
root=/dev/md3
alias=1
image=/vmlinuz
label=Linux
read-only
alias=2

image=/vmlinuz.old
label=LinuxOLD
read-only
optional

5.2 Test our new lilo.conf

lilo -t -v

(With a RAID installation, always run lilo -t first just to have Lilo tell you what it is about to do; use the -v flag, too, for verbose output.)

5.3 Run Lilo

Configure a one time Lilo boot via the -R flag and with a reboot with Kernel panic

The -R <boot-parameters-here> tells Lilo to only use the specified image for the next boot. So once you reboot it will revert to your old Kernel.

From 'man lilo':
-R command line
This option sets the default command for the boot loader the next time it executes. The boot loader will then erase this line: this is a once-only command. It is typically used in reboot scripts, just before calling `shutdown -r'. Used without any arguments, it will cancel a lock-ed or fallback command line.

Before you can do the 'lilo -v -R RAID' command, you must first do a 'lilo' command to update the Lilo boot record with the contents of your new lilo.conf. Otherwise Lilo does not know what you mean by 'RAID' and you just get a 'Fatal: No image "RAID" is defined' error message when you do 'lilo -v -R RAID'. So,

lilo
lilo -v -R RAID

5.4 Edit /mnt/md0/etc/fstab and reboot

to have /dev/md0 mount as root (/), when Lilo boots from our RAID device, /dev/md0.

Previous root (/) in fstab was:

/dev/hda1 / reiserfs defaults 0 0

Edit it to:

/dev/md0 / ext3 defaults 0 0

Note: edit /mnt/md0/etc/fstab, not /etc/fstab, because at the moment we are booted with hda1 as root (/) but we want to change the /etc/fstab that we currently have mounted on /mnt/md0/etc/fstab, our RAID device.

Reboot to check if system boots our RAID device, /dev/md0, as root (/). If it does not, just reboot again and you will come up with your previous boot partition courtesy of the -R flag in step 5.3 above.

reboot

Verify /dev/md0 is mounted as root (/)

mount

should show:

/dev/md0 on / type reiserfs (rw)
proc on /proc type proc (rw)
devpts on /dev/pts type devpts (rw,gid=5,mode=620)

'type reiserfs' is just my example; you will see whatever your file system type is.

Now we are booted into the new RAID device -- md0 as root (/). Our RAID device only has one disk in it at the moment because we earlier declared the other disk as 'missing'. That was because we needed that other disk, hda, to install Debian on or because it was our pre-existing Debian system.

6. Reformat hda as 'fd' and declare it as disk-one of your RAID

For step 6 reboots, we tell Lilo that

• as in step 5 above, our root (/) is now on md0.
• and now, /boot is also on md0,
• and MBR is on both hda and hdc.

Here we not only use md0's root (/) as in step 5, but also md0's /boot (it contains an identical kernel-image to the one on hda because we copied it here from hda in step 4, but we will be overwriting everything on hda in step 6 and can't continue relying on the stuff on hda) and MBR from either hda or hdc, whichever the BIOS can find (they will be identical MBRs and the BIOS will still find hda's MBR but in case the hda disk were to fail down the road we would want the BIOS to look on hdc as a fail over so that it could still boot up the system).

6.1 Change the signature on /dev/hda to software RAID

cfdisk /dev/hda

• Select "/dev/hda1"
• Then select "[Type]"
• Then hit "enter".
• Then type "FD".
• We are setting partition to "Software RAID"
• Should already be set.
• Then Select "Boot" if not set, so that you can boot off the device.
• All the boot partitions that are members of your bootable RAID device (hda1 and hdc1) should have the bootable flag set. If one is not set, set it here now
• Then select "Write" and enter 'yes'.
• Then select "Quite".

My two hard disks are from different manufacturers and as it happens, while both are roughly 40G, they have different architectures in terms of sectors and precise size. So cfdisk was unable to make the partitions precisely the same size and I had hda1 29,997.60MB and hdc1 30,000MB. This didn't work when I get to the 'mdadm --add /dev/md0 /dev/hda1' step. I got a, "failed: no space left on device!" error. So I ran cfdisk again and made hda1 slightly larger than hdc1, since I could not make them both exactly the same size. Now hda1 is 30,005.83MB and the 'mdadm -add /dev/md0 /dev/hda1' step works :-). (The remaining 10,000MB on each disk I am using for other purposes, including a md1 of 1,000MB composed of hda2 and hdc2.)

6.2 Add the first-disk to our existing RAID device

And watch the booted RAID system automatically mirror itself onto the new drive. We are currently booted from MBR and /boot device on /dev/hdc1, with /dev/md0 as root (/).

mdadm --add /dev/md0 /dev/hda1

Note: We are adding /dev/hda1 into our existing RAID device. See if it is syncing.

cat /proc/mdstat

should show that it is syncing.

6.3 Write new /etc/lilo.conf settings

these are from when we are booted onto RAID.

boot=/dev/md0
root=/dev/md0
#this writes the boot signatures to either disk.
raid-extra-boot=/dev/hda,/dev/hdc
image=/vmlinuz
label=RAID
read-only

YOU NEED THE raid-extra-boot to have it write the boot loader to all the disks.

YOU ARE OVERWRITING THE BOOT LOADER ON BOTH /dev/hda and /dev/hdc.

You can keep your old boot option to boot /dev/hda so you can boot RAID and /dev/hda.

But remember you don't want to boot into a RAID device in non RAID as it will hurt the synchronization. If you make changes on one disk and not the other.

6.4 Run Lilo with -R option and reboot

(we are currently booted into RAID)

lilo -t -v

lilo -R RAID

The -R option tells Lilo it to use the new Lilo setting only for the next reboot, and then revert back to previous setting.

reboot

and check

cat /proc/mdstat

and check

mount

to be sure all is as expected.

6.5 Now run Lilo normally (without -R) and reboot

See what Lilo will do.

lilo -t -v

If it looks okay, do it:

lilo

reboot

and check

cat /proc/mdstat

and check

mount

as a final system check.

Done.

Part II. RAID using initrd and grub

I used the following procedure with stock Debian 2.6.5, which has an initrd with all the modules ready to boot into RAID. The procedure also covers using grub as the boot loader. I built this from a bare install of Sarge using the new installer with grub as the boot loader, but most of this document is distro independent. My file system throughout is ext3 and it shouldn't take too much to use reiserfs.

These steps reference back to the procedure sections outlined above and indicate where things differ due to initrd or grub, so you will have to read/do/be familiar with the above steps. Also, make sure you currently use grub as your boot loader, if you are using LILO, install grub and make sure it works before proceeding!

Section - 2. Upgrade to a RAID savvy kernel

When using initrd the kernel does not need to have the RAID compiled in, they will be loaded as modules. Make sure the kernel loads the RAID modules.

Edit /etc/modules and add

md
raid1

Section - 3. Setup RAID 1

Section - 4. Copy your Debian system

Section - 5. Reboot to RAID device

Instead of section 5 using LILO, grub is used as the boot loader, and initrd used to load the kernel. A new kernel entry in the grub menu is created that refers to an initrd that is created which will start the md [raid] device. The original kernel entry will remain and can be reverted to if something goes wrong until RAID is running. This will still use grub loaded installed on the /dev/hda MBR.

5.1 Build a new RAID initrd

A) Make sure the initrd has the modules it needs, by editing /etc/mkinitrd/modules. Add the following [you can see what modules are available by mounting the initrd and looking in the lib/modules - see section 8.]:

md
raid1

B) Update the initrd so that the root device loaded is the raid device, not probed. Edit the /etc/mkinitrd/mkinitrd.conf, and update the ROOT line
ROOT=/dev/md0

C) Create the new initrd and a link to it.

mkinitrd -o /boot/initrd.img-2.6.5-raid

5.2 Update the grub boot menuorrectly. We do this with an entry in hda's /etc/fstab and a reboot. Note that by editing hda's /etc/fstab after the copy, instead of before, we leave the copy on md0 unaltered and only are editing hda's /etc/fstab.

NB: THIS IS A BRANCH IN OUR SYSTEM CONFIGURATION (eg temporary!), but it will overwritten later by the md0 version of /etc/fstab by the sync in step 6.

4.1 Create a mount point.

mkdir /mnt/md0

4.2 Mount your RAID device.

mount /dev/md0 /mnt/md0

4.3 Copy your Debian system to RAID device.

cp -axu / /mnt/md0

You don't need the -u switch; it just tells cp not to copy the files again if they exist. If you are running the command a second time it will run faster with the -u switch.

4.4 Edit /etc/fstab so that you mount your new RAID partition on boot up.

This verifies that you have the correct partition signatures on the partition and that your partition is correct. Sample Line in /etc/fstab:

/dev/md0 /mnt/md0 ext3 defaults 0 0

Then

reboot

And see if the RAID partition comes up.

mount

Should show /dev/md0 mounted on /mnt/md0.

5. Reboot to RAID device

For step 5 reboot, we will tell Lilo that

• as before, /boot and MBR are still on hda,
  
• and now we want root (/) to mount on md0.
  

We will, as before, be using hda's MBR (Master Boot Record is the first 512 bytes on a disk and is what the BIOS reads first in determining how to boot up a system) and hda's /boot dir (the kernel-image and some other stuff live here), but instead of mounting root (/) from hda, we will mount md0's root (/) (the root of our RAID device, currently running off of only hdc because we declared the first disk 'missing').

5.1 Configure Lilo to boot to the RAID device

(Later we will configure Lilo to write the boot sector to the RAID boot device also, so we can still boot even if either disk fails.)

Add a stanza labeled 'RAID' to /etc/lilo.conf on hda1 so that we can boot with /dev/md0, our RAID device, as root (/):

#the same boot drive as before.
boot=/dev/hda
image=/vmlinuz
label=RAID
read-only
#our new root partition.
root=/dev/md0

That makes an entry labeled 'RAID' specific to the RAID device, so you can still boot to /dev/hda if /dev/md0 does not work.

#sample working lilo.conf for raid.
#hda1,hdc1 are boot, hda2,hdc2 are swap
#hda3,hdc3 are the partition used by array
#root partition is /dev/md3 on / type reiserfs (rw)
#I named the raid volumes the same as the partition numbers
#this is the final lilo.conf file of a system completely finished,
#and booted into raid.


lba32
boot=/dev/md1
root=/dev/hda3
install=/boot/boot-menu.b
map=/boot/map
prompt
delay=50
timeout=50
vga=normal
raid-extra-boot=/dev/hda,/dev/hdd
default=RAID
image=/boot/vmlinuz-RAID
label=RAID
read-only
root=/dev/md3
alias=1
image=/vmlinuz
label=Linux
read-only
alias=2

image=/vmlinuz.old
label=LinuxOLD
read-only
optional

5.2 Test our new lilo.conf

lilo -t -v

(With a RAID installation, always run lilo -t first just to have Lilo tell you what it is about to do; use the -v flag, too, for verbose output.)

5.3 Run Lilo

Configure a one time Lilo boot via the -R flag and with a reboot with Kernel panic

The -R <boot-parameters-here> tells Lilo to only use the specified image for the next boot. So once you reboot it will revert to your old Kernel.

From 'man lilo':
-R command line
This option sets the default command for the boot loader the next time it executes. The boot loader will then erase this line: this is a once-only command. It is typically used in reboot scripts, just before calling `shutdown -r'. Used without any arguments, it will cancel a lock-ed or fallback command line.

Before you can do the 'lilo -v -R RAID' command, you must first do a 'lilo' command to update the Lilo boot record with the contents of your new lilo.conf. Otherwise Lilo does not know what you mean by 'RAID' and you just get a 'Fatal: No image "RAID" is defined' error message when you do 'lilo -v -R RAID'. So,

lilo
lilo -v -R RAID

5.4 Edit /mnt/md0/etc/fstab and reboot

to have /dev/md0 mount as root (/), when Lilo boots from our RAID device, /dev/md0.

Previous root (/) in fstab was:

/dev/hda1 / reiserfs defaults 0 0

Edit it to:

/dev/md0 / ext3 defaults 0 0

Note: edit /mnt/md0/etc/fstab, not /etc/fstab, because at the moment we are booted with hda1 as root (/) but we want to change the /etc/fstab that we currently have mounted on /mnt/md0/etc/fstab, our RAID device.

Reboot to check if system boots our RAID device, /dev/md0, as root (/). If it does not, just reboot again and you will come up with your previous boot partition courtesy of the -R flag in step 5.3 above.

reboot

Verify /dev/md0 is mounted as root (/)

mount

should show:

/dev/md0 on / type reiserfs (rw)
proc on /proc type proc (rw)
devpts on /dev/pts type devpts (rw,gid=5,mode=620)

'type reiserfs' is just my example; you will see whatever your file system type is.

Now we are booted into the new RAID device -- md0 as root (/). Our RAID device only has one disk in it at the moment because we earlier declared the other disk as 'missing'. That was because we needed that other disk, hda, to install Debian on or because it was our pre-existing Debian system.

6. Reformat hda as 'fd' and declare it as disk-one of your RAID

For step 6 reboots, we tell Lilo that

• as in step 5 above, our root (/) is now on md0.
• and now, /boot is also on md0,
• and MBR is on both hda and hdc.

Here we not only use md0's root (/) as in step 5, but also md0's /boot (it contains an identical kernel-image to the one on hda because we copied it here from hda in step 4, but we will be overwriting everything on hda in step 6 and can't continue relying on the stuff on hda) and MBR from either hda or hdc, whichever the BIOS can find (they will be identical MBRs and the BIOS will still find hda's MBR but in case the hda disk were to fail down the road we would want the BIOS to look on hdc as a fail over so that it could still boot up the system).

6.1 Change the signature on /dev/hda to software RAID

cfdisk /dev/hda

• Select "/dev/hda1"
• Then select "[Type]"
• Then hit "enter".
• Then type "FD".
• We are setting partition to "Software RAID"
• Should already be set.
• Then Select "Boot" if not set, so that you can boot off the device.
• All the boot partitions that are members of your bootable RAID device (hda1 and hdc1) should have the bootable flag set. If one is not set, set it here now
• Then select "Write" and enter 'yes'.
• Then select "Quite".

My two hard disks are from different manufacturers and as it happens, while both are roughly 40G, they have different architectures in terms of sectors and precise size. So cfdisk was unable to make the partitions precisely the same size and I had hda1 29,997.60MB and hdc1 30,000MB. This didn't work when I get to the 'mdadm --add /dev/md0 /dev/hda1' step. I got a, "failed: no space left on device!" error. So I ran cfdisk again and made hda1 slightly larger than hdc1, since I could not make them both exactly the same size. Now hda1 is 30,005.83MB and the 'mdadm -add /dev/md0 /dev/hda1' step works :-). (The remaining 10,000MB on each disk I am using for other purposes, including a md1 of 1,000MB composed of hda2 and hdc2.)

6.2 Add the first-disk to our existing RAID device

And watch the booted RAID system automatically mirror itself onto the new drive. We are currently booted from MBR and /boot device on /dev/hdc1, with /dev/md0 as root (/).

mdadm --add /dev/md0 /dev/hda1

Note: We are adding /dev/hda1 into our existing RAID device. See if it is syncing.

cat /proc/mdstat

should show that it is syncing.

6.3 Write new /etc/lilo.conf settings

these are from when we are booted onto RAID.

boot=/dev/md0
root=/dev/md0
#this writes the boot signatures to either disk.
raid-extra-boot=/dev/hda,/dev/hdc
image=/vmlinuz
label=RAID
read-only

YOU NEED THE raid-extra-boot to have it write the boot loader to all the disks.

YOU ARE OVERWRITING THE BOOT LOADER ON BOTH /dev/hda and /dev/hdc.

You can keep your old boot option to boot /dev/hda so you can boot RAID and /dev/hda.

But remember you don't want to boot into a RAID device in non RAID as it will hurt the synchronization. If you make changes on one disk and not the other.

6.4 Run Lilo with -R option and reboot

(we are currently booted into RAID)

lilo -t -v

lilo -R RAID

The -R option tells Lilo it to use the new Lilo setting only for the next reboot, and then revert back to previous setting.

reboot

and check

cat /proc/mdstat

and check

mount

to be sure all is as expected.

6.5 Now run Lilo normally (without -R) and reboot

See what Lilo will do.

lilo -t -v

If it looks okay, do it:

lilo

reboot

and check

cat /proc/mdstat

and check

mount

as a final system check.

Done.

Part II. RAID using initrd and grub

I used the following procedure with stock Debian 2.6.5, which has an initrd with all the modules ready to boot into RAID. The procedure also covers using grub as the boot loader. I built this from a bare install of Sarge using the new installer with grub as the boot loader, but most of this document is distro independent. My file system throughout is ext3 and it shouldn't take too much to use reiserfs.

These steps reference back to the procedure sections outlined above and indicate where things differ due to initrd or grub, so