Benrwong: Created Page based on "RAID 1 INSTALLATION STEPS" from Lab Manual

2016-06-30T19:52:43Z

Created Page based on "RAID 1 INSTALLATION STEPS" from Lab Manual

New page

=RAID 1 Installation Steps=
I ignored the swap completely (you don’t do anything with swap)

CURRENT SITUATION ON TAV:
[root@tav ~]# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/vg_system-lv_root
50G 6.0G 41G 13% /
tmpfs 3.9G 0 3.9G 0% /dev/shm
/dev/sda1 146M 112M 27M 81% /boot
/dev/mapper/vg_data-lv_share
860G 59G 758G 8% /opt
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
[root@tav ~]# cat /etc/fstab
/dev/mapper/vg_system-lv_root / ext4 defaults 1 1
UUID=d20b31a8-2cd3-4aee-ab9f-9d6bea4c0647 /boot ext2 defaults 1 2
#/dev/mapper/vg_data-lv_share /srv/share ext4 defaults 1 2
/dev/mapper/vg_data-lv_share /opt ext4 defaults 1 2
UUID=b89df73b-90e2-49e2-bae0-7707d7840111 swap swap defaults 0 0
tmpfs /dev/shm tmpfs defaults 0 0
devpts /dev/pts devpts gid=5,mode=620 0 0
sysfs /sys sysfs defaults 0 0
proc /proc proc defaults 0 0
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ <br />
[root@tav ~]# fdisk -l
Disk /dev/sdb: 1000.2 GB, 1000204886016 bytes
255 heads, 63 sectors/track, 121601 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x0002a4e1
Device Boot Start End Blocks Id System
/dev/sdb1 * 1 20 153600 83 Linux
Partition 1 does not end on cylinder boundary.
/dev/sdb2 20 6547 52428800 8e Linux LVM
/dev/sdb3 6547 7567 8192000 82 Linux swap / Solaris
/dev/sdb4 7567 121602 915987160 5 Extended
/dev/sdb5 7567 121602 915985408 8e Linux LVM
Disk /dev/sda: 1000.2 GB, 1000204886016 bytes
255 heads, 63 sectors/track, 121601 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x000ece4c
Device Boot Start End Blocks Id System
/dev/sda1 * 1 20 153600 83 Linux
Partition 1 does not end on cylinder boundary.
/dev/sda2 20 6547 52428800 8e Linux LVM
/dev/sda3 6547 7567 8192000 82 Linux swap / Solaris
/dev/sda4 7567 121602 915987160 5 Extended
/dev/sda5 7567 121602 915985408 8e Linux LVM
Disk /dev/mapper/vg_system-lv_root: 53.7 GB, 53653536768 bytes
255 heads, 63 sectors/track, 6523 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000
Disk /dev/mapper/vg_data-lv_share: 937.9 GB, 937947037696 bytes
255 heads, 63 sectors/track, 114032 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
[root@tav ~]# pvdisplay
--- Physical volume ---
PV Name /dev/sda5
VG Name vg_data
PV Size 873.55 GiB / not usable 21.00 MiB
Allocatable yes (but full)
PE Size 32.00 MiB
Total PE 27953
Free PE 0
Allocated PE 27953
PV UUID pcZ4WO-DnMD-WzNn-yXzp-s8u8-19Mf-Ug4vWD <br />
--- Physical volume ---
PV Name /dev/sda2
VG Name vg_system
PV Size 50.00 GiB / not usable 32.00 MiB
Allocatable yes (but full)
PE Size 32.00 MiB
Total PE 1599
Free PE 0
Allocated PE 1599
PV UUID dP0qZ0-ILST-aEMp-GN3t-ONqP-Y8OG-0fQgKR

@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
*** The reason why the both drives have the same layout (see sfdisk -l below) is because I've
already copied the partition tables from sda to sdb on tav: sfdisk -d /dev/sda | sfdisk --force /dev/sdb ******
[root@tav ~]# sfdisk -l
Disk /dev/sdb: 121601 cylinders, 255 heads, 63 sectors/track
Warning: extended partition does not start at a cylinder boundary.
DOS and Linux will interpret the contents differently.
Units = cylinders of 8225280 bytes, blocks of 1024 bytes, counting from 0
Device Boot Start End #cyls #blocks Id System
/dev/sdb1 * 0+ 19- 20- 153600 83 Linux
/dev/sdb2 19+ 6546- 6528- 52428800 8e Linux LVM
/dev/sdb3 6546+ 7566- 1020- 8192000 82 Linux swap / Solaris
/dev/sdb4 7566+ 121601- 114036- 915987160 5 Extended
/dev/sdb5 7566+ 121601- 114035- 915985408 8e Linux LVM
Disk /dev/sda: 121601 cylinders, 255 heads, 63 sectors/track
Warning: extended partition does not start at a cylinder boundary.
DOS and Linux will interpret the contents differently.
Units = cylinders of 8225280 bytes, blocks of 1024 bytes, counting from 0
Device Boot Start End #cyls #blocks Id System
/dev/sda1 * 0+ 19- 20- 153600 83 Linux
/dev/sda2 19+ 6546- 6528- 52428800 8e Linux LVM
/dev/sda3 6546+ 7566- 1020- 8192000 82 Linux swap / Solaris
/dev/sda4 7566+ 121601- 114036- 915987160 5 Extended
/dev/sda5 7566+ 121601- 114035- 915985408 8e Linux LVM
Disk /dev/mapper/vg_system-lv_root: 6523 cylinders, 255 heads, 63 sectors/track
Disk /dev/mapper/vg_data-lv_share: 114032 cylinders, 255 heads, 63 sectors/track
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
[root@tav ~]# vgdisplay -v
Finding all volume groups
Finding volume group "vg_data"
--- Volume group ---
VG Name vg_data
System ID
Format lvm2
Metadata Areas 1
Metadata Sequence No 2
VG Access read/write
VG Status resizable
MAX LV 0
Cur LV 1
Open LV 1
Max PV 0
Cur PV 1
Act PV 1
VG Size 873.53 GiB
PE Size 32.00 MiB
Total PE 27953
Alloc PE / Size 27953 / 873.53 GiB
Free PE / Size 0 / 0
VG UUID mWLha7-je5U-9Pjf-wO0y-JgmT-X12l-YF9e0c
--- Logical volume ---
LV Path /dev/vg_data/lv_share
LV Name lv_share
VG Name vg_data
LV UUID unYF5D-jvdd-LVXe-Ntuz-DbDU-FSE5-1mKJDM
LV Write Access read/write
LV Creation host, time tav.cluster.ucsf.bkslab.org, 2014-05-01 13:15:10 -0700
LV Status available
# open 1
LV Size 873.53 GiB
Current LE 27953
Segments 1
Allocation inherit
Read ahead sectors auto
- currently set to 256
Block device 253:1
--- Physical volumes ---
PV Name /dev/sda5
PV UUID pcZ4WO-DnMD-WzNn-yXzp-s8u8-19Mf-Ug4vWD
PV Status allocatable
Total PE / Free PE 27953 / 0
Finding volume group "vg_system"
--- Volume group ---
VG Name vg_system
System ID
Format lvm2
Metadata Areas 1
Metadata Sequence No 2
VG Access read/write
VG Status resizable
MAX LV 0
Cur LV 1
Open LV 1
Max PV 0
Cur PV 1
Act PV 1
VG Size 49.97 GiB
PE Size 32.00 MiB
Total PE 1599
Alloc PE / Size 1599 / 49.97 GiB
Free PE / Size 0 / 0
VG UUID B09ehJ-3G3g-8TEB-HFie-vrjW-5bRM-QT3Rlb
--- Logical volume ---
LV Path /dev/vg_system/lv_root
LV Name lv_root
VG Name vg_system
LV UUID TzNh0J-GWri-u2Or-YSjy-Uu9W-GnJm-X5JRR5
LV Write Access read/write
LV Creation host, time tav.cluster.ucsf.bkslab.org, 2014-05-01 13:14:36 -0700
LV Status available
# open 1
LV Size 49.97 GiB
Current LE 1599
Segments 1
Allocation inherit
Read ahead sectors auto
- currently set to 256
Block device 253:0
--- Physical volumes ---
PV Name /dev/sda2
PV UUID dP0qZ0-ILST-aEMp-GN3t-ONqP-Y8OG-0fQgKR
PV Status allocatable
Total PE / Free PE 1599 / 0
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
[root@tav ~]# iostat
Linux 2.6.32-431.20.3.0.1.el6.centos.plus.x86_64 (tav.ucsf.bkslab.org) 08/08/2014 _x86_64_ (8 CPU)
avg-cpu: %user %nice %system %iowait %steal %idle
0.05 0.00 0.03 0.06 0.00 99.85
Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn
sdb 0.00 0.04 0.00 11506 16
sda 0.85 4.47 14.99 1448876 4856642
dm-0 1.28 4.23 9.25 1370754 2996248
dm-1 0.72 0.15 5.74 48178 1860376
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
[root@tav ~]# cat /proc/mdstat
Personalities :
unused devices: <none>
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
[root@tav ~]# grub-install -v
grub-install (GNU GRUB 0.97)
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
SUMMARY:
sda1 => boot
sda2 => vg_system (or /)
sda3 => swap
sda4 => extended => Ignoring this partition
sda5 => vg_data (or /opt)
#####################################################################
THE FOLLOWING IS HOW TO SET UP A RAID 1 ARRAY WITH THE DRIVES SDA AND SDB, WITHOUT LOSING THE DATA ON SDA.
# First you have to load these kernel modules to avoid reboot.
modprobe linear
modprobe raid0
modprobe raid1
# Typically you would do the following step to copy the partition tables, but I already did it:
sfdisk -d /dev/sda | sfdisk --force /dev/sdb
# The following is to format the system file type for the second drive sdb:
fdisk /dev/sdb
Command (m for help): t # Set partition type
Command (m for help): 1 # Select partition number 1
Command (m for help): fd # Set file system type to Linux RAID type
Command (m for help): t # Set partition type
Command (m for help): 2 # Select partition number 2
Command (m for help): fd # Set file system type to Linux RAID type
Command (m for help): t # Set partition type
Command (m for help): 3 # Select partition number 3
Command (m for help): fd # Set file system type to Linux RAID type
Command (m for help): t # Set partition type
Command (m for help): 5 # Select partition number 5
Command (m for help): fd # Set file system type to Linux RAID type
Command (m for help): w # Write the changes and exit frisk
fdisk -l # Sanity check

# Creating RAID arrays /dev/md0 through /dev/md3.
# /dev/sdb1 will be added to /dev/md0, /dev/sdb2 to /dev/md1, etc.
# /dev/sda1, /dev/sda2, etc, can't be added right now because the system is currently running on them.
# Therefore we use the placeholder "missing" for sda1-5.
# You have to use the "--metadata=0.90" because the grub version is older than grub version 2.
mdadm --create /dev/md0 --metadata=0.90 --level=1 --raid-disks=2 missing /dev/sdb1
mdadm --create /dev/md1 --metadata=0.90 --level=1 --raid-disks=2 missing /dev/sdb2
mdadm --create /dev/md2 --metadata=0.90 --level=1 --raid-disks=2 missing /dev/sdb3
mdadm --create /dev/md3 --metadata=0.90 --level=1 --raid-disks=2 missing /dev/sdb5
# Create the filesystems for the newly created RAID arrays:
# md0 will be the boot partition, so ext2.
mkfs.ext2 /dev/md0
# md1 will contain the vg_system, so lvm.
pvcreate /dev/md1
vgextend vg_system /dev/md1
# md2 will contain the swap space.
mkswap /dev/md2
# md4 will contain vg_data, so lvm.
pvcreate /dev/md3
vgextend vg_data /dev/md3
mdadm --examine --scan > /etc/mdadm.conf
cat /etc/mdadm.conf # Sanity Check
vim /etc/fstab => Replace LABEL=/boot with /dev/md0
vim /etc/mtab => Replace /dev/sda1 with /dev/md0
vim /boot/grub/menu.lst
add "fallback=1" right underneath "default=0"
go to the bottom where you should find some kernel stanzas. Copy the first of them and paste the stanza before the first existing stanza; replace root (hd0,0) with root (hd1,0)
Delete "rd_NO_MD" if it exists, so that md modules will be loaded at boot time.
It should look something like this:
[...]
title CentOS (2.6.18-128.el5)
root (hd1,0)
kernel /vmlinuz-2.6.18-128.el5 ro root=/dev/VolGroup00/LogVol00
initrd /initrd-2.6.18-128.el5.img
title CentOS (2.6.18-128.el5)
root (hd0,0)
kernel /vmlinuz-2.6.18-128.el5 ro root=/dev/VolGroup00/LogVol00
initrd /initrd-2.6.18-128.el5.img
mv /boot/initramfs-$(uname -r).img /boot/initramfs-$(uname -r).img.bak
mkinitrd /boot/initramfs-$(uname -r).img $(uname -r)
# Moving the vg_system to RAID array md1
pvmove /dev/sda2 /dev/md1
vgreduce vg_system /dev/sda2
pvremove /dev/sda2
pvdisplay # Sanity check
# Moving the vg_data to RAID array md3
pvmove /dev/sda5 /dev/md3
vgreduce vg_data /dev/sda5
pvremove /dev/sda5
pvdisplay # Sanity check
swapoff /dev/sda3 # Deactivate previous swap space
# Convert all the sda* partitions to Linux RAID types, except for boot partition (sda1)
fdisk /dev/sda
Command (m for help): t # Set partition type
Command (m for help): 2 # Select partition number 2
Command (m for help): fd # Set file system type to Linux RAID type
Command (m for help): t # Set partition type
Command (m for help): 3 # Select partition number 3
Command (m for help): fd # Set file system type to Linux RAID type
Command (m for help): t # Set partition type
Command (m for help): 5 # Select partition number 5
Command (m for help): fd # Set file system type to Linux RAID type
Command (m for help): w # Write the changes and exit frisk
# Add the sda2-5 to the RAID
mdadm --add /dev/md1 /dev/sda2
mdadm --add /dev/md2 /dev/sda3
mdadm --add /dev/md3 /dev/sda5
watch cat /proc/mdstat
swapon /dev/md2
mkdir /mnt/raid
mount /dev/md0 /mnt/raid
cd /boot
cp -dpRx . /mnt/raid # Copy the boot directory to the RAID array
grub
# Set the root device temporarily to the second disk:
grub> root (hd0,0)
grub> setup (hd0)
# Repeat for the first disk:
grub> root (hd1,0)
grub> setup (hd1)
grub> quit
reboot
# If all is well, format boot partition, sda1:
fdisk /dev/sda
Command (m for help): t # Set partition type
Command (m for help): 1 # Select partition number 1
Command (m for help): fd # Set file system type to Linux RAID type
Command (m for help): w # Write the changes and exit frisk
# Add sda1 to the RAID array
mdadm --add /dev/md0 /dev/sda1
cat /proc/mdstat
mdadm --examine --scan > /etc/mdadm.conf
cat /etc/mdadm.conf
reboot
That should be it!

=Debugging Raid=
http://www.tldp.org/HOWTO/Software-RAID-HOWTO-6.html

http://www.techrepublic.com/blog/data-center/testing-your-software-raid-be-prepared/

About the md126 md127 thing how to fix it:
mv /boot/initramfs-$(uname -r).img /boot/initramfs-$(uname -r).img.bak1
mkinitrd /boot/initramfs-$(uname -r).img $(uname -r)

When I ran into problems with multiple pv’s (physical volumes) found, I did this:
mount -o remount / (If the root partition is on a lvm)
rm /etc/lvm/cache/.cache
vgscan
pvdisplay => to make sure it’s right
init 6
==To see the version of grub==
grub-install -v
=References=

These three were the best:

http://www.howtoforge.com/how-to-set-up-software-raid1-on-a-running-lvm-system-incl-grub-configuration-centos-5.3-p2

http://www.howtoforge.com/how-to-create-a-raid1-setup-on-an-existing-centos-redhat-6.0-system-p2

http://carlos-spitzer.com/2013/07/29/how-to-create-a-raid1-on-an-existing-centos-rhel-6-os-with-lvm/

The rest that I used:

http://community.spiceworks.com/how_to/show/340-lvm-single-drive-to-lvm-raid-1-mirror-migration

https://raid.wiki.kernel.org/index.php/RAID_setup

http://unix.stackexchange.com/questions/63928/can-i-create-a-software-raid-1-with-one-device

http://www.debian-administration.org/articles/238

http://feeding.cloud.geek.nz/posts/setting-up-raid-on-existing/

http://feeding.cloud.geek.nz/posts/setting-up-raid-on-existing/

Software Raid - Revision history

Benrwong: Created Page based on "RAID 1 INSTALLATION STEPS" from Lab Manual