需求
服务器上装了两块硬盘，其中一块上装了一个Debian，现在需要把两块硬盘做成软RAID。比较坑的地方在于，服务器只有SSH访问，没有任何带外管理。
环境
开了台类似配置的虚拟机做了个实验。系统是Debian 9，CSM方式启动，用GRUB作为bootloader。虚拟机的硬盘配置如下：
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 |
root@test-2xdisk ~ # lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT sr0 11:0 1 1024M 0 rom vda 254:0 0 30G 0 disk └─vda1 254:1 0 15G 0 part / vdb 254:16 0 30G 0 disk root@test-2xdisk ~ # fdisk -l /dev/vda Disk /dev/vda: 30 GiB, 32212254720 bytes, 62914560 sectors Units: sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disklabel type: dos Disk identifier: 0xbeefbeef Device Boot Start End Sectors Size Id Type /dev/vda1 * 2048 31454975 31452928 15G 83 Linux |
配置方法
这一方案的基本原理是，先在空硬盘上建立一个degraded的RAID阵列，把系统复制进去，重启从RAID阵列启动，再把原来的系统盘加入RAID阵列。
在开始之前，我们需要确认以下几点：
- 系统可以使用root用户远程登录
- 你到该服务器有稳定的网络连接，SSH不会频繁掉线
以下所有操作需要使用root用户进行。
安装所有必要的程序。
|
1 |
apt install mdadm psmisc wget xz-utils |
首先我们给空硬盘分区，建立一个FD00（Linux RAID autodetect）类型的分区。记得不要用完整个盘的空间，后面稍微留一点，以防不同硬盘的可用空间不同导致添加硬盘时出现问题。
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 |
$ fdisk /dev/vdb Welcome to fdisk (util-linux 2.29.2). Changes will remain in memory only, until you decide to write them. Be careful before using the write command. Device does not contain a recognized partition table. Created a new DOS disklabel with disk identifier 0x540a0e81. Command (m for help): n Partition type p primary (0 primary, 0 extended, 4 free) e extended (container for logical partitions) Select (default p): p Partition number (1-4, default 1): 1 First sector (2048-62914559, default 2048): Last sector, +sectors or +size{K,M,G,T,P} (2048-62914559, default 62914559): +28G Created a new partition 1 of type 'Linux' and of size 28 GiB. Command (m for help): t Selected partition 1 Partition type (type L to list all types): fd Changed type of partition 'Linux' to 'Linux raid autodetect'. Command (m for help): w The partition table has been altered. Calling ioctl() to re-read partition table. Syncing disks. |
创建RAID 1阵列
|
1 2 3 4 5 6 7 8 9 |
$ mdadm --create /dev/md0 --level=1 --raid-devices=2 missing /dev/vdb1 mdadm: Note: this array has metadata at the start and may not be suitable as a boot device. If you plan to store '/boot' on this device please ensure that your boot-loader understands md/v1.x metadata, or use --metadata=0.90 Continue creating array? y mdadm: Defaulting to version 1.2 metadata mdadm: array /dev/md0 started. |
给RAID阵列分区，创建文件系统。
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 |
$ fdisk /dev/md0 Welcome to fdisk (util-linux 2.29.2). Changes will remain in memory only, until you decide to write them. Be careful before using the write command. Device does not contain a recognized partition table. Created a new DOS disklabel with disk identifier 0x94f402e2. Command (m for help): n Partition type p primary (0 primary, 0 extended, 4 free) e extended (container for logical partitions) Select (default p): p Partition number (1-4, default 1): 1 First sector (2048-58687487, default 2048): Last sector, +sectors or +size{K,M,G,T,P} (2048-58687487, default 58687487): Created a new partition 1 of type 'Linux' and of size 28 GiB. Command (m for help): w The partition table has been altered. Calling ioctl() to re-read partition table. Syncing disks. $ mkfs -t ext4 /dev/md0p1 |
关掉系统上能关掉的非关键服务。（全新安装的Debian 9上应该只有这么多，如果有别的服务的话一并关掉。）
|
1 2 3 4 5 6 |
systemctl stop fail2ban systemctl stop cron systemctl stop rsyslog systemctl stop systemd-journald systemctl stop exim4 systemctl stop systemd-timesyncd |
卸载掉所有能卸载的文件系统。
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 |
$ umount -a umount: /sys/fs/cgroup/systemd: target is busy (In some cases useful info about processes that use the device is found by lsof(8) or fuser(1).) umount: /sys/fs/cgroup: target is busy (In some cases useful info about processes that use the device is found by lsof(8) or fuser(1).) umount: /: target is busy (In some cases useful info about processes that use the device is found by lsof(8) or fuser(1).) umount: /run: target is busy (In some cases useful info about processes that use the device is found by lsof(8) or fuser(1).) umount: /dev: target is busy (In some cases useful info about processes that use the device is found by lsof(8) or fuser(1).) $ swapoff -a |
这里报错是正常的。这一步完成以后，请确认除了根文件系统和虚拟文件系统以外的所有分区已经卸载；如果还有程序在占用这些分区，可以使用fuser来找到这些程序，然后手工关闭它们。
开一个tmpfs，做我们一会儿移动系统时候使用的根文件系统。这里你可以选择复制现有的系统进去，但是一个全新安装的Debian 9怎么也有1GiB左右，考虑到内存占用问题，我们从LXC那儿偷一个315MiB的根文件系统。
|
1 2 3 4 5 |
wget https://images.linuxcontainers.org/images/debian/stretch/amd64/default/20190406_05:24/rootfs.tar.xz mkdir /tmp/tmproot mount -t tmpfs none /tmp/tmproot tar -xvf rootfs.tar.xz -C /tmp/tmproot rm rootfs.tar.xz |
然后我们悄悄把现在的根文件系统换下来。
|
1 2 3 4 |
mkdir -p /tmp/tmproot/oldroot mount --make-rprivate / pivot_root /tmp/tmproot /tmp/tmproot/oldroot for i in dev proc sys run; do mount --move /oldroot/$i /$i; done |
在新的根文件系统里面装一个SSH daemon，替换掉外面那个。
|
1 2 3 4 5 6 7 |
cp /oldroot/etc/resolv.conf /etc apt update apt install ssh cp -axr /oldroot/etc/ssh /etc cp -a /oldroot/etc/{passwd,shadow} /etc cp -axr /oldroot/root/.ssh /root systemctl restart ssh |
在不断开原有SSH会话的情况下再连接一次服务器，确认能够正常连接。如果一切正常，那么断开原来的SSH会话。
最后检查一次原来的根文件系统还有谁在用，然后把它们一一关闭（kill -15 $PID）：
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 |
$ fuser -vm /oldroot USER PID ACCESS COMMAND /oldroot: root kernel mount /oldroot root 1 ....m systemd root 300 ....m agetty root 301 ....m agetty root 302 ....m agetty root 303 ....m agetty root 304 ....m agetty root 305 ....m agetty root 3555 f...m systemd-udevd root 11343 ....m mdadm root 11395 F...m rsyslogd root 11405 ....m systemd-journal |
最后应该只剩下一个systemd。
|
1 2 3 4 |
$ fuser -vm /oldroot USER PID ACCESS COMMAND /oldroot: root kernel mount /oldroot root 1 ....m systemd |
这时候我们重启一下systemd本身，这样原来的根文件系统就完全不会被占用了。（daemon-reexec的时候可能因为某些unit导致卡住，Ctrl+C即可。）
|
1 2 3 4 |
$ systemctl daemon-reexec $ fuser -vm /oldroot USER PID ACCESS COMMAND /oldroot: root kernel mount /oldroot |
复制原来的文件系统到RAID阵列上。
|
1 2 3 |
mkdir -p /newroot mount /dev/md0p1 /newroot cp -avrfx /oldroot/* /newroot |
重建fstab：打开/newroot/etc/fstab，找到原来的根文件系统，改成/dev/md0p1或者UUID。
chroot进新系统：
|
1 2 |
for i in dev proc sys run; do mount --bind /$i /newroot/$i; done chroot /newroot |
创建正确的mdadm配置：
|
1 |
mdadm --detail --scan >> /etc/mdadm/mdadm.conf |
重建initramfs：
|
1 |
update-initramfs -u -k all |
重建GRUB配置：打开/etc/default/grub，把GRUB_DEVICE（如有）改成/dev/md0p1，然后创建新的GRUB配置。
|
1 |
grub-mkconfig > /boot/grub/grub.cfg |
安装GRUB到新的物理硬盘。
|
1 2 3 4 5 |
$ grub-install --recheck /dev/vdb Installing for i386-pc platform. grub-install: warning: Couldn't find physical volume `(null)'. Some modules may be missing from core image.. grub-install: warning: Couldn't find physical volume `(null)'. Some modules may be missing from core image.. Installation finished. No error reported. |
为了让系统即使仍然从原系统盘引导，仍然能从新的mdadm阵列启动，我们需要把新的GRUB配置文件复制到原系统盘上。
|
1 2 |
exit # exit chroot cp /newroot/grub/grub.cfg /oldroot/grub/grub.cfg |
重启。如果一切正常的话，你将会看到RAID阵列被挂载到根上。
|
1 2 3 4 5 6 7 8 9 |
$ lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT sr0 11:0 1 1024M 0 rom vda 254:0 0 30G 0 disk └─vda1 254:1 0 15G 0 part vdb 254:16 0 30G 0 disk └─vdb1 254:17 0 28G 0 part └─md0 9:0 0 28G 0 raid1 └─md0p1 259:0 0 28G 0 md / |
清空原系统盘数据并加入RAID阵列：
|
1 2 3 4 5 |
swapoff -a sfdisk -d /dev/vdb | sfdisk /dev/vda grub-install --recheck /dev/vda # might need reboot to make kernel read the new partition table mdadm /dev/md0 -a /dev/vda1 |
mdadm会开始重建RAID阵列。
|
1 2 3 4 5 6 7 |
$ cat /proc/mdstat Personalities : [raid1] [linear] [multipath] [raid0] [raid6] [raid5] [raid4] [raid10] md0 : active raid1 vda1[2] vdb1[1] 29343744 blocks super 1.2 [2/1] [_U] [==>..................] recovery = 13.6% (4009344/29343744) finish=2.1min speed=200467K/sec unused devices: <none> |
至此，迁移工作结束。
参考：