mdadm是linux下用于创建和管理软件RAID的命令。
1 命令帮助
[dave@www.cndba.cn ~]# mdadm --help
[dave@www.cndba.cn ~]# mdadm --create --help
Options that are valid with --create (-C) are:
--bitmap= -b : Create a bitmap for the array with the given filename
: or an internal bitmap if 'internal' is given
--chunk= -c : chunk size in kibibytes
--rounding= : rounding factor for linear array (==chunk size)
--level= -l : raid level: 0,1,4,5,6,10,linear,multipath and synonyms
--parity= -p : raid5/6 parity algorithm: {left,right}-{,a}symmetric
--layout= : same as --parity, for RAID10: [fno]NN
--raid-devices= -n : number of active devices in array
--spare-devices= -x : number of spare (eXtra) devices in initial array
--size= -z : Size (in K) of each drive in RAID1/4/5/6/10 - optional
--data-offset= : Space to leave between start of device and start
: of array data.
--force -f : Honour devices as listed on command line. Don't
: insert a missing drive for RAID5.
--run -R : insist of running the array even if not all
: devices are present or some look odd.
--readonly -o : start the array readonly - not supported yet.
--name= -N : Textual name for array - max 32 characters
--bitmap-chunk= : bitmap chunksize in Kilobytes.
--delay= -d : bitmap update delay in seconds.
--write-journal= : Specify journal device for RAID-4/5/6 array
--consistency-policy= : Specify the policy that determines how the array
-k : maintains consistency in case of unexpected shutdown.
选项:-C 的几个常用专用选项:
- -l 级别
- -n 设备个数
- -a {yes|no} 自动为其创建设备文件
- -c 指定数据块大小(chunk)
- -x 指定空闲盘(热备磁盘)个数,空闲盘(热备磁盘)能在工作盘损坏后自动顶替
注意:创建阵列时,阵列所需磁盘数为-n参数和-x参数的个数和
2 环境说明
[dave@www.cndba.cn ~]# fdisk -l |grep /dev/sd
Disk /dev/sdl: 1 GiB, 1073741824 bytes, 2097152 sectors
Disk /dev/sdk: 1 GiB, 1073741824 bytes, 2097152 sectors
Disk /dev/sdh: 1 GiB, 1073741824 bytes, 2097152 sectors
Disk /dev/sdf: 1 GiB, 1073741824 bytes, 2097152 sectors
Disk /dev/sdi: 1 GiB, 1073741824 bytes, 2097152 sectors
Disk /dev/sdd: 1 GiB, 1073741824 bytes, 2097152 sectors
Disk /dev/sdb: 1 GiB, 1073741824 bytes, 2097152 sectors
Disk /dev/sde: 1 GiB, 1073741824 bytes, 2097152 sectors
Disk /dev/sdj: 1 GiB, 1073741824 bytes, 2097152 sectors
Disk /dev/sdg: 1 GiB, 1073741824 bytes, 2097152 sectors
Disk /dev/sdc: 1 GiB, 1073741824 bytes, 2097152 sectors
Disk /dev/sda: 50 GiB, 53687091200 bytes, 104857600 sectors
/dev/sda1 * 2048 2099199 2097152 1G 83 Linux
/dev/sda2 2099200 104857599 102758400 49G 8e Linux LVM
[dave@www.cndba.cn ~]#
3 创建raid0
3.1 创建raid
[dave@www.cndba.cn ~]# lvs
LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert
root klas -wi-ao---- <44.00g
swap klas -wi-ao---- 5.00g
[dave@www.cndba.cn ~]#
[dave@www.cndba.cn ~]# mdadm -C /dev/md0 -a yes -l 0 -n 2 /dev/sd{b,c}
mdadm: Defaulting to version 1.2 metadata
mdadm: array /dev/md0 started.
[dave@www.cndba.cn ~]# fdisk -l /dev/md0
Disk /dev/md0: 1.102 GiB, 2143289344 bytes, 4186112 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 524288 bytes / 1048576 bytes
[dave@www.cndba.cn ~]#
3.2 格式化:
[dave@www.cndba.cn ~]# mkfs.ext4 /dev/md0
mke2fs 1.45.6 (20-Mar-2020)
Creating filesystem with 523264 4k blocks and 130816 inodes
Filesystem UUID: de345027-6b8c-4674-ad6e-5da8de60be84
Superblock backups stored on blocks:
32768, 98304, 163840, 229376, 294912
Allocating group tables: done
Writing inode tables: done
Creating journal (8192 blocks): done
Writing superblocks and filesystem accounting information: done
[dave@www.cndba.cn ~]#
4 创建raid1:
4.1 创建raid
[dave@www.cndba.cn ~]# mdadm -C /dev/md1 -a yes -l 1 -n 2 /dev/sd{d,e}
mdadm: Note: this array has metadata at the start and
may not be suitable as a boot device. If you plan to
store '/boot' on this device please ensure that
your boot-loader understands md/v1.x metadata, or use
--metadata=0.90
Continue creating array? y
mdadm: Defaulting to version 1.2 metadata
mdadm: array /dev/md1 started.
[dave@www.cndba.cn ~]#
注意:这个提示是说软raid不能用作启动分区。
[dave@www.cndba.cn ~]# fdisk -l /dev/md1
Disk /dev/md1: 1022 MiB, 1071644672 bytes, 2093056 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
[dave@www.cndba.cn ~]#
4.2 格式化:
[dave@www.cndba.cn ~]# mkfs.ext4 /dev/md1
mke2fs 1.45.6 (20-Mar-2020)
Creating filesystem with 261632 4k blocks and 65408 inodes
Filesystem UUID: 954ab8d6-ffad-427b-85f2-2e703ea4d87f
Superblock backups stored on blocks:
32768, 98304, 163840, 229376
Allocating group tables: done
Writing inode tables: done
Creating journal (4096 blocks): done
Writing superblocks and filesystem accounting information: done
[dave@www.cndba.cn ~]#
5 创建raid5:
5.1 创建raid5
[dave@www.cndba.cn ~]# mdadm -C /dev/md5 -a yes -l 5 -n 3 /dev/sd{f,g,h}
mdadm: Defaulting to version 1.2 metadata
mdadm: array /dev/md5 started.
[dave@www.cndba.cn ~]# fdisk -l /dev/md5
Disk /dev/md5: 1.102 GiB, 2143289344 bytes, 4186112 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 524288 bytes / 1048576 bytes
[dave@www.cndba.cn ~]#
5.2 格式化:
[dave@www.cndba.cn ~]# mkfs.ext4 /dev/md5
mke2fs 1.45.6 (20-Mar-2020)
Creating filesystem with 523264 4k blocks and 130816 inodes
Filesystem UUID: 5a67350c-f625-414b-800f-675cc9acefae
Superblock backups stored on blocks:
32768, 98304, 163840, 229376, 294912
Allocating group tables: done
Writing inode tables: done
Creating journal (8192 blocks): done
Writing superblocks and filesystem accounting information: done
[dave@www.cndba.cn ~]#
5.3 增加热备磁盘:
[dave@www.cndba.cn ~]# mdadm -D /dev/md5
/dev/md5:
Version : 1.2
Creation Time : Fri Jun 2 14:10:26 2023
Raid Level : raid5
Array Size : 2093056 (2044.00 MiB 2143.29 MB)
Used Dev Size : 1046528 (1022.00 MiB 1071.64 MB)
Raid Devices : 3
Total Devices : 3
Persistence : Superblock is persistent
Update Time : Fri Jun 2 14:14:18 2023
State : clean
Active Devices : 3
Working Devices : 3
Failed Devices : 0
Spare Devices : 0
Layout : left-symmetric
Chunk Size : 512K
Consistency Policy : resync
Name : dcm3:5 (local to host dcm3)
UUID : 883963c1:83ca52ae:86763222:8bbdc797
Events : 18
Number Major Minor RaidDevice State
0 8 80 0 active sync /dev/sdf
1 8 96 1 active sync /dev/sdg
3 8 112 2 active sync /dev/sdh
[dave@www.cndba.cn ~]# mdadm /dev/md5 -a /dev/sdi
mdadm: added /dev/sdi
[dave@www.cndba.cn ~]# mdadm -D /dev/md5
/dev/md5:
Version : 1.2
Creation Time : Fri Jun 2 14:10:26 2023
Raid Level : raid5
Array Size : 2093056 (2044.00 MiB 2143.29 MB)
Used Dev Size : 1046528 (1022.00 MiB 1071.64 MB)
Raid Devices : 3
Total Devices : 4
Persistence : Superblock is persistent
Update Time : Fri Jun 2 14:15:19 2023
State : clean
Active Devices : 3
Working Devices : 4
Failed Devices : 0
Spare Devices : 1
Layout : left-symmetric
Chunk Size : 512K
Consistency Policy : resync
Name : dcm3:5 (local to host dcm3)
UUID : 883963c1:83ca52ae:86763222:8bbdc797
Events : 19
Number Major Minor RaidDevice State
0 8 80 0 active sync /dev/sdf
1 8 96 1 active sync /dev/sdg
3 8 112 2 active sync /dev/sdh
4 8 128 - spare /dev/sdi
[dave@www.cndba.cn ~]#
6 查看md状态:
6.1 查看RAID阵列的详细信息:
选项: -D = —detail
mdadm -D /dev/md# 查看指定RAID设备的详细信息
[dave@www.cndba.cn ~]# mdadm -D /dev/md0
/dev/md0:
Version : 1.2
Creation Time : Fri Jun 2 13:59:31 2023
Raid Level : raid0
Array Size : 2093056 (2044.00 MiB 2143.29 MB)
Raid Devices : 2
Total Devices : 2
Persistence : Superblock is persistent
Update Time : Fri Jun 2 13:59:31 2023
State : clean
Active Devices : 2
Working Devices : 2
Failed Devices : 0
Spare Devices : 0
Chunk Size : 512K
Consistency Policy : none
Name : dcm3:0 (local to host dcm3)
UUID : 1e9db374:a1454a28:b330ee33:c030cd37
Events : 0
Number Major Minor RaidDevice State
0 8 16 0 active sync /dev/sdb
1 8 32 1 active sync /dev/sdc
[dave@www.cndba.cn ~]#
6.2 查看raid状态
[dave@www.cndba.cn ~]# cat /proc/mdstat
Personalities : [raid0]
md0 : active raid0 sdc[1] sdb[0]
2093056 blocks super 1.2 512k chunks
unused devices: <none>
[dave@www.cndba.cn ~]#
注意:在创建raid前,应该先查看磁盘是否被识别,如果内核还为识别,创建Raid时会报错:
cat /proc/partitions
如果没有被识别,可以执行命令:
kpartx /dev/sdb或者partprobe/dev/sdb
7 管理模式
选项:-a(—add),-d(—del),-r(—remove),-f(—fail)
7.1 模拟损坏:
[dave@www.cndba.cn ~]# mdadm /dev/md1 -f /dev/sdd
mdadm: set /dev/sdd faulty in /dev/md1
[dave@www.cndba.cn ~]# mdadm -D /dev/md1
/dev/md1:
Version : 1.2
Creation Time : Fri Jun 2 14:03:34 2023
Raid Level : raid1
Array Size : 1046528 (1022.00 MiB 1071.64 MB)
Used Dev Size : 1046528 (1022.00 MiB 1071.64 MB)
Raid Devices : 2
Total Devices : 2
Persistence : Superblock is persistent
Update Time : Fri Jun 2 14:27:33 2023
State : clean, degraded
Active Devices : 1
Working Devices : 1
Failed Devices : 1
Spare Devices : 0
Consistency Policy : resync
Name : dcm3:1 (local to host dcm3)
UUID : 985dfa4e:c5f7a754:6bfc9d7e:ba1ac90d
Events : 19
Number Major Minor RaidDevice State
- 0 0 0 removed
1 8 64 1 active sync /dev/sde
0 8 48 - faulty /dev/sdd
[dave@www.cndba.cn ~]#
7.2 移除损坏的磁盘:
[dave@www.cndba.cn ~]# mdadm /dev/md1 -r /dev/sdd
mdadm: hot removed /dev/sdd from /dev/md1
[dave@www.cndba.cn ~]# mdadm -D /dev/md1
/dev/md1:
Version : 1.2
Creation Time : Fri Jun 2 14:03:34 2023
Raid Level : raid1
Array Size : 1046528 (1022.00 MiB 1071.64 MB)
Used Dev Size : 1046528 (1022.00 MiB 1071.64 MB)
Raid Devices : 2
Total Devices : 1
Persistence : Superblock is persistent
Update Time : Fri Jun 2 14:28:47 2023
State : clean, degraded
Active Devices : 1
Working Devices : 1
Failed Devices : 0
Spare Devices : 0
Consistency Policy : resync
Name : dcm3:1 (local to host dcm3)
UUID : 985dfa4e:c5f7a754:6bfc9d7e:ba1ac90d
Events : 20
Number Major Minor RaidDevice State
- 0 0 0 removed
1 8 64 1 active sync /dev/sde
[dave@www.cndba.cn ~]#
7.3 添加新的硬盘到已有阵列:
[dave@www.cndba.cn ~]# mdadm /dev/md1 -a /dev/sdd
mdadm: added /dev/sdd
[dave@www.cndba.cn ~]# mdadm -D /dev/md1
/dev/md1:
Version : 1.2
Creation Time : Fri Jun 2 14:03:34 2023
Raid Level : raid1
Array Size : 1046528 (1022.00 MiB 1071.64 MB)
Used Dev Size : 1046528 (1022.00 MiB 1071.64 MB)
Raid Devices : 2
Total Devices : 2
Persistence : Superblock is persistent
Update Time : Fri Jun 2 14:29:28 2023
State : clean, degraded, recovering
Active Devices : 1
Working Devices : 2
Failed Devices : 0
Spare Devices : 1
Consistency Policy : resync
Rebuild Status : 38% complete
Name : dcm3:1 (local to host dcm3)
UUID : 985dfa4e:c5f7a754:6bfc9d7e:ba1ac90d
Events : 28
Number Major Minor RaidDevice State
2 8 48 0 spare rebuilding /dev/sdd
# 注意这里的状态
1 8 64 1 active sync /dev/sde
[dave@www.cndba.cn ~]# mdadm -D /dev/md1
/dev/md1:
Version : 1.2
Creation Time : Fri Jun 2 14:03:34 2023
Raid Level : raid1
Array Size : 1046528 (1022.00 MiB 1071.64 MB)
Used Dev Size : 1046528 (1022.00 MiB 1071.64 MB)
Raid Devices : 2
Total Devices : 2
Persistence : Superblock is persistent
Update Time : Fri Jun 2 14:29:31 2023
State : clean
Active Devices : 2
Working Devices : 2
Failed Devices : 0
Spare Devices : 0
Consistency Policy : resync
Name : dcm3:1 (local to host dcm3)
UUID : 985dfa4e:c5f7a754:6bfc9d7e:ba1ac90d
Events : 39
Number Major Minor RaidDevice State
2 8 48 0 active sync /dev/sdd
1 8 64 1 active sync /dev/sde
[dave@www.cndba.cn ~]#
注意:
- 新增加的硬盘需要与原硬盘大小一致
- 如果原有阵列缺少工作磁盘(如raid1只有一块在工作,raid5只有2块在工作),这时新增加的磁盘直接变为工作磁盘,如果原有阵列工作正常,则新增加的磁盘为热备磁盘。
7.4 停止阵列:
选项:-S = —stop
[dave@www.cndba.cn ~]# mdadm -S /dev/md1
mdadm: stopped /dev/md1
[dave@www.cndba.cn ~]# fdisk -l /dev/md1
fdisk: cannot open /dev/md1: No such file or directory
[dave@www.cndba.cn ~]#
7.5 启动 RAID /dev/md1
[dave@www.cndba.cn ~]# mdadm -A /dev/md1 /dev/sdd /dev/sde
mdadm: /dev/md1 has been started with 2 drives.
[dave@www.cndba.cn ~]# mdadm -D /dev/md1
/dev/md1:
Version : 1.2
Creation Time : Fri Jun 2 14:03:34 2023
Raid Level : raid1
Array Size : 1046528 (1022.00 MiB 1071.64 MB)
Used Dev Size : 1046528 (1022.00 MiB 1071.64 MB)
Raid Devices : 2
Total Devices : 2
Persistence : Superblock is persistent
Update Time : Fri Jun 2 14:29:31 2023
State : clean
Active Devices : 2
Working Devices : 2
Failed Devices : 0
Spare Devices : 0
Consistency Policy : resync
Name : dcm3:1 (local to host dcm3)
UUID : 985dfa4e:c5f7a754:6bfc9d7e:ba1ac90d
Events : 39
Number Major Minor RaidDevice State
2 8 48 0 active sync /dev/sdd
1 8 64 1 active sync /dev/sde
[dave@www.cndba.cn ~]#
这里需要写明所有设备,可以创建/etc/mdadm.conf 文件,就不用创建了。
7.6 扫描 RAID 信息
[dave@www.cndba.cn ~]# mdadm -Ds
ARRAY /dev/md0 metadata=1.2 name=dcm3:0 UUID=1e9db374:a1454a28:b330ee33:c030cd37
ARRAY /dev/md5 metadata=1.2 spares=1 name=dcm3:5 UUID=883963c1:83ca52ae:86763222:8bbdc797
ARRAY /dev/md1 metadata=1.2 name=dcm3:1 UUID=985dfa4e:c5f7a754:6bfc9d7e:ba1ac90d
[dave@www.cndba.cn ~]#
7.7 开机自启动:创建配置文件 /etc/mdadm.conf
如果没有配置 /etc/mdadm.conf 文件,那么每次机器重启后都需要手工启动RADI。 可以通过创建配置文件,让RAID 自动启动。
[dave@www.cndba.cn ~]# mdadm -Ds
ARRAY /dev/md0 metadata=1.2 name=dcm3:0 UUID=1e9db374:a1454a28:b330ee33:c030cd37
ARRAY /dev/md5 metadata=1.2 spares=1 name=dcm3:5 UUID=883963c1:83ca52ae:86763222:8bbdc797
ARRAY /dev/md1 metadata=1.2 name=dcm3:1 UUID=985dfa4e:c5f7a754:6bfc9d7e:ba1ac90d
[dave@www.cndba.cn ~]#
[dave@www.cndba.cn ~]# mdadm --detail --scan
ARRAY /dev/md0 metadata=1.2 name=dcm3:0 UUID=1e9db374:a1454a28:b330ee33:c030cd37
ARRAY /dev/md5 metadata=1.2 spares=1 name=dcm3:5 UUID=883963c1:83ca52ae:86763222:8bbdc797
ARRAY /dev/md1 metadata=1.2 name=dcm3:1 UUID=985dfa4e:c5f7a754:6bfc9d7e:ba1ac90d
[dave@www.cndba.cn ~]#
创建文件:
[dave@www.cndba.cn ~]# mdadm -Ds >> /etc/mdadm.conf
修改文件 /etc/mdadm.conf, 增加 device 内容,修改后内容如下
[dave@www.cndba.cn ~]# cat /etc/mdadm.conf
ARRAY /dev/md0 metadata=1.2 name=dcm3:0 UUID=1e9db374:a1454a28:b330ee33:c030cd37 devices=/dev/sdb,/dev/sdc
ARRAY /dev/md5 metadata=1.2 spares=1 name=dcm3:5 UUID=883963c1:83ca52ae:86763222:8bbdc797 devices=/dev/sdf,/dev/sdg,/dev/sdh,/dev/sdi
ARRAY /dev/md1 metadata=1.2 name=dcm3:1 UUID=985dfa4e:c5f7a754:6bfc9d7e:ba1ac90d devices=/dev/sdd,/dev/sde
[dave@www.cndba.cn ~]#
备注:通过创建 /etc/mdadm.conf 文件后,那么启动RAID 时不需要指定 RAID设备和 RAID 成员。
7.8 删除阵列
[dave@www.cndba.cn ~]# mdadm -S /dev/md5
mdadm: stopped /dev/md5
[dave@www.cndba.cn ~]#
或
rm /dev/md0
修改/etc/mdadm.conf、/etc/fstab等设置文件,把相关的地方去掉;最后,用fdisk对磁盘进行重新分区即可。
8 增长模式
用于增加磁盘,为阵列扩容:
选项:-G
将上述raid5的热备磁盘增加到阵列工作磁盘中
[dave@www.cndba.cn ~]# mdadm -G /dev/md5 -n 4
[dave@www.cndba.cn ~]# mdadm -D /dev/md5
/dev/md5:
Version : 1.2
Creation Time : Fri Jun 2 14:10:26 2023
Raid Level : raid5
Array Size : 2093056 (2044.00 MiB 2143.29 MB)
Used Dev Size : 1046528 (1022.00 MiB 1071.64 MB)
Raid Devices : 4
Total Devices : 4
Persistence : Superblock is persistent
Update Time : Fri Jun 2 14:52:50 2023
State : clean, reshaping
Active Devices : 4
Working Devices : 4
Failed Devices : 0
Spare Devices : 0
Layout : left-symmetric
Chunk Size : 512K
Consistency Policy : resync
Reshape Status : 28% complete
Delta Devices : 1, (3->4)
Name : dcm3:5 (local to host dcm3)
UUID : 883963c1:83ca52ae:86763222:8bbdc797
Events : 37
Number Major Minor RaidDevice State
0 8 80 0 active sync /dev/sdf
1 8 96 1 active sync /dev/sdg
3 8 112 2 active sync /dev/sdh
4 8 128 3 active sync /dev/sdi
[dave@www.cndba.cn ~]#
给raid 1增加2个硬盘:
[dave@www.cndba.cn ~]# mdadm -G /dev/md1 -n 3 -a /dev/sdj
mdadm: added /dev/sdj
raid_disks for /dev/md1 set to 3
[dave@www.cndba.cn ~]# mdadm -D /dev/md1
/dev/md1:
Version : 1.2
Creation Time : Fri Jun 2 14:03:34 2023
Raid Level : raid1
Array Size : 1046528 (1022.00 MiB 1071.64 MB)
Used Dev Size : 1046528 (1022.00 MiB 1071.64 MB)
Raid Devices : 3
Total Devices : 3
Persistence : Superblock is persistent
Update Time : Fri Jun 2 14:55:21 2023
State : clean
Active Devices : 3
Working Devices : 3
Failed Devices : 0
Spare Devices : 0
Consistency Policy : resync
Name : dcm3:1 (local to host dcm3)
UUID : 985dfa4e:c5f7a754:6bfc9d7e:ba1ac90d
Events : 61
Number Major Minor RaidDevice State
2 8 48 0 active sync /dev/sdd
1 8 64 1 active sync /dev/sde
3 8 144 2 active sync /dev/sdj
[dave@www.cndba.cn ~]#
[dave@www.cndba.cn ~]# mdadm -G /dev/md1 -n 4 -a /dev/sdk
mdadm: added /dev/sdk
raid_disks for /dev/md1 set to 4
[dave@www.cndba.cn ~]# mdadm -D /dev/md1
/dev/md1:
Version : 1.2
Creation Time : Fri Jun 2 14:03:34 2023
Raid Level : raid1
Array Size : 1046528 (1022.00 MiB 1071.64 MB)
Used Dev Size : 1046528 (1022.00 MiB 1071.64 MB)
Raid Devices : 4
Total Devices : 4
Persistence : Superblock is persistent
Update Time : Fri Jun 2 14:58:07 2023
State : clean
Active Devices : 4
Working Devices : 4
Failed Devices : 0
Spare Devices : 0
Consistency Policy : resync
Name : dcm3:1 (local to host dcm3)
UUID : 985dfa4e:c5f7a754:6bfc9d7e:ba1ac90d
Events : 83
Number Major Minor RaidDevice State
2 8 48 0 active sync /dev/sdd
1 8 64 1 active sync /dev/sde
3 8 144 2 active sync /dev/sdj
4 8 160 3 active sync /dev/sdk
[dave@www.cndba.cn ~]#
9 装配模式(重新启动RAID)
软RAID是基于系统的,当原系统损坏了,需要重新装配RAID
选项:-A
[dave@www.cndba.cn ~]# mdadm -A /dev/md1 /dev/sdd /dev/sde
mdadm: /dev/md1 has been started with 2 drives.
版权声明:本文为博主原创文章,未经博主允许不得转载。