1

I have a 3 RAID arrays on my Linux server.

root@master:~# lsblk -b
NAME          MAJ:MIN RM           SIZE RO TYPE  MOUNTPOINTS
loop0           7:0    0    21474836480  0 loop  
loop1           7:1    0    10737418240  0 loop  
sda             8:0    0   512110190592  0 disk  
├─sda1          8:1    0        1048576  0 part  
└─sda2          8:2    0   512107741184  0 part  
  └─md0         9:0    0   511972474880  0 raid1 /
sdb             8:16   0   512110190592  0 disk  
├─sdb1          8:17   0        1048576  0 part  
└─sdb2          8:18   0   512107741184  0 part  
  └─md0         9:0    0   511972474880  0 raid1 /
sdc             8:32   0  3000592982016  0 disk  
└─sdc1          8:33   0  3000591450112  0 part  
  └─md127       9:127  0  9001368551424  0 raid5 
    └─vg0-lv0 252:0    0 11001479036928  0 lvm   /mnt/bigdrive
sdd             8:48   0  4000787029504  0 disk  
├─sdd1          8:49   0  3000591450112  0 part  
│ └─md127       9:127  0  9001368551424  0 raid5 
│   └─vg0-lv0 252:0    0 11001479036928  0 lvm   /mnt/bigdrive
└─sdd2          8:50   0  1000193654784  0 part  
  └─md126       9:126  0  2000116776960  0 raid5 
    └─vg0-lv0 252:0    0 11001479036928  0 lvm   /mnt/bigdrive
sde             8:64   0  4000787030016  0 disk  
├─sde1          8:65   0  3000591450112  0 part  
│ └─md127       9:127  0  9001368551424  0 raid5 
│   └─vg0-lv0 252:0    0 11001479036928  0 lvm   /mnt/bigdrive
└─sde2          8:66   0  1000193654784  0 part  
  └─md126       9:126  0  2000116776960  0 raid5 
    └─vg0-lv0 252:0    0 11001479036928  0 lvm   /mnt/bigdrive
sdf             8:80   0  4000787030016  0 disk  
├─sdf1          8:81   0  3000591450112  0 part  
└─sdf2          8:82   0  1000193654784  0 part  

md0 is a mirror raid and is irrelevant to my problem.

My other 2 raids are RAID 5 arrays.

root@master:~# cat /proc/mdstat
Personalities : [raid1] [raid6] [raid5] [raid4] [raid0] [raid10] 
md126 : active raid5 sdd2[0] sde2[1]
      1953239040 blocks super 1.2 level 5, 512k chunk, algorithm 2 [3/2] [UU_]
      bitmap: 0/8 pages [0KB], 65536KB chunk

md127 : active raid5 sdd1[1] sde1[2] sdc1[0]
      8790398976 blocks super 1.2 level 5, 512k chunk, algorithm 2 [4/3] [UUU_]
      bitmap: 2/22 pages [8KB], 65536KB chunk

md0 : active raid1 sdb2[1] sda2[0]
      499973120 blocks super 1.2 [2/2] [UU]
      
unused devices: <none>

root@master:~# mdadm --detail /dev/md126
/dev/md126:
           Version : 1.2
     Creation Time : Tue May 21 19:23:02 2024
        Raid Level : raid5
        Array Size : 1953239040 (1862.75 GiB 2000.12 GB)
     Used Dev Size : 976619520 (931.38 GiB 1000.06 GB)
      Raid Devices : 3
     Total Devices : 2
       Persistence : Superblock is persistent

     Intent Bitmap : Internal

       Update Time : Tue Jun 11 19:07:38 2024
             State : clean, degraded 
    Active Devices : 2
   Working Devices : 2
    Failed Devices : 0
     Spare Devices : 0

            Layout : left-symmetric
        Chunk Size : 512K

Consistency Policy : bitmap

              Name : master:2  (local to host master)
              UUID : 266af7f7:8915136a:c8630864:2c514c63
            Events : 3201

    Number   Major   Minor   RaidDevice State
       0       8       50        0      active sync   /dev/sdd2
       1       8       66        1      active sync   /dev/sde2
       -       0        0        2      removed

root@master:~# mdadm --detail /dev/md127
/dev/md127:
           Version : 1.2
     Creation Time : Tue May 21 19:22:19 2024
        Raid Level : raid5
        Array Size : 8790398976 (8.19 TiB 9.00 TB)
     Used Dev Size : 2930132992 (2.73 TiB 3.00 TB)
      Raid Devices : 4
     Total Devices : 3
       Persistence : Superblock is persistent

     Intent Bitmap : Internal

       Update Time : Tue Jun 11 19:05:40 2024
             State : clean, degraded 
    Active Devices : 3
   Working Devices : 3
    Failed Devices : 0
     Spare Devices : 0

            Layout : left-symmetric
        Chunk Size : 512K

Consistency Policy : bitmap

              Name : master:1  (local to host master)
              UUID : cb895040:3fa006aa:88ea4323:15194452
            Events : 25871

    Number   Major   Minor   RaidDevice State
       0       8       33        0      active sync   /dev/sdc1
       1       8       49        1      active sync   /dev/sdd1
       2       8       65        2      active sync   /dev/sde1
       -       0        0        3      removed

As seen above, one array has 4 drives and one has 3. Both arrays are degraded with one drive missing each.

I have inserted a new drive "/dev/sdf" that has 2 partitions that exactly matches the sizes of the missing "drives".

The status of the 2 arrays looks identical to me.

Now, when I try to add the new/replaced drives to the 2 arrays, only one works (!!!).

root@master:~# mdadm --manage /dev/md127 --add /dev/sdf1
mdadm: add new device failed for /dev/sdf1 as 4: Invalid argument
root@master:~# mdadm --manage /dev/md126 --add /dev/sdf2
mdadm: added /dev/sdf2

I am even able to add the "/dev/sdf1" drive to the "/dev/md126" array (even though it's too big), so there is no problem with the drive itself.

I am really at a loss here. No idea why "/dev/127" refuses to accept the drive and throws "Invalid argument" without any further details to why...

UPDATE:

root@master:~# cat /sys/block/md126/md/dev-*/state | xargs
in_sync in_sync
root@master:~# cat /sys/block/md127/md/dev-*/state | xargs
in_sync in_sync in_sync

In dmesg, I see this error. No idea why it thinks it would have a superblock already as I am adding a new drive:

md: sdf1 does not have a valid v1.2 superblock, not importing!
md: md_import_device returned -22
17
  • 1/2 What do you get for the problematic RAID arrays cat /sys/block/md126/md/dev-*/state | xargs and cat /sys/block/md127/md/dev-*/state | xargs? Please add these to your question Commented Jun 11, 2024 at 18:32
  • 2/2 If you've got missing as one of the parameters you need to replace it in the approprate dev-* subdirectory with remove. For example echo remove >/sys/block/md126/md/dev-YYY/state Commented Jun 11, 2024 at 18:32
  • Actually, I see this in the logs: md: sdf1 does not have a valid v1.2 superblock, not importing! Commented Jun 11, 2024 at 18:37
  • But now the drive crashed with lots of I/O errors, so I guess my new drive is simply faulty :( Commented Jun 11, 2024 at 18:37
  • @ChrisDavies I have updated the question is the data you requested. I am now using yet another new drive and still get the same problem. Commented Jun 13, 2024 at 11:17

0

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.