I have a 4 x 4 TB drive configured in a raid5 array.
Drives labeled as /dev/sdb1, /dev/sdc1, /dev/sdd1 & /dev/sde1 assembled as /dev/md0
mdadm assemble output
When I tried to assemble the array I get:
sudo mdadm --assemble /dev/md0
mdadm: failed to RUN_ARRAY /dev/md0: Invalid argument
dmesg -T output
The output of interest:
[Sun Dec 27 00:15:35 2020] md/raid:md0: device sdc1 operational as raid disk 0
[Sun Dec 27 00:15:35 2020] md/raid:md0: device sdd1 operational as raid disk 3
[Sun Dec 27 00:15:35 2020] md/raid:md0: device sdb1 operational as raid disk 2
[Sun Dec 27 00:15:35 2020] md/raid:md0: device sde1 operational as raid disk 1
[Sun Dec 27 00:15:35 2020] md/raid:md0: raid level 5 active with 4 out of 4 devices, algorithm 2
[Sun Dec 27 00:15:35 2020] md0: invalid bitmap file superblock: bad magic
[Sun Dec 27 00:15:35 2020] md0: failed to create bitmap (-22)
[Sun Dec 27 00:15:35 2020] md: md0 stopped.
mdadm examine output
All the drives seem normal to me with the exception of /dev/sdc1:
/dev/sdc1:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x0
Array UUID : 1546e1f3:18b8a99f:a6b9e7b9:f5ca4949
Name : server:0 (local to host server)
Creation Time : Mon Nov 19 17:36:22 2018
Raid Level : raid5
Raid Devices : 4
Avail Dev Size : 7813769216 (3725.90 GiB 4000.65 GB)
Array Size : 11720653824 (11177.69 GiB 12001.95 GB)
Data Offset : 264192 sectors
Super Offset : 8 sectors
Unused Space : before=264112 sectors, after=0 sectors
State : clean
Device UUID : 3cb78503:731a0dcb:8afc96d0:9a801d67
Update Time : Thu Dec 10 17:35:57 2020
Bad Block Log : 512 entries available at offset 24 sectors
Checksum : 4c2657ea - expected 4c2657e9
Events : 10242
Layout : left-symmetric
Chunk Size : 128K
Device Role : Active device 0
Array State : AAAA ('A' == active, '.' == missing, 'R' == replacing)
The checksum shows 4c2657ea - expected 4c2657e9
My thoughts
The checksum discrepancy is so similar that it makes me believe that a bit was flipped on my primary drive since any change in the 4TB drive would have changed the checksum dramatically. Thus, I think that the drive is OK and I've checked it with Ubuntu Disk utility (both SMART data and self-test) and using bad blocks. I've also checked it in windows using the Western Digital diagnostic tool. They all give it a pass.
My question is how do I go about restoring the array? Is it possible to mdadm to correct this? Or do I have to remove and add another drive to the array?
Thanks for your help in advance!