Although the system itself stated that the array functions normally, DiskSMART for Disk 2 was giving yellow warning at Reallocated_Sector_Ct line:
Value=100
Worst Value=100
Threshold=010
Raw value=8
I have replaced Disk 2 with ST6000VX0023, it began to rebuild and then got stuck at around 99.99%:
QNAP report on rebuild process
This time Disk 1 began showing warnings. Having assumed that new Disk 2 is somewhat wrong (it was 6Tb as other disks, but different type as I could not source the original ST6000NM0024). I made another attempt replacing Disk 2 with still other 6Tb. This ended up in two disks failure, Disk 1 did not survive second rebuild. Being mostly Windows user, I did not have ddrescue readily available (I didn’t even know such a tool exists), so I built an Ubuntu machine out of a Dell desktop I found in the attic. Disk 2 was copied to a new drive using commands:
Phase 1
sudo ddrescue --verbose --idirect --no-scrape -f /dev/sda /dev/sdc disk2.logfile
Phase 2
sudo ddrescue --verbose --idirect -r3 --no-scrape -f /dev/sda /dev/sdc disk2.logfile
Phase 3
sudo ddrescue --verbose --idirect -r10 -R -f /dev/sda /dev/sdc disk2.logfile
As far as I remember it recovered 99.99% of data. So I turned to the remains of Disk 1, and this time I confused source with destination, i.e. I have written about 6Gb of useless data to Disk 1. Now prerequisites are:
- I have no backup whatsoever
- I have partially ddrescued Drive 2
- Total array had a size of 24Tb, I need to recover probably 0.5Tb, unlikely more, mostly documents and photos
Disks 1 and 2 are physically failing, there were no warnings of filesystem problems. Although none of Disks 3, 4 and 5 showed any physical problem, I would guess they might be nearing their death, as they made similar to Disks 1 and 2 number of revolutions.
Actions I am planning to take:
- ddrescue whatever remained from Disk 1;
- ddrescue whatever remained from Disk 2. Here I have two source disks: initially failed original Disk 2, and new disk (say Disk 2a) that was partially rebuilt by QNAP, and a destination Disk 2 that was ddrescued from originally failed Disk 2. Is there any merging functionality in ddrescue? I mean is there any way I can enrich destination disk 2 with the one rebuilt by QNAP? If at all feasible;
- (may be) ddrescue Disk 3 through 5;
- Manually reassemble RAID 5 on the Ubuntu machine (dmsetup), have no idea how to do that;
- Attempt to recover data, at least partially, using testdisk/photorec, or whatever tools are out there.
So primary objective to get my docs/photos back. I understand that chances are slim. Questions:
- Any advice on plan items 4 and 5 are appreciated;
- I should probably not attempt to use QNAP for recovery, right?
- As my lab mouse Dell has only three SATA ports, I will need to add a PCI card extension, making sure the hardware is supported by Ubuntu (sanity check).
