I’ve got a 4-bay QNAP configured in passthrough mode. I got once drive cleared of and now it’s holding the dumps. btrfs RAID 1, with deduplication because there’s a lot of overlap between the dumps. As I clear & verify more drives I’ll chuck them into the array.
score 1 · canonical
I’m working through ~14 old hard drives I’ve accumulated over the years from various computer upgrades and decommissioned home NASes.
Each one is getting dumped, merged, and pruned until I have exactly one current, unified file tree representing all of my data. No more “old-w540-ubuntu-2017/” folders where I’m not sure if that’s the only copy of the data or not.
I’ve misplaced some data over the years, and my laptop drive is always full. I hope to address both problems.
score 1 · canonical
btrfs deduplication helps cut down on intermediate storage needs while I get all the file trees in one place so I can compare them.
Comparison is tricky since for trees evolve over time. I need to detect when two folders at arbitrary places in the file tree have highly similar content. I’ll roll up arrays of rapidhash values and throw each hierarchy level at Levenstein and see how far that gets me.
But I’m also going to try directly accessing the btrfs block identity data instead of doing a manual scan.
score 2 · canonical