Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
Any experience of dm-integrity ?
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Kernel & Hardware
View previous topic :: View next topic  
Author Message
Goverp
l33t
l33t


Joined: 07 Mar 2007
Posts: 837

PostPosted: Fri May 08, 2020 6:47 pm    Post subject: Any experience of dm-integrity ? Reply with quote

I'm about to upgrade my HDD stack; the plan is to
  1. dump old disks
  2. replace disks
  3. run smart transit check
  4. gpt partition
  5. create RAID array (actually 2, one for "spare" - disk is cheap)
  6. gpt partition the resulting device (my old arrangement, has served me well)
  7. mkfs
  8. restore data


I see there's a relatively new device mapper DM-Integrity target, loosely documented in the wiki and kernel docs. I say new, it seems to have been around quite some time, but not been used very much. IIUC, I'd put it below the RAID array, so that if one disk returns bad data, RAID will choose the result from the other. I'd probably use the bitmap option and rely on RAID to sort out problems.

Does anyone have any experience of using DM-Integrity? I like the idea of integrity, but not bleeding edge, and it looks a bit bloody ...

<edit 09/05/2020>
OK, so far so quiet. I'll edit the question, so as not to loose my valued "unanswered" status.

I've found someone doing tests, which is good.
They show virtually no read impact, but quite a hit on write speed. I suspect that's partly down to using journals for integrity, rather than bitmap. I'll try some tests on one drive. At least they show a similar setup (even a useless bash script) to my thinking - except I'm skipping crypto.
_________________
Greybeard
Back to top
View user's profile Send private message
Goverp
l33t
l33t


Joined: 07 Mar 2007
Posts: 837

PostPosted: Sat May 09, 2020 5:03 pm    Post subject: Reply with quote

Neddy Seagoon's comment makes the point that DM-Integrity and its ilk are really only useful in a limited number of circumstances - so far I can only think of two:
  1. transmission errors between a drive and the computer; and
  2. rogue applications writing directly to block devices, bypassing the RAID level

A drive's bit rot will be caught by its own hardware.

So, given the simple fact that preparing a 5 disk RAID array of 1TB disks for DM-Integrity will take writing about 10 hours (format writes 0s to the whole disk; at about 150 MB/sec, which is realistic for spinning rust, that's about 2 hours per disk), and the extra complexity only really adds a warm feeling, I'll stick to RAID 10 by itself.

Here's another article on the subject that makes interesting reading, though I don't agree with the conclusions - the test was a bit flawed, and the recommended FS isn't part of the standard kernel.
_________________
Greybeard
Back to top
View user's profile Send private message
Goverp
l33t
l33t


Joined: 07 Mar 2007
Posts: 837

PostPosted: Sun May 10, 2020 6:49 pm    Post subject: Some dodgy numbers Reply with quote

My kit now has a 5 disk RAID10 array, "near offset" layout, no DM-integrity, just RAID, formatted with a big ext4 partition. The disks are 1TB Toshiba P300.

I tried copying the performance test "methodology" in the first reference above.
Code:
# write:
echo 3 > /proc/sys/vm/drop_cache  # drop all caches
dd if=/dev/zero of=/path/to/testfile bs=1M count=8000 conv=fdatasync

# read
echo 3 > /proc/sys/vm/drop_cache  # drop all caches
dd if=/path/to/testfile of=/dev/zero conv=fdatasync

It's clearly not sorted, as I got steadily increasing performance on my system; I presume echo 3 > /proc/sys/vm/drop_caches is nowhere near good enough to clear all the relevant caches (not least the hard drives themselves).

My own tests with a real file were:
  1. copying a 20 GB file to /dev/null, which achieved 377 MB/s first time, 421 MB/s second time;
  2. copying 80 GB from /dev/zero to disk, which claimed to achieve 1 GB/s; I don't believe it; and
  3. copying the first 20 GB file to a new file; first test achieved 465 MB/s, 449 MB/s second time.

Whatever the validity of the tests, they're all a heck of a lot faster than in the first reference.
If I had nothing better to do, I might download something serious like phoronix test suite, but that would really only for claiming ricer points...
_________________
Greybeard
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Kernel & Hardware All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum