Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
Question about RAID performance
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Kernel & Hardware
View previous topic :: View next topic  
Author Message
allistarM
Tux's lil' helper
Tux's lil' helper


Joined: 22 Jul 2004
Posts: 84

PostPosted: Tue May 19, 2020 9:57 pm    Post subject: Question about RAID performance Reply with quote

Hi there,
I have an Asus Prime 390A, i7-9700 with 32Gb of RAM. The m/b has 6 SATA headers and I have a PCI SATA card that adds 4 more. I have 9 disks of varying sizes. Because of the different disk sizes I have multiple RAID arrays spread across the disks. Here's my current setup:

md1 : active raid1 sdb1[0] sdf1[5] sda1[4] sdc1[3] sdg1[1] sdh1[2]
md2 : active raid5 sda3[5] sdf3[1] sdb3[0] sdh3[3] sdc3[6] sdg3[2]
md3 : active raid10 sda4[2] sdf4[4] sdb4[3] sdc5[5]
md4 : active raid1 sdi1[1] sdd1[0]

These array are used for:

/ is not on RAID, it's on an SSD but is backed up daily. I figured I want this on a fast disk. This means my own home dir is also here. The other disks are spinning platters.
/md1, RAID1, 1.2Gb = /boot. Not normally mounted. I've spread this across 6 disks so grub never has an issue finding it. I know this probably isn't necessary but it's saved me in the past.
/md2, RAID5, 1.2Tb = /home/storage. This is RAID 5 to give me maximum storage. from the disks I have.
/md3, RAID10, 128G = /home/system. This is where /var, /usr/portage lives.
/md4, RAID1, 3Tb = /home/storage3. Plex files. Ahem.

Is there a performance hit in having a single disk (say, /dev/sda) contain portions of multiple RAID arrays (/dev/sda1 is in md1, /dev/sda3 is in md2, /dev/sda4 is in md3). Will such a system cause the different arrays to compete with each other and cause bottlenecks?

I know when a disk fails it will be a pain to get this all going again. I'm using mdadm.

Thanks,
Allistar.
Back to top
View user's profile Send private message
eccerr0r
Watchman
Watchman


Joined: 01 Jul 2004
Posts: 7446
Location: almost Mile High in the USA

PostPosted: Wed May 20, 2020 1:05 am    Post subject: Reply with quote

Yes whenever you share a disk to multiple requestors you'll definitely get a performance drop. However if only one requestor uses resource/array at a time, it will work as usual.

Doesn't matter if it's an array or not shared on a disk though you can get "lucky" if the resources are on the same volume (shared cache preload, potential locality hits, etc.)

I have only 3 disks that share a RAID1 and RAID5. The RAID1 is rarely if ever used, so that's thrown out of the equation. The RAID5 is partitioned (by LVM) which hurts cache locality - no different than if I had multiple RAID5s on the disk set - but it generally still provides acceptable performance for me at least.

Note that the md driver does notice that multiple arrays are on a specific disk and will serialize things such as resyncs and rebuilds when it detects sharing, so at least that isn't doing the worst possible method.
_________________
Intel Core i7 2700K@ 4.1GHz/HD3000 graphics/8GB DDR3/180GB SSD
What am I supposed watching?
Back to top
View user's profile Send private message
allistarM
Tux's lil' helper
Tux's lil' helper


Joined: 22 Jul 2004
Posts: 84

PostPosted: Wed May 20, 2020 1:48 am    Post subject: Reply with quote

Thanks for the explanation.
I was running VirtualBox on the RAID5 and this worked well (and outperformed my "real" Surface Pro) but I noticed when doing a lot of writes to this array (it's also got the root dir of an FTP server) it killed the VirtualBox performance. I then moved VirtualBox to a RAID 10 and performance seems better.
Back to top
View user's profile Send private message
eccerr0r
Watchman
Watchman


Joined: 01 Jul 2004
Posts: 7446
Location: almost Mile High in the USA

PostPosted: Wed May 20, 2020 3:18 am    Post subject: Reply with quote

Generally RAID10 has better performance than RAID5 especially without tuning.
But regardless if two servers are sharing different RAIDs on the same disk, performance will tank. The optimum is that each server has their own dedicated spindle(s). If you can keep each server from not touching the other, your performance will go up.

Even having two VM/disk consumers on the same RAID5 operating on different data sets tanks performance, at least on my system. Expected. By the nature of RAID5 where data is spread among all spindles, there's no guarantee any one VM will use one set of disks. In a RAID1+0 you can get lucky and the two VMs have their own spindle but even then it's very conditional.

I had one of those "this raid, that disk, all protected" setup you have at one point. It works okay if you only have one consumer. Once you have multiple consumers/VMs/etc. trying to share, performance will tank badly.

Best you can do is give io priorities to specific VM/servers but it's tough to gauge which one really should get higher priority...
_________________
Intel Core i7 2700K@ 4.1GHz/HD3000 graphics/8GB DDR3/180GB SSD
What am I supposed watching?
Back to top
View user's profile Send private message
allistarM
Tux's lil' helper
Tux's lil' helper


Joined: 22 Jul 2004
Posts: 84

PostPosted: Wed May 20, 2020 3:39 am    Post subject: Reply with quote

Thanks. I've decide to buy a 500Gb SSD and dedicate that to the virtual machine. I can pick one up for about $100 so it's not too expensive. Even though the current VM performs well on a spinning platter RAID10 I think it will perform better on an SSD.
Back to top
View user's profile Send private message
eccerr0r
Watchman
Watchman


Joined: 01 Jul 2004
Posts: 7446
Location: almost Mile High in the USA

PostPosted: Wed May 20, 2020 3:19 pm    Post subject: Reply with quote

You can share an SSD among two servers just fine too unlike HDDs. No need to dedicate.
_________________
Intel Core i7 2700K@ 4.1GHz/HD3000 graphics/8GB DDR3/180GB SSD
What am I supposed watching?
Back to top
View user's profile Send private message
allistarM
Tux's lil' helper
Tux's lil' helper


Joined: 22 Jul 2004
Posts: 84

PostPosted: Wed May 27, 2020 9:27 am    Post subject: Reply with quote

As expected the VM performance is much better when run on an SSD. The raw read and write rates for the VM are about the same as on the RAID10 spinning disks but the access times are drastically faster. I can't fit any more drives in this box now, I've got 10. sda through to sdj.
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Kernel & Hardware All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum