Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
FlashCache - SSD based cache for speeding up hard drives
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Unsupported Software
View previous topic :: View next topic  
Author Message
drescherjm
Advocate
Advocate


Joined: 05 Jun 2004
Posts: 2779
Location: Pittsburgh, PA, USA

PostPosted: Sun Nov 07, 2010 7:09 pm    Post subject: FlashCache - SSD based cache for speeding up hard drives Reply with quote

https://github.com/facebook/flashcache

http://www.mysqlperformanceblog.com/2010/05/10/flashcache-first-experiments/

http://www.facebook.com/note.php?note_id=388112370932

Has anyone tried this? I am on vacation in a week so I can not order an additional SSD so I doubt I will be able to test before I get back. Just wondering if anyone tried it and are there any ebuild. If not when I get back I may try ordering a $110 used 80GB Intel SSD and see how much that can speed up my disks.
_________________
John

My gentoo overlay
Instructons for overlay
Back to top
View user's profile Send private message
Shining Arcanine
Veteran
Veteran


Joined: 24 Sep 2009
Posts: 1110

PostPosted: Sun Nov 07, 2010 8:47 pm    Post subject: Reply with quote

You would probably be better off doing things yourself with a NAND Flash boot drive. Your system will be slower as long as your boot drive is a mechanical disk and adding another layer of overhead is not going to be much help.
Back to top
View user's profile Send private message
yoshi314
l33t
l33t


Joined: 30 Dec 2004
Posts: 848
Location: PL

PostPosted: Sun Nov 07, 2010 8:47 pm    Post subject: Reply with quote

afaik there are already sata drives with ssd cache coming out, so the idea must be pretty solid.
_________________
~amd64
shrink your /usr/portage with squashfs+aufs
Back to top
View user's profile Send private message
drescherjm
Advocate
Advocate


Joined: 05 Jun 2004
Posts: 2779
Location: Pittsburgh, PA, USA

PostPosted: Sun Nov 07, 2010 9:14 pm    Post subject: Reply with quote

Boot times are not a problem since I reboot less than 1 time per month..

I have an 80GB SSD on one of my gentoo boxes (not the one I want to try this on though) but I would like to speed up my KVM virtual machines so that it does not take forever to build 3 million lines of code under a different OS. The only problem is I would need a 256GB ssd to even begin to fit the VM since the build drive is over 100GB and that does not include the OS.
_________________
John

My gentoo overlay
Instructons for overlay
Back to top
View user's profile Send private message
drescherjm
Advocate
Advocate


Joined: 05 Jun 2004
Posts: 2779
Location: Pittsburgh, PA, USA

PostPosted: Sun Nov 07, 2010 9:18 pm    Post subject: Reply with quote

yoshi314 wrote:
afaik there are already sata drives with ssd cache coming out, so the idea must be pretty solid.


In addition to that there is a $1000 US RAID controller with a buitin Intel X25-E that uses the ssd as a cache.

http://www.theregister.co.uk/2009/09/09/adaptec_maxiq/
_________________
John

My gentoo overlay
Instructons for overlay
Back to top
View user's profile Send private message
Shining Arcanine
Veteran
Veteran


Joined: 24 Sep 2009
Posts: 1110

PostPosted: Sun Nov 07, 2010 9:32 pm    Post subject: Reply with quote

yoshi314 wrote:
afaik there are already sata drives with ssd cache coming out, so the idea must be pretty solid.


It is a great way to part people from their money. Unfortunately, parting people from their money does not require solid ideas. It just requires that you convince them to make a purchase.

There have been benchmarks of hybrid drives at Phoronix and the performance is not much better than a regular mechnical hard drive. One problem is knowing what programs need what files and without any conception of what processes are running and access logs for each one, there is no way to know which files need to be loaded at what time. A much more fundamental problem is that these drives have write limits and using them as cache means that the write amplification these drives will encounter is potentially astronomical. They will go long before the mechanical drives.


Last edited by Shining Arcanine on Sun Nov 07, 2010 9:36 pm; edited 1 time in total
Back to top
View user's profile Send private message
drescherjm
Advocate
Advocate


Joined: 05 Jun 2004
Posts: 2779
Location: Pittsburgh, PA, USA

PostPosted: Sun Nov 07, 2010 9:36 pm    Post subject: Reply with quote

I agree that the 1 drive that does this is not that great of an advancement. It was never meant to be a desktop replacement. It does make a good laptop drive if you need more than 80GB and can not afford a large ssd.
_________________
John

My gentoo overlay
Instructons for overlay
Back to top
View user's profile Send private message
drescherjm
Advocate
Advocate


Joined: 05 Jun 2004
Posts: 2779
Location: Pittsburgh, PA, USA

PostPosted: Sat Nov 27, 2010 5:28 am    Post subject: Reply with quote

I put up an ebuild in my overlay.

https://github.com/drescherjm/jmdgentoooverlay/tree/master/sys-fs/flashcache/

WARNING: I have not tested this yet. Waiting on a new 120GB SSD.
_________________
John

My gentoo overlay
Instructons for overlay
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 43562
Location: 56N 3W

PostPosted: Sat Nov 27, 2010 10:33 am    Post subject: Reply with quote

drescherjm,

Even a SSD is slow compared to a RAM cache. As an example, I find that building OOo with /var/tmp/portage on /dev/shm gives almost no speedup over /var/tmp/portage on raid5. I attribute that to all the built binaries staying in RAM anyway.

Such huge caches go against the the ideal of cache. The idea is a small amount of fast storage to mimic the area(s) of frequently accessed slower storage.
The bigger the cache, the higher the hit ratio but 90% isn't difficult to achieve now. You have to add a lot more cache to make any improvement at all.

Without doing the statistics, I would be surprised if investing the same money in more RAM did not provide a better speed up than a SSD.
That is, the single slower fetch from a 'spinny' would be more than offset by the multiple fetches from RAM, rather than SSD.
I remain to be convinced.

Putting cache on a spinny is a good idea but SSD???
Thats a questionable design decision. Its slower than RAM (normally used for cache) and why do you need the data retention of SSD.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
drescherjm
Advocate
Advocate


Joined: 05 Jun 2004
Posts: 2779
Location: Pittsburgh, PA, USA

PostPosted: Sat Nov 27, 2010 2:35 pm    Post subject: Reply with quote

Quote:
Even a SSD is slow compared to a RAM cache.


I know it is more than 10 times slower. But an SSD is many times cheaper than ram per GB and there are limitations to how much ram I can put in my computer. My current motherboard has 6GB (I could use more) of ram and supports 24GB. With SSDs I have no such limit.

On top of this the cache is lazy write back. So in theory on my 2TB 5400 RPM disks I can get SSD random write speeds which are 10 to 100 times that of a disk without having to risk data loss.

There are expensive hardware raid controllers that do exactly this. Although they use the 32 GB Enterprise Intel SSD instead of the mainstream version.

I will try and see how well this works this week if I get my drive. If it does not I will probably need to buy more SSDs because the problem that I am trying to solve is poor random disk performance inside a windows VM (which drastically slows down my work) when run under gentoo. I could put the entire windows VM on an SSD or 2.
_________________
John

My gentoo overlay
Instructons for overlay
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Unsupported Software All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum