Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
Reiser4, gzip or lzo compression?
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Unsupported Software
View previous topic :: View next topic  
Author Message
Need4Speed
Guru
Guru


Joined: 06 Jun 2004
Posts: 497

PostPosted: Thu May 01, 2008 8:39 pm    Post subject: Reiser4, gzip or lzo compression? Reply with quote

After looking at the benchmarks of reiser4 with compression I can't resist upgrading to reiser4 anymore. I understand gzip provides better compression than lzo at the cost of cpu usage. What are your thoughts on gzip vs lzo on an average modern desktop?
Back to top
View user's profile Send private message
dusanc
Apprentice
Apprentice


Joined: 19 Sep 2005
Posts: 248
Location: Serbia

PostPosted: Thu May 01, 2008 8:42 pm    Post subject: Reply with quote

I use LZO1 because Edward Shiskin recomended it to me cause it has slightly lower copmpression on some types of data, and much smaller CPU requirements.
Back to top
View user's profile Send private message
kernelOfTruth
Watchman
Watchman


Joined: 20 Dec 2005
Posts: 6111
Location: Vienna, Austria; Germany; hello world :)

PostPosted: Thu May 01, 2008 8:45 pm    Post subject: Reply with quote

dusanc wrote:
I use LZO1 because Edward Shiskin recomended it to me cause it has slightly lower copmpression on some types of data, and much smaller CPU requirements.


++

you could safe several hundreds of megabytes by using gzip (ideally for /usr/portage) in favor of lzo but the disadvantage would be the much higher cpu usage

I use lzo for / and gzip for /usr/portage

default reiser4 for data partitions ...
_________________
https://github.com/kernelOfTruth/ZFS-for-SystemRescueCD/tree/ZFS-for-SysRescCD-4.9.0
https://github.com/kernelOfTruth/pulseaudio-equalizer-ladspa

Hardcore Gentoo Linux user since 2004 :D
Back to top
View user's profile Send private message
Need4Speed
Guru
Guru


Joined: 06 Jun 2004
Posts: 497

PostPosted: Thu May 01, 2008 8:56 pm    Post subject: Reply with quote

So does storing data like mp3's and movies work worse with compression or does it just not provide any benefits?
Back to top
View user's profile Send private message
kernelOfTruth
Watchman
Watchman


Joined: 20 Dec 2005
Posts: 6111
Location: Vienna, Austria; Germany; hello world :)

PostPosted: Thu May 01, 2008 8:59 pm    Post subject: Reply with quote

Need4Speed wrote:
So does storing data like mp3's and movies work worse with compression or does it just not provide any benefits?


it doesn't work worse, but since mp3s and movies are already compressed there's little to no advantage for those kind of files

it (subjectively) also takes longer
_________________
https://github.com/kernelOfTruth/ZFS-for-SystemRescueCD/tree/ZFS-for-SysRescCD-4.9.0
https://github.com/kernelOfTruth/pulseaudio-equalizer-ladspa

Hardcore Gentoo Linux user since 2004 :D
Back to top
View user's profile Send private message
dusanc
Apprentice
Apprentice


Joined: 19 Sep 2005
Posts: 248
Location: Serbia

PostPosted: Fri May 02, 2008 1:19 pm    Post subject: Reply with quote

Reiser4 evaluates on first use if data is compressable. If it isn't it doesn't try to compress it again.
It would be interesting if someone could post some numbers about "slower" cryptocompress r4 against plain one.
There shouldn't be any differences as speed bottleneck is later down the path AFAIK and cryptocompress should be same or better in all situations.
Back to top
View user's profile Send private message
kernelOfTruth
Watchman
Watchman


Joined: 20 Dec 2005
Posts: 6111
Location: Vienna, Austria; Germany; hello world :)

PostPosted: Fri May 02, 2008 1:22 pm    Post subject: Reply with quote

dusanc wrote:
Reiser4 evaluates on first use if data is compressable. If it isn't it doesn't try to compress it again.
It would be interesting if someone could post some numbers about "slower" cryptocompress r4 against plain one.
There shouldn't be any differences as speed bottleneck is later down the path AFAIK and cryptocompress should be same or better in all situations.


it highly dependens how you set up the cryptcompress plugin during format

there is compressMode=conv, compressMode=latt, compressMode=force, compressMode=ultim

compressMode=force should give the worst performance (?)

if you set up
Code:
mkfs.reiser4 -o create=ccreg40,compress=gzip1
or
Code:
mkfs.reiser4 -o create=ccreg40,compress=lzo1
it shouldn't be much worse than default reiser4, perhaps even better

Quote:
compression_mode

Points to a compression mode plugin which defines some heuristic for data compression. Currently this is a set of handlers called by deflate manager in flush time. Properly assigned compression mode allows to reduce cpu and memory usage when handling incompressible data.

List of currently possible values (label/description):

*

"ultim", "Check ultimately". In this mode compression will be "turned off" forever as soon as deflate manager meet incompressible page cluster.
*

"latt", "Check on dynamic lattice". This is adaptive mode that manage special per-inode compression toggle. This toggle shows last compression experience: ->discard_hook() sets it to 0; ->accept_hook() sets it to 1. ->should_compress() first looks at this toggle. If it is 1, then ->should_compress() advises to compress current logical cluster. Otherwise ->should_compress() advises to compress it only in the case when its index belongs to the lattice of variable factor X (X = 2, 4, ..., 32) which get increased after each unsuccessful experience (successful one returns it back to minimal value).
*

"conv", "Convert to extent". This mode activates a special hook in reiser4 context of sys_write() (when writing from special offset in some write iteration) which tries to estimate whether a file is compressible by testing its first logical cluster (64K by default). Such evaluation is performed only once per file's life. If evaluation result is negative, then fragments will be converted to extents, and management will be passed to unix-file plugin. Back conversion does not take place. If evaluation result is positive, then file remains under cryptcompress plugin control, but compression mode plugin will be converted to "latt", i.e all future decisions about compressibility will be in the competence of the flush algorithm.
*

"force", "Compress everything". In this mode deflate manager is forced to compress every logical cluster (unless it is too small).
*

"none", "Do not compress". In this mode deflate manager does not invoke compression.

Default value is "conv".

_________________
https://github.com/kernelOfTruth/ZFS-for-SystemRescueCD/tree/ZFS-for-SysRescCD-4.9.0
https://github.com/kernelOfTruth/pulseaudio-equalizer-ladspa

Hardcore Gentoo Linux user since 2004 :D
Back to top
View user's profile Send private message
dusanc
Apprentice
Apprentice


Joined: 19 Sep 2005
Posts: 248
Location: Serbia

PostPosted: Fri May 02, 2008 1:27 pm    Post subject: Reply with quote

Ofcourse :)
I was talking about default behavior as people usually don't (and IMHO shouldn't :) )tinker too much with those options.
Back to top
View user's profile Send private message
MostAwesomeDude
Guru
Guru


Joined: 12 Aug 2007
Posts: 373

PostPosted: Fri May 02, 2008 2:52 pm    Post subject: Reply with quote

I find that LZMA works wonders on /usr/portage .
_________________
Don't believe the "n00b" under my name.
Back to top
View user's profile Send private message
Need4Speed
Guru
Guru


Joined: 06 Jun 2004
Posts: 497

PostPosted: Fri May 02, 2008 3:19 pm    Post subject: Reply with quote

MostAwesomeDude wrote:
I find that LZMA works wonders on /usr/portage .

Does standard reiser4 support LZMA? What's the format command?
I know gzip worked really well too. It decreased my ~600mb portage sparse file to ~90mb! :D
Back to top
View user's profile Send private message
energyman76b
Advocate
Advocate


Joined: 26 Mar 2003
Posts: 2045
Location: Germany

PostPosted: Fri May 02, 2008 7:01 pm    Post subject: Reply with quote

I tried both and the differences on a several gb /var (which includes portage) was not that big. So I'd say go with lzo
_________________
Study finds stunning lack of racial, gender, and economic diversity among middle-class white males

I identify as a dirty penismensch.
Back to top
View user's profile Send private message
MostAwesomeDude
Guru
Guru


Joined: 12 Aug 2007
Posts: 373

PostPosted: Fri May 02, 2008 9:09 pm    Post subject: Reply with quote

Need4Speed wrote:
MostAwesomeDude wrote:
I find that LZMA works wonders on /usr/portage .

Does standard reiser4 support LZMA? What's the format command?
I know gzip worked really well too. It decreased my ~600mb portage sparse file to ~90mb! :D


Nope, but squashfs does, and reduced my portage to ~35MB.
_________________
Don't believe the "n00b" under my name.
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Unsupported Software All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum