Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
ReiserFS tuning thread, the mother of all "ricer" threads ;)
View unanswered posts
View posts from last 24 hours

Goto page Previous  1, 2, 3, 4, 5  Next  
Reply to topic    Gentoo Forums Forum Index Unsupported Software
View previous topic :: View next topic  
Author Message
DigitalCorpus
Apprentice
Apprentice


Joined: 30 Jul 2007
Posts: 245

PostPosted: Fri Oct 02, 2009 7:00 pm    Post subject: Reply with quote

This is probably why I experience a massive slow down in my workflow when trying to test BFS then. I was hoping that Reiser4 would be introduced into the mainline by now as predicted at the beginning of summer.
*crosses fingers*
btrfs is similar in performance, but still doesn't match R4 for small files.
_________________
Atlas (HDTV PVR, HTTP & Media server)
http://atlas.selfip.net/Info/
Back to top
View user's profile Send private message
kernelOfTruth
Watchman
Watchman


Joined: 20 Dec 2005
Posts: 6108
Location: Vienna, Austria; Germany; hello world :)

PostPosted: Fri Oct 02, 2009 7:10 pm    Post subject: Reply with quote

zen-sources bug-tracker: "Reiser4 errors"
_________________
https://github.com/kernelOfTruth/ZFS-for-SystemRescueCD/tree/ZFS-for-SysRescCD-4.9.0
https://github.com/kernelOfTruth/pulseaudio-equalizer-ladspa

Hardcore Gentoo Linux user since 2004 :D
Back to top
View user's profile Send private message
energyman76b
Advocate
Advocate


Joined: 26 Mar 2003
Posts: 2045
Location: Germany

PostPosted: Sat Oct 03, 2009 5:43 am    Post subject: Reply with quote

'normal' sources are ok. So it is either a bug with zen or a bug in mmotm-reiser4 which is AFAIK pretty broken.
_________________
Study finds stunning lack of racial, gender, and economic diversity among middle-class white males

I identify as a dirty penismensch.
Back to top
View user's profile Send private message
seqizz
Tux's lil' helper
Tux's lil' helper


Joined: 14 Jan 2008
Posts: 103

PostPosted: Sat Oct 03, 2009 6:03 pm    Post subject: Reply with quote

unK wrote:
seqizz wrote:
i don't know why this is happening, but since i've started to using reiser4 for my root folder (/) , i'm taking some lockups when using resources heavily. Also i made a change on fs (e.g edit a file) and after one reboot, it's turning back to old one, like i've never touched the file.. (wtf? :mrgreen: ) i think i should change my mount options, btw i'm using KernelOfTruth's last recipe..

what kernel do you use? I once had such behaviour with zen-sources.


Using 2.6.31-zen1..
_________________
what about an answer here?
Back to top
View user's profile Send private message
energyman76b
Advocate
Advocate


Joined: 26 Mar 2003
Posts: 2045
Location: Germany

PostPosted: Sat Oct 03, 2009 6:13 pm    Post subject: Reply with quote

seqizz wrote:
unK wrote:
seqizz wrote:
i don't know why this is happening, but since i've started to using reiser4 for my root folder (/) , i'm taking some lockups when using resources heavily. Also i made a change on fs (e.g edit a file) and after one reboot, it's turning back to old one, like i've never touched the file.. (wtf? :mrgreen: ) i think i should change my mount options, btw i'm using KernelOfTruth's last recipe..

what kernel do you use? I once had such behaviour with zen-sources.


Using 2.6.31-zen1..


seriously, stay away from that. Reiser4 in mmotm is broken. They use reiser4 from mmotm. Use vanilla-sources, patch them with reiser4-for-2.6.30 and you won't have any problems.
_________________
Study finds stunning lack of racial, gender, and economic diversity among middle-class white males

I identify as a dirty penismensch.
Back to top
View user's profile Send private message
unK
l33t
l33t


Joined: 06 Feb 2007
Posts: 769

PostPosted: Sat Oct 03, 2009 6:20 pm    Post subject: Reply with quote

Well, then take gentoo/vanilla sources, patch it with reiser4 from here (afaik 2.6.30 version applies cleanly to 2.6.31) and see if the problem still occurs. Ah, and check your partition with fcsk immediately, I suspect your fs is far from being clean after these hang-ups.
_________________
ncmpcpp - featureful ncurses based MPD client inspired by ncmpc
Back to top
View user's profile Send private message
Ant P.
Watchman
Watchman


Joined: 18 Apr 2009
Posts: 5879

PostPosted: Sat Oct 03, 2009 7:25 pm    Post subject: Reply with quote

seqizz wrote:
Also i made a change on fs (e.g edit a file) and after one reboot, it's turning back to old one, like i've never touched the file.. (wtf? :mrgreen: )

This is a Good Thing. You don't get so lucky when ext4 crashes, it eats your files and you get a zero-byte .bash_history (hope you don't get a crash after reinstalling glibc!). ext4's behaviour is technically acceptable according to POSIX (and xfs also does it), but you can probably agree it's a really idiotic thing to do to 99% of users.
Back to top
View user's profile Send private message
DigitalCorpus
Apprentice
Apprentice


Joined: 30 Jul 2007
Posts: 245

PostPosted: Sun Oct 04, 2009 11:07 am    Post subject: Reply with quote

Okay, so I decided to do a full on emerge -e just for kicks and giggles despite the problems you all have had with the current/prior R4 patch with 2.6.31 and zen sources. Watching the clock and /proc/meminfo it turns out that R4 would not flush to disk when /proc/sys/vm/ dirty_ratio was being hit. It was only engaging when dirty_expire_centisecs was reached. I know you guys have a fix already, but seriously looking at what is configured here can greatly affect how your system responds with pending disk activity. A couple of these affect how much data you'll loose too if you loose power :P

One other thing workaround to use if this happens in a future release. just copy and paste this code into your command line and it'll force the kernel to sync any dirty pages:
Code:
echo 1 > /proc/sys/vm/drop_caches

_________________
Atlas (HDTV PVR, HTTP & Media server)
http://atlas.selfip.net/Info/
Back to top
View user's profile Send private message
kernelOfTruth
Watchman
Watchman


Joined: 20 Dec 2005
Posts: 6108
Location: Vienna, Austria; Germany; hello world :)

PostPosted: Tue Oct 06, 2009 10:55 pm    Post subject: Reply with quote

the problem should be fixed - at least with 2.6.31-zen* (zen-stable) - all of you who are/were affected by this problem and using zen-sources:

please update to 2.6.31-zen2 :idea:


if the fix isn't included yet:
http://thread.gmane.org/gmane.comp.file-systems.reiserfs.general/22617/focus=22626

<-- this fixes the introduced problem with reiser4-for-2.6.31 that reiser4-specific mount-options aren't recognized

kudos to Edward :)
_________________
https://github.com/kernelOfTruth/ZFS-for-SystemRescueCD/tree/ZFS-for-SysRescCD-4.9.0
https://github.com/kernelOfTruth/pulseaudio-equalizer-ladspa

Hardcore Gentoo Linux user since 2004 :D
Back to top
View user's profile Send private message
darklegion
Guru
Guru


Joined: 14 Nov 2004
Posts: 468

PostPosted: Thu Dec 03, 2009 2:32 am    Post subject: Reply with quote

I'm having a problem with saving a file with vim to a reiser4 + lzo compression filesystem.It is taking 5-10+ seconds to save a simple text file.If I save it to an ext2/ext3 partition it saves near instantly.If I use nano to save to the r4+lzo filesystem, it also saves instantly.

So this is probably an issue with vim, but since you guys are more familiar with the issues of transparent compression, I feel I'll get better answers here.

Thanks
Back to top
View user's profile Send private message
DigitalCorpus
Apprentice
Apprentice


Joined: 30 Jul 2007
Posts: 245

PostPosted: Thu Dec 03, 2009 6:31 am    Post subject: Reply with quote

run a check on your R4 partition
Code:
fsck.reiser4 -y --fix --build-fs --build-sb

Just make sure you tack on the /dev path to your unmounted partition. That shouldn't be happening. I've not experienced delays like that. Can you give us more information like the size of the file and your mount options?
_________________
Atlas (HDTV PVR, HTTP & Media server)
http://atlas.selfip.net/Info/
Back to top
View user's profile Send private message
darklegion
Guru
Guru


Joined: 14 Nov 2004
Posts: 468

PostPosted: Thu Dec 03, 2009 8:23 am    Post subject: Reply with quote

DigitalCorpus wrote:
run a check on your R4 partition
Code:
fsck.reiser4 -y --fix --build-fs --build-sb

Just make sure you tack on the /dev path to your unmounted partition. That shouldn't be happening. I've not experienced delays like that. Can you give us more information like the size of the file and your mount options?


File could be 1 byte or 100MB, it makes no difference (I just tested with a 0 byte file with nothing in it; same result).I should note that the drive is a 8GB Sandisk Ultra II SDHC card, and is in an Acer Aspire One netbook, which uses an Intel Atom N270 @ 1.6ghz.Write speed is 16-19 MB/s when copying a file from an ext2 fs to the r4-lzo filesystem.I don't have any other drives to test with, but copying data from the ext2 fs to the same ext2 fs, results in a write speed of 26.9 MB/s.So there is a performance hit, but it shouldn't be enough to cause these symptoms.

Filesystem is mounted with 'noatime' ; no other options are used.

EDIT: I just checked the filesystem.It had errors and they were fixed but it didn't help the issue.
Back to top
View user's profile Send private message
DigitalCorpus
Apprentice
Apprentice


Joined: 30 Jul 2007
Posts: 245

PostPosted: Thu Dec 03, 2009 10:43 am    Post subject: Reply with quote

I would venture to guess since you're writing to flash media, that is then your problem

normal flash memory use in USB sticks responds extrememly poorly to random reads and write. Even sequential reads and writes that are close to cluster size. Reiser4 is a journaled filesystem. this means before or while data is being written, the journal is being written to as well. journal transactions are tiny. you should only be using ext2 or btrfs w/ ssd mount option, or another flash-specific file system on that memory card.
_________________
Atlas (HDTV PVR, HTTP & Media server)
http://atlas.selfip.net/Info/
Back to top
View user's profile Send private message
darklegion
Guru
Guru


Joined: 14 Nov 2004
Posts: 468

PostPosted: Thu Dec 03, 2009 11:39 am    Post subject: Reply with quote

DigitalCorpus wrote:
I would venture to guess since you're writing to flash media, that is then your problem

normal flash memory use in USB sticks responds extrememly poorly to random reads and write. Even sequential reads and writes that are close to cluster size. Reiser4 is a journaled filesystem. this means before or while data is being written, the journal is being written to as well. journal transactions are tiny. you should only be using ext2 or btrfs w/ ssd mount option, or another flash-specific file system on that memory card.


Is there a benchmark for random writes? I'd like to see if it's a general system issue, or just a problem with vim, as nano saving instantly is still giving me pause.Perhaps vim's heaviness is the problem, maybe there are some tweaks to help it perform better?

Anyway, I'll check out btrfs, I've always wanted to try it out.It does have compression support, too, but zlib is probably a bit too heavy for the Atom.Shame really, as I could really use the space that compression saves.
Back to top
View user's profile Send private message
gerardo
Apprentice
Apprentice


Joined: 05 Feb 2004
Posts: 228
Location: Belgium

PostPosted: Sat Dec 05, 2009 5:42 pm    Post subject: Reply with quote

kernelOfTruth wrote:

best reiser4-partition creating options are so far:

mkfs.reiser4 -o create=ccreg40,compress=gzip1,compressMode=ultim,cluster=8K,fibration=lexic_fibre,formatting=smart for /usr/portage

and

mkfs.reiser4 -o create=ccreg40,compress=gzip1,compressMode=ultim,cluster=8K,formatting=smart (without the lexic ordering) for / (root / or system how you like to call it)


mount-options are so far:

noatime,nodiratime,tree.cbk_cache.nr_slots=48 (you might need to lower tree.cbk_cache.nr_slots to 24 or 32 if you have low memory, dunno right now how memory usage depends on it)


And what about the home partition?
I have a lot of mp3, avi, zip...
_________________
Windoze : Plug and Pay...
Back to top
View user's profile Send private message
DigitalCorpus
Apprentice
Apprentice


Joined: 30 Jul 2007
Posts: 245

PostPosted: Sat Dec 05, 2009 9:44 pm    Post subject: Reply with quote

Use something other than reiser4 for your home partition is you have already compressed content since the data will not compress further
_________________
Atlas (HDTV PVR, HTTP & Media server)
http://atlas.selfip.net/Info/
Back to top
View user's profile Send private message
kernelOfTruth
Watchman
Watchman


Joined: 20 Dec 2005
Posts: 6108
Location: Vienna, Austria; Germany; hello world :)

PostPosted: Sat Dec 05, 2009 10:17 pm    Post subject: Reply with quote

DigitalCorpus wrote:
Use something other than reiser4 for your home partition is you have already compressed content since the data will not compress further


++

someone also wrote about hash collisions when using the default r5 hash and recommends using the tea hash

so I'd go the safe road for now and use reiserfs v3.6 or any other long tested filesystem ...
_________________
https://github.com/kernelOfTruth/ZFS-for-SystemRescueCD/tree/ZFS-for-SysRescCD-4.9.0
https://github.com/kernelOfTruth/pulseaudio-equalizer-ladspa

Hardcore Gentoo Linux user since 2004 :D
Back to top
View user's profile Send private message
darklegion
Guru
Guru


Joined: 14 Nov 2004
Posts: 468

PostPosted: Sun Dec 06, 2009 7:18 am    Post subject: Reply with quote

DigitalCorpus wrote:
I would venture to guess since you're writing to flash media, that is then your problem

normal flash memory use in USB sticks responds extrememly poorly to random reads and write. Even sequential reads and writes that are close to cluster size. Reiser4 is a journaled filesystem. this means before or while data is being written, the journal is being written to as well. journal transactions are tiny. you should only be using ext2 or btrfs w/ ssd mount option, or another flash-specific file system on that memory card.


I just tried out Btrfs with compress and ssd mount options enabled, and vim now saves at a reasonable speed (around 1 sec).Space usage is about the same as reiser4+lzo although I expect cpu usage will probably be higher with zlib.
Back to top
View user's profile Send private message
HecHacker1
Apprentice
Apprentice


Joined: 26 Jun 2003
Posts: 213
Location: UCSD

PostPosted: Sun Dec 13, 2009 3:17 am    Post subject: Warning Reply with quote

Just as a warning, I wouldn't use btrfs yet. It is absolutely not stable and still randomly segfaults. Basically the devs have only started on working on all the edge cases and problems.

I myself was using btrfs w/compression in a Virtual Machine and the FS panics after a hard reset.

fsck.btrfs also segfaults when trying to check that filesystem. Luckily I didn't loose any data because I was able to mount it in read only mode to retrieve it.

If you check out the btrfs mailing list you will see daily errors and problems people have. The devs have not even responded to my problem.

Right now I'm trying to decide between reiserfs/ext4/xfs/ and reiser4/w lzo compression.

So I am actually going to copy my / partiton to each filesystem and compare. This is on a laptop, 320GB disk split into 8 partitons. 5400rpm.

I've already done:

BTRFS w/compress: Much slower to boot (over a minute). Slower to the desktop. But fsync's (and small random r/w) are surprisingly fast. Without compression the speed is not much better on 2.6.31. I tried 2.6.32 which is supposed to have improvements, but my laptop doesn't boot because of a bug in 2.6.32. It did survive hard resets and I never had to fsck.

XFS: Pretty good performance. Much faster than btrfs w/compression (on my older pentium-m 1.7GHz). bootchart 49s. Opens Firefox/chrome faster than reiser4. 20GB used.

Reiser4 w/lzo: Fastest when working with portage (syncing, searching). However I get small lag spikes when something like tracker is indexing. With xfs, Tracker doesn't distract. bootchart 40s. 10GB used. I should warn that I am getting compression errors in my dmesg, Reiser4 is saying the file/block is corrupt. Yet when I do a reiser4.fsck it comes back clean. I think this might be a 2.6.31/2.6.32-zen problem.

Reiserfs: 17GB used, 3GB improvement from XFS. Still haven't tested it. Will update.

Ext4: Feels like less lag when using a web-browser than reiser4 w/lzo. Portage sync is alright, similar to xfs, but reiser4 blows it away. A stable FS and I have never lost data with ext4. Still need to test bootchart.
Back to top
View user's profile Send private message
darklegion
Guru
Guru


Joined: 14 Nov 2004
Posts: 468

PostPosted: Sun Dec 13, 2009 7:48 am    Post subject: Re: Warning Reply with quote

HecHacker1 wrote:
Just as a warning, I wouldn't use btrfs yet. It is absolutely not stable and still randomly segfaults. Basically the devs have only started on working on all the edge cases and problems.


I'm doing daily backups, so I'm not too worried. :) I also didn't see any slowdowns when booting or anywhere else.I don't think fsck is supported at all yet, but that doesn't necessarily mean it will fail during the first hard reset/power failure (although I guess it did, for you).
I'm using 2.6.32, BTW.
Back to top
View user's profile Send private message
dusanc
Apprentice
Apprentice


Joined: 19 Sep 2005
Posts: 248
Location: Serbia

PostPosted: Tue Dec 15, 2009 11:04 am    Post subject: Re: Warning Reply with quote

HecHacker1 wrote:
...

Right now I'm trying to decide between reiserfs/ext4/xfs/ and reiser4/w lzo compression.

So I am actually going to copy my / partiton to each filesystem and compare. This is on a laptop, 320GB disk split into 8 partitons. 5400rpm.
...


Ofcourse you tested all filesystems on same partition?
_________________
Reiser4 Gentoo FAQ [25Dec2016]
Back to top
View user's profile Send private message
Dont Panic
Guru
Guru


Joined: 20 Jun 2007
Posts: 322
Location: SouthEast U.S.A.

PostPosted: Wed Dec 30, 2009 4:52 pm    Post subject: Reply with quote

What are the current solutions for Reiser4 for the 2.6.32 kernel.

I see where Edward Shishkin has a brief comment that the reiser4 in the zen-sources 2.6.32 kernel may not work correctly.

http://kerneltrap.org/mailarchive/reiserfs-devel/2009/12/11/6634703/thread

At any rate, I am having a hard time finding a stable configuration for a zen-sources kernel.

Are there other options?
Back to top
View user's profile Send private message
kernelOfTruth
Watchman
Watchman


Joined: 20 Dec 2005
Posts: 6108
Location: Vienna, Austria; Germany; hello world :)

PostPosted: Wed Dec 30, 2009 5:07 pm    Post subject: Reply with quote

Dont Panic wrote:
What are the current solutions for Reiser4 for the 2.6.32 kernel.

I see where Edward Shishkin has a brief comment that the reiser4 in the zen-sources 2.6.32 kernel may not work correctly.

http://kerneltrap.org/mailarchive/reiserfs-devel/2009/12/11/6634703/thread

At any rate, I am having a hard time finding a stable configuration for a zen-sources kernel.

Are there other options?


your best bet is to start from the following topic:

http://thread.gmane.org/gmane.comp.file-systems.reiserfs.general/22676

http://news.gmane.org/gmane.comp.file-systems.reiserfs.general

currently I'm not using reiser4 at all so I can't say much but the following:

* reiser4 with cryptcompress seems to work reasonably well enough (last time I used it only on /usr/portage; large partitions or file-transaction seem to result in BUG)

* reiser4 without cryptcompress seems to be broken - at least fsync, it will sync data to the partition during umount but in addition to that seems to handle it in a wrong way [I'm referring to zen-kernels !; try the latest if it got fixed - from what I read it works again !]
_________________
https://github.com/kernelOfTruth/ZFS-for-SystemRescueCD/tree/ZFS-for-SysRescCD-4.9.0
https://github.com/kernelOfTruth/pulseaudio-equalizer-ladspa

Hardcore Gentoo Linux user since 2004 :D


Last edited by kernelOfTruth on Thu Apr 15, 2010 1:27 am; edited 1 time in total
Back to top
View user's profile Send private message
lightvhawk0
Guru
Guru


Joined: 07 Nov 2003
Posts: 388

PostPosted: Thu Apr 15, 2010 1:14 am    Post subject: Reply with quote

anyways I decided to rice out reiser4

mkfs.reiser4 -o compressMode=none,fibration=lexic_fibre,formatting=extents,cluster=8K

I decided to hell with compression, and tail packing since I have 500gb hdd I'm running reiser4 on.

So far it seems to be immensely fast.

I was wondering if it would be better to change the hash algorithm as well?
_________________
If God has made us in his image, we have returned him the favor. - Voltaire
Back to top
View user's profile Send private message
kernelOfTruth
Watchman
Watchman


Joined: 20 Dec 2005
Posts: 6108
Location: Vienna, Austria; Germany; hello world :)

PostPosted: Thu Apr 15, 2010 1:28 am    Post subject: Reply with quote

better don't mess with the hash-algo unless you're fond of collisions ;)

for /usr/portage you could try another algo for a data-partition (with safety in mind) I wouldn't risk it,

r5 or tea should be safe

*) tea for less collisions

*) r5 [=default] for more performance in directory-lookups
_________________
https://github.com/kernelOfTruth/ZFS-for-SystemRescueCD/tree/ZFS-for-SysRescCD-4.9.0
https://github.com/kernelOfTruth/pulseaudio-equalizer-ladspa

Hardcore Gentoo Linux user since 2004 :D
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Unsupported Software All times are GMT
Goto page Previous  1, 2, 3, 4, 5  Next
Page 4 of 5

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum