Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
Official thread: "zen-sources" - Part IV
View unanswered posts
View posts from last 24 hours

Goto page Previous  1, 2, 3 ... , 15, 16, 17  Next  
Reply to topic    Gentoo Forums Forum Index Unsupported Software
View previous topic :: View next topic  
Author Message
Palatis
n00b
n00b


Joined: 07 Oct 2006
Posts: 23
Location: Taipei/Taiwan

PostPosted: Sat Aug 09, 2008 2:14 pm    Post subject: Reply with quote

kernelOfTruth wrote:
hi :),

could those of you amd64-users (64bit) give compcache (w. tlsf) a test-drive ? since now afaik it only was tested with x86 (I tested it some time ago and afaik it was working ok):

some info on compcache (for those who don't know what it is):

Quote:
Hi All,

(sending to lkml since I didn't get any reply at linux-mm).

This implements a RAM based block device which acts as swap disk.
Pages swapped to this disk are compressed and stored in memory itself.
This allows more applications to fit in given amount of memory. This is
especially useful for embedded devices, OLPC and small desktops
(aka virtual machines).

Project home: http://code.google.com/p/compcache/

It consists of following components:
- compcache.ko: Creates RAM based block device
- tlsf.ko: Two Level Segregate Fit (TLSF) allocator
- LZO de/compressor: (Already in mainline)

Project home contains some performance numbers for TLSF and LZO.
For general desktop use, this is giving *significant* performance gain
under memory pressure. For now, it has been tested only on x86.

Thanks,
Nitin


source: http://lwn.net/Articles/274653/

thanks

it works well and rocks!
except if CONFIG_TLSF_STATS is enabled, there's a compiling error.
patch to solve it:
Code:
--- mm/tlsf.c.orig 2008-08-09 21:47:57.875258433 +0800
+++ mm/tlsf.c      2008-08-09 19:34:22.618820684 +0800
@@ -663,7 +663,7 @@
 {
 #if defined(CONFIG_TLSF_STATS)
         if (proc)
-                remove_proc_entry("tlsfinfo", &proc_root);
+                remove_proc_entry("tlsfinfo", proc->parent);
 #endif
         return;
 }
Back to top
View user's profile Send private message
kernelOfTruth
Watchman
Watchman


Joined: 20 Dec 2005
Posts: 6108
Location: Vienna, Austria; Germany; hello world :)

PostPosted: Sun Aug 10, 2008 9:41 pm    Post subject: Reply with quote

latest zen-sources with reiser4-devel works fine so far :)

so it really was the "base" going crazy (mm-patchset)

thanks :)

update:

Quote:
[ 3987.637185] BUG: scheduling while atomic: ifconfig/15858/0x00000100
[ 3987.637195] Modules linked in: usb_storage nf_conntrack_ftp nf_conntrack_irc nvidia(P) coretemp snd_pcm_oss snd_mixer_oss snd_seq_dummy snd_seq_oss snd_seq_midi_event snd_seq snd_seq_device lrw gf128mul sha256_generic ipwraw uvcvideo compat_ioctl32 videodev v4l1_compat usblp iwl3945 mac80211 led_class tg3 ohci1394 sdhci_pci ieee1394 libphy sdhci uhci_hcd i2c_i801 ehci_hcd cfg80211 wmi mmc_core
[ 3987.637267] Pid: 15858, comm: ifconfig Tainted: P 2.6.27-rc1-zen0-01232-g8e47063 #2
[ 3987.637272]
[ 3987.637274] Call Trace:
[ 3987.637288] [<ffffffff8077f220>] thread_return+0x2ad/0x65d
[ 3987.637296] [<ffffffff80781949>] error_exit+0x0/0x51
[ 3987.637305] [<ffffffff80249844>] lock_timer_base+0x34/0x70
[ 3987.637312] [<ffffffff80249b27>] __mod_timer+0xb7/0xd0
[ 3987.637319] [<ffffffff8077f978>] schedule_timeout+0x58/0xd0
[ 3987.637328] [<ffffffff80249440>] process_timeout+0x0/0x10
[ 3987.637334] [<ffffffff8077f973>] schedule_timeout+0x53/0xd0
[ 3987.637341] [<ffffffff80249b5d>] msleep+0x1d/0x40
[ 3987.637351] [<ffffffff804f79c0>] pci_set_power_state+0x2d0/0x380
[ 3987.637368] [<ffffffffa0075399>] tg3_set_power_state+0x49/0xa70 [tg3]
[ 3987.637381] [<ffffffffa007f3ba>] tg3_open+0x3a/0x690 [tg3]
[ 3987.637390] [<ffffffff8066f479>] dev_open+0x89/0xf0
[ 3987.637397] [<ffffffff8066ecef>] dev_change_flags+0x9f/0x1e0
[ 3987.637406] [<ffffffff806c6d94>] devinet_ioctl+0x784/0x790
[ 3987.637414] [<ffffffff8065ed06>] sock_ioctl+0x66/0x270
[ 3987.637423] [<ffffffff802bacef>] vfs_ioctl+0x2f/0xb0
[ 3987.637430] [<ffffffff802badec>] do_vfs_ioctl+0x7c/0x470
[ 3987.637437] [<ffffffff802bb281>] sys_ioctl+0xa1/0xb0
[ 3987.637445] [<ffffffff8020c36b>] system_call_fastpath+0x16/0x1b
[ 3987.637451]
[ 3987.703584] ADDRCONF(NETDEV_UP): eth2: link is not ready
[ 3989.399985] tg3: eth2: Link is up at 100 Mbps, full duplex.
[ 3989.399999] tg3: eth2: Flow control is on for TX and on for RX.
[ 3989.401882] ADDRCONF(NETDEV_CHANGE): eth2: link becomes ready


dead simple to reproduce this one:

Code:
ifconfig eth2 down


wait few seconds

Code:
ifconfig eth2 up


this also happens on zenmm :idea: :!:

http://lkml.org/lkml/2008/8/4/310
potential fix :idea:
_________________
https://github.com/kernelOfTruth/ZFS-for-SystemRescueCD/tree/ZFS-for-SysRescCD-4.9.0
https://github.com/kernelOfTruth/pulseaudio-equalizer-ladspa

Hardcore Gentoo Linux user since 2004 :D
Back to top
View user's profile Send private message
kernelOfTruth
Watchman
Watchman


Joined: 20 Dec 2005
Posts: 6108
Location: Vienna, Austria; Germany; hello world :)

PostPosted: Sun Aug 10, 2008 10:09 pm    Post subject: Reply with quote

Palatis wrote:


it works well and rocks!
except if CONFIG_TLSF_STATS is enabled, there's a compiling error.
patch to solve it:
Code:
--- mm/tlsf.c.orig 2008-08-09 21:47:57.875258433 +0800
+++ mm/tlsf.c      2008-08-09 19:34:22.618820684 +0800
@@ -663,7 +663,7 @@
 {
 #if defined(CONFIG_TLSF_STATS)
         if (proc)
-                remove_proc_entry("tlsfinfo", &proc_root);
+                remove_proc_entry("tlsfinfo", proc->parent);
 #endif
         return;
 }


I just downloaded the newest patch against 2.6.26 and your compiler error is already fixed upstream :idea: :)
_________________
https://github.com/kernelOfTruth/ZFS-for-SystemRescueCD/tree/ZFS-for-SysRescCD-4.9.0
https://github.com/kernelOfTruth/pulseaudio-equalizer-ladspa

Hardcore Gentoo Linux user since 2004 :D
Back to top
View user's profile Send private message
rmh3093
Advocate
Advocate


Joined: 06 Aug 2003
Posts: 2138
Location: Albany, NY

PostPosted: Sun Aug 10, 2008 10:16 pm    Post subject: Reply with quote

kernelOfTruth wrote:
latest zen-sources with reiser4-devel works fine so far :)

so it really was the "base" going crazy (mm-patchset)

thanks :)

update:

Quote:
[ 3987.637185] BUG: scheduling while atomic: ifconfig/15858/0x00000100
[ 3987.637195] Modules linked in: usb_storage nf_conntrack_ftp nf_conntrack_irc nvidia(P) coretemp snd_pcm_oss snd_mixer_oss snd_seq_dummy snd_seq_oss snd_seq_midi_event snd_seq snd_seq_device lrw gf128mul sha256_generic ipwraw uvcvideo compat_ioctl32 videodev v4l1_compat usblp iwl3945 mac80211 led_class tg3 ohci1394 sdhci_pci ieee1394 libphy sdhci uhci_hcd i2c_i801 ehci_hcd cfg80211 wmi mmc_core
[ 3987.637267] Pid: 15858, comm: ifconfig Tainted: P 2.6.27-rc1-zen0-01232-g8e47063 #2
[ 3987.637272]
[ 3987.637274] Call Trace:
[ 3987.637288] [<ffffffff8077f220>] thread_return+0x2ad/0x65d
[ 3987.637296] [<ffffffff80781949>] error_exit+0x0/0x51
[ 3987.637305] [<ffffffff80249844>] lock_timer_base+0x34/0x70
[ 3987.637312] [<ffffffff80249b27>] __mod_timer+0xb7/0xd0
[ 3987.637319] [<ffffffff8077f978>] schedule_timeout+0x58/0xd0
[ 3987.637328] [<ffffffff80249440>] process_timeout+0x0/0x10
[ 3987.637334] [<ffffffff8077f973>] schedule_timeout+0x53/0xd0
[ 3987.637341] [<ffffffff80249b5d>] msleep+0x1d/0x40
[ 3987.637351] [<ffffffff804f79c0>] pci_set_power_state+0x2d0/0x380
[ 3987.637368] [<ffffffffa0075399>] tg3_set_power_state+0x49/0xa70 [tg3]
[ 3987.637381] [<ffffffffa007f3ba>] tg3_open+0x3a/0x690 [tg3]
[ 3987.637390] [<ffffffff8066f479>] dev_open+0x89/0xf0
[ 3987.637397] [<ffffffff8066ecef>] dev_change_flags+0x9f/0x1e0
[ 3987.637406] [<ffffffff806c6d94>] devinet_ioctl+0x784/0x790
[ 3987.637414] [<ffffffff8065ed06>] sock_ioctl+0x66/0x270
[ 3987.637423] [<ffffffff802bacef>] vfs_ioctl+0x2f/0xb0
[ 3987.637430] [<ffffffff802badec>] do_vfs_ioctl+0x7c/0x470
[ 3987.637437] [<ffffffff802bb281>] sys_ioctl+0xa1/0xb0
[ 3987.637445] [<ffffffff8020c36b>] system_call_fastpath+0x16/0x1b
[ 3987.637451]
[ 3987.703584] ADDRCONF(NETDEV_UP): eth2: link is not ready
[ 3989.399985] tg3: eth2: Link is up at 100 Mbps, full duplex.
[ 3989.399999] tg3: eth2: Flow control is on for TX and on for RX.
[ 3989.401882] ADDRCONF(NETDEV_CHANGE): eth2: link becomes ready


dead simple to reproduce this one:

Code:
ifconfig eth2 down


wait few seconds

Code:
ifconfig eth2 up


this also happens on zenmm :idea: :!:

http://lkml.org/lkml/2008/8/4/310
potential fix :idea:


did you try this patch? i have a tg3 on another box I can test in a few days
_________________
Do not meddle in the affairs of wizards, for they are subtle and quick to anger.
Back to top
View user's profile Send private message
rmh3093
Advocate
Advocate


Joined: 06 Aug 2003
Posts: 2138
Location: Albany, NY

PostPosted: Sun Aug 10, 2008 10:17 pm    Post subject: Reply with quote

kernelOfTruth wrote:
latest zen-sources with reiser4-devel works fine so far :)


well thats funny it works for you, it must be because you have no ext* partitions, my attempts at todo #14 dont seem to play with ext3/ext4
_________________
Do not meddle in the affairs of wizards, for they are subtle and quick to anger.
Back to top
View user's profile Send private message
kernelOfTruth
Watchman
Watchman


Joined: 20 Dec 2005
Posts: 6108
Location: Vienna, Austria; Germany; hello world :)

PostPosted: Sun Aug 10, 2008 10:46 pm    Post subject: Reply with quote

rmh3093 wrote:
kernelOfTruth wrote:
latest zen-sources with reiser4-devel works fine so far :)

so it really was the "base" going crazy (mm-patchset)

thanks :)

update:

Quote:
[ 3987.637185] BUG: scheduling while atomic: ifconfig/15858/0x00000100
[ 3987.637195] Modules linked in: usb_storage nf_conntrack_ftp nf_conntrack_irc nvidia(P) coretemp snd_pcm_oss snd_mixer_oss snd_seq_dummy snd_seq_oss snd_seq_midi_event snd_seq snd_seq_device lrw gf128mul sha256_generic ipwraw uvcvideo compat_ioctl32 videodev v4l1_compat usblp iwl3945 mac80211 led_class tg3 ohci1394 sdhci_pci ieee1394 libphy sdhci uhci_hcd i2c_i801 ehci_hcd cfg80211 wmi mmc_core
[ 3987.637267] Pid: 15858, comm: ifconfig Tainted: P 2.6.27-rc1-zen0-01232-g8e47063 #2
[ 3987.637272]
[ 3987.637274] Call Trace:
[ 3987.637288] [<ffffffff8077f220>] thread_return+0x2ad/0x65d
[ 3987.637296] [<ffffffff80781949>] error_exit+0x0/0x51
[ 3987.637305] [<ffffffff80249844>] lock_timer_base+0x34/0x70
[ 3987.637312] [<ffffffff80249b27>] __mod_timer+0xb7/0xd0
[ 3987.637319] [<ffffffff8077f978>] schedule_timeout+0x58/0xd0
[ 3987.637328] [<ffffffff80249440>] process_timeout+0x0/0x10
[ 3987.637334] [<ffffffff8077f973>] schedule_timeout+0x53/0xd0
[ 3987.637341] [<ffffffff80249b5d>] msleep+0x1d/0x40
[ 3987.637351] [<ffffffff804f79c0>] pci_set_power_state+0x2d0/0x380
[ 3987.637368] [<ffffffffa0075399>] tg3_set_power_state+0x49/0xa70 [tg3]
[ 3987.637381] [<ffffffffa007f3ba>] tg3_open+0x3a/0x690 [tg3]
[ 3987.637390] [<ffffffff8066f479>] dev_open+0x89/0xf0
[ 3987.637397] [<ffffffff8066ecef>] dev_change_flags+0x9f/0x1e0
[ 3987.637406] [<ffffffff806c6d94>] devinet_ioctl+0x784/0x790
[ 3987.637414] [<ffffffff8065ed06>] sock_ioctl+0x66/0x270
[ 3987.637423] [<ffffffff802bacef>] vfs_ioctl+0x2f/0xb0
[ 3987.637430] [<ffffffff802badec>] do_vfs_ioctl+0x7c/0x470
[ 3987.637437] [<ffffffff802bb281>] sys_ioctl+0xa1/0xb0
[ 3987.637445] [<ffffffff8020c36b>] system_call_fastpath+0x16/0x1b
[ 3987.637451]
[ 3987.703584] ADDRCONF(NETDEV_UP): eth2: link is not ready
[ 3989.399985] tg3: eth2: Link is up at 100 Mbps, full duplex.
[ 3989.399999] tg3: eth2: Flow control is on for TX and on for RX.
[ 3989.401882] ADDRCONF(NETDEV_CHANGE): eth2: link becomes ready


dead simple to reproduce this one:

Code:
ifconfig eth2 down


wait few seconds

Code:
ifconfig eth2 up


this also happens on zenmm :idea: :!:

http://lkml.org/lkml/2008/8/4/310
potential fix :idea:


did you try this patch? i have a tg3 on another box I can test in a few days


yeah, currently I'm running a kernel with this patch applied,
the driver survives removes & insertions and ifconfig ups and downs :idea:

the following days will show if there's still any issues ... (works fine so far)

thanks :)
_________________
https://github.com/kernelOfTruth/ZFS-for-SystemRescueCD/tree/ZFS-for-SysRescCD-4.9.0
https://github.com/kernelOfTruth/pulseaudio-equalizer-ladspa

Hardcore Gentoo Linux user since 2004 :D
Back to top
View user's profile Send private message
kernelOfTruth
Watchman
Watchman


Joined: 20 Dec 2005
Posts: 6108
Location: Vienna, Austria; Germany; hello world :)

PostPosted: Sun Aug 10, 2008 10:50 pm    Post subject: Reply with quote

rmh3093 wrote:
kernelOfTruth wrote:
latest zen-sources with reiser4-devel works fine so far :)


well thats funny it works for you, it must be because you have no ext* partitions, my attempts at todo #14 dont seem to play with ext3/ext4


note:

I'm using the kernel with commit fbf5ead9c7b4ce5efc28ca18061152a37e1ed0e5 (afaik; you reverted that & replaced by #14 v2), so that might be a difference

any need for me to update ?

(a 'return' was removed)
_________________
https://github.com/kernelOfTruth/ZFS-for-SystemRescueCD/tree/ZFS-for-SysRescCD-4.9.0
https://github.com/kernelOfTruth/pulseaudio-equalizer-ladspa

Hardcore Gentoo Linux user since 2004 :D
Back to top
View user's profile Send private message
rmh3093
Advocate
Advocate


Joined: 06 Aug 2003
Posts: 2138
Location: Albany, NY

PostPosted: Sun Aug 10, 2008 11:09 pm    Post subject: Reply with quote

kernelOfTruth wrote:
rmh3093 wrote:
kernelOfTruth wrote:
latest zen-sources with reiser4-devel works fine so far :)


well thats funny it works for you, it must be because you have no ext* partitions, my attempts at todo #14 dont seem to play with ext3/ext4


note:

I'm using the kernel with commit fbf5ead9c7b4ce5efc28ca18061152a37e1ed0e5 (afaik; you reverted that & replaced by #14 v2), so that might be a difference

any need for me to update ?

(a 'return' was removed)


no need for you to update since u dont get the bug, i get the bug with all versions I try
_________________
Do not meddle in the affairs of wizards, for they are subtle and quick to anger.
Back to top
View user's profile Send private message
kernelOfTruth
Watchman
Watchman


Joined: 20 Dec 2005
Posts: 6108
Location: Vienna, Austria; Germany; hello world :)

PostPosted: Mon Aug 11, 2008 9:13 am    Post subject: Reply with quote

ok, a little status update:

I tried out every reiser4-devel kernel, and each and every of those [except the latest, still testing] has catastrophic fsync "performance" (if you can call that), it takes ages to sync data to the hdd, even with magic sysrq + sync it the clattering doesn't halt; before TODO #15v4 also pdflush wreaks havok (a lot of activity)

seems you broke something :P

the latest with TODO #15v4 looks a little better, though; I will let you know more later ...

thanks :)

update:

yeah, this one's fine again :P
(commit: ecd8e87664e1c549a352c29343c2fe64701628d0)

it's TODO #14 right ?
so there's a little typo (in git) :P

update2:

it's still worse than standard reiser4 tree (before those patches): e.g. after
Code:
paludis -s && emerge --regen && update-eix


then
Code:
sync
will take pretty long :?

as long as it doesn't take hours or 15-30 minutes every time it's fine I guess :roll:

"the show (development) must go on" :!:
_________________
https://github.com/kernelOfTruth/ZFS-for-SystemRescueCD/tree/ZFS-for-SysRescCD-4.9.0
https://github.com/kernelOfTruth/pulseaudio-equalizer-ladspa

Hardcore Gentoo Linux user since 2004 :D
Back to top
View user's profile Send private message
Gregoire
Apprentice
Apprentice


Joined: 15 Apr 2006
Posts: 292

PostPosted: Mon Aug 11, 2008 12:38 pm    Post subject: Reply with quote

Hello,

any hope to have http://liplianindvb.sourceforge.net/cgi-bin/hgwebdir.cgi/liplianindvb/ included ?

Thanks.
Back to top
View user's profile Send private message
rmh3093
Advocate
Advocate


Joined: 06 Aug 2003
Posts: 2138
Location: Albany, NY

PostPosted: Mon Aug 11, 2008 1:38 pm    Post subject: Reply with quote

kernelOfTruth wrote:
ok, a little status update:

I tried out every reiser4-devel kernel, and each and every of those [except the latest, still testing] has catastrophic fsync "performance" (if you can call that), it takes ages to sync data to the hdd, even with magic sysrq + sync it the clattering doesn't halt; before TODO #15v4 also pdflush wreaks havok (a lot of activity)

seems you broke something :P

the latest with TODO #15v4 looks a little better, though; I will let you know more later ...

thanks :)

update:

yeah, this one's fine again :P
(commit: ecd8e87664e1c549a352c29343c2fe64701628d0)

it's TODO #14 right ?
so there's a little typo (in git) :P

update2:

it's still worse than standard reiser4 tree (before those patches): e.g. after
Code:
paludis -s && emerge --regen && update-eix


then
Code:
sync
will take pretty long :?

as long as it doesn't take hours or 15-30 minutes every time it's fine I guess :roll:

"the show (development) must go on" :!:


so you are saying v4 is the best so far

EDIT: just wanted to say I appreciate you testing all these changes
_________________
Do not meddle in the affairs of wizards, for they are subtle and quick to anger.
Back to top
View user's profile Send private message
kernelOfTruth
Watchman
Watchman


Joined: 20 Dec 2005
Posts: 6108
Location: Vienna, Austria; Germany; hello world :)

PostPosted: Mon Aug 11, 2008 2:13 pm    Post subject: Reply with quote

rmh3093 wrote:


so you are saying v4 is the best so far



considering the changes you made (TODO points) definitely yes ! :)

I think we can leave the fine-tuning / tuning up to Edward :P :wink:

rmh3093 wrote:


EDIT: just wanted to say I appreciate you testing all these changes



glad I can help :D

update:


ha ! didn't see v5 :lol:
_________________
https://github.com/kernelOfTruth/ZFS-for-SystemRescueCD/tree/ZFS-for-SysRescCD-4.9.0
https://github.com/kernelOfTruth/pulseaudio-equalizer-ladspa

Hardcore Gentoo Linux user since 2004 :D
Back to top
View user's profile Send private message
rmh3093
Advocate
Advocate


Joined: 06 Aug 2003
Posts: 2138
Location: Albany, NY

PostPosted: Mon Aug 11, 2008 2:28 pm    Post subject: Reply with quote

kernelOfTruth wrote:
rmh3093 wrote:


so you are saying v4 is the best so far



considering the changes you made (TODO points) definitely yes ! :)

I think we can leave the fine-tuning / tuning up to Edward :P :wink:

rmh3093 wrote:


EDIT: just wanted to say I appreciate you testing all these changes



glad I can help :D

update:


ha ! didn't see v5 :lol:


well v5 is about as good as I think it can get...
_________________
Do not meddle in the affairs of wizards, for they are subtle and quick to anger.
Back to top
View user's profile Send private message
kernelOfTruth
Watchman
Watchman


Joined: 20 Dec 2005
Posts: 6108
Location: Vienna, Austria; Germany; hello world :)

PostPosted: Mon Aug 11, 2008 6:34 pm    Post subject: Reply with quote

rmh3093 wrote:

well v5 is about as good as I think it can get...


it surely is, but it's still somewhat slower than the reiser4 (non-devel) [which is a very subjective impression :wink: ]

thanks ! :)

I'd be glad if zenmm / mm-patchset wasn't that likely to hardlock, that way the system would be a little faster


yesterday I tested compcache and it doesn't seem to play well with latest zen-sources, on shutdown/reboot it produces clattering & A bug / kernel panic (like on .26-zen* afaik, something doesn't seem to play well with either reiser4 or lockless)
_________________
https://github.com/kernelOfTruth/ZFS-for-SystemRescueCD/tree/ZFS-for-SysRescCD-4.9.0
https://github.com/kernelOfTruth/pulseaudio-equalizer-ladspa

Hardcore Gentoo Linux user since 2004 :D
Back to top
View user's profile Send private message
rmh3093
Advocate
Advocate


Joined: 06 Aug 2003
Posts: 2138
Location: Albany, NY

PostPosted: Mon Aug 11, 2008 7:56 pm    Post subject: Reply with quote

kernelOfTruth wrote:
rmh3093 wrote:

well v5 is about as good as I think it can get...


it surely is, but it's still somewhat slower than the reiser4 (non-devel) [which is a very subjective impression :wink: ]

thanks ! :)

I'd be glad if zenmm / mm-patchset wasn't that likely to hardlock, that way the system would be a little faster


yesterday I tested compcache and it doesn't seem to play well with latest zen-sources, on shutdown/reboot it produces clattering & A bug / kernel panic (like on .26-zen* afaik, something doesn't seem to play well with either reiser4 or lockless)


thats good to hear r4 is working, when u say its a little slower are you comparing -zenmm to -zen or just within -zen

compcache was non included in .27 kernels so far cause I noticed it was buggy
_________________
Do not meddle in the affairs of wizards, for they are subtle and quick to anger.
Back to top
View user's profile Send private message
kernelOfTruth
Watchman
Watchman


Joined: 20 Dec 2005
Posts: 6108
Location: Vienna, Austria; Germany; hello world :)

PostPosted: Mon Aug 11, 2008 9:33 pm    Post subject: Reply with quote

rmh3093 wrote:
kernelOfTruth wrote:
rmh3093 wrote:

well v5 is about as good as I think it can get...


it surely is, but it's still somewhat slower than the reiser4 (non-devel) [which is a very subjective impression :wink: ]

thanks ! :)

I'd be glad if zenmm / mm-patchset wasn't that likely to hardlock, that way the system would be a little faster


yesterday I tested compcache and it doesn't seem to play well with latest zen-sources, on shutdown/reboot it produces clattering & A bug / kernel panic (like on .26-zen* afaik, something doesn't seem to play well with either reiser4 or lockless)


thats good to hear r4 is working, when u say its a little slower are you comparing -zenmm to -zen or just within -zen

compcache was non included in .27 kernels so far cause I noticed it was buggy


I'd say it's noticably slower than on zenmm (before those weird changes & v5 / "vanilla" reiser4 [non reiser4-devel]; I haven't tested it anymore),

but not a too big difference within -zen

ok, here real-world numbers:

dell xps m1330, 320 GB samsung hdd (sata), throughput max. 63 MB/s (hdparm),

i/o scheduler: bfq/cfq; cat /sys/block/sda/queue/read_ahead_kb -> 256 (default 128);

kernel: 2.6.27-rc1-zen0 (commit: 46eab0b4193a475ad8a2eddd9452b61b1ddbb614 )


time needed to sync data to hdd (on a 20 GB reiser4 partition w. cryptcompress lzo)

1.)
after emerge of glibc-2.8_p20080602 & openrc-9999

(during compilation of glibc & openrc; awn, compiz, epiphany and 2 instances of gnome-terminal were running; in the middle of the compilation of glibc I once issued sync-command in order to mostly empty the cache)

Quote:
time sync

real 4m37.362s
user 0m0.000s
sys 0m3.155s


2.)
deleting a ~2.7 GB large kernel-folder (after having filled it during kernel-compilation) with the following config: kernel-config of 2.6.27-rc1-zen0-01244-g46eab0b-dirty for dell xps m1330 (~amd64 / x64)

~ 2m 30s, I believe (will correct this if was wrong, later)

system:
gentoo ~amd64, 2008.0, hardened glibc-2.7, hardened gcc-4.3.1
_________________
https://github.com/kernelOfTruth/ZFS-for-SystemRescueCD/tree/ZFS-for-SysRescCD-4.9.0
https://github.com/kernelOfTruth/pulseaudio-equalizer-ladspa

Hardcore Gentoo Linux user since 2004 :D
Back to top
View user's profile Send private message
rmh3093
Advocate
Advocate


Joined: 06 Aug 2003
Posts: 2138
Location: Albany, NY

PostPosted: Mon Aug 11, 2008 10:03 pm    Post subject: Reply with quote

kernelOfTruth wrote:
rmh3093 wrote:
kernelOfTruth wrote:
rmh3093 wrote:

well v5 is about as good as I think it can get...


it surely is, but it's still somewhat slower than the reiser4 (non-devel) [which is a very subjective impression :wink: ]

thanks ! :)

I'd be glad if zenmm / mm-patchset wasn't that likely to hardlock, that way the system would be a little faster


yesterday I tested compcache and it doesn't seem to play well with latest zen-sources, on shutdown/reboot it produces clattering & A bug / kernel panic (like on .26-zen* afaik, something doesn't seem to play well with either reiser4 or lockless)


thats good to hear r4 is working, when u say its a little slower are you comparing -zenmm to -zen or just within -zen

compcache was non included in .27 kernels so far cause I noticed it was buggy


I'd say it's noticably slower than on zenmm (before those weird changes & v5 / "vanilla" reiser4 [non reiser4-devel]; I haven't tested it anymore),

but not a too big difference within -zen

ok, here real-world numbers:

dell xps m1330, 320 GB samsung hdd (sata), throughput max. 63 MB/s (hdparm),

i/o scheduler: bfq/cfq; cat /sys/block/sda/queue/read_ahead_kb -> 256 (default 128);

kernel: 2.6.27-rc1-zen0 (commit: 46eab0b4193a475ad8a2eddd9452b61b1ddbb614 )


time needed to sync data to hdd (on a 20 GB reiser4 partition w. cryptcompress lzo)

1.)
after emerge of glibc-2.8_p20080602 & openrc-9999

(during compilation of glibc & openrc; awn, compiz, epiphany and 2 instances of gnome-terminal were running; in the middle of the compilation of glibc I once issued sync-command in order to mostly empty the cache)

Quote:
time sync

real 4m37.362s
user 0m0.000s
sys 0m3.155s


2.)
deleting a ~2.7 GB large kernel-folder (after having filled it during kernel-compilation) with the following config: kernel-config of 2.6.27-rc1-zen0-01244-g46eab0b-dirty for dell xps m1330 (~amd64 / x64)

~ 2m 30s, I believe (will correct this if was wrong, later)

system:
gentoo ~amd64, 2008.0, hardened glibc-2.7, hardened gcc-4.3.1


idk what those numbers mean really, i need to see times from v4 vs. v5
_________________
Do not meddle in the affairs of wizards, for they are subtle and quick to anger.
Back to top
View user's profile Send private message
kernelOfTruth
Watchman
Watchman


Joined: 20 Dec 2005
Posts: 6108
Location: Vienna, Austria; Germany; hello world :)

PostPosted: Mon Aug 11, 2008 10:07 pm    Post subject: Reply with quote

the biggest problem / difference from reiser4-devel compared to reiser4 (old branch) is still the following:

during shutdown/reboot it takes ages (several minutes) to "sync" (?) data to hdd (during that there's also heavy pdflush-activity)

even if shortly before that a successful sync-command was issued,

doing a magic sysrq reisub-combo also doesn't really help, since it takes pretty long, too, until Emergency sync complet(ed) is shown

reiser4 / reiser4-code seems like a really fragile thing :o
_________________
https://github.com/kernelOfTruth/ZFS-for-SystemRescueCD/tree/ZFS-for-SysRescCD-4.9.0
https://github.com/kernelOfTruth/pulseaudio-equalizer-ladspa

Hardcore Gentoo Linux user since 2004 :D
Back to top
View user's profile Send private message
rmh3093
Advocate
Advocate


Joined: 06 Aug 2003
Posts: 2138
Location: Albany, NY

PostPosted: Mon Aug 11, 2008 10:45 pm    Post subject: Reply with quote

kernelOfTruth wrote:
the biggest problem / difference from reiser4-devel compared to reiser4 (old branch) is still the following:

during shutdown/reboot it takes ages (several minutes) to "sync" (?) data to hdd (during that there's also heavy pdflush-activity)

even if shortly before that a successful sync-command was issued,

doing a magic sysrq reisub-combo also doesn't really help, since it takes pretty long, too, until Emergency sync complet(ed) is shown

reiser4 / reiser4-code seems like a really fragile thing :o


thats odd because my computer reboots pretty fast (normal i would say, i dont notice a slow down) now I am only using reiser4 for my /usr/portage partition but I cant really notice anything....

one of the patches (todo #9 I think) had to do with a loop in reiser4 exiting prematurely, i fixed that (at least I think I did) so what might be happening is that the reiser4 code for the first time is doing everything that it should be doing.... but maybe not, idk.... i submitted those 3 patches to lkml, lets see what they have to say
_________________
Do not meddle in the affairs of wizards, for they are subtle and quick to anger.
Back to top
View user's profile Send private message
Saundersx
Apprentice
Apprentice


Joined: 11 Apr 2005
Posts: 247

PostPosted: Tue Aug 12, 2008 5:53 am    Post subject: Reply with quote

Just thought I would throw this in, using 2.6.26-zen2.1

Code:
Aug 12 02:10:09 homebox ------------[ cut here ]------------
Aug 12 02:10:09 homebox WARNING: at drivers/acpi/tables/tbfadt.c:348 acpi_tb_create_local_fadt+0x144/0x2fd()
Aug 12 02:10:09 homebox Modules linked in:
Aug 12 02:10:09 homebox Pid: 0, comm: swapper Not tainted 2.6.26-zen2 #1
Aug 12 02:10:09 homebox
Aug 12 02:10:09 homebox Call Trace:
Aug 12 02:10:09 homebox [<ffffffff80231874>] warn_on_slowpath+0x64/0xa0
Aug 12 02:10:09 homebox [<ffffffff804493d5>] acpi_os_vprintf+0x16/0x2c
Aug 12 02:10:09 homebox [<ffffffff8045e607>] acpi_ut_info+0x67/0x70
Aug 12 02:10:09 homebox [<ffffffff8070d46c>] dmi_string_nosave+0x5c/0x80
Aug 12 02:10:09 homebox [<ffffffff8070d61d>] dmi_save_ident+0x1d/0x30
Aug 12 02:10:09 homebox [<ffffffff8070d8b0>] dmi_decode+0x280/0x4a0
Aug 12 02:10:09 homebox [<ffffffff8045d504>] acpi_tb_create_local_fadt+0x144/0x2fd
Aug 12 02:10:09 homebox [<ffffffff8045d702>] acpi_tb_parse_fadt+0x45/0x97
Aug 12 02:10:09 homebox [<ffffffff80709756>] acpi_tb_parse_root_table+0x26f/0x2c1
Aug 12 02:10:09 homebox [<ffffffff80708d12>] acpi_table_init+0x17/0x7f
Aug 12 02:10:09 homebox [<ffffffff804dbc76>] dmi_check_system+0x36/0x70
Aug 12 02:10:09 homebox [<ffffffff806f909f>] acpi_boot_table_init+0x1f/0xd0
Aug 12 02:10:09 homebox [<ffffffff806f4dcb>] setup_arch+0x2eb/0x3f0
Aug 12 02:10:09 homebox [<ffffffff806f39fc>] start_kernel+0x5c/0x330
Aug 12 02:10:09 homebox [<ffffffff806f3214>] x86_64_start_kernel+0x204/0x240
Aug 12 02:10:09 homebox
Aug 12 02:10:09 homebox ---[ end trace 4eaa2a86a8e2da22 ]---


Does it on every boot. Continues to boot fine.

Also a vmware module issue.

Code:
Aug 12 02:10:02 homebox /dev/vmmon[3773]: VMCI: Driver initialized.
Aug 12 02:10:02 homebox /dev/vmmon[3773]: Module vmmon: registered with major=10 minor=165
Aug 12 02:10:02 homebox /dev/vmmon[3773]: Initial HV check: anyNotCapable=0 anyUnlocked=0 anyEnabled=0 anyDisabled=1
Aug 12 02:10:02 homebox /dev/vmmon[3773]: HV check: anyNotCapable=0 anyUnlocked=0 anyEnabled=0 anyDisabled=1
Aug 12 02:10:02 homebox /dev/vmmon[3773]: Module vmmon: initialized
Aug 12 02:10:03 homebox proc_dir_entry 'fs' already registered
Aug 12 02:10:03 homebox Pid: 3794, comm: modprobe Tainted: P        W 2.6.26-zen2 #1
Aug 12 02:10:03 homebox
Aug 12 02:10:03 homebox Call Trace:
Aug 12 02:10:03 homebox [<ffffffff8040b20b>] idr_get_new+0xb/0x30
Aug 12 02:10:03 homebox [<ffffffff802d69c2>] proc_register+0xf2/0x180
Aug 12 02:10:03 homebox [<ffffffff802d6b68>] create_proc_entry+0x58/0xa0
Aug 12 02:10:03 homebox [<ffffffffa0a0a5f3>] :vmblock:VMBlockInitControlOps+0x23/0x120
Aug 12 02:10:03 homebox [<ffffffffa0a0b166>] :vmblock:init_module+0x6/0x40
Aug 12 02:10:03 homebox [<ffffffff80257120>] sys_init_module+0xb0/0x1e0
Aug 12 02:10:03 homebox [<ffffffff8020b5eb>] system_call_after_swapgs+0x7b/0x80
Aug 12 02:10:03 homebox


this is using the patch posted in the bugtracker, so it is probably flaky anyway.
Back to top
View user's profile Send private message
kernelOfTruth
Watchman
Watchman


Joined: 20 Dec 2005
Posts: 6108
Location: Vienna, Austria; Germany; hello world :)

PostPosted: Tue Aug 12, 2008 8:42 am    Post subject: Reply with quote

rmh3093 wrote:


thats odd because my computer reboots pretty fast (normal i would say, i dont notice a slow down) now I am only using reiser4 for my /usr/portage partition but I cant really notice anything....



yeah, that's no big deal since you're only using it on /usr/portage:
try to do a
Code:
emerge --sync
, afterwards issue
Code:
shutdown -h now
now it should take much longer until it can shutdown / reboot

you'll notice in the future that there's a big difference in behavior whether you're using reiser4 on data-partitions only (until now always has worked pretty flawless) or also on / (root): usage on the system-partition seems to put much more load / pressure on the filesystem (obviously) & it's (I'm sorry to say) still not perfect yet on that cases

if these issues are sorted out nothing big should prevent reiser4 from wide usage anymore :)
_________________
https://github.com/kernelOfTruth/ZFS-for-SystemRescueCD/tree/ZFS-for-SysRescCD-4.9.0
https://github.com/kernelOfTruth/pulseaudio-equalizer-ladspa

Hardcore Gentoo Linux user since 2004 :D


Last edited by kernelOfTruth on Tue Aug 12, 2008 10:39 am; edited 1 time in total
Back to top
View user's profile Send private message
kernelOfTruth
Watchman
Watchman


Joined: 20 Dec 2005
Posts: 6108
Location: Vienna, Austria; Germany; hello world :)

PostPosted: Tue Aug 12, 2008 8:44 am    Post subject: Reply with quote

rmh3093 wrote:

idk what those numbers mean really, i need to see times from v4 vs. v5


you're right, those numbers would be better suited in an reiser4-thread,
these are for those purists who need numbers to decide whether a filesystem (at a certain stage) is worth trying / using :idea:

unfortunately I don't have the time to extract a stage4-tarball, then re-emerge glibc, gcc & time all of those (which would of course be a nice real-world "benchmark" between the reiser4 patch-stages)

edit:

rmh3093, does reiser4 still work for you after removing all of those typedef enums ?

I haven't tried that commit / stage out & will do probably later ...

thanks

edit2:

hm, those changes (excluding typedef enums) don't do THAT good :(
seems like reiser4 has lost some of it's parallelism capabilities:

I can't do a simple git pull on the linux kernel-tree and an emerge --regen,
my hdd is keeping on clattering all the time for several minutes already (after having interrupted emerge --regen)

in following cases it takes significant longer:
- emergency syncing
- emergency remount (sometimes it wouldn't even finish)
- parallel tasks
- hdparm -t /dev/sda (seemingly cache-flushing)
- sync

so even if it's doing now what it's supposed to do (thanks for fixing ! :) ) it's doing that really bad so there must be additional errors / problems lying somewhere else, perhaps the guys on lkml have suggestions for an alternative fixing approach / additional fixes

those fsync-issues (before reiser4-devel) were mostly fixed by simply changing from non-lockless -> lockless, but now they're present again & even more noticable ;(

you definitely should backup your system (with a stage4-tarball) & try reiser4 out on system-partition, that way you'll soon see that it's not that easy ... :cry:

thanks

edit3:
there's no need to test v4 compared to v5, it's significantly worse than v5
you were right writing that it's the best it can get

unfortunately it's stlll not good enough
I copied over a linux-sources folder over from zen-sources repository (also on root) to /usr/src
Code:
cp -r /usr/src/sources/kernel/zen-sources /usr/src/linux-2.6.27-rc1-zen0_test
,
then compiled the kernel (config-file provided above),

after that syncing took:
Quote:
time sync

real 29m25.340s
user 0m0.000s
sys 0m22.106s

that's much too long ! 8O

<-- I'll test this again with zen-sources with old reiser4-branch (commit: 3b02066b9f62c9d71c443fe830c61082b9d71d7d )

edit4:
ok, starting to fiddle a little with VFS, let's see what we can make of it :idea:

here some more info on the whole thing (in fact I'm tuning the pagecache / pdflush behavior):
The Linux Page Cache and pdflush:
Theory of Operation and Tuning for Write-Heavy Loads
kudos to Gregory Smith !
_________________
https://github.com/kernelOfTruth/ZFS-for-SystemRescueCD/tree/ZFS-for-SysRescCD-4.9.0
https://github.com/kernelOfTruth/pulseaudio-equalizer-ladspa

Hardcore Gentoo Linux user since 2004 :D
Back to top
View user's profile Send private message
kernelOfTruth
Watchman
Watchman


Joined: 20 Dec 2005
Posts: 6108
Location: Vienna, Austria; Germany; hello world :)

PostPosted: Tue Aug 12, 2008 7:21 pm    Post subject: Reply with quote

it works now fine again and also a little faster: 30 seconds less during removal of a "2.7 GB" kernel-directory (normally it takes around 3 minutes to do so)

it's a pity they didn't approve the patches

but: if there's a problem with ext3/ext4 due to usage of reiser4 on a non-root partition it's most likely a problem with ext3/ext4 itself, VFS, ... rather than with reiser4 I believe (just my 2 ct)

thanks :)
_________________
https://github.com/kernelOfTruth/ZFS-for-SystemRescueCD/tree/ZFS-for-SysRescCD-4.9.0
https://github.com/kernelOfTruth/pulseaudio-equalizer-ladspa

Hardcore Gentoo Linux user since 2004 :D
Back to top
View user's profile Send private message
kernelOfTruth
Watchman
Watchman


Joined: 20 Dec 2005
Posts: 6108
Location: Vienna, Austria; Germany; hello world :)

PostPosted: Wed Aug 13, 2008 9:15 am    Post subject: Reply with quote

kernelOfTruth wrote:
it works now fine again and also a little faster: 30 seconds less during removal of a "2.7 GB" kernel-directory (normally it takes around 3 minutes to do so)

it's a pity they didn't approve the patches

but: if there's a problem with ext3/ext4 due to usage of reiser4 on a non-root partition it's most likely a problem with ext3/ext4 itself, VFS, ... rather than with reiser4 I believe (just my 2 ct)

thanks :)


and they DID ! I'm currently testing [PATCH 1/3][reiser4] use wake_up_process() instead of wake_up() when possible

could you please sync/update the zen-tree to .27-rc3 ?

many thanks in advance :)
_________________
https://github.com/kernelOfTruth/ZFS-for-SystemRescueCD/tree/ZFS-for-SysRescCD-4.9.0
https://github.com/kernelOfTruth/pulseaudio-equalizer-ladspa

Hardcore Gentoo Linux user since 2004 :D
Back to top
View user's profile Send private message
kernelOfTruth
Watchman
Watchman


Joined: 20 Dec 2005
Posts: 6108
Location: Vienna, Austria; Germany; hello world :)

PostPosted: Wed Aug 13, 2008 9:32 am    Post subject: Reply with quote

ok guys, here are the setting with which I had the best experience / performance with reiser4 so far:
(this is for desktops, laptops with 1 SATA-hdd, for RAIDs you might need to tweak read_ahead_kb and max_sectors_kb in another way :idea: )

Code:
echo 10 > /proc/sys/vm/page-cluster
# default: 3
#

echo 20 > /proc/sys/vm/swappiness
# default: 60
# By default, Linux will aggressively swap processes out of physical memory onto disk in order to keep the disk cache as large as possible.
# This means that pages that haven't been used recently will be pushed into swap long before the system even comes close to running out of memory, which is an unexpected behavior compared to some operating systems.
# The /proc/sys/vm/swappiness parameter controls how aggressive Linux is in this area.

echo 3000 > /proc/sys/vm/dirty_expire_centisecs
# default: 3000 (30 seconds)
#2 how long data can be in the page cache before it is considered expired and must be written at the next opportunity. Note that this default is very long: a full 30 seconds. That means that under normal circumstances, unless you write enough to trigger the other pdflush method, Linux won't actually commit anything you write until 30 seconds later.

echo 1500  > /proc/sys/vm/dirty_writeback_centisecs
# default: 500 (5 seconds)
#1 how often pdflush wakes up to write data to disk. The default wakes up the two (or more) active threads every five seconds.

echo 10   > /proc/sys/vm/dirty_background_ratio
# default: 10
#3 Maximum percentage of active memory that can be filled with dirty pages before pdflush begins to write them

echo 40   > /proc/sys/vm/dirty_ratio
# default: 40
#4 Maximum percentage of total memory that can be filled with dirty pages before processes are forced to write dirty buffers themselves during their time slice instead of being allowed to do more writes.

echo 50 > /proc/sys/vm/vfs_cache_pressure

for i in /sys/block/sd*; do
         /bin/echo "256" >  $i/queue/read_ahead_kb
done

for i in /sys/block/sd*; do
         /bin/echo "256" >  $i/queue/max_sectors_kb
done


happy testing :)
_________________
https://github.com/kernelOfTruth/ZFS-for-SystemRescueCD/tree/ZFS-for-SysRescCD-4.9.0
https://github.com/kernelOfTruth/pulseaudio-equalizer-ladspa

Hardcore Gentoo Linux user since 2004 :D
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Unsupported Software All times are GMT
Goto page Previous  1, 2, 3 ... , 15, 16, 17  Next
Page 16 of 17

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum