Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
Some ext3 Filesystem Tips
View unanswered posts
View posts from last 24 hours

Goto page Previous  1, 2, 3 ... 9, 10, 11 ... 13, 14, 15  Next  
Reply to topic    Gentoo Forums Forum Index Documentation, Tips & Tricks
View previous topic :: View next topic  
Author Message
PaulBredbury
Watchman
Watchman


Joined: 14 Jul 2005
Posts: 7310

PostPosted: Sun Jan 29, 2006 1:52 am    Post subject: Reply with quote

wrc1944 wrote:
Maybe a few other ext3 users wouldn't mind temporarily setting their partitions to force a boot check every 2-3 mounts and see if this ocurrs on their systems? :lol:

There's no need - my hard drives get checked every few (I think it's 25-ish) reboots by default (which happened a lot when I was experimenting with kernel options), and I didn't have any problems shown.

The cause could be something as trivial as a slight kink in the hard drive cable (i.e. a hardware problem).
Back to top
View user's profile Send private message
codergeek42
Bodhisattva
Bodhisattva


Joined: 05 Apr 2004
Posts: 5142
Location: Anaheim, CA (USA)

PostPosted: Wed Feb 01, 2006 6:05 pm    Post subject: Reply with quote

I've recently noticed that the Gentoo Handbook has added a note about using -O dir_index when creating the filesystems if you wish to use Ext3. Thanks, docs team! :D
_________________
~~ Peter: Programmer, Mathematician, STEM & Free Software Advocate, Enlightened Agent, Transhumanist, Fedora contributor
Who am I? :: EFF & FSF
Back to top
View user's profile Send private message
wrc1944
Advocate
Advocate


Joined: 15 Aug 2002
Posts: 3257
Location: Gainesville, Florida

PostPosted: Wed Feb 01, 2006 8:20 pm    Post subject: Reply with quote

PaulBredbury wrote:
Quote:
There's no need - my hard drives get checked every few (I think it's 25-ish) reboots by default (which happened a lot when I was experimenting with kernel options), and I didn't have any problems shown.

The cause could be something as trivial as a slight kink in the hard drive cable (i.e. a hardware problem).


I tried your suggestion about the cable and replaced it with a new one, but I still have the errors. I also removed the nodiratime option, and reduced commit=600 to commit=60 from all partitions in fstab & grub.conf- still no improvement. Guess I'll try going back to the default 5 second commit for the next boot. These partitions are data=writeback- could that be a factor? I still have no clue as to why the i_block fix is always 8 less for each seemingly random inode, and not some other number.

Actually, since I noticed this, I haven't hooked up my other "backup" box to check on my other ext3 similarly tuned drives (I only have one monitor), so if that box doesn't exhibit this "problem," maybe it really is as you suggest- a specific hard drive problem, and not ext3 or my tuning options related at all.

One other point: If people have the default 25-ish mount check (mine seems to be 30), it seems to me they probably would never realize they had these inode problems being fixed, unless they happened to be watching the boot screen on the 25th mount of the particular partition. I know in my case that was the only reason I noticed, which prompted me set the checks to every 2 mounts to see if this was happening regularly, and sure enough, it is.
_________________
Main box- AsRock x370 Gaming K4
Ryzen 1700, 3.0GHz, 16GB GSkill Flare DDR4 3200mhz
Samsung SATA 1000GB, Radeon HD R7 350 2GB DDR5
Gentoo ~amd64 plasma, glibc-2.29-r5, gcc-9.2.0 kernel-5.3.6-gentoo-r1 USE=experimental
Back to top
View user's profile Send private message
Massimo B.
Veteran
Veteran


Joined: 09 Feb 2005
Posts: 1564
Location: Germany

PostPosted: Sun Feb 05, 2006 12:22 pm    Post subject: Reply with quote

Eventhough I have / and /home as ext3 I decided to have a 4GB file, formatted with reiserfs, and mounted as loop to /var. Also /usr/portage I moved to /var/spool/portage. So I have things like portage, ccache, and proxy cache with lots of small files on a reiserfs file. I prefered a file (on an ext3 fs) to be flexible. portage/ccache is running much quicker now.

Question about atime: Is that atime important for a desktop system? I've tried it also for / in fstab, but it's not going to be mounted with 'atime'.
_________________
ppc:PowerBook5,8 15"(1440)-G4/1.67,2G|amd64:HP EliteBook 8560w,i7-2620M,16G|Acer Z5610 (Core2QuadQ8200),8G|amd64-prefix:OpenSuse|HP EliteDesk 800G1 i7-4790|HP Compaq Pro 6300 i7-3770
Lila-Theme
Back to top
View user's profile Send private message
mudrii
l33t
l33t


Joined: 26 Jun 2003
Posts: 789
Location: Singapore

PostPosted: Mon Feb 06, 2006 9:11 am    Post subject: Reply with quote

This ext3 Optimisation work with RAID 0 and RAID 1 volume ?
_________________
www.gentoo.ro
Back to top
View user's profile Send private message
Massimo B.
Veteran
Veteran


Joined: 09 Feb 2005
Posts: 1564
Location: Germany

PostPosted: Mon Feb 06, 2006 9:35 am    Post subject: Reply with quote

That is what I meant about ext3 with Raid0. I think with mirrored Raid1 this problem doesn't exist, because the write process is the same on each drive.
_________________
ppc:PowerBook5,8 15"(1440)-G4/1.67,2G|amd64:HP EliteBook 8560w,i7-2620M,16G|Acer Z5610 (Core2QuadQ8200),8G|amd64-prefix:OpenSuse|HP EliteDesk 800G1 i7-4790|HP Compaq Pro 6300 i7-3770
Lila-Theme
Back to top
View user's profile Send private message
mudrii
l33t
l33t


Joined: 26 Jun 2003
Posts: 789
Location: Singapore

PostPosted: Mon Feb 06, 2006 3:42 pm    Post subject: Reply with quote

If a stripe size of 64 it should not be a problem thought.
_________________
www.gentoo.ro
Back to top
View user's profile Send private message
Sachankara
l33t
l33t


Joined: 11 Jun 2004
Posts: 696
Location: Stockholm, Sweden

PostPosted: Wed Feb 08, 2006 12:50 pm    Post subject: Reply with quote

"journal_data" has worse performance on large files than ReiserFS. Don't use it unless you want worse performance. Try it for yourselves if you don't believe me.
_________________
Gentoo Hardened Linux 2.6.21 + svorak (Swedish dvorak)
Back to top
View user's profile Send private message
Ainvar
n00b
n00b


Joined: 06 Aug 2004
Posts: 11

PostPosted: Wed Feb 08, 2006 1:48 pm    Post subject: Reply with quote

I have switched to the ext3 with tweaks from reiserfs and have seen increases in a lot of areas except for unpacking large compressed files. This is the only place I have seen a performance hit.
Back to top
View user's profile Send private message
Xk2c
Apprentice
Apprentice


Joined: 17 Jul 2004
Posts: 240

PostPosted: Sat Feb 18, 2006 1:40 pm    Post subject: Reply with quote

Sachankara wrote:
"journal_data" has worse performance on large files than ReiserFS. Don't use it unless you want worse performance. Try it for yourselves if you don't believe me.


Yes you are right. I have tried it and my system is slightly slower.

How can i undo:
Code:
tune2fs -O has_journal -o journal_data /dev/hdXY


and revert to "normal Mode"?
_________________
useful Xterm, Aterm and RXVT-Unicode features
Back to top
View user's profile Send private message
Xk2c
Apprentice
Apprentice


Joined: 17 Jul 2004
Posts: 240

PostPosted: Sat Feb 18, 2006 2:45 pm    Post subject: Reply with quote

Xk2c wrote:
How can i undo:
Code:
tune2fs -O has_journal -o journal_data /dev/hdXY


and revert to "normal Mode"?


Code:
# tune2fs -o ^journal_data /dev/hdXY
# tune2fs -O has_journal -o journal_data_ordered /dev/hdXY


Apart from that nice Howto codergeek42. Thanks. :D 8)
_________________
useful Xterm, Aterm and RXVT-Unicode features
Back to top
View user's profile Send private message
Massimo B.
Veteran
Veteran


Joined: 09 Feb 2005
Posts: 1564
Location: Germany

PostPosted: Sat Feb 18, 2006 2:57 pm    Post subject: Reply with quote

Somewhere at redhat I read about benchmarks, that data=journal should give better performance than writeback, especially write processes. I can't find the article anymore. Here is the only thing I've found about it.
Moreover with data=journal I notice more cpu load from kjournald.
_________________
ppc:PowerBook5,8 15"(1440)-G4/1.67,2G|amd64:HP EliteBook 8560w,i7-2620M,16G|Acer Z5610 (Core2QuadQ8200),8G|amd64-prefix:OpenSuse|HP EliteDesk 800G1 i7-4790|HP Compaq Pro 6300 i7-3770
Lila-Theme
Back to top
View user's profile Send private message
get sirius
Guru
Guru


Joined: 27 Apr 2002
Posts: 316
Location: Madison, WI

PostPosted: Sun Mar 05, 2006 11:23 pm    Post subject: Reply with quote

I notice, from using tune2fs -l, that sparse_super is enabled by default. From man tune2fs, I see that sparse_super is primarily aimed at partitions filled with large files. From this, I infer that sparse_super would be most useful in /usr/portage/distfiles. Has anyone experimented with turning sparse_super off and thereby perhaps making the filesystem more suitable for small files :?: . Turning this option off might make up some of the apparent performance difference between ext3 and reiser 3.6 on partitions containing large numbers of small files (like /usr/portage, for example). :)

If no one has come across tests to measure the performance diff, could anyone suggest a test protocol that would?
Back to top
View user's profile Send private message
wrc1944
Advocate
Advocate


Joined: 15 Aug 2002
Posts: 3257
Location: Gainesville, Florida

PostPosted: Mon Mar 06, 2006 2:48 pm    Post subject: Reply with quote

I'm not sure sparse_super has anything to do with the size of an actual file(s) on a partition.

Man tune2fs:

sparse_super
Limit the number of backup superblocks to save space on large filesystems.

-s [0|1]
Turn the sparse super feature off or on. Turning this feature on saves space on really big filesystems. This is the same as using the -O sparse_super option.

Unless I'm mistaken, I read this to mean it only limits the number of backup superblocks, for a really large partition and it's files, whatever the size and number of files there may be stored on the specific partition.

I don't know what they mean by "large filesystems," other than a filesystem on a very large partition, and they don't specify what that means. I guess it might be 4GB, or 20GB plus- who knows?
_________________
Main box- AsRock x370 Gaming K4
Ryzen 1700, 3.0GHz, 16GB GSkill Flare DDR4 3200mhz
Samsung SATA 1000GB, Radeon HD R7 350 2GB DDR5
Gentoo ~amd64 plasma, glibc-2.29-r5, gcc-9.2.0 kernel-5.3.6-gentoo-r1 USE=experimental
Back to top
View user's profile Send private message
Drysh
Apprentice
Apprentice


Joined: 06 Apr 2005
Posts: 203
Location: São Paulo, Brazil

PostPosted: Mon Mar 06, 2006 7:23 pm    Post subject: Reply with quote

sparse_super has nothing to do with file size, but only with partition size. I tested turning it on and off, and I started to notice some change after 100 GB. I don't recall exactly how much. You should set it if you are partitionng the whole disk as a single ext3 (or using a huge disk array), but I don't think it will hurt if you set sparse_super for anything larger than 5 GB.
Back to top
View user's profile Send private message
get sirius
Guru
Guru


Joined: 27 Apr 2002
Posts: 316
Location: Madison, WI

PostPosted: Tue Mar 07, 2006 12:41 am    Post subject: Reply with quote

I see what you (both of you :) ) mean - I read it wrong, forgetting the distinction between filesystem and file :oops:. Thanks for setting me straight!
Back to top
View user's profile Send private message
pointers
Tux's lil' helper
Tux's lil' helper


Joined: 18 Apr 2004
Posts: 123

PostPosted: Tue Mar 07, 2006 10:54 am    Post subject: High ratio of non-contiguous Reply with quote

Hello,
Yesterday I have shutdown my linux web server after 194 days. We haven't rebooted the server since we have bought and set up the server and when I reboot the server I have seen 24.5% con-contiguous partition. Read speed was so low and I hadn't realised it yet. Shame on me... I have read some documents Drobbins document and others about fragmentation but
it is really bad if you have a web server and the only way you can avoid this is to archive entire partition then restore files again on a production server, in fact in some partitions we have 1-2 million image files and if you have 2nd level of subdirectory, archiving is a pain...
I just would like to ask there will be any way of avoiding fragmentation or to keep the ratio on acceptable levels. Is there any
work for this lots of fragmentation such as dynamic relocation etc. (in the near future)


Thanks a lot.
Back to top
View user's profile Send private message
codergeek42
Bodhisattva
Bodhisattva


Joined: 05 Apr 2004
Posts: 5142
Location: Anaheim, CA (USA)

PostPosted: Tue Mar 07, 2006 4:51 pm    Post subject: Reply with quote

@pointers, none that I am aware of, aside from re-indexing the directory structure if it's unmounted:
Code:
# /sbin/e2fsck -Dy /dev/partition

_________________
~~ Peter: Programmer, Mathematician, STEM & Free Software Advocate, Enlightened Agent, Transhumanist, Fedora contributor
Who am I? :: EFF & FSF
Back to top
View user's profile Send private message
pointers
Tux's lil' helper
Tux's lil' helper


Joined: 18 Apr 2004
Posts: 123

PostPosted: Tue Mar 07, 2006 10:01 pm    Post subject: Reply with quote

Hello,
I think the root of the problem is putting files into non-contiguous blocks. If files are distributed into non-contiguous blocks
index's help is descreasing because seeks are not eliminated that much with index right? I hope one day dynamic relocation can be possible

Thanks for your suggestion.
Back to top
View user's profile Send private message
codergeek42
Bodhisattva
Bodhisattva


Joined: 05 Apr 2004
Posts: 5142
Location: Anaheim, CA (USA)

PostPosted: Wed Mar 08, 2006 1:27 am    Post subject: Reply with quote

@pointers: Yes. From what I understand, directory-indexing merely helps the VFS/Ext3 subsystem to locate the file block(s) on disk faster (Logarithmic versus polynomial search/sort running time, you're familiar with Big O Notation and the like). The disk itself still requires the same seek time in most cases.

If I'm not mistaken, Extents and Dynamic-Relocation are in planning for the next incarnation of Ext3 (Ext4?).
_________________
~~ Peter: Programmer, Mathematician, STEM & Free Software Advocate, Enlightened Agent, Transhumanist, Fedora contributor
Who am I? :: EFF & FSF
Back to top
View user's profile Send private message
XenoTerraCide
Veteran
Veteran


Joined: 18 Jan 2004
Posts: 1418
Location: MI, USA

PostPosted: Wed Mar 08, 2006 3:06 am    Post subject: Reply with quote

yes it's ext4. I wonder when it's going to be ready and how long it will take before it's stable...
_________________
I don't hang out here anymore, try asking on http://unix.stackexchange.com/ if you want my help.
Back to top
View user's profile Send private message
Special K
n00b
n00b


Joined: 24 Aug 2004
Posts: 20

PostPosted: Wed Mar 08, 2006 3:11 am    Post subject: ext3/reiser on 24/7 Reply with quote

when using ext3 on a server which will reboot let's say every 200days only, should I run fsck sometimes?
Is there anything wich sould be done so ext3 is not defragmenting? collecting errors until crash?

Does reiserfs need any maintenance during 24/7?
_________________
Sam & Max 2 development has been resurrected.
http://www.telltalegames.com/samandmax
Back to top
View user's profile Send private message
XenoTerraCide
Veteran
Veteran


Joined: 18 Jan 2004
Posts: 1418
Location: MI, USA

PostPosted: Wed Mar 08, 2006 3:23 am    Post subject: Reply with quote

lol I switched to reiserfs because I kept getting errors I couldn't recover the lost data from... I don't recommend reiserfs, for anything that can't easily be replaced.
_________________
I don't hang out here anymore, try asking on http://unix.stackexchange.com/ if you want my help.
Back to top
View user's profile Send private message
Q-collective
Advocate
Advocate


Joined: 22 Mar 2004
Posts: 2056

PostPosted: Wed Mar 08, 2006 3:25 am    Post subject: Reply with quote

ext4? I missed something?
Back to top
View user's profile Send private message
XenoTerraCide
Veteran
Veteran


Joined: 18 Jan 2004
Posts: 1418
Location: MI, USA

PostPosted: Wed Mar 08, 2006 3:27 am    Post subject: Reply with quote

ext4 is under development. It hasn't been released yet.
_________________
I don't hang out here anymore, try asking on http://unix.stackexchange.com/ if you want my help.
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Documentation, Tips & Tricks All times are GMT
Goto page Previous  1, 2, 3 ... 9, 10, 11 ... 13, 14, 15  Next
Page 10 of 15

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum