Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
Some ext3 Filesystem Tips
View unanswered posts
View posts from last 24 hours

Goto page Previous  1, 2, 3 ... 8, 9, 10 ... 13, 14, 15  Next  
Reply to topic    Gentoo Forums Forum Index Documentation, Tips & Tricks
View previous topic :: View next topic  
Author Message
XenoTerraCide
Veteran
Veteran


Joined: 18 Jan 2004
Posts: 1418
Location: MI, USA

PostPosted: Wed Jan 18, 2006 12:39 am    Post subject: Reply with quote

yeah... my problem is inodes...
_________________
I don't hang out here anymore, try asking on http://unix.stackexchange.com/ if you want my help.
Back to top
View user's profile Send private message
XenoTerraCide
Veteran
Veteran


Joined: 18 Jan 2004
Posts: 1418
Location: MI, USA

PostPosted: Wed Jan 18, 2006 1:02 am    Post subject: Reply with quote

k I got the problem fixed
Code:
mke2fs -b 1024 -i 1024
df -h
Code:
Filesystem            Size  Used Avail Use% Mounted on
/dev/sda3              21G  8.1G   12G  42% /
udev                  379M  236K  378M   1% /dev
/dev/sda4              78G   52G   27G  66% /mnt/winntfs
/dev/hda1              38G  9.9G   28G  27% /mnt/windows
shm                   379M     0  379M   0% /dev/shm
/dev/sda6             838M  197M  593M  25% /usr/portage
/dev/sda7             1.9G   33M  1.8G   2% /usr/portage/distfiles
df -hi
Code:
Filesystem            Inodes   IUsed   IFree IUse% Mounted on
/dev/sda3               2.7M    444K    2.2M   17% /
udev                     95K    5.2K     90K    6% /dev
/dev/sda4                72K     18K     55K   24% /mnt/winntfs
/dev/hda1                59K     59K      10  100% /mnt/windows
shm                      95K       1     95K    1% /dev/shm
/dev/sda6               958K    132K    826K   14% /usr/portage
/dev/sda7               239K      10    239K    1% /usr/portage/distfiles
more info coming soon...
_________________
I don't hang out here anymore, try asking on http://unix.stackexchange.com/ if you want my help.
Back to top
View user's profile Send private message
XenoTerraCide
Veteran
Veteran


Joined: 18 Jan 2004
Posts: 1418
Location: MI, USA

PostPosted: Wed Jan 18, 2006 1:04 am    Post subject: Reply with quote

one question should I have used a bigger number for -i? like 2048?
_________________
I don't hang out here anymore, try asking on http://unix.stackexchange.com/ if you want my help.
Back to top
View user's profile Send private message
XenoTerraCide
Veteran
Veteran


Joined: 18 Jan 2004
Posts: 1418
Location: MI, USA

PostPosted: Wed Jan 18, 2006 2:49 am    Post subject: Reply with quote

is there a way to resize ext2/3 partitions?
_________________
I don't hang out here anymore, try asking on http://unix.stackexchange.com/ if you want my help.
Back to top
View user's profile Send private message
codergeek42
Bodhisattva
Bodhisattva


Joined: 05 Apr 2004
Posts: 5142
Location: Anaheim, CA (USA)

PostPosted: Wed Jan 18, 2006 3:11 am    Post subject: Reply with quote

XenoTerraCide wrote:
is there a way to resize ext2/3 partitions?
Yes. You can add the option while creating the fileystem:
Code:
# /sbin/mkfs.ext3 -O dir_index -E resize=max_online_resize /dev/hdXY
This allows the filesystem to grow to a size of max_online_resize blocks. (Check the mke2fs(8) man page for more information.)

Alternatively, you can use something like LVM to manage the partitioning. I think parted can do it too, but I'm not certain.
_________________
~~ Peter: Programmer, Mathematician, STEM & Free Software Advocate, Enlightened Agent, Transhumanist, Fedora contributor
Who am I? :: EFF & FSF
Back to top
View user's profile Send private message
wrc1944
Advocate
Advocate


Joined: 15 Aug 2002
Posts: 3260
Location: Gainesville, Florida

PostPosted: Fri Jan 20, 2006 11:38 pm    Post subject: Reply with quote

Well, I've run into a strange bothersome ext3 mystery.
Suddenly (I haven't noticed this before), I get this on my ext3 tuned partitions (changed from reiserfs 3-4 weeks ago) at the end of dmesg:

JBD: barrier-based sync failed on hda7 - disabling barriers
JBD: barrier-based sync failed on hda5 - disabling barriers
JBD: barrier-based sync failed on hda3 - disabling barriers
spurious 8259A interrupt: IRQ7.
-----------------------------------------------------------------------------------
I searched the forum and googled everywhere for many hours trying to figure this out, and came up with some info, but no mention of what this really means in terms of "should I use the ext3 barrier=1 mount option in fstab to re-enable," or not. I need to find out if there are any serious consequences to enabling barrier again, because I feel my system must be disabling it for some important reason I can't seem to understand. I don't want to modprobe JBD, or add the fstab barrier option until I really know what's going on. lsmod has never shown JBD.

On a similar box set up the same way (except it's a gcc-4.1 rig), this doesn't happen. The only difference is the commit is default 5 with data=journal, and on the box with the above disabling barrier lines commit=600, and data=writeback. On the disabling barriers box there is also an ext2 /mnt/portage directory, and another old reiserfs data partition I didn't change yet.

I guess I'm asking experienced ext3 users what all this actually means- can it be ignored, or is it something serious I need to contend with? My system seems to function normally, but I subjectively feel it has become less responsive than before I noticed this.

When I booted the next time, there was a forced check, and it corrected many inodes on hda3 (my / partition), and the JBD: barrier-based sync failed on hda3 - disabling barriers line wasn't in dmesg for 3-4 boots. Now it has returned on hda3, along with one inode fix on / at last boot.

I've been finding things like this on the internet:

Quote:
You are running a 2.6 kernel. Is this something new? Or did you
recently change the filesystem type on hdb1? This jbd message can be
considered a warning. Does an lsmod show jbd? If not, modprobing jbd
and adding it to your initrd will probably fix it for you. If you
have questions about how to do this, I can walk you through it.

If you are interested in what this is all about... request/write
barriers work by guaranteeing that disk operations before the barrier
will be written before those after the barrier. In other words, the
device won't reorder the disk writes across the barrier even though it
might be more efficient to write that way (like less movement of the
heads). Write barriers speed up I/O for applications that care very
much about write order (like journalling filesystems), because instead
of writing one chunk at a time and waiting for the write to finish
before sending the next chunk, the application can write multiple
chunks with barriers between them. If the application were to write
multiple chunks without the barriers and without waiting for each
individual chunk to be written before sending the next, the device
might reorder the chunks being written for efficiency. Most
applications wouldn't care, but some would... like a journaling
filesystem that cares very much what blocks are updated on the device
when.


At this point, I'm not sure I understand what this means :?
_________________
Main box- AsRock x370 Gaming K4
Ryzen 1700, 3.0GHz, 16GB GSkill Flare DDR4 3200mhz
Samsung SATA 1000GB, Radeon HD R7 350 2GB DDR5
Gentoo ~amd64 plasma, glibc-2.30-r2, gcc-9.2.0 kernel-5.3.9-gentoo USE=experimental
Back to top
View user's profile Send private message
wrc1944
Advocate
Advocate


Joined: 15 Aug 2002
Posts: 3260
Location: Gainesville, Florida

PostPosted: Sat Jan 21, 2006 1:46 am    Post subject: Reply with quote

Now I'm thinking this is kernel related. I currently have 5 kernels on this box, compiled in this order (first to last, date-wise).

2.6.13-gvivid
2.6.15-mm3
2.6.15-ext3 (my own patched vanilla 2.6.15 with individually applied ext3 multiple block allocation patches)
2.6.15-nitro2
2.6.15-nitro3

My above problems were all when booted with nitro3 (maybe I didn't notice with nitro2).
Booted with nitro2, only hda5 and hda7 had the barriers disabled.

Booted with 2.6.15-ext3, at boot, once again 1 inode needed fixing, and the message afterwards was reboot needed. After reboot, dmesg reported no disabled barriers.

After the 2.6.15-ext3 boot (all OK), I rebooted with 2.6.15-mm3 (which also has the ext3 multi-block patches), all is also OK- no disabled barriers. (Of course the "OK" assumes any of this even matters). :?

I booted in reverse order to test, last to first.
----------------------------------------------------------------------------------
Nitro3 has the:
# fix for the ext3 multiblock patch
27_ext3-get-blocks-maping-multiple-blocks-at-a-once-ext3_getblk-fix.patch

Nitro2 has:
ext3-get-blocks-adjust-accounting-info-in-build-fix.patch
ext3-get-blocks-adjust-accounting-info-in.patch
ext3-get-blocks-adjust-reservation-window-size-for.patch
ext3-get-blocks-maping-multiple-blocks-at-a-once-vs-ext3_readdir-use-generic-readahead.patch
ext3-get-blocks-maping-multiple-blocks-at-a-once.patch
ext3-get-blocks-multiple-block-allocation.patch
ext3-get-blocks-support-multiple-blocks-allocation-in.patch
ext3_readdir-use-generic-readahead.patch

My own ext3 multiblock patched kernel had all of ext3 patches included in nitro2 (and IIRC one more from mm3, broken out), but I did that a day or two before nitro2 was posted.
----------------------------------------------------------------------------------------
Considering both of my last posts on this ext3 stuff, has anyone got opinions, conclusions, advice, on exactly why this would be happening? I built this box, and have compiled many hundreds of testing kernels, so I doubt it's any config anomaly I missed. I'm thinking the ext3 multiblock stuff is OK, and it's something in the Nitro patch set that conflicts???

Just to be thorough, here's my dumpe2fs for hda3, /, and fstab.
One thing I'm rethinking is the wisdom of disabling the ext3 boot checks with tune2fs -c 0 -i 0 /dev/hdXY. If it means slower boot time to insure the apparently often occurring ext3 inode problems are fixed immediately, so be it- I can live with that.( I hope this post is considered "on topic," as it is all about ext3 problems). :)

/dev/hda1 /boot ext3 noauto,noatime 1 2
/dev/hda3 / ext3 noatime,nodiratime 0 1
/dev/hda2 none swap sw 0 0
/dev/hda5 /home ext3 noatime,nodiratime,commit=600 0 2
/dev/hda6 /mnt/portage ext2 noatime,nodiratime 0 2
/dev/hda7 /mnt/rwstorage ext3 noatime,nodiratime,commit=600 0 2
/dev/hda8 /mnt/dump ext3 noatime,nodiratime,commit=600 0 2
/dev/hda9 /mnt/data2 reiserfs notail,noatime,user 0 2
/dev/sda1 /mnt/sda1 vfat auto,rw,user 0 0
/dev/sda5 /mnt/sda5 vfat auto,rw,user 0 0
/dev/sda6 /mnt/sda6 vfat auto,rw,user 0 0
/dev/sda7 /mnt/sda7 vfat auto,rw,user 0 0

-----------------------------------------------------------------
mymachine wrc # dumpe2fs /dev/hda3
dumpe2fs 1.38 (30-Jun-2005)
Filesystem volume name: <none>
Last mounted on: <not available>
Filesystem UUID: 484ed04b-a858-425c-a917-0a25a1b28990
Filesystem magic number: 0xEF53
Filesystem revision #: 1 (dynamic)
Filesystem features: has_journal dir_index filetype needs_recovery sparse_super
Default mount options: journal_data_writeback
Filesystem state: clean
Errors behavior: Continue
Filesystem OS type: Linux
Inode count: 1221600
Block count: 2441376
Reserved block count: 122068
Free blocks: 1598676
Free inodes: 1000005
First block: 0
Block size: 4096
Fragment size: 4096
Blocks per group: 32768
Fragments per group: 32768
Inodes per group: 16288
Inode blocks per group: 509
Filesystem created: Sat Dec 10 09:35:31 2005
Last mount time: Fri Jan 20 15:13:34 2006
Last write time: Fri Jan 20 15:13:34 2006
Mount count: 2
Maximum mount count: 3
Last checked: Fri Jan 20 14:28:45 2006
Check interval: 0 (<none>)
Reserved blocks uid: 0 (user root)
Reserved blocks gid: 0 (group root)
First inode: 11
Inode size: 128
Journal inode: 8
Default directory hash: tea
Directory Hash Seed: 36681716-6609-4e78-a3f2-0699a3adb1e9
Journal backup: inode blocks

EDIT: (forgot)
mymachine wrc # df -hi
Filesystem Inodes IUsed IFree IUse% Mounted on
/dev/hda3 1.2M 217K 977K 19% /
udev 111K 883 110K 1% /dev
cachedir 1.2M 217K 977K 19% /lib/splash/cache
/dev/hda5 597K 119K 478K 20% /home
/dev/hda6 1.4M 132K 1.3M 10% /mnt/portage
/dev/hda7 358K 65K 294K 18% /mnt/rwstorage
/dev/hda8 478K 131K 347K 28% /mnt/dump
/dev/hda9 0 0 0 - /mnt/data2
/dev/sda1 0 0 0 - /mnt/sda1
/dev/sda5 0 0 0 - /mnt/sda5
/dev/sda6 0 0 0 - /mnt/sda6
/dev/sda7 0 0 0 - /mnt/sda7
none 111K 1 111K 1% /dev/shm
_________________
Main box- AsRock x370 Gaming K4
Ryzen 1700, 3.0GHz, 16GB GSkill Flare DDR4 3200mhz
Samsung SATA 1000GB, Radeon HD R7 350 2GB DDR5
Gentoo ~amd64 plasma, glibc-2.30-r2, gcc-9.2.0 kernel-5.3.9-gentoo USE=experimental
Back to top
View user's profile Send private message
wrc1944
Advocate
Advocate


Joined: 15 Aug 2002
Posts: 3260
Location: Gainesville, Florida

PostPosted: Tue Jan 24, 2006 7:01 am    Post subject: Reply with quote

I just ran across this little gem- it's a disk allocation/fragmentaion viewer for ext2/3. DAVL can collect the fragmentation status information for a partition, a directory, and a file, regardless of whether filesystem is mounted, and can output its text data or visualize it.

Check out the screen shots- Look Great, with full info in a nice gui!
http://davl.sourceforge.net/

I looked, but there is no ebuild in portage.

Perhaps somebody experienced in creating ebuilds could whip one up in no time. It requires GTK+1.2 or GTK+2, and looks pretty simple, but I've never created an ebuild before, or I'd do it myself. I'll give it a try (something I need to learn anyway- and this is a good excuse), but it might take me some time to succeeed. I'm pretty sure ext2/3 users will likely want this before I can create one.
_________________
Main box- AsRock x370 Gaming K4
Ryzen 1700, 3.0GHz, 16GB GSkill Flare DDR4 3200mhz
Samsung SATA 1000GB, Radeon HD R7 350 2GB DDR5
Gentoo ~amd64 plasma, glibc-2.30-r2, gcc-9.2.0 kernel-5.3.9-gentoo USE=experimental
Back to top
View user's profile Send private message
PaulBredbury
Watchman
Watchman


Joined: 14 Jul 2005
Posts: 7310

PostPosted: Tue Jan 24, 2006 2:38 pm    Post subject: Reply with quote

I've submitted an ebuild for davl to bugzilla :)
Back to top
View user's profile Send private message
wrc1944
Advocate
Advocate


Joined: 15 Aug 2002
Posts: 3260
Location: Gainesville, Florida

PostPosted: Tue Jan 24, 2006 6:19 pm    Post subject: Reply with quote

Many thanks!
I just looked at your ebuild, and amazingly it seems like even I could have eventually figured it out.

When I first extracted the davl tarball, and looked at the Gentoo ebuild howto and wiki documentation, and at the davl makefiles and read me, creating an ebuild appeared to be pretty complicated procedure. I couldn't figure out how I could decide which of the ebuild options davl actually needed, and looking at yours, it appears davl needs virtually none of them, and is essentially just a "bare-bones" ebuild.

I don't want to go OT here (ext3), so I'll leave all my ebuild related questions except one for another thread. My one question is:
Is this ebuild going to also modify the kernel so it installs the optional "davl_liveinfo" module mentioned in the davl README, so that it also reports on mounted partitions?
_________________
Main box- AsRock x370 Gaming K4
Ryzen 1700, 3.0GHz, 16GB GSkill Flare DDR4 3200mhz
Samsung SATA 1000GB, Radeon HD R7 350 2GB DDR5
Gentoo ~amd64 plasma, glibc-2.30-r2, gcc-9.2.0 kernel-5.3.9-gentoo USE=experimental
Back to top
View user's profile Send private message
PaulBredbury
Watchman
Watchman


Joined: 14 Jul 2005
Posts: 7310

PostPosted: Tue Jan 24, 2006 7:21 pm    Post subject: Reply with quote

wrc1944 wrote:
Is this ebuild going to also modify the kernel so it installs the optional "davl_liveinfo" module

Not in its current state. I think it would need the linux-mod eclass.
Back to top
View user's profile Send private message
wrc1944
Advocate
Advocate


Joined: 15 Aug 2002
Posts: 3260
Location: Gainesville, Florida

PostPosted: Tue Jan 24, 2006 7:26 pm    Post subject: Reply with quote

Just tried your ebuild in my overlay- apparently it works great, and does report on a mounted partition, but I don't seem to have the kernel module as described below from the davl README in /lib/modules/my running kernel.

I build my kernels in /home/wrc/kern/linux-2.6.xx, and always have the /usr/src/linux symlink pointing to the running kernel. The davl path_list seemed Ok to me, but since I have no kernel davl module, I must have done something wrong, or left out a step (beginning at # 4. in the README below), or the ebuild needs more instructions? Also, since I have no davl module, I don't understand how gdavl is apparently working on mounted partitions, and am worried about even running this on a mounted partition.

davl path_list:

KERN_DIR = /lib/modules/$(shell uname -r)/build
BIN_DIR = /usr/local/bin
DRV_DIR = /lib/modules/$(shell uname -r)/kernel/drivers/davl
MAN_DIR = /usr/local/man
GTK_VER = GTK2
# If you want to use GTK1.2 for the GUI tool, comment out the above line,
# and uncomment the below line.
#GTK_VER = GTK1
-------------------------------------------------------------------------------
From the davl README- source section: (I figured with Gentoo, we aren't concerned with all of these steps, but possibly some of them, particularly the KERN_DIR = line ?).

B. Install from source code
---------------------------

1. Preparing kernel source code.
(optional for preparing the davl_liveinfo build --- case of Fedora Core 2)

a. Change to super-user

$ su
(input super-user password here)

b. Download and install the kernel source package.

# rpm -i kernel-2.6.5-1.358.src.rpm

c. Extract the kernel source

# rpmbuild -bp --target i686 /usr/src/redhat/SPECS/kernel-2.6.spec

d. Move to the kernel source directory

# cd /usr/src/redhat/BUILD/kernel-2.6.5/linux-2.6.5

e. Edit Makefile, and modify "EXTRAVERSION" line

# vi Makefile
("EXTRAVERSION = -1.358"
'-1.358' is part of the "uname -r" command result)

f. Prepare the kernel source for module build
# make prepare-all

g. Change to normal-user

# exit

2. Download davl-XXX.tar.bz2 from the project page.
(XXX is version number of davl)

3. Extract archive file.

$ cd $(WHERE_YOUR_WORK_DIRECTORY)
$ tar jxvf davl-XXX.tar.bz2
$ cd davl

4. Modify the macro values in the "path_file" according to your system.

macro meaning
-------- ---------------------------------
* KERN_DIR kernel source directory
* BIN_DIR executable file install directory
* DRV_DIR kernel module install directory
* MAN_DIR man page install directory
* GTK_VER GTK version (use "GTK1" or "GTK2")

5. Build executable file.

$ make

or If you want to build the kernel module too, do next

$ make WITH_DRV=1

6. Install executable file.

$ su
(input super-user password here)
# make install

or If you want to install the kernel module too, do next

# make WITH_DRV=1 install

III. How to use
===============
1. If you installed the kernel module, load the davl_liveinfo module before
executing cdavl or gdavl.

# modprobe davl_liveinfo

2. Use cdavl or gdavl. For displaying the usage, use -h option.
_________________
Main box- AsRock x370 Gaming K4
Ryzen 1700, 3.0GHz, 16GB GSkill Flare DDR4 3200mhz
Samsung SATA 1000GB, Radeon HD R7 350 2GB DDR5
Gentoo ~amd64 plasma, glibc-2.30-r2, gcc-9.2.0 kernel-5.3.9-gentoo USE=experimental
Back to top
View user's profile Send private message
XenoTerraCide
Veteran
Veteran


Joined: 18 Jan 2004
Posts: 1418
Location: MI, USA

PostPosted: Wed Jan 25, 2006 6:01 am    Post subject: Reply with quote

although you've all prolly read this I thought it would be good to post the link. http://www.gentoo.org/doc/en/articles/l-afig-p8.xml it further explain's our ext3 optimization's.
_________________
I don't hang out here anymore, try asking on http://unix.stackexchange.com/ if you want my help.
Back to top
View user's profile Send private message
BeteNoire
Veteran
Veteran


Joined: 25 Sep 2005
Posts: 1827

PostPosted: Wed Jan 25, 2006 10:25 am    Post subject: Reply with quote

I decided to try ext3 tips again. I've created one big partition and made ext3 on it and tuned it. I've done what is written in the first post of this thread.
Code:
Filesystem features:      has_journal dir_index filetype needs_recovery sparse_super
Default mount options:    journal_data

One thing is still unacceptable for me : long time when changing directory to that partition in MC/Krusader. Takes about 5-8 seconds!
Partition is 70GB large and stores about 12 tousands files.
What would you advise me to do to decrease access time to this dir/partition?
_________________
powered by power plant
Back to top
View user's profile Send private message
XenoTerraCide
Veteran
Veteran


Joined: 18 Jan 2004
Posts: 1418
Location: MI, USA

PostPosted: Thu Jan 26, 2006 11:39 am    Post subject: Reply with quote

I got nothing for your MC problem. wish i could help.

in further news. I am currently conducting benchmark test of putting /usr/portage and /usr/portage/distfiles in there own partition's vs not doing so.
_________________
I don't hang out here anymore, try asking on http://unix.stackexchange.com/ if you want my help.
Back to top
View user's profile Send private message
XenoTerraCide
Veteran
Veteran


Joined: 18 Jan 2004
Posts: 1418
Location: MI, USA

PostPosted: Thu Jan 26, 2006 12:30 pm    Post subject: Reply with quote

all of these test's were done on a freshly wiped /usr/portage directory. The first date is the time the sync command executed the second is the time it finished

in this one /usr/portage is part of the / filesystem which has a 4096 blocksize and (dir_index,has_journal,data_journal)
Code:

Thu Jan 26 06:34:01 EST 2006
Thu Jan 26 06:40:03 EST 2006


this one was done with a seperate 1GB /usr/portage, and a seperate distfiles directory. blocksize 1024 (dir_index,has_journal,data_journal) (created with -i 1024 "don't want to run out of inodes")
Code:

Thu Jan 26 06:42:05 EST 2006
Thu Jan 26 06:47:46 EST 2006


this is an ext2 test, blocksize 1024 (created with -i 1024) same 1GB partition
Code:

Thu Jan 26 06:51:21 EST 2006
Thu Jan 26 06:57:00 EST 2006


I added dir_index to the same fs as prior test.
Code:
 
Thu Jan 26 07:02:55 EST 2006
Thu Jan 26 07:08:36 EST 2006


one variable I am unable to control is the speed I get them from the mirror. they all came out pretty close. but I think... the second test one over the last by a hair. but I'm tired and I'm doing fuzzy math in my head. I may be wrong the numbers are there if anyone want's to do the math.
_________________
I don't hang out here anymore, try asking on http://unix.stackexchange.com/ if you want my help.
Back to top
View user's profile Send private message
RuiP
l33t
l33t


Joined: 15 Jan 2005
Posts: 643

PostPosted: Thu Jan 26, 2006 1:04 pm    Post subject: Reply with quote

They all take 5 to 6 minutes, with that average times the variations on seconds are irrelevant (as they are usually in any process that involves network connections).

The diferences in time are note relevant too, taking in count that you do only one sync for measure something that had huge fluctuations and a lot of variables associated. You should have done measures over a certain period of time, like 1-2 weeks each case (you can check times on emerge.log) and make a estimative of averages.
Benchmarks are a fastidious and boring thing to do and delicate cases take a lot of time, if one wants some accuracy.

Anyway the main purpose to separate distfiles from portage was avoid fragmentation. The prejudice or beneficts from that could only be measuread on systems after a certain period of time.
Only moving files to another partition wont affect imediattly fragmentation on that partition.
Back to top
View user's profile Send private message
carpman
Advocate
Advocate


Joined: 20 Jun 2002
Posts: 2202
Location: London - UK

PostPosted: Thu Jan 26, 2006 4:39 pm    Post subject: Reply with quote

XenoTerraCide wrote:
all of these test's were done on a freshly wiped /usr/portage directory. The first date is the time the sync command executed the second is the time it finished

in this one /usr/portage is part of the / filesystem which has a 4096 blocksize and (dir_index,has_journal,data_journal)
Code:

Thu Jan 26 06:34:01 EST 2006
Thu Jan 26 06:40:03 EST 2006


this one was done with a seperate 1GB /usr/portage, and a seperate distfiles directory. blocksize 1024 (dir_index,has_journal,data_journal) (created with -i 1024 "don't want to run out of inodes")
Code:

Thu Jan 26 06:42:05 EST 2006
Thu Jan 26 06:47:46 EST 2006


this is an ext2 test, blocksize 1024 (created with -i 1024) same 1GB partition
Code:

Thu Jan 26 06:51:21 EST 2006
Thu Jan 26 06:57:00 EST 2006


I added dir_index to the same fs as prior test.
Code:
 
Thu Jan 26 07:02:55 EST 2006
Thu Jan 26 07:08:36 EST 2006


one variable I am unable to control is the speed I get them from the mirror. they all came out pretty close. but I think... the second test one over the last by a hair. but I'm tired and I'm doing fuzzy math in my head. I may be wrong the numbers are there if anyone want's to do the math.



You should set up a local sync server, as i already have, this mean you can control the speed of sync as you know it is on local network and will not have had any changes, provided you don't sync master.
_________________
Work Station - 64bit
Gigabyte GA X48-DQ6 Core2duo E8400
8GB GSkill DDR2-1066
SATA Areca 1210 Raid
BFG OC2 8800 GTS 640mb
--------------------------------
Notebook
Samsung Q45 7100 4gb
Back to top
View user's profile Send private message
XenoTerraCide
Veteran
Veteran


Joined: 18 Jan 2004
Posts: 1418
Location: MI, USA

PostPosted: Thu Jan 26, 2006 8:46 pm    Post subject: Reply with quote

problem is the machine I would use as the server is the one I did the test on. the only other machine I own is a 64-bit laptop with a 4200 rpm HD. it's not really the ideal system for a benchmark test.
_________________
I don't hang out here anymore, try asking on http://unix.stackexchange.com/ if you want my help.
Back to top
View user's profile Send private message
wrc1944
Advocate
Advocate


Joined: 15 Aug 2002
Posts: 3260
Location: Gainesville, Florida

PostPosted: Fri Jan 27, 2006 1:02 am    Post subject: Reply with quote

Question for experienced ext3 users:
I just switched over from reiser about a month ago, and left my tuned ext3 / partition set at the default boot check count, and disabled the boot check on my four other ext3 partitions, as recommended on several websites I looked at.

I reboot at least once a day, and when the first check came up on /, it had to fix 20-30 inodes (like "inode 1110604 was 336, should be 328, fixed- reboot"). This had me concerned, and led me to question the wisdom of disabling the boot check, so I enabled it all partitions to check every 3 boots, to see what was going on.

Since then, at nearly every boot, at least one partition has an inode fix or two to be done. Is this normal ext3 behavior, or is there something wrong with my system causing this? This never happened with reiserfs (or at least it never was on the bootup screen that I noticed)
_________________
Main box- AsRock x370 Gaming K4
Ryzen 1700, 3.0GHz, 16GB GSkill Flare DDR4 3200mhz
Samsung SATA 1000GB, Radeon HD R7 350 2GB DDR5
Gentoo ~amd64 plasma, glibc-2.30-r2, gcc-9.2.0 kernel-5.3.9-gentoo USE=experimental
Back to top
View user's profile Send private message
carpman
Advocate
Advocate


Joined: 20 Jun 2002
Posts: 2202
Location: London - UK

PostPosted: Fri Jan 27, 2006 6:32 am    Post subject: Reply with quote

XenoTerraCide wrote:
problem is the machine I would use as the server is the one I did the test on. the only other machine I own is a 64-bit laptop with a 4200 rpm HD. it's not really the ideal system for a benchmark test.


use the 64bit a sync server, you know that syncing will then be consistant.

http://gentoo-wiki.com/HOWTO_Local_Rsync_Mirror
_________________
Work Station - 64bit
Gigabyte GA X48-DQ6 Core2duo E8400
8GB GSkill DDR2-1066
SATA Areca 1210 Raid
BFG OC2 8800 GTS 640mb
--------------------------------
Notebook
Samsung Q45 7100 4gb
Back to top
View user's profile Send private message
wrc1944
Advocate
Advocate


Joined: 15 Aug 2002
Posts: 3260
Location: Gainesville, Florida

PostPosted: Sat Jan 28, 2006 12:44 pm    Post subject: Reply with quote

Hmmmm. No responses on my ext3 question posted above? I thought this would be an easy one for our ext3 gurus.:wink: Maybe some excerpts from my var/log/boot.msg log will help. The problems on hda7 this boot are typical, and appear randomly on all the other ext3 partitions nearly every boot. It's always that the i_blocks are always exactly 8 blocks more than they should be, whatever the Inode is, or on whatever partition they need fixing. I'm very curious as to what this means, and if I need to figure out how to correct it. Any insight is much appreciated. :D

(hda7 is on /mnt/rwstorage, which contains /var and /tmp, and I just did one little emerge, so it appears activity stimulates the fixes. On the other hand, hda6 is /mnt/portage, ext2, and had no fixes needed this time)


* Checking root filesystem ...
/dev/hda3: clean, 221607/1221600 files, 842732/2441376 blocks (check after next mount)
* Checking all filesystems ...
/dev/hda1: clean, 352/24480 files, 23472/97744 blocks
/dev/hda5: clean, 121310/610432 files, 606374/1220680 blocks (check after next mount)

/dev/hda6 has been mounted 5 times without being checked, check forced.
/dev/hda6: 134299/1465920 files (0.1% non-contiguous), 255079/1464860 blocks

/dev/hda7 has been mounted 2 times without being checked, check forced.
/dev/hda7: Inode 136520, i_blocks is 232, should be 224. FIXED.
/dev/hda7: Inode 132551, i_blocks is 216, should be 208. FIXED.
/dev/hda7: Inode 132522, i_blocks is 456, should be 448. FIXED.
/dev/hda7: Inode 132569, i_blocks is 144, should be 136. FIXED.

/dev/hda8: 133997/488640 files (0.1% non-contiguous), 344067/976492 blocks

Reiserfs super block in block 16 on 0x309 of format 3.6 with standard journal
Blocks (total/free): 3173792/1405058 by 4096 bytes
Filesystem is clean
Replaying journal..
Reiserfs journal '/dev/hda9' in blocks [18..8211]: 0 transactions replayed
Checking internal tree..finished
* Filesystem errors corrected.
[ !! ]
* Mounting local filesystems ...
[ ok ]
_________________
Main box- AsRock x370 Gaming K4
Ryzen 1700, 3.0GHz, 16GB GSkill Flare DDR4 3200mhz
Samsung SATA 1000GB, Radeon HD R7 350 2GB DDR5
Gentoo ~amd64 plasma, glibc-2.30-r2, gcc-9.2.0 kernel-5.3.9-gentoo USE=experimental
Back to top
View user's profile Send private message
PaulBredbury
Watchman
Watchman


Joined: 14 Jul 2005
Posts: 7310

PostPosted: Sat Jan 28, 2006 2:54 pm    Post subject: Reply with quote

wrc1944 wrote:
Since then, at nearly every boot, at least one partition has an inode fix or two to be done. Is this normal ext3 behavior, or is there something wrong with my system causing this?

I don't have a huge amount of insight to give. I've been using ext3 exclusively for about 4 years, but I expect that my comments apply to any filesystem: If, after a clean reboot, the filesystem regularly has "problems", then of course something is wrong. As to exactly what is wrong, and how to fix it, those are extremely difficult questions. It could be hardware or software, or a combination of both :?

Recommendation: Backup, reformat, restore. If your instincts suspect the hardware, then buy a new drive - the good news is, they're getting better and cheaper all the time.
Back to top
View user's profile Send private message
codergeek42
Bodhisattva
Bodhisattva


Joined: 05 Apr 2004
Posts: 5142
Location: Anaheim, CA (USA)

PostPosted: Sat Jan 28, 2006 6:31 pm    Post subject: Reply with quote

@wrc1944: Did you put grub or another bootloader on the MBR of that partition? That's the only time I've personally ever seen such an error.
_________________
~~ Peter: Programmer, Mathematician, STEM & Free Software Advocate, Enlightened Agent, Transhumanist, Fedora contributor
Who am I? :: EFF & FSF
Back to top
View user's profile Send private message
wrc1944
Advocate
Advocate


Joined: 15 Aug 2002
Posts: 3260
Location: Gainesville, Florida

PostPosted: Sun Jan 29, 2006 12:50 am    Post subject: Reply with quote

codergeek42 & PaulBredbury,
Thanks for the responses! All my drives are fairly new and have been rock solid with many distros, and I just recently set these partitions up on this box, so this box has just gone through "backup, reformat, restore." They show no other problems, and run normally. I'm very experienced, and have built many computers and installed Linux and windows many hundreds of times, on countless systems, so I'm virtually certain this is not hardware related.

There doesn't seem to be any actual problem in the normal operation of the computer, or the fine-tuned Gentoo installation, except that every 2-3 boots one partition or another apparently needs to fix that 8 block "oversized" detected thing.

I'm really becoming convinced this is perfectly normal ext3 behavior (in fact, very beneficial- like early prevention), and I just never noticed it before, since I hadn't used ext3 for years before last month, when I converted everything, and tuned it up.

Grub is on the MBR- I've always done it that way, as well as when I use Lilo, and/or dual-boot. This particular box is not dual-booted.

Maybe a few other ext3 users wouldn't mind temporarily setting their partitions to force a boot check every 2-3 mounts and see if this ocurrs on their systems? :lol:
_________________
Main box- AsRock x370 Gaming K4
Ryzen 1700, 3.0GHz, 16GB GSkill Flare DDR4 3200mhz
Samsung SATA 1000GB, Radeon HD R7 350 2GB DDR5
Gentoo ~amd64 plasma, glibc-2.30-r2, gcc-9.2.0 kernel-5.3.9-gentoo USE=experimental
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Documentation, Tips & Tricks All times are GMT
Goto page Previous  1, 2, 3 ... 8, 9, 10 ... 13, 14, 15  Next
Page 9 of 15

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum