Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
Has anybody tried ext4?
View unanswered posts
View posts from last 24 hours

Goto page Previous  1, 2, 3  Next  
Reply to topic    Gentoo Forums Forum Index Unsupported Software
View previous topic :: View next topic  
Author Message
mikkoc
Apprentice
Apprentice


Joined: 24 May 2007
Posts: 231

PostPosted: Sun Oct 26, 2008 9:10 pm    Post subject: Reply with quote

Schwinni wrote:

You want to use extents:
Code:
tune2fs -O extents /dev/sda8

Do that before you mount it as ext4.


Thanks, I already have 256 inode size.
One more question:
can I do "tune2fs -O extents /dev/sda8" while the partition is mounted?
If not, is it ok to do it from an ubuntu liveCD?

EDIT: ok, it seems everything went fine:

EXT3-fs: sda8: couldn't mount because of unsupported optional features (40).
EXT4-fs: barriers enabled
EXT4-fs: delayed allocation enabled
EXT4-fs: file extents enabled
EXT4-fs: mballoc enabled
EXT4-fs: mounted filesystem with ordered data mode.
EXT4 FS on sda8, internal journal on sda8:8

What about the first line? Does it try to mount as ext3 or what?
Back to top
View user's profile Send private message
Schwinni
Apprentice
Apprentice


Joined: 02 Sep 2004
Posts: 214
Location: quadrant1.earth. germany.wuerzburg

PostPosted: Sun Oct 26, 2008 11:01 pm    Post subject: Reply with quote

Have a look at the thread, I have exactly the same line in my log
The funny thing is that it only appears for my root partition.
For the home partition everything is normal.
If I wouldn't have a separate boot partition with ext2 I would say it is because grub doesn't handle ext4, but this way...
I really don't know what it is.

I also fscked my partitions now several times and everything seems to be fine.

I think I try to find out something about that message on the ext4 mailing list or on the IRC channel...
Back to top
View user's profile Send private message
boris64
Veteran
Veteran


Joined: 04 Oct 2003
Posts: 1770
Location: vechelde / peine

PostPosted: Mon Oct 27, 2008 1:56 pm    Post subject: Reply with quote

sleipner wrote:
boris64 wrote:
There is just one tiny thing that needs to be resolved to use
it on my root partition.
I didn't find any grub version, that can boot an ext4 root partition (w/o extra /boot).
Is there anyone who found a working patch etc.?


http://ext4.wiki.kernel.org/index.php/Ext4_Howto#Booting_from_an_ext4_filesystem

Yeah, that's what i found, too. The patches from those sites (grub4ext4) don't apply cleanly to
an unpatched grub-0.97 and grub-1.97, which claims to have preliminary ext4 support, doesn't even
build on amd64 because of missing 32bit lzo.
Did i miss something important? So, anyone out there who got it compiled
and who is already able to boot from an ext4 partition?
_________________
boris64.net 200x / visit my desktop / try these tiny kernel patches ;)
Back to top
View user's profile Send private message
octoploid
n00b
n00b


Joined: 21 Oct 2006
Posts: 65

PostPosted: Mon Oct 27, 2008 5:17 pm    Post subject: Reply with quote

boris64 wrote:
sleipner wrote:
boris64 wrote:
There is just one tiny thing that needs to be resolved to use
it on my root partition.
I didn't find any grub version, that can boot an ext4 root partition (w/o extra /boot).
Is there anyone who found a working patch etc.?


http://ext4.wiki.kernel.org/index.php/Ext4_Howto#Booting_from_an_ext4_filesystem

Yeah, that's what i found, too. The patches from those sites (grub4ext4) don't apply cleanly to
an unpatched grub-0.97 and grub-1.97, which claims to have preliminary ext4 support, doesn't even
build on amd64 because of missing 32bit lzo.
Did i miss something important? So, anyone out there who got it compiled
and who is already able to boot from an ext4 partition?


Yes, I switched my root partition to ext4 yesterday and my system (amd64) boots without
problems with grub2.
You'll have to install grub2 by hand (the version in portage does not work), but is is rather straightforward.
Code:
 svn co svn://svn.savannah.gnu.org/grub/trunk/grub2/
cd grub2
./configure
make
# make install
# grub-install /dev/sda
# vim /boot/grub/grub.cfg


The grub2 syntax is different. This is my boot/grub/grub.cfg:
Code:
# Timeout for menu
set timeout=6

# Set default boot entry as Entry 0
set default=0

# Fallback to the second entry.
set fallback=2

# Entry 0 - Load Linux kernel
menuentry "Gentoo" {
    set root=(hd0,1)
    linux /boot/kernel254 root=/dev/sda1 rootflags=commit=60 hpet=force fbcon=rotate:3                                           
}

menuentry "git kernel" {
    set root=(hd0,1)
    linux /usr/src/linux/arch/x86_64/boot/bzImage root=/dev/sda1 rootflags=commit=60 hpet=force fbcon=rotate:3
}

menuentry "old_kernel" {
    set root=(hd0,1)
    linux /boot/kernel_old root=/dev/sda1 rootflags=commit=60 hpet=force
}

menuentry "memtest" {
    set root=(hd0,1)
    linux /boot/memtest86plus/memtest.bin
}


More info here: http://www.gnu.org/software/grub/grub-2.en.html
_________________
Myself and mine gymnastic ever
Back to top
View user's profile Send private message
devsk
Advocate
Advocate


Joined: 24 Oct 2003
Posts: 2870
Location: Bay Area, CA

PostPosted: Tue Oct 28, 2008 3:44 am    Post subject: Reply with quote

is there a real world comparison of ext3 and ext4 somewhere? what I am interested in is:

1. large number of small files. How well does ext4 handle this case? space consumption and speed wise. This is one case where both speed can be increased and space consumption reduced like reiser4 does
.
2. small number of large files. e.g. how many fragments does the ext4 create for a 4GB file on a 70% used file system of 30GB? how fast is the delete of large files? Its slow on ext3/ext2.

3. there is no online de-fragmentation tool. Please don't flame me with "linux FSs don't fragment". Anybody who has created multiple virtual machines and have a full enough FS, knows that all FSs fragment.

These are the things that bother me on ext3. Does ext4 help with any of these?
Back to top
View user's profile Send private message
wrc1944
Advocate
Advocate


Joined: 15 Aug 2002
Posts: 3253
Location: Gainesville, Florida

PostPosted: Tue Oct 28, 2008 8:14 am    Post subject: Reply with quote

Here's some I just looked at:
http://www.linuxinsight.com/first_benchmarks_of_the_ext4_file_system.html

http://en.wikipedia.org/wiki/Ext4
_________________
Main box- AsRock x370 Gaming K4
Ryzen 1700, 3.0GHz, 16GB GSkill Flare DDR4 3200mhz
Samsung SATA 1000GB, Radeon HD R7 350 2GB DDR5
Gentoo ~amd64 plasma, glibc-2.29-r5, gcc-9.2.0 kernel-5.2.10-gentoo USE=experimental
Back to top
View user's profile Send private message
octoploid
n00b
n00b


Joined: 21 Oct 2006
Posts: 65

PostPosted: Tue Oct 28, 2008 8:40 am    Post subject: Reply with quote

Another up-to-date benchmark:

http://btrfs.boxacle.net/repository/single-disk/Initial-compare/Initial-Compare-Single_disk.html
_________________
Myself and mine gymnastic ever
Back to top
View user's profile Send private message
devsk
Advocate
Advocate


Joined: 24 Oct 2003
Posts: 2870
Location: Bay Area, CA

PostPosted: Thu Oct 30, 2008 7:05 am    Post subject: Reply with quote

So, I tested ext4dev in 2.6.27.4. Stay away from it. Its worse than ext3 in every sense. Large files have many many (41 against 770) more fragments compared to ext3 (so, what's up with extents?). Seq write b/w drops 20%. And pdflush locks up cpu during large (5GB) file copy and kjournald2 goes into 'D' state, requiring a reboot. About the only thing going for it was that the delete of large files was much quicker than ext3. The tests were done on the same physical hardware and same partition, on an amd64 system.
Back to top
View user's profile Send private message
lucke
n00b
n00b


Joined: 11 May 2005
Posts: 35

PostPosted: Thu Oct 30, 2008 10:11 am    Post subject: Reply with quote

They apparently fixed that bug with excessive number of extents. It's comparable to xfs now in this regard. All in all, it's dandy on .28.
Back to top
View user's profile Send private message
octoploid
n00b
n00b


Joined: 21 Oct 2006
Posts: 65

PostPosted: Thu Oct 30, 2008 10:33 am    Post subject: Reply with quote

devsk wrote:
So, I tested ext4dev in 2.6.27.4. Stay away from it. Its worse than ext3 in every sense. Large files have many many (41 against 770) more fragments compared to ext3 (so, what's up with extents?). Seq write b/w drops 20%. And pdflush locks up cpu during large (5GB) file copy and kjournald2 goes into 'D' state, requiring a reboot. About the only thing going for it was that the delete of large files was much quicker than ext3. The tests were done on the same physical hardware and same partition, on an amd64 system.


Hmm, strange. I don't observe any of these symptoms on my machine (amd64).
I'm running 2.6.28-rc2 and started with fresh filesystems ("mkfs.ext4") to which
I copied all my data from ext3 backups, including my ~250GB music collection.

fsck.ext4 is dramatically faster than the ext3 version. Firing up my music player (cmus)
now only takes a second (reading the entire directory structure of my collection); on ext3
I had to wait 10-15 sec before I could use it. update-eix is blazingly fast.

Granted, these may all be effects of a freshly populated file system, but I definitely don't
see any downsides compared to ext3.
_________________
Myself and mine gymnastic ever
Back to top
View user's profile Send private message
devsk
Advocate
Advocate


Joined: 24 Oct 2003
Posts: 2870
Location: Bay Area, CA

PostPosted: Thu Oct 30, 2008 3:33 pm    Post subject: Reply with quote

may be the difference comes between ext4dev of 2.6.27.4 and ext4 of 2.6.28. But my experiments on the same hardware and partition under the same OS, same tools, same everything, paints ext4dev in bad shape. I will wait for 2.6.28 and see if it changes.
Back to top
View user's profile Send private message
asturm
Developer
Developer


Joined: 05 Apr 2007
Posts: 7113
Location: Austria

PostPosted: Thu Oct 30, 2008 8:24 pm    Post subject: Reply with quote

That's because ext4dev != ext4
_________________
backend.cpp:92:2: warning: #warning TODO - this error message is about as useful as a cooling unit in the arctic
Back to top
View user's profile Send private message
kernelOfTruth
Watchman
Watchman


Joined: 20 Dec 2005
Posts: 6108
Location: Vienna, Austria; Germany; hello world :)

PostPosted: Thu Oct 30, 2008 8:53 pm    Post subject: Reply with quote

octoploid wrote:
Another up-to-date benchmark:

http://btrfs.boxacle.net/repository/single-disk/Initial-compare/Initial-Compare-Single_disk.html


whoa! if those write-benchmarks of btrfs are reproducible it better gets ready fast ! (especially now after compression-support has been included),

ext4 really doesn't seem to be that much better than ext3, cpu-load however seems to be significantly lower, though
_________________
https://github.com/kernelOfTruth/ZFS-for-SystemRescueCD/tree/ZFS-for-SysRescCD-4.9.0
https://github.com/kernelOfTruth/pulseaudio-equalizer-ladspa

Hardcore Gentoo Linux user since 2004 :D
Back to top
View user's profile Send private message
devsk
Advocate
Advocate


Joined: 24 Oct 2003
Posts: 2870
Location: Bay Area, CA

PostPosted: Thu Oct 30, 2008 11:32 pm    Post subject: Reply with quote

genstorm wrote:
That's because ext4dev != ext4
do you have data to support that? I doubt they put in so much of speed between 2.6.27.4 and 2.6.28-rc2.
Back to top
View user's profile Send private message
agent_jdh
Veteran
Veteran


Joined: 08 Aug 2002
Posts: 1779
Location: Scotland

PostPosted: Thu Oct 30, 2008 11:58 pm    Post subject: Reply with quote

devsk wrote:
genstorm wrote:
That's because ext4dev != ext4
do you have data to support that? I doubt they put in so much of speed between 2.6.27.4 and 2.6.28-rc2.


When you make menuconfig your kernel and you see ext4dev, it's not at the latest level. There's a link to the kernel.org repository for the latest level of patches for ext4 with 2.6.27 on the first page of this topic. People using 2.6.28-rc kernels already have them. Once the patches are applied, when you make menuconfig, ext4dev is gone, and has been replaced by simply ext4 (albeit with an option for compatability with ext4dev partitions).

This would probably fix the performance issues you're seeing. Although YMMV.
_________________
Jingle Jangle Jewellery
Back to top
View user's profile Send private message
Schwinni
Apprentice
Apprentice


Joined: 02 Sep 2004
Posts: 214
Location: quadrant1.earth. germany.wuerzburg

PostPosted: Fri Oct 31, 2008 9:20 am    Post subject: Reply with quote

But be careful:
2.6.27-ext4-2 cannot patch "ext4/dir.c" because this file itself was patched from 2.6.27.3 to 2-6-27-4.
I didn't have the time to look at the rejected hunk yesterday.
The safest way is to wait for the next patch set (2.6.27-ext4-3).

devsk wrote:
genstorm wrote:
That's because ext4dev != ext4
do you have data to support that? I doubt they put in so much of speed between 2.6.27.4 and 2.6.28-rc2.

These are the broken out patches from 2.6.27-ext4-2. They do more than just renaming extdev to ext4:
http://www2.kernel.org/pub/linux/kernel/people/tytso/ext4-patches/2.6.27-ext4-2/broken-out/

Best regards
Chris
Back to top
View user's profile Send private message
Schwinni
Apprentice
Apprentice


Joined: 02 Sep 2004
Posts: 214
Location: quadrant1.earth. germany.wuerzburg

PostPosted: Fri Oct 31, 2008 1:30 pm    Post subject: Reply with quote

Seems like the failing hunk is included both in the kernel patch set for 2.6.27.4 and in 2.6.27-ext4-2:

From patch-2.6.27.4:
Code:
diff --git a/fs/ext4/dir.c b/fs/ext4/dir.c
index ec8e33b..d1d6487 100644
--- a/fs/ext4/dir.c
+++ b/fs/ext4/dir.c
@@ -102,6 +102,7 @@ static int ext4_readdir(struct file * filp,
    int err;
    struct inode *inode = filp->f_path.dentry->d_inode;
    int ret = 0;
+   int dir_has_error = 0;
 
    sb = inode->i_sb;
 
@@ -148,9 +149,13 @@ static int ext4_readdir(struct file * filp,
        * of recovering data when there's a bad sector
        */
       if (!bh) {
-         ext4_error (sb, "ext4_readdir",
-            "directory #%lu contains a hole at offset %lu",
-            inode->i_ino, (unsigned long)filp->f_pos);
+         if (!dir_has_error) {
+            ext4_error(sb, __func__, "directory #%lu "
+                  "contains a hole at offset %Lu",
+                  inode->i_ino,
+                  (unsigned long long) filp->f_pos);
+            dir_has_error = 1;
+         }
          /* corrupt size?  Maybe no more blocks to read */
          if (filp->f_pos > inode->i_blocks << 9)
             break;


From 2.6.27-ext4-2:
Code:
diff --git a/fs/ext4/dir.c b/fs/ext4/dir.c
index ec8e33b..3ca6a2b 100644
--- a/fs/ext4/dir.c
+++ b/fs/ext4/dir.c
.
.
.
@@ -102,6 +102,7 @@ static int ext4_readdir(struct file * filp,
    int err;
    struct inode *inode = filp->f_path.dentry->d_inode;
    int ret = 0;
+   int dir_has_error = 0;
 
    sb = inode->i_sb;
 
@@ -148,9 +149,13 @@ static int ext4_readdir(struct file * filp,
        * of recovering data when there's a bad sector
        */
       if (!bh) {
-         ext4_error (sb, "ext4_readdir",
-            "directory #%lu contains a hole at offset %lu",
-            inode->i_ino, (unsigned long)filp->f_pos);
+         if (!dir_has_error) {
+            ext4_error(sb, __func__, "directory #%lu "
+                  "contains a hole at offset %Lu",
+                  inode->i_ino,
+                  (unsigned long long) filp->f_pos);
+            dir_has_error = 1;
+         }
          /* corrupt size?  Maybe no more blocks to read */
          if (filp->f_pos > inode->i_blocks << 9)
             break;
.
.
.
Back to top
View user's profile Send private message
devsk
Advocate
Advocate


Joined: 24 Oct 2003
Posts: 2870
Location: Bay Area, CA

PostPosted: Sat Nov 01, 2008 12:24 am    Post subject: Reply with quote

Schwinni wrote:
But be careful:
2.6.27-ext4-2 cannot patch "ext4/dir.c" because this file itself was patched from 2.6.27.3 to 2-6-27-4.
I didn't have the time to look at the rejected hunk yesterday.
The safest way is to wait for the next patch set (2.6.27-ext4-3).

devsk wrote:
genstorm wrote:
That's because ext4dev != ext4
do you have data to support that? I doubt they put in so much of speed between 2.6.27.4 and 2.6.28-rc2.

These are the broken out patches from 2.6.27-ext4-2. They do more than just renaming extdev to ext4:
http://www2.kernel.org/pub/linux/kernel/people/tytso/ext4-patches/2.6.27-ext4-2/broken-out/

Best regards
Chris
Have all these patches been accepted in 2.6.28-rc2?
Back to top
View user's profile Send private message
Schwinni
Apprentice
Apprentice


Joined: 02 Sep 2004
Posts: 214
Location: quadrant1.earth. germany.wuerzburg

PostPosted: Sat Nov 01, 2008 12:55 am    Post subject: Reply with quote

I don't know. At least on http://ext4.wiki.kernel.org/index.php/Ext4_patchsets it says
Quote:
Periodically, Ted will take the ext4 patch queue and create a stable patchset for people to apply.

Because these patch sets doesn't contain all tests from the git tree but only the stable things, I tend to say yes, probably they are accepted in 2.6.28.
But to make sure you'd have to ask that on the ext4 mailing list.
Back to top
View user's profile Send private message
devsk
Advocate
Advocate


Joined: 24 Oct 2003
Posts: 2870
Location: Bay Area, CA

PostPosted: Sat Nov 01, 2008 1:35 am    Post subject: Reply with quote

I got this reject while applying the patch:
Code:
fs/ext4/dir.c.rej
***************
*** 102,107 ****
        int err;
        struct inode *inode = filp->f_path.dentry->d_inode;
        int ret = 0;

        sb = inode->i_sb;

--- 102,108 ----
        int err;
        struct inode *inode = filp->f_path.dentry->d_inode;
        int ret = 0;
+       int dir_has_error = 0;

        sb = inode->i_sb;

***************
*** 148,156 ****
                 * of recovering data when there's a bad sector
                 */
                if (!bh) {
-                       ext4_error (sb, "ext4_readdir",
-                               "directory #%lu contains a hole at offset %lu",
-                               inode->i_ino, (unsigned long)filp->f_pos);
                        /* corrupt size?  Maybe no more blocks to read */
                        if (filp->f_pos > inode->i_blocks << 9)
                                break;
--- 149,161 ----
                 * of recovering data when there's a bad sector
                 */
                if (!bh) {
+                       if (!dir_has_error) {
+                               ext4_error(sb, __func__, "directory #%lu "
+                                          "contains a hole at offset %Lu",
+                                          inode->i_ino,
+                                          (unsigned long long) filp->f_pos);
+                               dir_has_error = 1;
+                       }
                        /* corrupt size?  Maybe no more blocks to read */
                        if (filp->f_pos > inode->i_blocks << 9)
                                break;


As per Schwinni's post above, I think this is fine. so, I should be able to continue, right?
Back to top
View user's profile Send private message
devsk
Advocate
Advocate


Joined: 24 Oct 2003
Posts: 2870
Location: Bay Area, CA

PostPosted: Sat Nov 01, 2008 3:15 am    Post subject: Reply with quote

OK, just booted with the patch and the tests on the same physical partition are much more favorable now. Number of fragments went down substantially. Sequential write/read b/w on large files went up beyond ext3 by 20%, I think mostly because of reduced number of fragments.

Had no effect on portage tree compared to ext2: same results, give or take few millisecs.

I can live with that.

Does anybody know anything about this error from bonnie++?
Code:
Delete files in sequential order...Can't delete file 0004996uWK9H6o
Cleaning up test directory after error.
It doesn't happen on ext[23].
Back to top
View user's profile Send private message
octoploid
n00b
n00b


Joined: 21 Oct 2006
Posts: 65

PostPosted: Sat Nov 01, 2008 5:54 am    Post subject: Reply with quote

devsk wrote:
OK, just booted with the patch and the tests on the same physical partition are much more favorable now. Number of fragments went down substantially. Sequential write/read b/w on large files went up beyond ext3 by 20%, I think mostly because of reduced number of fragments.

Had no effect on portage tree compared to ext2: same results, give or take few millisecs.

I can live with that.

Does anybody know anything about this error from bonnie++?
Code:
Delete files in sequential order...Can't delete file 0004996uWK9H6o
Cleaning up test directory after error.
It doesn't happen on ext[23].


Yes, this bug is already fixed in 2.6.28-rc2:
http://git.kernel.org/?p=linux/kernel/git/tytso/ext4.git;a=commitdiff;h=3c37fc86d20fe35be656f070997d62f75c2e4874
_________________
Myself and mine gymnastic ever
Back to top
View user's profile Send private message
devsk
Advocate
Advocate


Joined: 24 Oct 2003
Posts: 2870
Location: Bay Area, CA

PostPosted: Sat Nov 01, 2008 6:10 am    Post subject: Reply with quote

doesn't apply on top of the other patch ext4-2. Did someone generate that patch against ext4-2?

edit: it applied with "patch -p1 -l". Cut&paste mangled the tabs.


Last edited by devsk on Sat Nov 01, 2008 4:33 pm; edited 1 time in total
Back to top
View user's profile Send private message
fldc2
Tux's lil' helper
Tux's lil' helper


Joined: 14 Jun 2006
Posts: 112
Location: Motala, Sweden

PostPosted: Sat Nov 01, 2008 12:19 pm    Post subject: Reply with quote

The only problem i've had with ext4 so far is that the torrents i download get lots of failed chunks for some reason, no matter what client is used, my workaround at this time is to download on a partition with a different filesystem :)
Back to top
View user's profile Send private message
devsk
Advocate
Advocate


Joined: 24 Oct 2003
Posts: 2870
Location: Bay Area, CA

PostPosted: Sat Nov 01, 2008 4:39 pm    Post subject: Reply with quote

ext4 is really shining with large files. I used to get upto 3000 fragments for a 650MB VM vmem file after repeated suspend/restore cycles. It has got stuck with 2 so far, which means suspend/resume of VM takes just 5-10 seconds, instead of worst cases 30-60 seconds. I just hope it doesn't oops on me....:-)
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Unsupported Software All times are GMT
Goto page Previous  1, 2, 3  Next
Page 2 of 3

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum