Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
Pappy's Kernel Seeds Part III<Closed. Please use new thread>
View unanswered posts
View posts from last 24 hours

Goto page Previous  1, 2, 3 ... 11, 12, 13 ... 22, 23, 24  Next  
This topic is locked: you cannot edit posts or make replies.    Gentoo Forums Forum Index Unsupported Software
View previous topic :: View next topic  
Author Message
Moriah
Advocate
Advocate


Joined: 27 Mar 2004
Posts: 2117
Location: Kentucky

PostPosted: Sat Jan 01, 2011 3:42 pm    Post subject: Reply with quote

You actually will have a pair of 2 TB drives on the way, so I would make a RAID1 mirror out of it for that server. ;-)

BTW Those 2 TB drives are WD "Green Power" drives that are hard sectored at 4096 bytes/sector, even though the firmware makes it look like 512 bytes/sector. Be sure you set them up partitioned on a 4096 byte boundary, and use 4096 bytes/sector with them, or they will be very slooooowwww. :x

And yes, HAPPY NEW YEAR TO ALL :!: :!: :!:
_________________
The MyWord KJV Bible tool is at http://www.elilabs.com/~myword

Foghorn Leghorn is a Warner Bros. cartoon character.
Back to top
View user's profile Send private message
pappy_mcfae
Watchman
Watchman


Joined: 27 Dec 2007
Posts: 5997
Location: Pomona, California.

PostPosted: Sun Jan 02, 2011 8:45 am    Post subject: Reply with quote

The flowers have bloomed, and a new batch of seed settings has been created, and uploaded to the sites. The main site is currently up and available with the new seeds, and the failover servers will be up within an hour. Enjoy!

EDIT All the failover servers are now updated.

Cheers,
Pappy
_________________
This space left intentionally blank, except for these ASCII symbols.
Back to top
View user's profile Send private message
pappy_mcfae
Watchman
Watchman


Joined: 27 Dec 2007
Posts: 5997
Location: Pomona, California.

PostPosted: Thu Jan 06, 2011 8:40 am    Post subject: Reply with quote

It's a new year, and thanks to the kernel.org folks, it's also a year that starts with new source. I've just uploaded .configs for 2.6.37 and 2.6.37-gentoo in both x86 and x86_64 flavors. Enjoy!

I'm already in the process of compiling a .37 kernel for this machine to see if it coughs up any hairballs, then it's on to core-too with nvidia-drivers. Will they work properly? Will they require an update to deal with .37 kernels? Soon, we shall all know.

Cheers,
Pappy
_________________
This space left intentionally blank, except for these ASCII symbols.
Back to top
View user's profile Send private message
pappy_mcfae
Watchman
Watchman


Joined: 27 Dec 2007
Posts: 5997
Location: Pomona, California.

PostPosted: Thu Jan 06, 2011 9:07 pm    Post subject: Reply with quote

Well, perhaps a little later than intended, but the news is good from the .37 kernel front is good from this chair. Compilation of ndiswrapper and nvidia-drivers went off without so much as a sniffle. I love it when I don't have to wait to use my new kernels.

2.6.36-zen2 has found its way to git, and that means there's a seed sure to follow sometime later on this evening/tomorrow, depending on where you are on the planet. Firing up core-too with that very source even as I type this message. Everything looks normal.

Cool. I'm about to reboot this into 2.6.36-zen2.

Cheers,
Pappy
_________________
This space left intentionally blank, except for these ASCII symbols.
Back to top
View user's profile Send private message
pappy_mcfae
Watchman
Watchman


Joined: 27 Dec 2007
Posts: 5997
Location: Pomona, California.

PostPosted: Fri Jan 07, 2011 9:18 am    Post subject: Reply with quote

2.6.36-zen2 is still not in portage, but is in git. I've uploaded .configs for 2.6.36-zen2 in both x86 and x86_64 flavors. Enjoy!

It may be in portage later on today. I'm not sure. Maybe .37 zen will be coming soon, too. :crossing fingers:

Cheers,
Pappy
_________________
This space left intentionally blank, except for these ASCII symbols.
Back to top
View user's profile Send private message
pappy_mcfae
Watchman
Watchman


Joined: 27 Dec 2007
Posts: 5997
Location: Pomona, California.

PostPosted: Sat Jan 08, 2011 7:36 am    Post subject: Reply with quote

Well, no zen-sources in portage, and no .37 zen (yet). That doesn't mean there was no new source today. As a matter of fact, I've just uploaded .configs for 2.6.32-hardened-r33, 2.6.34.8, 2.6.36.3, and 2.6.36-hardened-r8 in both x86 and x86_64 flavors. Enjoy!

Cheers,
Pappy
_________________
This space left intentionally blank, except for these ASCII symbols.
Back to top
View user's profile Send private message
audiodef
Watchman
Watchman


Joined: 06 Jul 2005
Posts: 6313
Location: /usr/lib64/lv2

PostPosted: Sat Jan 08, 2011 5:29 pm    Post subject: Reply with quote

Too many sources too fast! Stop the truck! Eat some cookies and cake! :P
_________________
Gentoo Studio: A Gentoo-based, professional digital audio workstation OS.
Back to top
View user's profile Send private message
pappy_mcfae
Watchman
Watchman


Joined: 27 Dec 2007
Posts: 5997
Location: Pomona, California.

PostPosted: Sat Jan 08, 2011 6:26 pm    Post subject: Reply with quote

hehehehehhe

Cheers,
Pappy
_________________
This space left intentionally blank, except for these ASCII symbols.
Back to top
View user's profile Send private message
pappy_mcfae
Watchman
Watchman


Joined: 27 Dec 2007
Posts: 5997
Location: Pomona, California.

PostPosted: Mon Jan 10, 2011 9:19 am    Post subject: Reply with quote

And still more. I've just uploaded .configs for 2.6.32.28 and 2.6.32-gentoo-r25 in both x86 and x86_64 flavors. Enjoy!

Cheers,
Pappy
_________________
This space left intentionally blank, except for these ASCII symbols.
Back to top
View user's profile Send private message
dufeu
l33t
l33t


Joined: 30 Aug 2002
Posts: 896
Location: US-FL-EST

PostPosted: Tue Jan 11, 2011 4:47 pm    Post subject: Pappy 'seeds' settings - page 2 Reply with quote

Hi Pappy,

I've been upgrading kernels recently and reviewing the kernel settings pages as I go along. I'd like to point out the following on 'page 2':
Quote:
When you <Select> at this point, you bring up the following options:
--- Enable the block layer
[ ] Block layer SG support v4
[ ] Block layer data integrity support

You have the Block layer SG support as being unchecked. You probably want this checked on by default.

Despite the fact the help has "SCSI" written all over it, it's really talking about 'SCSI generic'. Virtually everything with memory now works through SCSI generic to at least obtain IDs. This includes USB attached hard drives and memory cards, SATA and eSATA, modern tape devices etc.

In fact, the help for 'Block layer SG support v4' now includes:
Code:
This option is required by recent UDEV versions to properly
access device serial numbers, etc.

This option isn't required the same way that the generic SCSI is required out of the SCSI section of the kernel settings. But it is complementary to that. As I understand it, the 'Block layer SG support v4' provides the support for advanced generic SCSI functions such as out of order command queuing and advanced device serial numbers. For example, SATA drives are finally starting to appear which actually support these commands. While you can still use these drives {at their lowest common command denominator} without this option turned on, you won't be able to achieve maximum hard drive performance without access to these advanced commands.

In general, this option won't make any performance difference with older drives/devices of any type but new drives/devices which support advanced commands will automatically benefit if this option is on.

As far as I have been able to tell, there has been no downside to having this option on.

BTW - so far, the kernel seeds settings pages are looking good! I'm glad to see you're still keeping up with them.
_________________
People whom think M$ is mediocre, don't know the half of it.
Back to top
View user's profile Send private message
pappy_mcfae
Watchman
Watchman


Joined: 27 Dec 2007
Posts: 5997
Location: Pomona, California.

PostPosted: Tue Jan 11, 2011 9:14 pm    Post subject: Reply with quote

Thanks for the info. I've never really had a problem with that setting one way or another, but I'll give it a go to see what happens. I'm always up for improving the seeds. There has to be a version nine, don't you know? hehehehe

Cheers,
Pappy
_________________
This space left intentionally blank, except for these ASCII symbols.
Back to top
View user's profile Send private message
pross
Apprentice
Apprentice


Joined: 30 Jul 2003
Posts: 156
Location: Ireland

PostPosted: Tue Jan 11, 2011 10:01 pm    Post subject: Reply with quote

I had to enable cpu hotplugging to get BFS to compile
_________________
My Overlay [pross] | bfs-sources
Back to top
View user's profile Send private message
pappy_mcfae
Watchman
Watchman


Joined: 27 Dec 2007
Posts: 5997
Location: Pomona, California.

PostPosted: Tue Jan 11, 2011 10:31 pm    Post subject: Reply with quote

A little friendly Heads Up from yours truly. This specifically concerns Western Digital GreenPower hard drives >=1TB.

In order to pack more data on the disk, WD uses 4k sectors instead of the traditional 512 Byte sectors. While it is said that Linux is immune to the effects of this, in actual practice, that's not ABSOLUTELY correct.

I recently acquired some WD GreenPower drives from a contributor to the site. He warned me about this, but as per my standard Modus Operendi, I decided I'd try my usual methodology to see if Linux was unaffected. Well, suffice it to say the evidence I see before me shows that, indeed, the impact to Linux can be incredible if these disks are not properly partitioned.

Properly partitioning these drives requires a bit of number fudging in order to get the 4k sector boundaries to line up properly. There is a very slight loss of space, but you make up for that by having a drive that can actually approach its reported speed numbers. Because of the required fudging, cfdisk will NOT work as a means of partitioning these drives. Let me say that again, cfdisk will NOT work as a means of partitioning these drives. Get it? Got it? Good.

parted and fdisk are your only options. Since I know a bit more about fdisk than parted, I used it. I started fdisk thusly:
Code:
fdisk -H 224 -S 56


That forces the 4k sectors to line up properly, and boy howdy, does it make a difference.

Core-too, my amd64 machine contains unarchived tar balls of all the versions of kernel source code from 2.6.25 to the present, in five separate flavors. Roughly translated to space, that's 287 Gigs of data that are now flowing from the old drive to the new.

Yesterday, when the drives arrived and I decided it was time to play, using my old methodology, the drive took over twenty hours to move thirty Gigs. I know old 512 meg drives faster than that. After redoing the partitioning with fdisk as invoked above, I've moved the same amount of data in about three hours. Can you say big difference?

I was impressed. I am still impressed. I can't wait 'til all the data's moved, and I can see how this drive boots.

Please check out this link for the most complete information on fixing the issue.

Cheers,
Pappy
_________________
This space left intentionally blank, except for these ASCII symbols.
Back to top
View user's profile Send private message
pappy_mcfae
Watchman
Watchman


Joined: 27 Dec 2007
Posts: 5997
Location: Pomona, California.

PostPosted: Wed Jan 12, 2011 12:10 am    Post subject: Reply with quote

pross wrote:
I had to enable cpu hotplugging to get BFS to compile


Please post that at the zen-kernel discussion thread. They will be much more likely to have the information you need to figure out why that is.

Cheers,
Pappy
_________________
This space left intentionally blank, except for these ASCII symbols.
Back to top
View user's profile Send private message
myceliv
Apprentice
Apprentice


Joined: 29 Nov 2007
Posts: 178

PostPosted: Wed Jan 12, 2011 4:42 am    Post subject: Using parted on 4KB sector drives Reply with quote

pappy_mcfae wrote:
A little friendly Heads Up from yours truly. This specifically concerns Western Digital GreenPower hard drives >=1TB.

In order to pack more data on the disk, WD uses 4k sectors instead of the traditional 512 Byte sectors. While it is said that Linux is immune to the effects of this, in actual practice, that's not ABSOLUTELY correct.

.... cfdisk will NOT work.... parted and fdisk are your only options. Since I know a bit more about fdisk than parted, I used it. I started fdisk thusly:
    fdisk -H 224 -S 56
That forces the 4k sectors to line up properly, and boy howdy, does it make a difference. Please check out this link for the most complete information on fixing the issue.


Coincidentally, I was just researching this same issue yesterday, and picked that same link as the best one. (I'm a fdisk guy too.) For people using recent versions of parted, this link to a WD article might be helpful, too.

Pappy I haven't said thanks in a long time. THANKS!!! Your new site is quite a resource. Thanks for putting all the creativity and effort into explaining so many kernel options, and maintaining all the seeds. You rock!
Back to top
View user's profile Send private message
pappy_mcfae
Watchman
Watchman


Joined: 27 Dec 2007
Posts: 5997
Location: Pomona, California.

PostPosted: Wed Jan 12, 2011 6:18 am    Post subject: Reply with quote

You're most welcome. I'm a little over halfway finished moving all that data. It is amazing how much difference there is between setting the partition properly, and not. I never knew such a thing could be, but had I not started over, I'd be at around maybe fifty gigs copied. At this point, I'm at 164 out of 287 Gigs a little less than twelve hours since I started. I think I can call that acceptable considering how fast the file names are shooting past on the term screen.

Cheers,
Pappy
_________________
This space left intentionally blank, except for these ASCII symbols.
Back to top
View user's profile Send private message
dufeu
l33t
l33t


Joined: 30 Aug 2002
Posts: 896
Location: US-FL-EST

PostPosted: Wed Jan 12, 2011 6:42 am    Post subject: Reply with quote

pappy_mcfae wrote:
At this point, I'm at 164 out of 287 Gigs a little less than twelve hours since I started. I think I can call that acceptable considering how fast the file names are shooting past on the term screen.

287Gs? That's all?

Piffle!

{hehe}

Code:
pyrodyno # df
Filesystem           1K-blocks      Used Available Use% Mounted on
rootfs               380497160 303184744  57984216  84% /
/dev/root            380497160 303184744  57984216  84% /
rc-svcdir                 1024       132       892  13% /lib64/rc/init.d
udev                     10240       268      9972   3% /dev
shm                    4090208      1892   4088316   1% /dev/shm
/dev/sdb1            961432072 864744256  96687816  90% /home
/dev/sda4            586051200 552484512  33566688  95% /pub00
/dev/sdc1            961432072 914602496  44387676  96% /pub01
/dev/sdd1            961432072 846143240  66450832  93% /pub02
/dev/sde1            961432072 882990740  29603332  97% /pub03
192.168.10.199:/public
                     3175415712 2983168456 192247256  94% /mnt/slizard/public
192.168.10.199:/pubroot
                     237185024 175182120  49954560  78% /mnt/slizard/pubroot
192.168.10.199:/home 237185024 175182120  49954560  78% /mnt/slizard/home
192.168.10.105:/home 183997648 113665064  60985968  66% /mnt/pyrotekk/home
192.168.10.105:/public
                     1271657520 1263052384   8605136 100% /mnt/pyrotekk/public
192.168.10.105:/pubroot
                     183997648 113665064  60985968  66% /mnt/pyrotekk/pubroot

pyrodyno is my primary workstation. pyrotekk is my secondary. And slizard is the house server.

Slizard is supposed to be my backup. But you can't cram 4.5Ts into a 3T sack. Oh. And the 1T on pyrotekk not back up either. It's in addition to the 4.5Ts.

All I wanted for Christmas was 2T drives. :D 10 of them would have been a treat.
_________________
People whom think M$ is mediocre, don't know the half of it.
Back to top
View user's profile Send private message
pappy_mcfae
Watchman
Watchman


Joined: 27 Dec 2007
Posts: 5997
Location: Pomona, California.

PostPosted: Wed Jan 12, 2011 7:39 am    Post subject: Reply with quote

I'm filling as fast as I can. Since we don't have a computer added into the "home theater" system, I don't really need a lot of space. I'm just really glad to know I can keep on untarring kernel source for a lot longer without having to worry about running out of space.

Cheers,
Pappy
_________________
This space left intentionally blank, except for these ASCII symbols.
Back to top
View user's profile Send private message
pappy_mcfae
Watchman
Watchman


Joined: 27 Dec 2007
Posts: 5997
Location: Pomona, California.

PostPosted: Wed Jan 12, 2011 11:08 am    Post subject: Reply with quote

Well, that's done. Next time I think about copying an entire hard drive over, I'm going to make extra sure to get rid of crap. Still, loving the new drive. It's up and running...and core-too has lots of room to open lots more kernel source code.

Cheers,
Pappy
_________________
This space left intentionally blank, except for these ASCII symbols.
Back to top
View user's profile Send private message
d2_racing
Bodhisattva
Bodhisattva


Joined: 25 Apr 2005
Posts: 13047
Location: Ste-Foy,Canada

PostPosted: Wed Jan 12, 2011 3:16 pm    Post subject: Reply with quote

That's why WD Black serie are a must and also they are more expansive , they don't have that kind of problem, at least AFAIK :P
Back to top
View user's profile Send private message
Moriah
Advocate
Advocate


Joined: 27 Mar 2004
Posts: 2117
Location: Kentucky

PostPosted: Wed Jan 12, 2011 3:54 pm    Post subject: Reply with quote

Black are also 7200 rpm, which is the major difference. Seagate has 7200 rpm 2tb drives for considerably cheaper, and Hitachi's are even cheaper still. Whenever you are trusting a mechanical contraption with that much intellectual property, you should at least have a good backup plan in action, and I always RAID-1 these drives, as they are pretty cheap, and that way if a drive dies, its no big deal: just yank the sick drive and shove in a fresh one, then send the bad drive back under warranty repair. Always keep a drive or 2 on hand as spares to account for such failures.
_________________
The MyWord KJV Bible tool is at http://www.elilabs.com/~myword

Foghorn Leghorn is a Warner Bros. cartoon character.
Back to top
View user's profile Send private message
jbouzan
Tux's lil' helper
Tux's lil' helper


Joined: 23 Nov 2007
Posts: 138

PostPosted: Wed Jan 12, 2011 5:31 pm    Post subject: Reply with quote

pappy_mcfae wrote:
A little friendly Heads Up from yours truly. This specifically concerns Western Digital GreenPower hard drives >=1TB.

In order to pack more data on the disk, WD uses 4k sectors instead of the traditional 512 Byte sectors. While it is said that Linux is immune to the effects of this, in actual practice, that's not ABSOLUTELY correct.

I recently acquired some WD GreenPower drives from a contributor to the site. He warned me about this, but as per my standard Modus Operendi, I decided I'd try my usual methodology to see if Linux was unaffected. Well, suffice it to say the evidence I see before me shows that, indeed, the impact to Linux can be incredible if these disks are not properly partitioned.

Properly partitioning these drives requires a bit of number fudging in order to get the 4k sector boundaries to line up properly. There is a very slight loss of space, but you make up for that by having a drive that can actually approach its reported speed numbers. Because of the required fudging, cfdisk will NOT work as a means of partitioning these drives. Let me say that again, cfdisk will NOT work as a means of partitioning these drives. Get it? Got it? Good.

parted and fdisk are your only options. Since I know a bit more about fdisk than parted, I used it. I started fdisk thusly:
Code:
fdisk -H 224 -S 56


That forces the 4k sectors to line up properly, and boy howdy, does it make a difference.

Core-too, my amd64 machine contains unarchived tar balls of all the versions of kernel source code from 2.6.25 to the present, in five separate flavors. Roughly translated to space, that's 287 Gigs of data that are now flowing from the old drive to the new.

Yesterday, when the drives arrived and I decided it was time to play, using my old methodology, the drive took over twenty hours to move thirty Gigs. I know old 512 meg drives faster than that. After redoing the partitioning with fdisk as invoked above, I've moved the same amount of data in about three hours. Can you say big difference?

I was impressed. I am still impressed. I can't wait 'til all the data's moved, and I can see how this drive boots.

Please check out this link for the most complete information on fixing the issue.

Cheers,
Pappy

Wow, thanks. I just bought one of these and installed it this morning, but I haven't moved data to it yet. This is great to know. I'd read about the 4k sectors, but I assumed that just running fdisk would set up the partitions correctly.
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 43072
Location: 56N 3W

PostPosted: Wed Jan 12, 2011 6:55 pm    Post subject: Reply with quote

jbouzan,

Unaligned writes force the drive to read the 4k sector first, modify what it just read, then write it back. Thus a write that should be done in one revolution of the platter now takes at least two.
Its all done inside the drive and if you use Native Command Queuing (NCQ) the drive will do other things meanwhile but that may mean moving the head to other tracks ... which can make the unaligned write even slower.

If the drive does not do write combining, then you can get eight of these read/modify/write cycles for every 4k block you try to write as each 4k block holds 8 of the o;d 512b sectors.
Now you have 16 or more revolutions of the platter to write 4k of unaligned data

You get different speed answers depending on the degree of misalignment.

By default, fdisk is out by 512 bytes ... thats due to the way the Cylinder/Head/Sector addressing scheme was desihned. If you look at a normal 512b sector drive in fdisk, the first partition starts at sector 63. With 4kb sectors you need it to start at sector 64 (in 512b sectors).

I'm surprised by pappys timings as I would expect the drive to work out for itself that only the first and last 4kb sectors need to use read/modify/writes, so with big files, I'm surprised its an issue.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
Moriah
Advocate
Advocate


Joined: 27 Mar 2004
Posts: 2117
Location: Kentucky

PostPosted: Wed Jan 12, 2011 7:16 pm    Post subject: Reply with quote

I used green power drives on my backup server for a while, and with the default sectoring (I did not realize they used non-standard sectoring at first) it really takes forever to delete hard linked directory entries. I use hard links as a poor man's form of file level dedup, so I have literally tens of millions of them to delete when I purge an old month from the backup storage RAID-1 3-way mirror. I have since given up on 5400 rpm drives altogether and now onlyuse 7200 rpm drives for this reason.
_________________
The MyWord KJV Bible tool is at http://www.elilabs.com/~myword

Foghorn Leghorn is a Warner Bros. cartoon character.
Back to top
View user's profile Send private message
pappy_mcfae
Watchman
Watchman


Joined: 27 Dec 2007
Posts: 5997
Location: Pomona, California.

PostPosted: Thu Jan 13, 2011 7:25 am    Post subject: Reply with quote

Neddy,

Wow, was it an issue! I moved 287 Gigs in about twenty hours after setting the drive up for the 4k sectors. The night before, doing it in my normal way, I moved about thirty Gigs in roughly the same space of time. What a difference! I must say though, when set properly, it moves quite well. I've got all kinds of room for source code now! When not set properly, I shudder to even consider the experience.

And I even got to try out the new drive while opening a kernel source archive. It way outperformed the older drive that had been in there. I have to wait to really judge until I get like three or more kernel source archives opening or being created at the same time. That will tell a tale!

Speaking of new source, I've just uploaded .configs for 2.6.36-gentoo-r7 in both x86 and x86_64 flavors. Enjoy! The IP for the main site is in transition. I expect I should be back online here in five to ten minutes.

Cheers,
Pappy
_________________
This space left intentionally blank, except for these ASCII symbols.
Back to top
View user's profile Send private message
Display posts from previous:   
This topic is locked: you cannot edit posts or make replies.    Gentoo Forums Forum Index Unsupported Software All times are GMT
Goto page Previous  1, 2, 3 ... 11, 12, 13 ... 22, 23, 24  Next
Page 12 of 24

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum