Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
Kernel compilation always starts from beginning? [SOLVED]
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Installing Gentoo
View previous topic :: View next topic  
Author Message
ville.aakko
Tux's lil' helper
Tux's lil' helper


Joined: 06 Aug 2006
Posts: 100
Location: Oulu, Finland

PostPosted: Sat Mar 09, 2019 11:51 am    Post subject: Kernel compilation always starts from beginning? [SOLVED] Reply with quote

Hi!

Years ago, when was the last time I used Gentoo, I remember making several minor tweaks to the Kernel config and re-compiling. It re-compiled only the changed part and whatever overhead (for the lack of a better word I'd know to use here), but certainly not the whole Kernel.

Now I'm compiling for a AMD K6-3 box. I already compiled the Kernel, but forgot to include ext?fs support built-in - and don't want to use initramfs (for now at least). I though - I'll just chroot, include the ext2-3-4fs support and re-compile....

But, make starts from beginning. :( . I enabled ccache -> restarted compiling. Noticed it is even slower with ccache on this box (subsequent builds might be faster, but this is not acceptable) -> disabled ccache, and tweaked distcc settings while doing that. But - I noticed it starts from beginning every single time 8O .

I even re-started once after I noticed this (in the beginning of the compile process) to make sure there were no changes made to the Makefile or .config
- and still, it is a complete restart of the make / compilation process! 8O

Now, has the Kernel compilation process somehow changed over the years, so that it always forces to compile everything (despite parts being already compiled)? Or, is this a side-effect (bug?) of using distcc (cross-compiling for i586@i686 slave) here?

I'm surprised it re-compiles everything after minor changes to .config, but also after no changes were made there. EDIT: At least on Arch it still works as I though it should work?

Cheers!

p.s. Also - if some mod things another subforum is more appropriate, please feel free to move - I wasn't sure to post here, or in the Kernel subforum...
_________________
- Ville


Last edited by ville.aakko on Fri May 03, 2019 6:25 pm; edited 1 time in total
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 43213
Location: 56N 3W

PostPosted: Sat Mar 09, 2019 12:04 pm    Post subject: Reply with quote

ville.aakko,

Welcome back!
We knew you would come, just not when :)

make is very lazy. It does indeed start from the beginning every time. Make has to work out what needs to be done.
It looks at the timestamp on every output file and if that is newer than the timestamp on all the input files, make does nothing other than that check.

There are a few caveats.
a) you must not use make clean or make mrproper. That would remove all the output files from the last build.
b) system time must be monotonic. Backwards steps in time confuse make.
c) there are a small number of kernel options that will force a kernel rebuild if they are changed.

You may be seeing make checking every file and doing nothing.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
ville.aakko
Tux's lil' helper
Tux's lil' helper


Joined: 06 Aug 2006
Posts: 100
Location: Oulu, Finland

PostPosted: Sat Mar 09, 2019 12:38 pm    Post subject: Reply with quote

Hi Neddy,

Thanks for your reply.

This is exactly my conception what should be happening. But it isn't!

System time and timestamps was something I hadn't double-checked (but should be setup correctly) and I checked those right now. The system is in correct time, and with a quick glance, there is nothing wrong with the timestamps of files in the Kernel source tree.

I also made the restart I mentioned previously, to rule out it is something I had made to the kernel source tree myself confusing make; i.e. start make -> wait until it is compiling the "regular" parts of the kernel (I know from previous resumes I've seen should be skipped) -> CTRL+C and immediately restart make -> everything is recompiled again!

I know it recompiles from previous expereince, in more detail because 1) the file names are not printed if they are skipped my make (although, of course they are checked) and 2) same jobs are re-distributed to the distcc slave again.

I presume this has to be distcc confusing make somehow, as that is the only thing "special" on this setup? I will test that hypothesis later when I have the time (can not stop the ongoing kernel compile).

I guess I will try with a more simpler program (since I know this CTRl+C -> re-run make is pretty standard way to interrupt and resume) if I can reproduce this that way - is this generic issue on the box or only with kernel source tree.
_________________
- Ville
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 43213
Location: 56N 3W

PostPosted: Sat Mar 09, 2019 1:17 pm    Post subject: Reply with quote

ville.aakko,

I cross compile kernels for weaker systems. Mostly 64 bit raspberry Pi. Since you have a distcc helper, you have the bits needed to cross compile too.
Still, that does not answer the question of what's happening for you.

Exactly how do you get kernel sources, configure, build, install then modify your kernel?
A list of commands it what I'm looking for.

You don't sound like a genkernel user but if you are, it starts with a make clean by default, You have to tell it not to.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
ville.aakko
Tux's lil' helper
Tux's lil' helper


Joined: 06 Aug 2006
Posts: 100
Location: Oulu, Finland

PostPosted: Mon Mar 11, 2019 10:40 am    Post subject: Reply with quote

Hi,

The process I compile Kernel is very standard - except that I use pump-mode with distcc (in hindsight, perhaps I should have mentioned that in the beginning). I.e. it is "make menuconfig ; make && make install && make modules_install" - except, I've modified the Makefile according to this thread - and since despite using distcc, preprosessing is really slow on this box, I also use the pump mode. Now, I think what is happening is that make checks if the part is already compiled during preprocessing. If this is distributed to slave hosts, this detection breaks. I'm travelling at the moment, I can check this hypothesis when I get back home.

But, indeed, I realized as I've installed the toolchain with crossdev, cross-compiling is the way to go, as you pointed out - and really simple. I see little point in using distcc to cross-compile (since the toolchain is needed on the slave machine in any case!).[/url]
_________________
- Ville
Back to top
View user's profile Send private message
ville.aakko
Tux's lil' helper
Tux's lil' helper


Joined: 06 Aug 2006
Posts: 100
Location: Oulu, Finland

PostPosted: Fri May 03, 2019 6:25 pm    Post subject: Reply with quote

Hi!

Only now I've had time to play with this again :-)

It seems definitely the case that using pump mode was the culprit. Using distcc without pump and kernel compilation resumes as it is supposed to.


Haven't tried yet to reproduce this with pump, that would make it 100% certain. However, marking this as [SOLVED] as the issue is gone for me. If it resurfaces without pump, then that is something new and I will report back if that happens :D
_________________
- Ville
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Installing Gentoo All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum