Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
distcc only using localhost.
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Installing Gentoo
View previous topic :: View next topic  
Author Message
1clue
Advocate
Advocate


Joined: 05 Feb 2006
Posts: 2562

PostPosted: Tue Aug 05, 2014 5:00 am    Post subject: distcc only using localhost. Reply with quote

Hi,

I have been using this guide: http://wiki.gentoo.org/wiki/Raspberry_Pi_Cross_building

I have a raspberry pi B+ with most of a Gentoo install.

When I do distcc-watch on the rpi I get:
Code:

  1322  Compile                                                   localhost[0]
  1354  Compile                                                   localhost[0]
  1775  Compile     conftest.cpp                                  localhost[0]
  2530  Compile                                                   localhost[0]
  2557  Compile                                                   localhost[0]
  2582  Compile     conftest.c                                    localhost[0]
  2600  Compile     conftest.c                                    localhost[0]
  2627  Compile                                                   localhost[0]
  2642  Compile                                                   localhost[0]
  2677  Compile                                                   localhost[0]
  2703  Compile                                                   localhost[0]
  2728  Compile                                                   localhost[0]
  2747  Compile                                                   localhost[0]


and so on. On my i7 I have several processes listed as distccd but none ever get time.

I suspect my problem is with config but it matches the example almost exactly:

Code:

localhost ~ # cat /etc/distcc/hosts
# --- /etc/distcc/hosts -----------------------
# See the "Hosts Specification" section of
# "man distcc" for the format of this file.
#
# By default, just test that it works in loopback mode.
192.168.1.10
#127.0.0.1


Code:

localhost ~ # cat /etc/conf.d/distccd
# /etc/conf.d/distccd: config file for /etc/init.d/distccd

# this is the distccd executable
DISTCCD_EXEC="/usr/bin/distccd"

# this is where distccd will store its pid file
DISTCCD_PIDFILE="/var/run/distccd/distccd.pid"

# Ports:
# Default port is 3632.  For most people the default is okay.
#
# Logging:
# You can change some logging options here:
# --log-file FILE
# --log-level LEVEL  [critical,error,warning, notice, info, debug]
#
# Leaving --log-file blank will log to syslog
# example: --log-file /dev/null --log-level warning
# example: --log-level critical
#
# Nice level:
# The default nice level is 15. You can change it by adding: -N xx
# where xx is the nice level.

# SECURITY NOTICE:
# It is HIGHLY recomended that you use the --listen option
# for increased security. You can specify an IP to permit connections
# from or a CIDR mask
# --listen accepts only a single IP
# --allow is now mandatory as of distcc-2.18.
# example:  --allow 192.168.0.0/24
# example:  --allow 192.168.0.5 --allow 192.168.0.150
# example:  --listen 192.168.0.2

DISTCCD_OPTS="--port 3632 --log-level notice --log-file /var/log/distccd.log -N 15"
DISTCCD_OPTS="${DISTCCD_OPTS} --allow 192.168.1.0/24"


The good news is raspberry pi is faster at compiling than I ever thought it could be. The bad news is that it's still pretty darn slow.

Need advice. Thanks.
Back to top
View user's profile Send private message
shazeal
Apprentice
Apprentice


Joined: 03 May 2006
Posts: 198
Location: New Zealand

PostPosted: Tue Aug 05, 2014 5:09 am    Post subject: Reply with quote

I just set this up on my server to use my desktop for compiling, same deal distcc never contacts my desktop. I am guessing the docs are out of date? :(

EDIT: My config is virtually identical to yours, with the exception i am not cross compiling as well.
_________________
CFLAGS="-OmgWTFR1CE --fun-lol-loops --march=asmx86go"
Back to top
View user's profile Send private message
1clue
Advocate
Advocate


Joined: 05 Feb 2006
Posts: 2562

PostPosted: Tue Aug 05, 2014 5:22 am    Post subject: Reply with quote

You doing it on multiple pi's or are you doing it with some other arm processor?
Back to top
View user's profile Send private message
wraeth
Developer
Developer


Joined: 08 May 2007
Posts: 71
Location: Australia

PostPosted: Tue Aug 05, 2014 5:57 am    Post subject: Reply with quote

I've recently set up a similar configuration as you, so lets see if I can help. :)

Just to double check, do you have a sane cross-compiler toolchain on your i7? By sane I mean that, while some builds may fail for one reason or another, you can build some packages for the target platform successfully?

What init system are you running on your i7? I found that the systemd unit for distccd did not use /etc/conf.d/distccd, but instead used /etc/systemd/system/distccd.service.d/00gentoo.conf

Do you have a firewall running? If so, try disabling that and seeing if that makes a difference.

Have you set FEATURES="distcc distcc-pump" in make.conf on your RPi?

Have you set the distcc hosts on the RPi?
Code:
distcc-config --get-hosts
nemesis/7,cpp,lzo localhost/1

Do you have any telling output from a build on the RPi? For example, when mine is having trouble with distcc, I get:

Code:
armv6j-hardfloat-linux-gnueabi-g++ -c -O2 -pipe -march=armv6j -mfpu=vfp -mfloat-abi=hard -I/usr/include/glib-2.0 -I/usr/lib/glib-2.0/include -I/usr/include/freetype2 -fPIC -Wall -W -DQT_BOOTSTRAPPED -DQT_LITE_UNICODE -DQT_NO_CAST_FROM_ASCII -DQT_NO_CAST_TO_ASCII -DQT_NO_CODECS -DQT_NO_DATASTREAM -DQT_NO_GEOM_VARIANT -DQT_NO_LIBRARY -DQT_NO_QOBJECT -DQT_NO_STL -DQT_NO_SYSTEMLOCALE -DQT_NO_TEXTSTREAM -DQT_NO_THREAD -DQT_NO_UNICODETABLES -DQT_NO_USING_NAMESPACE -DQT_NO_DEPRECATED -D_LARGEFILE64_SOURCE -D_LARGEFILE_SOURCE -I../../../mkspecs/linux-g++ -I. -I../../../include -I../../../include/QtCore -I../../../include/QtXml -o .obj/release-static/qabstractfileengine.o ../../corelib/io/qabstractfileengine.cpp
distcc[4561] Warning: include server gave up analyzing
distcc[4561] (dcc_build_somewhere) Warning: failed to get includes from include server, preprocessing locally
armv6j-hardfloat-linux-gnueabi-g++ -c -O2 -pipe -march=armv6j -mfpu=vfp -mfloat-abi=hard -I/usr/include/glib-2.0 -I/usr/lib/glib-2.0/include -I/usr/include/freetype2 -fPIC -Wall -W -DQT_BOOTSTRAPPED -DQT_LITE_UNICODE -DQT_NO_CAST_FROM_ASCII -DQT_NO_CAST_TO_ASCII -DQT_NO_CODECS -DQT_NO_DATASTREAM -DQT_NO_GEOM_VARIANT -DQT_NO_LIBRARY -DQT_NO_QOBJECT -DQT_NO_STL -DQT_NO_SYSTEMLOCALE -DQT_NO_TEXTSTREAM -DQT_NO_THREAD -DQT_NO_UNICODETABLES -DQT_NO_USING_NAMESPACE -DQT_NO_DEPRECATED -D_LARGEFILE64_SOURCE -D_LARGEFILE_SOURCE -I../../../mkspecs/linux-g++ -I. -I../../../include -I../../../include/QtCore -I../../../include/QtXml -o .obj/release-static/qbuffer.o ../../corelib/io/qbuffer.cpp
distcc[4562] ERROR: compile ../../corelib/codecs/qutfcodec.cpp on nemesis/7,cpp,lzo failed
distcc[4562] (dcc_build_somewhere) Warning: remote compilation of '../../corelib/codecs/qutfcodec.cpp' failed, retrying locally
distcc[4562] Warning: failed to distribute ../../corelib/codecs/qutfcodec.cpp to nemesis/7,cpp,lzo, running locally instead
Back to top
View user's profile Send private message
1clue
Advocate
Advocate


Joined: 05 Feb 2006
Posts: 2562

PostPosted: Tue Aug 05, 2014 3:43 pm    Post subject: Reply with quote

wraeth wrote:
I've recently set up a similar configuration as you, so lets see if I can help. :)


This is my first time with distcc, and my first time with crossdev. I'm confident that I know almost nothing about it other than following the directions.

Quote:

Just to double check, do you have a sane cross-compiler toolchain on your i7? By sane I mean that, while some builds may fail for one reason or another, you can build some packages for the target platform successfully?


I built the tool chain on my i7, but my only arm boxes are 3x rpi B+. One of them has half a Gentoo install. I don't know how to tell if the toolchain is sane.

Quote:

What init system are you running on your i7? I found that the systemd unit for distccd did not use /etc/conf.d/distccd, but instead used /etc/systemd/system/distccd.service.d/00gentoo.conf


I'm using the default openrc.

Quote:

Do you have a firewall running? If so, try disabling that and seeing if that makes a difference.


No firewalls yet on rpi, firewall on i7 disabled, firewall on Internet still engaged but not involved in traffic on localnet.

Quote:

Have you set FEATURES="distcc distcc-pump" in make.conf on your RPi?


It was just distcc, I just added distcc-pump.

Quote:

Have you set the distcc hosts on the RPi?
Code:
distcc-config --get-hosts
nemesis/7,cpp,lzo localhost/1



Code:
localhost portage # distcc-config --get-hosts
# --- /etc/distcc/hosts -----------------------
# See the "Hosts Specification" section of
# "man distcc" for the format of this file.
#
# By default, just test that it works in loopback mode.
192.168.1.10
#127.0.0.1


Quote:

Do you have any telling output from a build on the RPi? For example, when mine is having trouble with distcc, I get:

Code:
armv6j-hardfloat-linux-gnueabi-g++ -c -O2 -pipe -march=armv6j -mfpu=vfp -mfloat-abi=hard -I/usr/include/glib-2.0 -I/usr/lib/glib-2.0/include -I/usr/include/freetype2 -fPIC -Wall -W -DQT_BOOTSTRAPPED -DQT_LITE_UNICODE -DQT_NO_CAST_FROM_ASCII -DQT_NO_CAST_TO_ASCII -DQT_NO_CODECS -DQT_NO_DATASTREAM -DQT_NO_GEOM_VARIANT -DQT_NO_LIBRARY -DQT_NO_QOBJECT -DQT_NO_STL -DQT_NO_SYSTEMLOCALE -DQT_NO_TEXTSTREAM -DQT_NO_THREAD -DQT_NO_UNICODETABLES -DQT_NO_USING_NAMESPACE -DQT_NO_DEPRECATED -D_LARGEFILE64_SOURCE -D_LARGEFILE_SOURCE -I../../../mkspecs/linux-g++ -I. -I../../../include -I../../../include/QtCore -I../../../include/QtXml -o .obj/release-static/qabstractfileengine.o ../../corelib/io/qabstractfileengine.cpp
distcc[4561] Warning: include server gave up analyzing
distcc[4561] (dcc_build_somewhere) Warning: failed to get includes from include server, preprocessing locally
armv6j-hardfloat-linux-gnueabi-g++ -c -O2 -pipe -march=armv6j -mfpu=vfp -mfloat-abi=hard -I/usr/include/glib-2.0 -I/usr/lib/glib-2.0/include -I/usr/include/freetype2 -fPIC -Wall -W -DQT_BOOTSTRAPPED -DQT_LITE_UNICODE -DQT_NO_CAST_FROM_ASCII -DQT_NO_CAST_TO_ASCII -DQT_NO_CODECS -DQT_NO_DATASTREAM -DQT_NO_GEOM_VARIANT -DQT_NO_LIBRARY -DQT_NO_QOBJECT -DQT_NO_STL -DQT_NO_SYSTEMLOCALE -DQT_NO_TEXTSTREAM -DQT_NO_THREAD -DQT_NO_UNICODETABLES -DQT_NO_USING_NAMESPACE -DQT_NO_DEPRECATED -D_LARGEFILE64_SOURCE -D_LARGEFILE_SOURCE -I../../../mkspecs/linux-g++ -I. -I../../../include -I../../../include/QtCore -I../../../include/QtXml -o .obj/release-static/qbuffer.o ../../corelib/io/qbuffer.cpp
distcc[4562] ERROR: compile ../../corelib/codecs/qutfcodec.cpp on nemesis/7,cpp,lzo failed
distcc[4562] (dcc_build_somewhere) Warning: remote compilation of '../../corelib/codecs/qutfcodec.cpp' failed, retrying locally
distcc[4562] Warning: failed to distribute ../../corelib/codecs/qutfcodec.cpp to nemesis/7,cpp,lzo, running locally instead


I don't know. It's still building from last night's deep @world rebuild when I noticed it was only on localhost.

I have been having trouble emerging raspberrypi-sources and some other basic requirements on the pi. I thought maybe a world rebuild would clear up whatever was wrong.

I seriously am missing screen, it's like my arm is broken. Can't emerge that either.
Back to top
View user's profile Send private message
wraeth
Developer
Developer


Joined: 08 May 2007
Posts: 71
Location: Australia

PostPosted: Tue Aug 05, 2014 11:01 pm    Post subject: Reply with quote

1clue wrote:
This is my first time with distcc, and my first time with crossdev. I'm confident that I know almost nothing about it other than following the directions.

While this is also my first time with crossdev, I've had some experience at fumbling about with distcc before. It can be a little picky, but manageable. :)

1clue wrote:

I don't know how to tell if the toolchain is sane.

The simplest test is: can you build anything with the crossdev toolchain? It doesn't have to be anything complex or big, just something to test that it can build anything. The problem comes in when a build requires one of its built binaries to be run as part of the build process, because the crossdev host won't be able to run it.

1clue wrote:
It was just distcc, I just added distcc-pump.

The "distcc-pump" FEATURE allows portage/emerge to function with DistCC's pump mode, which is enabled by running emerge like below. With just the "distcc" feature enabled, normal emerges should use distcc hosts in non-pump mode.
Code:
pump emerge -av <something>


1clue wrote:
I don't know. It's still building from last night's deep @world rebuild when I noticed it was only on localhost.

When you can, have a look at the build output from the RPi. It should indicate what problems it's having with distcc.

1clue wrote:

I seriously am missing screen, it's like my arm is broken. Can't emerge that either.

I feel the same way when I can't use byobu or vim! :wink:

When you say you can't emerge them, however, do you mean on the i7 using crossdev, or on the RPi? Care to paste a build.log to have a look?
Back to top
View user's profile Send private message
1clue
Advocate
Advocate


Joined: 05 Feb 2006
Posts: 2562

PostPosted: Wed Aug 06, 2014 1:26 am    Post subject: Reply with quote

I don't know how to build something crossdev from the i7. Frankly I didn't know you could. I thought this whole distcc/crossdev had to be driven from the hardware you were building for.

Typically, I've installed Gentoo on i7 or similar, and never bothered with distcc because compiling was never that lengthy. And never did crossdev because I always buy intel. I'm pretty clueless on both.

I'm still compiling glibc. It was compiling glibc this morning, still going. I'm tempted to kill it and try to figure this thing out, surely if I can get distcc and crossdev going in a few hours it will do a world rebuild in no more than an hour.

I'll try the pump emerge thing asap.

I'm looking at emerge output as it prints on the terminal window through ssh, but I don't see errors.

Vim is a must. I got that as soon as I did the chroot. Well, after distcc I guess. It's usually the first thing. Don't use byobu.

I can't emerge screen or raspberrypi-sources on the pi. Again, don't know how to do it on the i7 with crossdev. I'll paste something if the problem still persists when I get the system back.
Back to top
View user's profile Send private message
wraeth
Developer
Developer


Joined: 08 May 2007
Posts: 71
Location: Australia

PostPosted: Wed Aug 06, 2014 1:43 am    Post subject: Reply with quote

crossdev creates a chroot (sort-of - you can't actually chroot into it without some sort of emulation layer) in /usr/${CTARGET} (so for me it's /usr/armv6j-hardfloat-linux-gnueabi). If I want to build something using crossdev on my desktop, I use the below command. One thing in particular to note is that you would need to keep the filesystem from the device and the chroot in /usr up-to-date (which I do by either compiling on the desktop then installing binaries on the RPi, or by just mounting the RPi's filesystem and copying root into /usr/armv6j-hardfloat-linux-gnueabi). When I first started this, I actually extracted the armv6j stage3 into /usr/${CTARGET} and started building there, then copied across to the SD card once I started getting build failures caused by trying to run native binaries.
Code:
mount --bind /usr/portage /usr/armv6j-hardfloat-linux-gnueabi/usr/portage
armv6j-hardfloat-linux-gnueabi-emerge --config-root=/usr/armv6j-hardfloat-linux-gnueabi --root=/usr/armv6j-hardfloat-linux-gnueabi <emerge_opts> <atom>

Because I try to do this with binary packages, and because I'm a lazy typist, I usually wrap this up with an alias:
Code:
alias pimerge='armv6j-hardfloat-linux-gnueabi-emerge --config-root=/usr/armv6j-hardfloat-linux-gnueabi --root=/usr/armv6j-hardfloat-linux-gnueabi --buildpkg'
pimerge -av <atom>


1clue wrote:
I'm looking at emerge output as it prints on the terminal window through ssh, but I don't see errors.

It should be noted that some packages are known to not work well with distcc, and so forcefully turn it of in the ebuild. I'm not sure if glibc is one of those, but it is something to be aware of. Similarly, some packages simply cannot be compiled using crossdev.
Back to top
View user's profile Send private message
1clue
Advocate
Advocate


Joined: 05 Feb 2006
Posts: 2562

PostPosted: Thu Aug 07, 2014 9:39 pm    Post subject: Reply with quote

What profile are you using?

Code:

# pimerge -av --ask app-misc/screen

 * IMPORTANT: 3 news items need reading for repository 'gentoo'.
 * Use eselect news to read news items.


These are the packages that would be merged, in order:

Calculating dependencies... done!
[ebuild  N     ] sys-libs/ncurses-5.9-r3:5 to /usr/armv6j-hardfloat-linux-gnueabi/ USE="cxx minimal -abi_ppc_32 -abi_ppc_64 -abi_s390_32 -abi_s390_64 -ada -debug -doc -gpm -profile -static-libs -tinfo -trace -unicode" 0 kB
[ebuild  N     ] app-misc/screen-4.0.3-r7 to /usr/armv6j-hardfloat-linux-gnueabi/ USE="-debug -multiuser -nethack -pam (-selinux)" 0 kB

Total: 2 packages (2 new), Size of downloads: 0 kB

Would you like to merge these packages? [Yes/No]
 * ARCH is not set... Are you missing the '/usr/armv6j-hardfloat-linux-
 * gnueabi/etc/portage/make.profile' symlink? Is the symlink correct? Is
 * your portage tree complete?


I don't have a make.profile. Which one would you use? eselect profile list shows my i7's amd64 profiles.
Back to top
View user's profile Send private message
1clue
Advocate
Advocate


Joined: 05 Feb 2006
Posts: 2562

PostPosted: Thu Aug 07, 2014 9:43 pm    Post subject: Reply with quote

Never mind, dumb question. I used the same one as the pi has.
Back to top
View user's profile Send private message
1clue
Advocate
Advocate


Joined: 05 Feb 2006
Posts: 2562

PostPosted: Thu Aug 07, 2014 9:45 pm    Post subject: Reply with quote

I tried pimerge screen and it seems to be working at typical i7 speeds.

BTW my glibc emerge is still happening on the pi. Not sure how many days now. 2 or 3.
Back to top
View user's profile Send private message
wraeth
Developer
Developer


Joined: 08 May 2007
Posts: 71
Location: Australia

PostPosted: Thu Aug 07, 2014 11:26 pm    Post subject: Reply with quote

I manually created the symlink with
Code:
ln -s ../../usr/portage/profiles/arch/arm/armv6j /usr/armv6j-hardfloat-linux-gnueabi/usr/portage/make.profile

I haven't tried rebuilding glibc (not that I can remember), but I have found some builds are painfully slow, even just to get started... :?
Back to top
View user's profile Send private message
krinn
Watchman
Watchman


Joined: 02 May 2003
Posts: 7071

PostPosted: Fri Aug 08, 2014 12:10 am    Post subject: Reply with quote

wraeth wrote:
It should be noted that some packages are known to not work well with distcc, and so forcefully turn it of in the ebuild. I'm not sure if glibc is one of those, but it is something to be aware of. Similarly, some packages simply cannot be compiled using crossdev.

It also should be note you shouldn't care that much for your toolchain, you just shouldn't use distcc to build the toolchain. As of course the two toolchain will be different.
That's something that will gave random results, and it's worst when crossbuilding. Only first run can be safely made with distcc, so you better do it on the slow computer then.
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Installing Gentoo All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum