Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
[SOLVED] How do I use distcc to compile packages?
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Installing Gentoo
View previous topic :: View next topic  
Author Message
alogim
Tux's lil' helper
Tux's lil' helper


Joined: 21 Aug 2015
Posts: 127

PostPosted: Wed May 01, 2019 11:52 am    Post subject: [SOLVED] How do I use distcc to compile packages? Reply with quote

I am following this guide. Now, I have two machines:
  • a desktop PC, with an Intel Core i5-3570k, with 4 physical cores (4 logical cores in total) and 8 GB RAM, with IP address 192.168.1.5
  • a laptop PC, with an Intel Core i7-8550U, with 4 physical cores (8 logical cores in total) with IP address 192.168.1.4

I have installed sys-devel/distcc on both machines. Now, how do I proceed?

I changed my /etc/portage/make.conf on my laptop, adding
Code:
MAKEOPTS="-j17 -l4" FEATURES="distcc"
and changing my CFLAGS so that it does not use -march=native but rather the output of this command. I also added
Code:
DISTCCD_OPTS="--port 3632 --log-level notice --log-file /var/log/distccd.log -N 15 --allow 192.168.1.5"
in my laptop's /etc/conf.d/distccd, and started the distcc service on both machines. Now what shall I do?
_________________
Desktop: i5-3570k - 2x4 GB - Sapphire HD 7950 Dual-X - 500 GB WD Caviar Black - Dell SE2717H 27"
Laptop: HP EliteBook 850 G5 - i7-8550U - 1x16GB - Intel UHD 620 - 512 GB Samsung NVMe SSD
Phone: OnePlus 5T - SD835 - 8 GB - Adreno 540 - 128 GB


Last edited by alogim on Sat May 04, 2019 9:22 am; edited 1 time in total
Back to top
View user's profile Send private message
krinn
Watchman
Watchman


Joined: 02 May 2003
Posts: 7195

PostPosted: Wed May 01, 2019 12:35 pm    Post subject: Reply with quote

Better understand who ask what.

Your setup is confusing.
* the computer that will use distcc to get help need have its cflags set, it need feature="distcc" and it doesn't need distccd running
* the computer that will be used by distcc need distccd running, and properly configure to allow this (--allow...)

you have setup laptop cflags and feature, like you will want use desktop to help laptop doing something
and you have setup laptop to allow to be use to help the desktop (with --allow).

desktop is not setup to be use as helper yet, so basically, you endup with laptop able to use distcc, and also able to be use as helper for desktop ; but desktop is not able to use distcc, nor able to be use as helper.
Back to top
View user's profile Send private message
alogim
Tux's lil' helper
Tux's lil' helper


Joined: 21 Aug 2015
Posts: 127

PostPosted: Wed May 01, 2019 2:46 pm    Post subject: Reply with quote

krinn wrote:
Better understand who ask what.

Your setup is confusing.
* the computer that will use distcc to get help need have its cflags set, it need feature="distcc" and it doesn't need distccd running
* the computer that will be used by distcc need distccd running, and properly configure to allow this (--allow...)

you have setup laptop cflags and feature, like you will want use desktop to help laptop doing something
and you have setup laptop to allow to be use to help the desktop (with --allow).

desktop is not setup to be use as helper yet, so basically, you endup with laptop able to use distcc, and also able to be use as helper for desktop ; but desktop is not able to use distcc, nor able to be use as helper.

Honestly, the Wiki says the following:
  • Install sys-devel/distcc on each machine, so I installed it on both my desktop and my laptop, and so far so good.
  • Then they say to configure distcc. The Wiki states that "The following example allows the distcc clients running at 192.168.0.4 and 192.168.0.5 to connect to the distccd server running locally:
    Code:
    DISTCCD_OPTS="--port 3632 --log-level notice --log-file /var/log/distccd.log -N 15 --allow 192.168.0.4 --allow 192.168.0.5"
    Now, I thought the distcc clients were the helpers and the distcc server was the one requesting the compilation, but maybe I am wrong. In fact, a server is usually always on and waiting for clients' requests, so maybe it is all the way around.
So, could you please guide me into the process? I will make sure to rewrite the wiki in a proper way, because as it is right now, it is very confusing and there is not a clear distinction of what you actually need to do. That is my humble opinion, of course, but I was always able to follow the wiki without problems, while here it is pretty abstract.
_________________
Desktop: i5-3570k - 2x4 GB - Sapphire HD 7950 Dual-X - 500 GB WD Caviar Black - Dell SE2717H 27"
Laptop: HP EliteBook 850 G5 - i7-8550U - 1x16GB - Intel UHD 620 - 512 GB Samsung NVMe SSD
Phone: OnePlus 5T - SD835 - 8 GB - Adreno 540 - 128 GB
Back to top
View user's profile Send private message
krinn
Watchman
Watchman


Joined: 02 May 2003
Posts: 7195

PostPosted: Wed May 01, 2019 3:29 pm    Post subject: Reply with quote

alogim wrote:
"The following example allows the distcc clients running at 192.168.0.4 and 192.168.0.5 to connect to the distccd server running locally:
Code:
DISTCCD_OPTS="--port 3632 --log-level notice --log-file /var/log/distccd.log -N 15 --allow 192.168.0.4 --allow 192.168.0.5"


No you read it bad: it says "with that example configuration on a server, the server will allow two others clients to contact it and use that server"
In that example, you have no idea of the server ip, but its not 192.168.0.5 and not 192.168.0.4 that are two other hosts that wish use that server to get help in building something.

Like i have told you earlier, you configure the laptop with --allow 192.168.0.4 which imply you want desktop (192.168.0.4) to use distcc with the help of the laptop

if you want laptop to use distcc:
in latop:
/etc/distcc/hosts
localhost/8 192.168.1.5/4
cflags set
features="distcc" set
on desktop:
run distccd
configure /etc/conf.d/distccd with --allow 192.168.1.4

the server is the one that get asked something, in my example the desktop
and the client is the one that is asking something, in my example, the laptop
it is no different than using some web server and firefox, the server is the one that will be asked something, while firefox is the one asking something to it.
Back to top
View user's profile Send private message
alogim
Tux's lil' helper
Tux's lil' helper


Joined: 21 Aug 2015
Posts: 127

PostPosted: Wed May 01, 2019 3:50 pm    Post subject: Reply with quote

Okay, so in my case the desktop is the server and laptop is the client. My desktop has IP address 192.168.1.5, my laptop has IP address 192.168.1.4 and I want to use the desktop to help the laptop compiling packages.
  • Install sys-devel/distcc on the desktop and on the laptop
  • Set -march=native -O2 -pipe and FEATURES="distcc" on the laptop in the file /etc/portage/make.conf
  • Run /usr/bin/distcc-config --set-hosts "localhost 192.168.1.5" on the laptop
  • Update /etc/conf.d/distccd on the desktop as follows:
    Code:
    DISTCCD_OPTS="--port 3632 --log-level notice --log-file /var/log/distccd.log -N 15 --allow 192.168.1.4"
  • Set the CFLAGS variable to the output of gcc -v -E -x c -march=native -mtune=native - < /dev/null 2>&1 | grep cc1 | perl -pe 's/^.* - //g;' in file /etc/portage/make.conf on the desktop
  • Add distccd to the default runlevel and start it (rc-update add distccd default and rc-service distccd start) on the desktop
  • Start compiling on the laptop
Is this correct?
_________________
Desktop: i5-3570k - 2x4 GB - Sapphire HD 7950 Dual-X - 500 GB WD Caviar Black - Dell SE2717H 27"
Laptop: HP EliteBook 850 G5 - i7-8550U - 1x16GB - Intel UHD 620 - 512 GB Samsung NVMe SSD
Phone: OnePlus 5T - SD835 - 8 GB - Adreno 540 - 128 GB


Last edited by alogim on Wed May 01, 2019 4:06 pm; edited 1 time in total
Back to top
View user's profile Send private message
Jaglover
Watchman
Watchman


Joined: 29 May 2005
Posts: 7328
Location: Saint Amant, Acadiana

PostPosted: Wed May 01, 2019 4:02 pm    Post subject: Reply with quote

No, distcc will not work with -march=native. Server CFLAGS do not matter.
_________________
Please learn how to denote units correctly!
Back to top
View user's profile Send private message
alogim
Tux's lil' helper
Tux's lil' helper


Joined: 21 Aug 2015
Posts: 127

PostPosted: Wed May 01, 2019 4:11 pm    Post subject: Reply with quote

You are saying this:
Jaglover wrote:
No, distcc will not work with -march=native. Server CFLAGS do not matter.
But then he said:
krinn wrote:
* the computer that will use distcc to get help need have its cflags set, it need feature="distcc" and it doesn't need distccd running
So, I need to only set CFLAGS for the laptop to the output of
Code:
gcc -v -E -x c -march=native -mtune=native - < /dev/null 2>&1 | grep cc1 | perl -pe 's/^.* - //g;'
?
_________________
Desktop: i5-3570k - 2x4 GB - Sapphire HD 7950 Dual-X - 500 GB WD Caviar Black - Dell SE2717H 27"
Laptop: HP EliteBook 850 G5 - i7-8550U - 1x16GB - Intel UHD 620 - 512 GB Samsung NVMe SSD
Phone: OnePlus 5T - SD835 - 8 GB - Adreno 540 - 128 GB
Back to top
View user's profile Send private message
Ant P.
Watchman
Watchman


Joined: 18 Apr 2009
Posts: 5963

PostPosted: Wed May 01, 2019 7:57 pm    Post subject: Reply with quote

Those CFLAGS should work, yes.
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 44010
Location: 56N 3W

PostPosted: Wed May 01, 2019 9:17 pm    Post subject: Reply with quote

alogim,

distcc is in two pieces. distccd, that listens for cries for help and the client that distributes jobs to helpers
Code:
emerge distcc
installs both pieces.

Its quite possible to build a 'compile farm' so that when any machine in the farm builds something, all the others help.
If you want this setup, all the machines in the compile farm need both pieces of distcc set up and distccd running.

Helpers only need distccd configured and running.
Clients need the distcc client configured. There is nothing to run.

distcc no longer distributes when you have -march=native set. That's a good thing. When your helpers are a different arch, you don't get back code built for the helpers arch.
Helpers need not be the same arch are the client.
e.g. My amd64 desktop system is a helper for my arm64 Raspberry Pi.

The CFLAGS that
Code:
gcc -v -E -x c -march=native -mtune=native - < /dev/null 2>&1 | grep cc1 | perl -pe 's/^.* - //g;'
produces will me excessive.
All the -mno-* options can be omitted as that's the default anyway.

Try
Code:
gcc -### -E - -march=native 2>&1 | sed -r '/cc1/!d;s/(")|(^.* - )|( -mno-[^\ ]+)//g'

_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Installing Gentoo All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum