forked from distcc/distcc
-
Notifications
You must be signed in to change notification settings - Fork 0
/
INSTALL
356 lines (235 loc) · 10.6 KB
/
INSTALL
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
Installation instructions for distcc -*- indented-text -*-
distcc is free software; you can redistribute it and/or modify it
under the terms of the GNU General Public License as published by the
Free Software Foundation; either version 2 of the License, or (at your
option) any later version. distcc comes with ABSOLUTELY NO WARRANTY,
for details see the licence.
Please report any problems to [email protected].
QUICK SUMMARY
=============
0. Install prerequisites. For example, on Debian or Ubuntu, you could use
sudo apt-get install gcc make python python-dev libiberty-dev
Note that past versions of Debian used binutils-dev instead of
libiberty-dev.
1. Build and install
./autogen.sh # If "configure" does not already exist.
./configure
make
make check # Optional! Should have python >= 2.4 installed.
make install # You may need to use "sudo" for this command.
make installcheck # Optional! Should have python >= 2.4 installed.
Repeat installation for each server machine.
2. Run distccd on each server machine,
with --allow options to limit access.
3. On your client machine, set DISTCC_POTENTIAL_HOSTS to the list
of distcc server machines:
export DISTCC_POTENTIAL_HOSTS='localhost red green blue'
4. Build your software using distcc:
cd ~/my_sources/my_project
pump make -j40 CC="distcc gcc" <your target>
DETAILED INSTRUCTIONS
=====================
Prerequisites
-------------
To build distcc you need
GNU Make
A C compiler
GNU libiberty (distributed with GNU Binutils and GCC)
It is also highly desireable to have
Python >=2.4
To use "pump" mode, or to run the full test suite with "make check",
or to run the benchmarks.
If you have any version of Python installed, then you need to have
the corresponding Python.h file installed (e.g. sudo apt-get install python-dev).
You can optionally have
Python >=2.2
To run a partial test suite with "make check" (it leaves out
the pump-mode functionality, which requires Python 2.4).
libpopt
If this is not found on your system, the popt version that is part of
the distcc distribution will be statically linked in.
linuxdoc SGML tools
To rebuild the documentation from its SGML source.
autoconf >=2.5
To rebuild the configure scripts if you edit configure.ac.
To build the optional GNOME monitor (--with-gnome), you need the
GNOME2 development libraries, and in particular
gtk+ >=2.0
libgnome >=2.0
libgnomeui >= 2.0
libglade >= 2.0
libpango
The monitor can also be built without GNOME desktop integration
(--with-gtk), in which case you need only
gtk+ >=2.0
libglade >= 2.0
Preliminaries
-------------
distcc can be installed and used without requiring root access.
Adjust directories appropriately when installing.
Configuring distcc
------------------
Note that the default GNU "sysconfdir" is /usr/local/etc. You may
want to change this to /etc.
$ ./configure --sysconfdir=/etc
You can set an installation prefix if you want to put distcc in /opt:
$ ./configure --prefix=/opt/distcc/
distcc needs to be installed on all client and server machines.
If you would like a graphical client-side monitor to show running
jobs, you can choose either --with-gnome or --with-gtk.
If you would like to try running distcc over IPv6, use
--enable-rfc2553. You must have a reasonably recent operating system
or this is likely to fail in complex ways.
Building distcc
---------------
The simple way to build distcc is to just do
./autogen.sh # if "configure" does not already exist in this directory
./configure && make
but the recommended way is to build in a different directory tree
than the source tree:
mkdir obj
cd obj
../autogen.sh
../configure && make
You can optionally run "make check" afterwards to verify that everything
built OK.
Installing distcc
-----------------
If your system supports RPM packages, you can build RPMs with "make rpm"
and then install them with "rpm -i packaging/*.rpm".
If your system supports Debian packages, you can build them with "make deb"
and then install them with "make install-deb".
Otherwise, you can use the regular "make install". But it is preferable
to install via an RPM or Debian package if possible, because those
methods will do a bit more of the setup work.
You can optionally run "make installcheck" afterwards to verify that
was installed OK. This works regardless of which installation method
you used.
Starting the daemon
-------------------
This stage is only required if you want to run distcc over TCP
sockets, rather than SSH connections. TCP is faster but less secure
and should only be used on trusted networks.
In TCP mode distccd can run either from inetd or as a standalone
daemon. Running standalone is recommended.
If you installed via the debian or RPM package, then the daemon will be
installed as a service, running as a standalone daemon, and it should get
started up automatically, so you can skip the rest of this section
"Starting the daemon". The rest of this section only applies if you
installed via "make install".
To run standalone, run a command like this, either from the command
line or from the system startup scripts.
distccd --daemon
If the daemon is started from an rc script, then make sure that it
sees a PATH setting which can find any compilers installed in
nonstandard directories.
You should create a "distcc" account on server machines so that distcc
can run with minimal privilege. It is not necessary for this account
to own any files or have a home directory. If this account doesn't
exist, distccd uses the "nobody" account.
By default distccd writes messages to the "daemon" syslog, which
typically ends up in /var/log/messages or /var/log/daemon.
You can set IP-based access control using the --allow and --listen
options, in either inetd or daemon mode:
distccd --allow 10.4.20.0/24
distccd does not need to run on machines that will only act as
clients.
See the manual for more information.
Editing the daemon configuration files
--------------------------------------
If you installed via the RPM or Debian package, then you
will have an /etc/init.d/distcc script which reads several
configuration files to decide if and how to invoke distccd.
These configuration files are
/etc/default/distcc
/etc/distcc/clients.allow
/etc/distcc/commands.allow.sh
If you installed via the RPM or Debian package, then you
should edit those configuration files, especially the
clients.allow file.
If you installed via "make install", then those configuration
files will be installed, but they will NOT be used, unless
you manually install the init.d/distcc script from
packaging/RedHat/init.d/distcc and tailor it for your system.
If you start distccd manually, rather than via the init.d/distcc script,
then distccd won't use any configuration files; the allowed client IP
addresses will be determined by the --allow options that you pass to
distccd and the allowed commands will be determined by the DISTCC_CMDLIST
environment variable; see the distccd(1) man page for details.
See also the doc/example directory and its README file, which
has examples of the configuration files that you need.
Setting up the host list
------------------------
On the client machines, store a list of servers names in
~/.distcc/hosts or /etc/hosts, or in the DISTCC_HOSTS
environment variable.
If you're using TCP connections, it should look like
this:
localhost red green blue
For SSH connections
localhost @red @green @blue
The hosts should be listed in descending order of speed. localhost
should normally be first, unless it is signficantly slower than
another machine. If you many hosts (say ten or more), it's probably
better to leave localhost out of the list.
The host list also needs ",cpp,lzo" after each host if you're using
pump mode - see README.pump for details. Another alternative if you're
using pump mode is to set the DISTCC_POTENTIAL_HOSTS environment
variable; in that case, the pump script will use "lsdistcc" to
set DISTCC_HOSTS, automatically eliminating hosts that are down
or inaccessible or that don't have distccd running.
See the distcc(1) manual and the pump(1) manual for more information.
Using distcc
------------
Distcc will only improve performance if your build is parallelized.
So you need to use the "-j" option to make, or its equivalent with
your build tools. If your build contains too many sequential steps,
e.g. if your Makefile contains
all:
for subdir in $(SUBDIRS); do make -C $$subdir all; done
then you may need to rewrite your Makefile to get better parallelism.
This is especially important if you're using pump mode.
Using pump mode
---------------
To use pump mode, invoke your build command via the "pump" script, e.g.
export DISTCC_POTENTIAL_HOSTS="localhost red blue green"
cd ~/my_sources/my_project
pump make -j40 CC="distcc gcc" my_target
Pump mode assumes that source files are not modified during the build.
If this assumption does not hold, you should not use pump mode.
You can disable pump mode by not using the pump script.
In that case, you need to set DISTCC_HOSTS rather than
DISTCC_POTENTIAL_HOSTS.
export DISTCC_HOSTS="localhost red blue green"
cd ~/my_sources/my_project
make -j40 CC="distcc gcc" my_target
Creating the masquerade directories
-----------------------------------
The easiest way to use distcc is in "masquerade" mode, where it is
installed on the PATH to "catch" calls to the compiler and redirect
them over the network. Other options are discussed in the manual.
For instance, you could create the directory named /usr/lib/distcc
and populate it with links.
# mkdir /usr/lib/distcc
# cd /usr/lib/distcc
# ln -s /usr/bin/distcc gcc
# ln -s /usr/bin/distcc cc
# ln -s /usr/bin/distcc g++
# ln -s /usr/bin/distcc c++
If you installed via the RedHat or Debian package, then this masquerade
directory is already set up automatically in /usr/lib/distcc.
Do this for all compiler names that you use.
Then, to use distcc, a user just needs to put the directory
/usr/lib/distcc/bin early in the PATH and distcc will handle the rest.
export PATH=/usr/lib/distcc/bin:$PATH
Use with ccache
---------------
The best way to use is to set up a similar masquerade directory for
ccache and put it on the PATH before distcc.
NOTE: This use of ccache is incompatible with use of distcc's "pump" mode.
(If you're using "pump" mode, it might be possible to use ccache on
the distcc server machines. But we haven't tried that setup.)
Complete the survey
-------------------
Once you have distcc working for your own application, please complete
and mail in the survey in survey.txt.