archlinuxarm/wiki
1
0
Fork 0
mirror of https://github.com/archlinuxarm/wiki.git synced 2024-11-08 22:45:42 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

Update the distcc and cross-compiling guides

Browse source
This commit is contained in:
Kevin Mihelich 2018-03-27 21:29:01 -06:00
parent 72517b0190
commit 1272fa2304
2 changed files with 68 additions and 31 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

19
Distcc_Cross-Compiling.md
View file

@ -6,7 +6,7 @@ This is the official cross-compiling method used at Arch Linux ARM. If you plan
This guide makes use of distcc in order to not have to build a full ARM development environment on x86. As the distcc project website states, "distcc does not require all machines to share a filesystem, have synchronized clocks, or to have the same libraries or header files installed." This is particularly advantageous to us since all that is needed is a working cross-compiler for ARM on a faster machine, while controlling the build from an ARM computer that has all of the current libraries and headers. This guide makes use of distcc in order to not have to build a full ARM development environment on x86. As the distcc project website states, "distcc does not require all machines to share a filesystem, have synchronized clocks, or to have the same libraries or header files installed." This is particularly advantageous to us since all that is needed is a working cross-compiler for ARM on a faster machine, while controlling the build from an ARM computer that has all of the current libraries and headers.
## Distcc ## Distcc
Before beginning, a distcc environment needs to be set up. Follow the [[Distributed Compiling]] guide to establish a master device. The x86 machines will be known as the clients. Before beginning, a distcc environment needs to be set up. Follow the [[Distributed Compiling]] guide to establish a master system. The x86 machines will be known as the clients.
## Pre-built crosstool-ng toolchains ## Pre-built crosstool-ng toolchains
In lieu of building the toolchain as detailed below, if you are running a 64-bit Linux installation you can use these packaged toolchains that are employed in the official build system. In lieu of building the toolchain as detailed below, if you are running a 64-bit Linux installation you can use these packaged toolchains that are employed in the official build system.
@ -18,9 +18,7 @@ It is *highly recommended* to use these tarballs as they have been thoroughly te
* [ARMv7l hard](/builder/xtools/x-tools7h.tar.xz) (4a934847291fc7469c3ef26e4ada0ba0) * [ARMv7l hard](/builder/xtools/x-tools7h.tar.xz) (4a934847291fc7469c3ef26e4ada0ba0)
* [ARMv8](/builder/xtools/x-tools8.tar.xz) (6af0035de121c0b962a78bb0cb45ff71) * [ARMv8](/builder/xtools/x-tools8.tar.xz) (6af0035de121c0b962a78bb0cb45ff71)
If you want to save yourself time and configuration, see [WarheadsSE's distccd-alarm](https://github.com/WarheadsSE/PKGs/tree/master/distccd-alarm) package. This will generate 3 packages (one for each architecture), and contains configuration and systemd service units for each. It is for x86_64 only, like these toolchain tarballs. ## Building a crosstool-ng toolchain
## Install crosstool-ng
This process is very automated, courtesy of [crosstool-ng](http://crosstool-ng.org). As a normal user (<b>not root!</b>), clone revision 1dbb06f of the git repository into a directory called "cross" in your home directory. Enter the source directory and configure with a prefix for the "cross" directory, make, and make install. If you are missing any pre-requisites, the configure script will let you know what they are. This process is very automated, courtesy of [crosstool-ng](http://crosstool-ng.org). As a normal user (<b>not root!</b>), clone revision 1dbb06f of the git repository into a directory called "cross" in your home directory. Enter the source directory and configure with a prefix for the "cross" directory, make, and make install. If you are missing any pre-requisites, the configure script will let you know what they are.
``` ```
@ -76,19 +74,10 @@ for file in `ls`; do
done done
``` ```
Now the "bin" directory contains links with names that distcc will play nice with. To get distcc to use these binaries instead of the default system ones, we need to place this directory into the path for the distcc daemon: Now the "bin" directory contains links with names that distcc will play nice with.
* Debian/Ubuntu: Edit "/etc/init.d/distcc"
* Arch Linux: Edit "/etc/conf.d/distccd"
After the initial header block, add this line or modify PATH if it already exists in the file. Note that we are placing our binary directory at the very front. After making the change, restart the distccd daemon.
```
PATH=/home/your_user/x-tools[6h|7h|8]/arm-unknown-linux-gnueabi[hf]/bin:$PATH
```
## Compile! ## Compile!
Back on ARM "master" device, make sure that distcc has been enabled in makepkg.conf per the [[Distributed Compiling]] guide, and specify the cross-compiler computer's IP address in the DISTCC_HOSTS variable. Now all builds using makepkg will make use of the distcc and cross compiler setup. Back on ARM master device, make sure that distcc has been enabled in makepkg.conf per the [[Distributed Compiling]] guide. Now all builds using makepkg will make use of the distcc and cross compiler setup.
## Crap, it's not working ## Crap, it's not working
If you've followed this guide to the letter and you know you've done everything right, the likely problem is that the user distccd is running as does not have permission to access the location of the crosstool-ng binaries. Either change distccd's user or relocate the x-tools directory to somewhere it can read it, then be sure the PATH set above reflects the new location. If you've followed this guide to the letter and you know you've done everything right, the likely problem is that the user distccd is running as does not have permission to access the location of the crosstool-ng binaries. Either change distccd's user or relocate the x-tools directory to somewhere it can read it, then be sure the PATH set above reflects the new location.

78
Distributed_Compiling.md
View file

@ -1,8 +1,10 @@
If you have multiple devices, using distcc is a very simple way to get all of them in on compiling packages. makepkg is already set up for using distcc, all that's needed is to enable the functionality. If you have spare ARM or x86 systems, using distcc is a very simple way to get all of them in on compiling packages. makepkg is already set up for using distcc, all that's needed is to enable the functionality.
Before you begin, you will need to make sure you have base-devel installed. Each device will need to have the same packages installed that are required for compiling the package you're about to build using this method. See the [Arch Wiki](https://wiki.archlinux.org) for makepkg, pacman, etc. For distributing to other ARM systems, you will need to have base-devel installed. Each system needs to have the same packages installed that are required for compiling the package you're about to build using this method. See the [Arch Wiki](https://wiki.archlinux.org) for makepkg, pacman, etc.
Install distcc on all devices to be used: For distributing to x86 systems, install the cross compilers for the needed targets from the [[Distcc Cross-Compiling]] page.
Install distcc on all systems to be used:
``` ```
pacman -S distcc pacman -S distcc
@ -11,45 +13,91 @@ pacman -S distcc
## Configuration: Introduction ## Configuration: Introduction
For the purposes of this example, all of the hosts are in the local network of 10.3.0.0/24, and access is allowed from that entire subnet. If you want to restrict access further, you will have to specify individual IPs. Be sure to change the address from the examples to fit your local network configuration. For the purposes of this example, all of the hosts are in the local network of 10.3.0.0/24, and access is allowed from that entire subnet. If you want to restrict access further, you will have to specify individual IPs. Be sure to change the address from the examples to fit your local network configuration.
Additionally, this example assumes that you are starting compiles from one specific device (master), which is sending out to the other device(s) (client). If you want to compile from any device, simply apply the client configuration to the master device, and the master device configuration to the client device(s). Additionally, this example assumes that you are starting compiles from one specific system (master), which is sending compiles out to the other system(s) (client). If you want to compile from any system, simply apply the client configuration to the master system, and the master system configuration to the client system(s).
If you want to use the master device to compile as well, apply the client configuration to it. If you want to use the master system to compile as well, apply the client configuration to it.
## Configuration: Master Device ## Configuration: Master System
This is what you run on the ARM device you'll be running "makepkg" from and using to coordinate builds. These are the configurations for the ARM system you'll be running *makepkg* from.
In /etc/makepkg.conf, change BUILDENV to un-negate the distcc option: In /etc/makepkg.conf, or in makepkg.conf in your chroot, change BUILDENV to un-negate the distcc option:
``` ```
BUILDENV=(distcc color !ccache check !sign) BUILDENV=(distcc color !ccache check !sign)
``` ```
Change DISTCC_HOSTS to a space-delimited list of the distcc clients on your network. If you want to use the master plug, be sure to put it's IP address in there too. The hosts listed will be prioritized in the order listed. To keep configurations simple, use the LAN IP address for the local plug, not "localhost" or "127.0.0.1". Change DISTCC_HOSTS to a space-delimited list of the distcc clients on your network. If you want to use the master system, be sure to put it's IP address in there too. The hosts listed will be prioritized in the order listed. To keep configurations simple, use the LAN IP address for the local system, not "localhost" or "127.0.0.1".
``` ```
DISTCC_HOSTS="10.3.0.6 10.3.0.2" DISTCC_HOSTS="10.3.0.6 10.3.0.2"
``` ```
Change MAKEFLAGS -j flag to reflect the total number of processors available +1. For example, with 2 plugs (1 master and 1 client): Additional options can be specified in DISTCC_HOSTS to control how distcc uses the hosts. For example, to randomize which host is used, limit the maximum number of jobs sent to the first host to 4 and the second host to 8, and to specify a different port that distccd is running on:
```
DISTCC_HOSTS="--randomize 10.3.0.6:3932/4 10.3.0.2:3932/8"
```
Change the MAKEFLAGS -j flag to reflect the total number of processors available on the master system. The common wisdom is to set this to the number of physical cores + 1. Keep in mind that only compiles are distributed; preprocessing and linking still takes place on the master system. Therefore, this number should reflect the capabilities of the master system and not the total number of distributed cores available.
``` ```
MAKEFLAGS="-j3" MAKEFLAGS="-j3"
``` ```
## Configuration: Client Device(s) ## Configuration: Client System(s)
This is what you'd run on a second ARM device or a cross-compiling computer. These are the configurations for a second ARM system or x86 cross-compiling computer.
In /etc/conf.d/distccd, change DISTCC_ARGS to reflect the hosts or network you're allowing to connect: In /etc/conf.d/distccd, change DISTCC_ARGS to reflect the hosts or network you're allowing to connect and the number of distccd instances (jobs) to start. These arguments will be passed to distccd when it's started, so this is a good place to add any additional arguments you want.
``` ```
DISTCC_ARGS="--user nobody --allow 10.3.0.0/24" DISTCC_ARGS="--allow 10.3.0.0/24 --jobs 20"
``` ```
Enable and start the distcc daemon: As of distcc 3.3, protections for [CVE-2004-2687](https://nvd.nist.gov/vuln/detail/CVE-2004-2687) have been implemented. Specifically, the symlinks to distcc in /usr/lib/distcc are a whitelist of what executables are allowed to be run through distcc. For cross-compiling systems, this means that symlinks of each ARM architecture's compiler must be created (c++, cc, cpp, g++, and gcc). For example:
```
ln -s /usr/bin/distcc /usr/lib/distcc/aarch64-unknown-linux-gnu-gcc
```
Alternately, if you are confident in your security, you can disable the protections by adding `--make-me-a-botnet` to DISTCC_ARGS.
### systemd services
On ARM systems where the installed toolchain will be used, simply enable the shipped distccd service:
``` ```
systemctl enable distccd systemctl enable distccd
systemctl start distccd systemctl start distccd
``` ```
On x86 systems where a cross toolchain will be used, create a new systemd service file for the toolchain. When setting up for cross-compiling multiple ARM architectures, each service file should start distcc with different ports from the default (3632) and specify a unique log file. The new port number must be set in DISTCC_HOSTS in makepkg.conf. This example shows a service file for the AArch64 toolchain in a multiple toolchain environment:
`/etc/systemd/system/distccd8.service`
```
[Unit]
Description=A distributed C/C++ compiler (ARMv8)
Documentation=man:distccd(1)
After=network.target
[Service]
User=myuser
EnvironmentFile=/etc/conf.d/distccd
Environment="PATH=/home/myuser/x-tools8/aarch64-unknown-linux-gnueabi/bin:/usr/bin"
ExecStart=/usr/bin/distccd --no-detach --daemon $DISTCC_ARGS --log-file=/var/log/distccd8.log --port 3932 --stats-port 3933
[Install]
WantedBy=multi-user.target
```
This service sets the user that distccd will run as, which must be able to access and execute the toolchain binaries. It sources the contents of `/etc/conf.d/distcc` into the environment so the DISTCC_ARGS variable is available. It then sets the path, placing the bin directory of the AArch64 toolchain at the front. The distccd daemon is started with the options from DISTCC_ARGS, a unique log file, and port numbers for distccd and the stats server (which can be enabled with the `--stats` option).
Depending on the user distccd is run as and the permissions of the log directory, you may need to create the log file prior to starting the service and set its owner to that user.
The service can then be enabled and started:
```
systemctl enable distccd8
systemctl start distccd8
```
You can now run *makepkg* as you normally would and the hosts specified will start receiving work. If the hosts are unavailable, the package will simply compile locally. You can now run *makepkg* as you normally would and the hosts specified will start receiving work. If the hosts are unavailable, the package will simply compile locally.