From mboxrd@z Thu Jan 1 00:00:00 1970
Return-Path: <gentoo-commits+bounces-1015402-garchives=archives.gentoo.org@lists.gentoo.org>
Received: from lists.gentoo.org (pigeon.gentoo.org [208.92.234.80])
(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
(No client certificate requested)
by finch.gentoo.org (Postfix) with ESMTPS id 15B9F138331
for <garchives@archives.gentoo.org>; Thu, 5 Apr 2018 22:16:17 +0000 (UTC)
Received: from pigeon.gentoo.org (localhost [127.0.0.1])
by pigeon.gentoo.org (Postfix) with SMTP id 2D036E0918;
Thu, 5 Apr 2018 22:16:16 +0000 (UTC)
Received: from smtp.gentoo.org (mail.gentoo.org [IPv6:2001:470:ea4a:1:5054:ff:fec7:86e4])
(using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits))
(No client certificate requested)
by pigeon.gentoo.org (Postfix) with ESMTPS id EA0CBE0918
for <gentoo-commits@lists.gentoo.org>; Thu, 5 Apr 2018 22:16:15 +0000 (UTC)
Received: from oystercatcher.gentoo.org (unknown [IPv6:2a01:4f8:202:4333:225:90ff:fed9:fc84])
(using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits))
(No client certificate requested)
by smtp.gentoo.org (Postfix) with ESMTPS id 77D23335C51
for <gentoo-commits@lists.gentoo.org>; Thu, 5 Apr 2018 22:16:14 +0000 (UTC)
Received: from localhost.localdomain (localhost [IPv6:::1])
by oystercatcher.gentoo.org (Postfix) with ESMTP id 0662C274
for <gentoo-commits@lists.gentoo.org>; Thu, 5 Apr 2018 22:16:13 +0000 (UTC)
From: "Sergei Trofimovich" <slyfox@gentoo.org>
To: gentoo-commits@lists.gentoo.org
Content-Transfer-Encoding: 8bit
Content-type: text/plain; charset=UTF-8
Reply-To: gentoo-dev@lists.gentoo.org, "Sergei Trofimovich" <slyfox@gentoo.org>
Message-ID: <1522966526.7ea765f7ff98a0f07ee1f21cc83d2b4db81c02b2.slyfox@gentoo>
Subject: [gentoo-commits] proj/crossdev:master commit in: /
X-VCS-Repository: proj/crossdev
X-VCS-Files: README
X-VCS-Directories: /
X-VCS-Committer: slyfox
X-VCS-Committer-Name: Sergei Trofimovich
X-VCS-Revision: 7ea765f7ff98a0f07ee1f21cc83d2b4db81c02b2
X-VCS-Branch: master
Date: Thu, 5 Apr 2018 22:16:13 +0000 (UTC)
Precedence: bulk
List-Post: <mailto:gentoo-commits@lists.gentoo.org>
List-Help: <mailto:gentoo-commits+help@lists.gentoo.org>
List-Unsubscribe: <mailto:gentoo-commits+unsubscribe@lists.gentoo.org>
List-Subscribe: <mailto:gentoo-commits+subscribe@lists.gentoo.org>
List-Id: Gentoo Linux mail <gentoo-commits.gentoo.org>
X-BeenThere: gentoo-commits@lists.gentoo.org
X-Archives-Salt: 3169dc86-6562-43bb-9441-fbfea4029efe
X-Archives-Hash: 30a21e0fe568de1aff6a70fbddab9742
commit: 7ea765f7ff98a0f07ee1f21cc83d2b4db81c02b2
Author: Sergei Trofimovich <slyfox <AT> gentoo <DOT> org>
AuthorDate: Thu Apr 5 22:15:26 2018 +0000
Commit: Sergei Trofimovich <slyfox <AT> gentoo <DOT> org>
CommitDate: Thu Apr 5 22:15:26 2018 +0000
URL: https://gitweb.gentoo.org/proj/crossdev.git/commit/?id=7ea765f7
README: basic description of what crossdev
Signed-off-by: Sergei Trofimovich <slyfox <AT> gentoo.org>
README | 281 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 281 insertions(+)
diff --git a/README b/README
new file mode 100644
index 0000000..4924258
--- /dev/null
+++ b/README
@@ -0,0 +1,281 @@
+What is crossdev
+----------------
+
+crossdev is a cross-compiler environment generator for Gentoo.
+
+It is useful for various purposes:
+
+- build cross-compiler toolchain for an operating system
+- build cross-compiler toolchain for embedded targets (bare metal)
+- cross-compile whole Gentoo on a new (or existing) target
+- cross-compile your favourite tool for every target out there
+ just to make sure it still compiles and works. Countless bugs
+ were found fixed like that :)
+
+Crossdev nano HOWTO
+-------------------
+
+So you want to cross-compile a Gentoo package (say busybox):
+
+# crossdev -t s390x-unknown-linux-gnu
+# (optional) ARCH=s390 PORTAGE_CONFIGROOT=/usr/s390x-unknown-linux-gnu eselect profile set default/linux/s390/17.0/s390x
+# USE=static s390x-unknown-linux-gnu emerge -v1 busybox
+# file /usr/s390x-unknown-linux-gnu/bin/busybox
+/usr/s390x-unknown-linux-gnu/bin/busybox: ELF 64-bit MSB executable, IBM S/390, version 1 (GNU/Linux), statically linked, for GNU/Linux 3.2.0, stripped
+
+Done!
+
+You can even use qemu-user to run this binary (or even chroot
+to /usr/s390x-unknown-linux-gnu!
+
+https://wiki.gentoo.org/wiki/Crossdev_qemu-static-user-chroot
+
+Supported platforms
+-------------------
+
+Cross-compilation is fairly well supported to linux targets.
+Windows is not too broken either.
+
+But prepare for rough corners. This doc will try to help you
+understand what crossdev does and does not do.
+
+A few examples of targets that worked today (produce running
+executables or kernels if applies):
+
+ aarch64-unknown-linux-gnu-7.3.0
+ alpha-unknown-linux-gnu-7.3.0
+ arm-none-eabi-7.3.0
+ armv5tel-softfloat-linux-gnueabi-7.3.0
+ armv7a-hardfloat-linux-gnueabi-7.3.0
+ armv7a-unknown-linux-gnueabi-7.3.0
+ avr-7.3.0
+ hppa-unknown-linux-gnu-7.3.0
+ hppa2.0-unknown-linux-gnu-7.3.0
+ hppa64-unknown-linux-gnu-7.3.0
+ i686-pc-gnu-7.3.0
+ i686-w64-mingw32-7.3.0
+ ia64-unknown-linux-gnu-7.3.0
+ m68k-unknown-linux-gnu-7.3.0
+ mips-unknown-linux-gnu-7.3.0
+ mips64-unknown-linux-gnu-7.3.0
+ mips64el-unknown-linux-gnu-7.3.0
+ mipsel-unknown-linux-gnu-7.3.0
+ mmix-7.3.0
+ nios2-unknown-linux-gnu-7.3.0
+ powerpc-unknown-linux-gnu-7.3.0
+ powerpc64-unknown-linux-gnu-7.3.0
+ powerpc64le-unknown-linux-gnu-7.3.0
+ s390-unknown-linux-gnu-7.3.0
+ s390x-unknown-linux-gnu-7.3.0
+ sh4-unknown-linux-gnu-7.3.0
+ sparc-unknown-linux-gnu-7.3.0
+ sparc64-unknown-linux-gnu-7.3.0
+ x86_64-HEAD-linux-gnu-7.3.0
+ x86_64-UNREG-linux-gnu-7.3.0
+ x86_64-pc-linux-gnu-7.3.0
+ x86_64-w64-mingw32-7.3.0
+
+A few more things are likely to Just Work
+and many more can be made work with a litle touch.
+
+How crossdev works (high-level overview)
+----------------------------------------
+
+crossdev is a tiny shell wrapper around emerge. Wrapper overrides
+a few variabled to aim emerge to another target.
+
+Crossdev leverages following features of portage (and ::gentoo ebulds):
+
+- ability to override ROOT=/usr/<target> to install cross-compiled
+ packages into a new root on a filesystem to avoid cluttering host.
+
+- ability to override PORTAGE_CONFIGROOT=/usr/<target> to untangle
+ from host's portage configuration.
+
+ Namely crossdev populates /usr/<target>/etc/portage/ with defaults
+ suitable for cross-compiling (ARCH, KERNEL, ELIBC variables and so on).
+ You can change all of it.
+
+- set CBUILD/CHOST/CTARGET variables accordingly to force build
+ system into cross-compiling mode. For autotools-based system
+ it means running the following configure script:
+ ./configure --build=${CBUILD} --host=${CHOST} --target=${CTARGET}
+
+If toolchains were simple programs crossdev would be a one-liner script:
+
+ ARCH=... \
+ CBUILD=... \
+ CHOST=... \
+ CTARGET=... \
+ ROOT=... \
+ emerge "$@"
+
+Unfortunately todays' toolchains have loops in their build-time dependencies:
+
+- cross-compiler itself normally needs a libc built for <target> because
+ libc defines various aspects of userland ABI and features provided.
+- and libc for <target> needs a cross-comiler because it's written in C
+
+That's where crossdev comes in useful. It unties this vicious compiler<->libc
+circle by carefully rinning the following emerge commands (assume s390x-linux
+example).
+
+Here is what crossdev actually does:
+
+1. create an overlay with new ebuilds (symlinks to existing ebuilds)
+2. build cross-binutils
+ $ emerge cross-s390x-unknown-linux-gnu/binutils
+3. Install minimal set of system headers (kernel and libc)
+ $ USE="headers-only" emerge cross-s390x-unknown-linux-gnu/linux-headers
+ $ USE="headers-only" emerge cross-s390x-unknown-linux-gnu/glibc
+4. Build minimal GCC without libc support (not able link final executables yet)
+ $ USE="-*" emerge cross-s390x-unknown-linux-gnu/gcc
+5. Build complete libc
+ $ USE="-headers-only" emerge cross-s390x-unknown-linux-gnu/linux-headers
+ $ USE="-headers-only" emerge cross-s390x-unknown-linux-gnu/glibc
+6. Build full GCC (able to link final binaries and can do c++)
+ $ USE="" emerge cross-s390x-unknown-linux-gnu/gcc
+
+Done!
+
+How crossdev works (more details)
+---------------------------------
+
+This section contains more details on what actually happens.
+Here we expand on each step briefly outlined in previous section:
+
+1. create an overlay with new ebuilds (symlinks to existing ebuilds)
+ <skipping actual commands>. After this step the outcomes are:
+
+ - overlay layout is formed:
+
+ $ ls -l cross-overlay/cross-s390x-unknown-linux-gnu
+ binutils -> /gentoo-ebuilds/gentoo/sys-devel/binutils
+ gcc -> /gentoo-ebuilds/gentoo/sys-devel/gcc
+ glibc -> /gentoo-ebuilds/gentoo/sys-libs/glibc
+ linux-headers -> /gentoo-ebuilds/gentoo/sys-kernel/linux-headers
+
+ - /usr/cross-s390x-unknown-linux-gnu (aka $SYSROOT) layout is set:
+
+ $ ls -l /usr/s390x-unknown-linux-gnu/etc/portage/
+ make.conf
+ make.profile -> /gentoo-ebuilds/gentoo/profiles/embedded
+ profile/
+
+ Here we override ARCH, LIBC, KERNEL, CBUILD, CHOST, CTARGET and a
+ few other variables.
+
+
+ - a few convenience wrappers are created:
+
+ /usr/bin/s390x-unknown-linux-gnu-emerge -> cross-emerge
+ /usr/bin/s390x-unknown-linux-gnu-pkg-config -> cross-pkg-config
+ /usr/bin/s390x-unknown-linux-gnu-fix-root -> cross-fix-root
+
+ This way we share ebuild code and still can install cross-compilers
+ independently. Each with it's owv version of libc.
+
+2. build cross-binutils
+ $ emerge cross-s390x-unknown-linux-gnu/binutils
+
+ This way we can install the same version of binutils aiming different
+ targets. As a result we get tools like:
+ s390x-unknown-linux-gnu-ar (static library archiver)
+ s390x-unknown-linux-gnu-as (assembler)
+ s390x-unknown-linux-gnu-ld (linker)
+ ... <many others>
+
+ Nothing special here.
+
+3. Install minimal set of system headers (kernel and libc)
+ $ USE="headers-only" emerge cross-s390x-unknown-linux-gnu/linux-headers
+ $ USE="headers-only" emerge cross-s390x-unknown-linux-gnu/glibc
+
+ As we don't have cross-compiler yet we just copy a bunch on
+ header files into
+ /usr/s390x-unknown-linux-gnu/usr/include
+ and setup symlinks like:
+ /usr/s390x-unknown-linux-gnu/sys-include -> usr/include
+ to make cross-gcc happy.
+
+ These symlinks are target-dependent. A few unusual examples:
+
+ - windows (mingw): /usr/x86_64-w64-mingw32/mingw -> usr
+ - hurd: /usr/i686-pc-gnu/include -> usr/include
+ - DOS (didn't try yet): /usr/i686-pc-gnu/dev/env/DJDIR/include -> ../../../usr/include
+
+ Side note: we could have omited symlink creation completely
+ and build gcc with parameter:
+ --with-native-system-header-dir=${EPREFIX}/usr/include
+ That way ${SYSROOT} would be even more like normal root.
+ Worth a try this one line :)
+
+4. Build minimal GCC without libc support (not able link final executables yet)
+ $ USE="-*" emerge cross-s390x-unknown-linux-gnu/gcc
+
+ Here gcc uses headers from step [3.] to find out what
+ target libc can do:
+
+ - POSIX support
+ - trigonometry functions
+ - threading
+ - vital constants
+
+ As a result we only get C code generator. No knowledge
+ of how to link executables or shared libraries as those
+ require bits of libc.
+
+ For tiniest targets (bare-metal) this can be a final step to get basic toolchain.
+
+5. Build complete libc
+ $ USE="-headers-only" emerge cross-s390x-unknown-linux-gnu/linux-headers
+ $ USE="-headers-only" emerge cross-s390x-unknown-linux-gnu/glibc
+
+ Here we rebuild full libc against system headers. As a result we get C startup
+ files and can link full programs!
+
+6. Build full GCC (able to link final binaries and can do c++)
+ $ USE="" emerge cross-s390x-unknown-linux-gnu/gcc
+
+ Here we get full c++ support, various default flags enabled (pie, sanitizers,
+ stack protectors and many other things).
+
+ This thing is ready for large-scale operations.
+
+Various notes (AKA dirty little tricks)
+---------------------------------------
+
+- config.site
+
+ Some ./configure scripts rely on runtime feature testing. We would
+ still like to enable things even in cross-environment.
+
+ crossdev installs /usr/share/config.site with a bunch of cache
+ variables preset for targets. It might be a nice place to drop
+ more things into. Or a source of all your cross-compilation
+ problems :)
+
+- eclass inheritance
+
+ To find out various things about target crossdev loads multilib.eclass
+ and tries to find out default ABI supported by the target.
+
+- crossdev is just a tiny shell script around emerge :)
+
+ It's full source code of comparable to the size of this README.
+
+- USE=headers-only
+
+ Many toolchain ebuilds (mostly libcs and kernel headers) are aware
+ of headers-only install specifically for crossdev and similar tools
+ to be able to build cross-toolchains.
+
+- How to test crossdev layout generation:
+
+ $ mkdir -p foo
+ $ PORTAGE_CONFIGROOT=$(pwd)/foo EPREFIX=$(pwd)/foo ./crossdev -t mmix -P -p
+
+ This needs some local patching. TODO: fix it to Just Work (perhaps with
+ additional --test options).
+
+Happy cross-compiling!