[gentoo-dev-announce] New developer features in portage: cgroup, network-sandbox, ipc-sandbox

[gentoo-dev-announce] New developer features in portage: cgroup, network-sandbox, ipc-sandbox

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 3547 bytes --]

Hello, fellow developers.

I've added a few new fancy features for Gentoo developers to portage
git. Sadly, since Zac isn't planning another release until 2.2.0 goes
stable, you need to switch to -9999 to use them. But I say to you, it's
worth the hassle.

The features are off by default since they need proper testing and can
break a lot of ebuilds. And FEATURES=network-sandbox does.

It should be noted that all of the features follow the systemd idea of
supporting Linux only and require fancy kernel features.


The following new FEATURES have been added:

1. FEATURES=cgroup

Requires: CONFIG_CGROUPS

Applies to: all src_* phases

Enables long-awaited cgroup support in portage. Each ebuild is confined
within a control group and all spawned processes are tracked. Once
the phase exits, all remaining orphans are killed.

This helps especially with multiprocessing/multibuild stuff and some
test phases that need to spawn servers. It ensures that portage does
not leave any orphans that would otherwise need to be separately
tracked and killed.

Control groups are applied to src_* phases only, since we expect that
pkg_postinst() may restart external daemons, and those could end up
being attached to the cgroup.

I doubt this could break something.


2. FEATURES=network-sandbox

Requires: CONFIG_NAMESPACES, CONFIG_NET_NS

Applies to: src_* except for src_unpack

This one uses the unshare() syscall to detach the build process from
host's network stack. This effectively means that each of the listed
phases will be able only to access a detached, 'local' loopback
interface and nothing else.

This has a few implications. First of all, ebuilds that used to access
the Internet won't be able to do that anymore. In the Python world,
this would mean that some packages will start to fail properly instead
of downloading missing dependencies. It will also break or skip all
the tests that rely on the network being available.

Secondly, this will prevent any kind of communication between host
network and ebuild, including services running on 127.0.0.1. That is,
ebuild will no longer be able to access production services running
on the host. This affects e.g. old mongodb frontend ebuilds which used
to run tests on the live database server (yep, create and delete
databases there).

Thirdly, this will prevent the daemons spawned within ebuild from being
publicly accessible. That is, if test phase spawns some kind of TCP/IP
server, even local users won't be able to connect to it (outside of
the namespace). This should improve the security.

This does not apply to pkg_* phases where networking may be needed for
some kind of IPC, and src_unpack where it is used for VCS fetching. If
we introduce separate src_fetch in a future EAPI, the exclude will
move there.

This one's going to trigger a lot of breakage in ebuilds. Therefore,
I'd appreciate if developers started using it early and fixing
the ebuilds.


3. FEATURES=ipc-sandbox

Requires: CONFIG_NAMESPACES, CONFIG_IPC_NS

Applies to: src_*

This one separates the ebuild's *nix IPC stuff from host. This includes
semaphores, shared memory etc. Similarly to network-sandbox, this could
prevent ebuilds from communicating with some production servers.

But honestly, I have no idea if anything really does it or relies on it.
I doubt this could break something but it's worth testing.


I'd really appreciate some testing and feedback. Thanks.

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 966 bytes --]