[gentoo-dev] PSA: switching default tmpfiles virtual provider

[gentoo-dev] PSA: switching default tmpfiles virtual provider

[Georgy Yakovlev]

[-- Attachment #1: Type: text/plain, Size: 1509 bytes --]

Hi,

In case you don't know, opentmpfiles has an open CVE
CVE-2017-18925: root privilege escalation by symlink attack
https://github.com/OpenRC/opentmpfiles/issues/4
It has been an issue for quite a while, reported 3 years ago,
and not much changed since.
Also it lacks any sort of testing, and master branch is in a non-working
state at time of writing, latest version is masked.[0]

Due to nature of opentmpfiles (it's a POSIX sh script),
it may be impossible to fix symlink handling and TOCTOU races.
As a consequence I'll be switching default tmpfiles
provider to sys-apps/systemd-tmpfiles by the end of the week by updating
virtual/tmpfiles ebuild.

pros of systemd-tmpfiles:
0) Secure.
1) Reference implementation.
2) Supports all features, because ^.
3) Has working tests.
4) Has millions of users as part of systemd.
5) upstream supports standalone usecase/build our ebuild uses. [1][2]
6) drop-in replacement, just emerge and forget.

systemd-tmpfiles does not depend on any systemd-isms, does not need dbus,
and is just a drop-in replacement, the only step needed is to emerge the
package.
it's a simple single binary + manpage, binary links to libacl and couple other
system libs.

existing installations will not be affected, but openrc users are welcome to
opt-in by running 'emerge --oneshot systemd-tmpfiles'

[0] https://bugs.gentoo.org/751739
[1] https://github.com/systemd/systemd/pull/16061
[2] https://github.com/systemd/systemd/pull/16061/commits/db64ba81c62afa0e0d3e95c4a3e1ec3dd9a471a4

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 902 bytes --]

Re: [gentoo-dev] PSA: switching default tmpfiles virtual provider

[Piotr Karbowski]

Hi,

On 25/11/2020 22.57, Georgy Yakovlev wrote:
> systemd-tmpfiles does not depend on any systemd-isms, does not need dbus,
> and is just a drop-in replacement, the only step needed is to emerge the
> package.
> it's a simple single binary + manpage, binary links to libacl and couple other
> system libs.

Can confirm that systemd-tmpfiles works fine on OpenRC systems. Been
using it since end of October.

Two things that are different in terms of interface to opentmpfiles is
that systemd-tmpfiles does not have --dry-run runtime option, and it
will complain if any /usr/lib/tmpfiles.d/*.conf uses /var/run instead of
/run, but that's just an warning.

Regardless, it's just a drop-in replacement, have not noticed any issues.

-- Piotr.

Re: [gentoo-dev] PSA: switching default tmpfiles virtual provider

[Thomas Deutschmann]

[-- Attachment #1.1: Type: text/plain, Size: 3106 bytes --]

Hi,

I don't have any objections regarding the change of the default tmpfiles 
provider but I would like to classify the vulnerability:

On 2020-11-25 22:57, Georgy Yakovlev wrote:
> In case you don't know, opentmpfiles has an open CVE CVE-2017-18925:
> root privilege escalation by symlink attack 
> https://github.com/OpenRC/opentmpfiles/issues/4 It has been an issue
> for quite a while, reported 3 years ago, and not much changed since.

Don't get scared by 'root privilege escalation': *Any* problem in *any* 
tmpfiles provider will *always* allow for root privilege escalation 
because this service is run by root early at boot.

In theory you could create a user for this service but you would need 
CAP_DAC_OVERRIDE privileges which would again allow for root privilege 
escalation.

Regarding CVE-2017-18925 itself: First you have to understand that 
anyone can request a CVE and that it isn't CNA's job to verify your 
report. That's it, having a CVE doesn't mean that a problem was 
confirmed. A CVE is just an identifier which should allow anyone who 
want to talk about the same problem to do that. For example when we file 
bug 123 in bugs.gentoo.org and Fedora would have the same package and 
experience the same issue they would file bug 456 in their bug tracker 
-- the goal of a CVE is just to connect information regarding the same 
issue -- in this example, the CVE would get references to Gentoo's bug 
123 and Fedora's bug 456.

The bug itself is about a race condition. This race condition is real.

However, the impact is questionable: tmpfiles service will only process 
files from

   /etc/tmpfiles.d/*.conf
   /run/tmpfiles.d/*.conf
   /usr/lib/tmpfiles.d/*.conf

Only root is allowed to write to these directories. In other words: To 
exploit this, a malicious local user (or a remote attacker who already 
gained user access) would have to trick root into creating specially 
crafted tmpfiles config allowing for race conditions first (according to 
the 10 immutable laws of security, if this is already possible, you are 
already lost).

If root doesn't install any tmpfiles config which will create such a 
race condition and if package maintainer will take care that their 
packages won't do the same, you are fine.

Rule of thumb: Just make sure that you only create top level 
directories. If something already exists, error out. Because whenever 
you try to work in a directory where any other user is able to write to 
at the same time, you are always vulnerable to such a race condition 
(that's why you should have a second level for actual user data and keep 
first level for ACL handling -- the service user must only be allowed to 
pass through this directory).


PS: Just to avoid any misunderstandings: OpenTmpfiles should of course 
try to fix/avoid this problem if possible. Security is a layered process 
(like an onion) and having multiple safe-guards is always a good thing.


-- 
Regards,
Thomas Deutschmann / Gentoo Security Team
fpr: C4DD 695F A713 8F24 2AA1 5638 5849 7EE5 1D5D 74A5


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 495 bytes --]

Re: [gentoo-dev] PSA: switching default tmpfiles virtual provider

[Michael Orlitzky]

On 11/26/20 10:07 AM, Thomas Deutschmann wrote:
> 
> Only root is allowed to write to these directories. In other words: To
> exploit this, a malicious local user (or a remote attacker who already
> gained user access) would have to trick root into creating specially
> crafted tmpfiles config allowing for race conditions first (according to
> the 10 immutable laws of security, if this is already possible, you are
> already lost).

Most of these security issues were fixed in systemd-tmpfiles years ago,
and you can easily find upstream tmpfiles.d entries that contain e.g.
"Z" entries. In that case, the upstream file is not in error, and root
doesn't have to be actively tricked into installing anything -- it will
just happen.

Opentmpfiles literally cannot fix this. There is no POSIX API to safely
handle hardlinks. At best it can be reduced to the same race condition
we have in checkpath, but the entire project would have to be rewritten
in C to accomplish even that.

Corollary: the tmpfiles.d specification can only be implemented (safely)
on Linux after all.

Re: [gentoo-dev] PSA: switching default tmpfiles virtual provider

[Peter Stuge]

Georgy Yakovlev wrote:
> I'll be switching default tmpfiles provider to sys-apps/systemd-tmpfiles
> by the end of the week by updating virtual/tmpfiles ebuild.

Michael Orlitzky wrote:
> Corollary: the tmpfiles.d specification can only be implemented (safely)
> on Linux after all.

So should virtual/tmpfiles differentiate based on system?


//Peter

Re: [gentoo-dev] PSA: switching default tmpfiles virtual provider

[Sam James]

[-- Attachment #1: Type: text/plain, Size: 412 bytes --]


> On 26 Nov 2020, at 22:37, Peter Stuge <peter@stuge.se> wrote:
> Michael Orlitzky wrote:
>> Corollary: the tmpfiles.d specification can only be implemented (safely)
>> on Linux after all.
> 
> So should virtual/tmpfiles differentiate based on system?
> 

It won’t be keyworded where it’s not available so Portage will skip over it. systemd only supports certain platforms.

> 
> //Peter
> 


[-- Attachment #2: Message signed with OpenPGP --]
[-- Type: application/pgp-signature, Size: 358 bytes --]

Re: [gentoo-dev] PSA: switching default tmpfiles virtual provider

[Michael Orlitzky]

On 11/26/20 5:37 PM, Peter Stuge wrote:
> Georgy Yakovlev wrote:
>> I'll be switching default tmpfiles provider to sys-apps/systemd-tmpfiles
>> by the end of the week by updating virtual/tmpfiles ebuild.
> 
> Michael Orlitzky wrote:
>> Corollary: the tmpfiles.d specification can only be implemented (safely)
>> on Linux after all.
> 
> So should virtual/tmpfiles differentiate based on system?
> 

There's no scenario where opentmpfiles is preferable.

systemd-tmpfiles with the fs.protected_hardlinks=1 sysctl is secure on
Linux. On other kernels, you're out of luck -- none of the options are
secure. Securing the service manager on other kernels would require
dropping tmpfiles entirely, and major changes to OpenRC.

Re: [gentoo-dev] PSA: switching default tmpfiles virtual provider

[Thomas Deutschmann]

[-- Attachment #1.1: Type: text/plain, Size: 2092 bytes --]

On 2020-11-26 21:36, Michael Orlitzky wrote:
> Most of these security issues were fixed in systemd-tmpfiles years ago,
> and you can easily find upstream tmpfiles.d entries that contain e.g.
> "Z" entries. In that case, the upstream file is not in error, and root
> doesn't have to be actively tricked into installing anything -- it will
> just happen.

I disagree here: Packages installing tmpfiles configs requiring 
recursive chown on each boot are doing something wrong from  my P.O.V. 
like you can never safely do that (you can only take precaution like not 
following symlinks but in this case you don't do what you were asked to 
do so you shouldn't return 'Yup, I chowned everything like you asked me 
to do').


> Opentmpfiles literally cannot fix this. There is no POSIX API to safely
> handle hardlinks. At best it can be reduced to the same race condition
> we have in checkpath, but the entire project would have to be rewritten
> in C to accomplish even that.

Note that hardlinks aren't even fixed for systemd's tmpfiles provider. 
It will always rely on fs.protected_hardlinks for example. And checking 
for hardlinks like happened to address CVE-2017-18078 will create 
another TOCTOU race. Where is the follow-up report for this?

In short: As long as it is possible for attacker to write to directory 
you are working on you can never do mentioned things in a safe way. You 
first have to revoke access for everyone except you and then you can 
start checking file per file... but *no* implementation is doing 
something like that.

And keep in mind: We are talking about an attack vector where we already 
assume someone successfully compromised an application and can now do 
everything the application user can do for which we do the work in 
tmpfiles config. Saying that systemd's implementation is more secure 
than OpenTmpfiles' implementation when you are still able to escalate 
privileges is very misleading.


-- 
Regards,
Thomas Deutschmann / Gentoo Linux Developer
C4DD 695F A713 8F24 2AA1 5638 5849 7EE5 1D5D 74A5


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 495 bytes --]

Re: [gentoo-dev] PSA: switching default tmpfiles virtual provider

[David Seifert]

On Thu, 2020-11-26 at 17:45 -0500, Michael Orlitzky wrote:
> On 11/26/20 5:37 PM, Peter Stuge wrote:
> > Georgy Yakovlev wrote:
> > > I'll be switching default tmpfiles provider to sys-apps/systemd-
> > > tmpfiles
> > > by the end of the week by updating virtual/tmpfiles ebuild.
> > 
> > Michael Orlitzky wrote:
> > > Corollary: the tmpfiles.d specification can only be implemented
> > > (safely)
> > > on Linux after all.
> > 
> > So should virtual/tmpfiles differentiate based on system?
> > 
> 
> There's no scenario where opentmpfiles is preferable.
> 
> systemd-tmpfiles with the fs.protected_hardlinks=1 sysctl is secure on
> Linux. On other kernels, you're out of luck -- none of the options are
> secure. Securing the service manager on other kernels would require
> dropping tmpfiles entirely, and major changes to OpenRC.
> 

...which is mostly a theoretical exercise, because we only support Linux
anyways.

Re: [gentoo-dev] PSA: switching default tmpfiles virtual provider

[Michael Orlitzky]

On 11/26/20 5:57 PM, Thomas Deutschmann wrote:
> 
> I disagree here: Packages installing tmpfiles configs requiring
> recursive chown on each boot are doing something wrong from  my P.O.V.

No argument there, but me thinking they're wrong doesn't stop people
from doing it.


> Note that hardlinks aren't even fixed for systemd's tmpfiles provider.
> It will always rely on fs.protected_hardlinks for example. And checking
> for hardlinks like happened to address CVE-2017-18078 will create
> another TOCTOU race. Where is the follow-up report for this?

Systemd only supports Linux, and sets fs.protected_hardlinks=1 itself.
There's not much more we can ask from them.

I normally err on the side of caution, but if someone goes out of their
way to disable a security setting, I don't consider it CVE-worthy if the
thing that setting was preventing is now exploitable.


> In short: As long as it is possible for attacker to write to directory
> you are working on you can never do mentioned things in a safe way. You
> first have to revoke access for everyone except you and then you can
> start checking file per file... but *no* implementation is doing
> something like that.

This came up in the old (late 1990s, early 2000s) LKML discussions about
the protected_* sysctls. The Right Thing To Do is to drop privileges to
the user who owns the directory if you need to do stuff in a directory
that a user owns or can write to.

To quote your earlier message:

> Rule of thumb: Just make sure that you only create top level directories.

...and then drop privileges.

Re: [gentoo-dev] PSA: switching default tmpfiles virtual provider

[Georgy Yakovlev]

[-- Attachment #1: Type: text/plain, Size: 1779 bytes --]

On 25.11.2020 13:57, Georgy Yakovlev wrote:
> Hi,
> 
> In case you don't know, opentmpfiles has an open CVE
> CVE-2017-18925: root privilege escalation by symlink attack
> https://github.com/OpenRC/opentmpfiles/issues/4
> It has been an issue for quite a while, reported 3 years ago,
> and not much changed since.
> Also it lacks any sort of testing, and master branch is in a non-working
> state at time of writing, latest version is masked.[0]
> 
> Due to nature of opentmpfiles (it's a POSIX sh script),
> it may be impossible to fix symlink handling and TOCTOU races.
> As a consequence I'll be switching default tmpfiles
> provider to sys-apps/systemd-tmpfiles by the end of the week by updating
> virtual/tmpfiles ebuild.
> 
> pros of systemd-tmpfiles:
> 0) Secure.
> 1) Reference implementation.
> 2) Supports all features, because ^.
> 3) Has working tests.
> 4) Has millions of users as part of systemd.
> 5) upstream supports standalone usecase/build our ebuild uses. [1][2]
> 6) drop-in replacement, just emerge and forget.
> 
> systemd-tmpfiles does not depend on any systemd-isms, does not need dbus,
> and is just a drop-in replacement, the only step needed is to emerge the
> package.
> it's a simple single binary + manpage, binary links to libacl and couple other
> system libs.
> 
> existing installations will not be affected, but openrc users are welcome to
> opt-in by running 'emerge --oneshot systemd-tmpfiles'
> 
> [0] https://bugs.gentoo.org/751739
> [1] https://github.com/systemd/systemd/pull/16061
> [2] https://github.com/systemd/systemd/pull/16061/commits/db64ba81c62afa0e0d3e95c4a3e1ec3dd9a471a4

This is done in
https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=ab23417927d8454c8bb1c0ae52a5cac79d140b94

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 902 bytes --]

Re: [gentoo-dev] PSA: switching default tmpfiles virtual provider

[William Hubbs]

[-- Attachment #1: Type: text/plain, Size: 1072 bytes --]

On Thu, Nov 26, 2020 at 07:55:33AM +0100, Piotr Karbowski wrote:
> Hi,
> 
> On 25/11/2020 22.57, Georgy Yakovlev wrote:
> > systemd-tmpfiles does not depend on any systemd-isms, does not need dbus,
> > and is just a drop-in replacement, the only step needed is to emerge the
> > package.
> > it's a simple single binary + manpage, binary links to libacl and couple other
> > system libs.
> 
> Can confirm that systemd-tmpfiles works fine on OpenRC systems. Been
> using it since end of October.
> 
> Two things that are different in terms of interface to opentmpfiles is
> that systemd-tmpfiles does not have --dry-run runtime option, and it
> will complain if any /usr/lib/tmpfiles.d/*.conf uses /var/run instead of
> /run, but that's just an warning.

Also, have we tested this on musl systems?

My plan is to take the tmpfiles code from systemd, like eudev and elogin
have done, and rewrite it to not use the systemd libraries so it will be
more portable.

Once this happens, I'll probably switch the provider back to
opentmpfiles.

William

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

Re: [gentoo-dev] PSA: switching default tmpfiles virtual provider

[Mike Gilbert]

On Sun, Nov 29, 2020 at 4:50 PM William Hubbs <williamh@gentoo.org> wrote:
>
> On Thu, Nov 26, 2020 at 07:55:33AM +0100, Piotr Karbowski wrote:
> > Hi,
> >
> > On 25/11/2020 22.57, Georgy Yakovlev wrote:
> > > systemd-tmpfiles does not depend on any systemd-isms, does not need dbus,
> > > and is just a drop-in replacement, the only step needed is to emerge the
> > > package.
> > > it's a simple single binary + manpage, binary links to libacl and couple other
> > > system libs.
> >
> > Can confirm that systemd-tmpfiles works fine on OpenRC systems. Been
> > using it since end of October.
> >
> > Two things that are different in terms of interface to opentmpfiles is
> > that systemd-tmpfiles does not have --dry-run runtime option, and it
> > will complain if any /usr/lib/tmpfiles.d/*.conf uses /var/run instead of
> > /run, but that's just an warning.
>
> Also, have we tested this on musl systems?
>
> My plan is to take the tmpfiles code from systemd, like eudev and elogin
> have done, and rewrite it to not use the systemd libraries so it will be
> more portable.

I think this is a bigger task than you realize. Even if you manage to
get an initial rewrite done, you would also need to keep up with
feature updates systemd releases. That hasn't worked out for eudev,
and I doubt it would work out for this yet-to-exist tmpfiles project.

Maintaining a patchset against the systemd sources is probably the
best path forward on this.