[gentoo-dev] Reviving the Sandbox project

[gentoo-dev] Reviving the Sandbox project

[Michał Górny]

Hi, everyone.

TL;DR: I'm looking for new people to resume work on sandbox, and I will
most likely branch it at v2.10 and start keep-alive work on top of that.


The state of our sandbox is not very well. Gentoo is currently using
the old v2.10 with local patches. v2.11 is p.masked for a long time
because of multiple unsolved bugs. The git repository also includes
v2.12 tag that has not even been packaged for Gentoo. From what I've
been able to establish, the bugs causing v2.11 mask are still present
in git head.

To add to this, the only person maintaining the code (vapier) has not
touched it since March, and AFAIA he's not responding to any contact
attempts from within Gentoo. Given this and the importance of sandbox to
Gentoo at the moment, I think it's reasonable to presume he's MIA
and start looking for volunteers to join the effort.

I have already added myself to the project page [1] and I'm going to try
to put some effort into improving the state of things. However, I'm not
really an expert in the high magic used in sandbox. If anyone is
interested in helping out, feel free to add yourself as well.


The above considered, I don't think I'm really going to be able to solve
the problems introduced by v2.11. If nobody has a better idea, I'm going
to branch sandbox at v2.10 and look into backporting whatever's feasible
and resuming development in hotfix mode on top of that.

In other words, don't expect any major developments. Only minor bugfixes
and whatever is necessary to keep the project semi-alive before we
deploy a suitable replacement.


By the way, if anybody is interested in a side Portage project, I've
added sys-apps/sydbox a few days ago and it's waiting for proper
integration into Portage. It's very neat but might be a bit hard to
figure out at first, so I can be of help with that.


[1]:https://wiki.gentoo.org/wiki/Project:Sandbox

-- 
Best regards,
Michał Górny

Re: [gentoo-dev] Reviving the Sandbox project

[Mart Raudsepp]

Ühel kenal päeval, N, 21.09.2017 kell 21:56, kirjutas Michał Górny:
> Hi, everyone.
> 
> TL;DR: I'm looking for new people to resume work on sandbox, and I
> will
> most likely branch it at v2.10 and start keep-alive work on top of
> that.
> 
> 
> The state of our sandbox is not very well. Gentoo is currently using
> the old v2.10 with local patches. v2.11 is p.masked for a long time
> because of multiple unsolved bugs. The git repository also includes
> v2.12 tag that has not even been packaged for Gentoo. From what I've
> been able to establish, the bugs causing v2.11 mask are still present
> in git head.
> 
> To add to this, the only person maintaining the code (vapier) has not
> touched it since March, and AFAIA he's not responding to any contact
> attempts from within Gentoo. Given this and the importance of sandbox
> to
> Gentoo at the moment, I think it's reasonable to presume he's MIA
> and start looking for volunteers to join the effort.
> 
> I have already added myself to the project page [1] and I'm going to
> try
> to put some effort into improving the state of things. However, I'm
> not
> really an expert in the high magic used in sandbox. If anyone is
> interested in helping out, feel free to add yourself as well.
> 
> 
> The above considered, I don't think I'm really going to be able to
> solve
> the problems introduced by v2.11. If nobody has a better idea, I'm
> going
> to branch sandbox at v2.10 and look into backporting whatever's
> feasible
> and resuming development in hotfix mode on top of that.

Do you have a handy list of problems with v2.11?
Perhaps all is well for Gentoo usages after the more ptrace-happy
fallbacking commit (apparently was needed by ChromeOS) is reverted if
ptrace-path problems don't get fixes?


Mart

Re: [gentoo-dev] Reviving the Sandbox project

[Michał Górny]

W dniu czw, 21.09.2017 o godzinie 23∶33 +0300, użytkownik Mart Raudsepp
napisał:
> Ühel kenal päeval, N, 21.09.2017 kell 21:56, kirjutas Michał Górny:
> > Hi, everyone.
> > 
> > TL;DR: I'm looking for new people to resume work on sandbox, and I
> > will
> > most likely branch it at v2.10 and start keep-alive work on top of
> > that.
> > 
> > 
> > The state of our sandbox is not very well. Gentoo is currently using
> > the old v2.10 with local patches. v2.11 is p.masked for a long time
> > because of multiple unsolved bugs. The git repository also includes
> > v2.12 tag that has not even been packaged for Gentoo. From what I've
> > been able to establish, the bugs causing v2.11 mask are still present
> > in git head.
> > 
> > To add to this, the only person maintaining the code (vapier) has not
> > touched it since March, and AFAIA he's not responding to any contact
> > attempts from within Gentoo. Given this and the importance of sandbox
> > to
> > Gentoo at the moment, I think it's reasonable to presume he's MIA
> > and start looking for volunteers to join the effort.
> > 
> > I have already added myself to the project page [1] and I'm going to
> > try
> > to put some effort into improving the state of things. However, I'm
> > not
> > really an expert in the high magic used in sandbox. If anyone is
> > interested in helping out, feel free to add yourself as well.
> > 
> > 
> > The above considered, I don't think I'm really going to be able to
> > solve
> > the problems introduced by v2.11. If nobody has a better idea, I'm
> > going
> > to branch sandbox at v2.10 and look into backporting whatever's
> > feasible
> > and resuming development in hotfix mode on top of that.
> 
> Do you have a handy list of problems with v2.11?
> Perhaps all is well for Gentoo usages after the more ptrace-happy
> fallbacking commit (apparently was needed by ChromeOS) is reverted if
> ptrace-path problems don't get fixes?
> 

There are two bugs listed in p.mask reason. There's a lot of bugs on
Bugzilla but I don't know which versions they affect.

-- 
Best regards,
Michał Górny

Re: [gentoo-dev] Reviving the Sandbox project

[Mart Raudsepp]

Ühel kenal päeval, N, 21.09.2017 kell 22:54, kirjutas Michał Górny:
> W dniu czw, 21.09.2017 o godzinie 23∶33 +0300, użytkownik Mart
> Raudsepp
> napisał:
> > Ühel kenal päeval, N, 21.09.2017 kell 21:56, kirjutas Michał Górny:
> > > Hi, everyone.
> > > 
> > > TL;DR: I'm looking for new people to resume work on sandbox, and
> > > I
> > > will
> > > most likely branch it at v2.10 and start keep-alive work on top
> > > of
> > > that.
> > > 
> > > 
> > > The state of our sandbox is not very well. Gentoo is currently
> > > using
> > > the old v2.10 with local patches. v2.11 is p.masked for a long
> > > time
> > > because of multiple unsolved bugs. The git repository also
> > > includes
> > > v2.12 tag that has not even been packaged for Gentoo. From what
> > > I've
> > > been able to establish, the bugs causing v2.11 mask are still
> > > present
> > > in git head.
> > > 
> > > To add to this, the only person maintaining the code (vapier) has
> > > not
> > > touched it since March, and AFAIA he's not responding to any
> > > contact
> > > attempts from within Gentoo. Given this and the importance of
> > > sandbox
> > > to
> > > Gentoo at the moment, I think it's reasonable to presume he's MIA
> > > and start looking for volunteers to join the effort.
> > > 
> > > I have already added myself to the project page [1] and I'm going
> > > to
> > > try
> > > to put some effort into improving the state of things. However,
> > > I'm
> > > not
> > > really an expert in the high magic used in sandbox. If anyone is
> > > interested in helping out, feel free to add yourself as well.
> > > 
> > > 
> > > The above considered, I don't think I'm really going to be able
> > > to
> > > solve
> > > the problems introduced by v2.11. If nobody has a better idea,
> > > I'm
> > > going
> > > to branch sandbox at v2.10 and look into backporting whatever's
> > > feasible
> > > and resuming development in hotfix mode on top of that.
> > 
> > Do you have a handy list of problems with v2.11?
> > Perhaps all is well for Gentoo usages after the more ptrace-happy
> > fallbacking commit (apparently was needed by ChromeOS) is reverted
> > if
> > ptrace-path problems don't get fixes?
> > 
> 
> There are two bugs listed in p.mask reason. There's a lot of bugs on
> Bugzilla but I don't know which versions they affect.


#580726 comes from the ptrace thing I mentioned. I identified the
commit that makes it fallback to ptrace for firefox case, while it
seemed to work fine before without fallback, and then there are issues
in that ptrace codepath that might have always been there, but they
just didn't get hit due to ptrace fallback being much rarer before
this. I believe the commit hash is mentioned on the bug.

#578582 seems to be just patrick being special and refusing to provide
any information about the bug that no-one else hits.

Maybe if we revert that more easy ptrace fallback stuff for now, the
rest in sandbox git master is fine (if my opendir fix gets applied
that's only patched in via ebuild right now)?

Mart

Re: [gentoo-dev] Reviving the Sandbox project

[Michał Górny]

W dniu pią, 22.09.2017 o godzinie 00∶07 +0300, użytkownik Mart Raudsepp
napisał:
> Ühel kenal päeval, N, 21.09.2017 kell 22:54, kirjutas Michał Górny:
> > W dniu czw, 21.09.2017 o godzinie 23∶33 +0300, użytkownik Mart
> > Raudsepp
> > napisał:
> > > Ühel kenal päeval, N, 21.09.2017 kell 21:56, kirjutas Michał Górny:
> > > > Hi, everyone.
> > > > 
> > > > TL;DR: I'm looking for new people to resume work on sandbox, and
> > > > I
> > > > will
> > > > most likely branch it at v2.10 and start keep-alive work on top
> > > > of
> > > > that.
> > > > 
> > > > 
> > > > The state of our sandbox is not very well. Gentoo is currently
> > > > using
> > > > the old v2.10 with local patches. v2.11 is p.masked for a long
> > > > time
> > > > because of multiple unsolved bugs. The git repository also
> > > > includes
> > > > v2.12 tag that has not even been packaged for Gentoo. From what
> > > > I've
> > > > been able to establish, the bugs causing v2.11 mask are still
> > > > present
> > > > in git head.
> > > > 
> > > > To add to this, the only person maintaining the code (vapier) has
> > > > not
> > > > touched it since March, and AFAIA he's not responding to any
> > > > contact
> > > > attempts from within Gentoo. Given this and the importance of
> > > > sandbox
> > > > to
> > > > Gentoo at the moment, I think it's reasonable to presume he's MIA
> > > > and start looking for volunteers to join the effort.
> > > > 
> > > > I have already added myself to the project page [1] and I'm going
> > > > to
> > > > try
> > > > to put some effort into improving the state of things. However,
> > > > I'm
> > > > not
> > > > really an expert in the high magic used in sandbox. If anyone is
> > > > interested in helping out, feel free to add yourself as well.
> > > > 
> > > > 
> > > > The above considered, I don't think I'm really going to be able
> > > > to
> > > > solve
> > > > the problems introduced by v2.11. If nobody has a better idea,
> > > > I'm
> > > > going
> > > > to branch sandbox at v2.10 and look into backporting whatever's
> > > > feasible
> > > > and resuming development in hotfix mode on top of that.
> > > 
> > > Do you have a handy list of problems with v2.11?
> > > Perhaps all is well for Gentoo usages after the more ptrace-happy
> > > fallbacking commit (apparently was needed by ChromeOS) is reverted
> > > if
> > > ptrace-path problems don't get fixes?
> > > 
> > 
> > There are two bugs listed in p.mask reason. There's a lot of bugs on
> > Bugzilla but I don't know which versions they affect.
> 
> 
> #580726 comes from the ptrace thing I mentioned. I identified the
> commit that makes it fallback to ptrace for firefox case, while it
> seemed to work fine before without fallback, and then there are issues
> in that ptrace codepath that might have always been there, but they
> just didn't get hit due to ptrace fallback being much rarer before
> this. I believe the commit hash is mentioned on the bug.
> 
> #578582 seems to be just patrick being special and refusing to provide
> any information about the bug that no-one else hits.
> 
> Maybe if we revert that more easy ptrace fallback stuff for now, the
> rest in sandbox git master is fine (if my opendir fix gets applied
> that's only patched in via ebuild right now)?
> 

Maybe it is. However, I don't feel confident releasing a completely new
version myself, and I won't be able to review the changes thouroughly
enough to become confident.

Given that sandbox is utterly broken by design, I don't really want to
put too much effort in trying to make it a little better. I'd rather put
the minimal effort required to make it not-much-worse.

-- 
Best regards,
Michał Górny

Re: [gentoo-dev] Reviving the Sandbox project

[Matt Turner]

On Thu, Sep 21, 2017 at 2:25 PM, Michał Górny <mgorny@gentoo.org> wrote:
> Given that sandbox is utterly broken by design, I don't really want to
> put too much effort in trying to make it a little better. I'd rather put
> the minimal effort required to make it not-much-worse.

You said in your initial email that you weren't an expert in its
internals, but here you say it's broken by design. Why do you think
that?

Re: [gentoo-dev] Reviving the Sandbox project

[Michał Górny]

W dniu czw, 21.09.2017 o godzinie 15∶41 -0700, użytkownik Matt Turner
napisał:
> On Thu, Sep 21, 2017 at 2:25 PM, Michał Górny <mgorny@gentoo.org> wrote:
> > Given that sandbox is utterly broken by design, I don't really want to
> > put too much effort in trying to make it a little better. I'd rather put
> > the minimal effort required to make it not-much-worse.
> 
> You said in your initial email that you weren't an expert in its
> internals, but here you say it's broken by design. Why do you think
> that?
> 

Because it uses LD_PRELOAD which is a huge hack and which causes
guaranteed issues we can't really fix. All we can do is disable it for
emacs, for compiler-rt and I'm afraid this list will grow because
overriding random library functions is never a good idea.

-- 
Best regards,
Michał Górny

Re: [gentoo-dev] Reviving the Sandbox project

[Alexis Ballier]

On Fri, 22 Sep 2017 06:07:18 +0200
Michał Górny <mgorny@gentoo.org> wrote:

> W dniu czw, 21.09.2017 o godzinie 15∶41 -0700, użytkownik Matt Turner
> napisał:
> > On Thu, Sep 21, 2017 at 2:25 PM, Michał Górny <mgorny@gentoo.org>
> > wrote:  
> > > Given that sandbox is utterly broken by design, I don't really
> > > want to put too much effort in trying to make it a little better.
> > > I'd rather put the minimal effort required to make it
> > > not-much-worse.  
> > 
> > You said in your initial email that you weren't an expert in its
> > internals, but here you say it's broken by design. Why do you think
> > that?
> >   
> 
> Because it uses LD_PRELOAD which is a huge hack and which causes
> guaranteed issues we can't really fix. All we can do is disable it for
> emacs, for compiler-rt and I'm afraid this list will grow because
> overriding random library functions is never a good idea.
> 

I think we're all ears for a better solution. There are probably much
better ways to do sandboxing these days than 15 years ago.

LD_PRELOAD does not work with static binaries. Hence the non
portable ptrace stuff. Hence bugs. Etc. The point is, that's the
best we have now.


Alexis.

Re: [gentoo-dev] Reviving the Sandbox project

[Sergei Trofimovich]

[-- Attachment #1: Type: text/plain, Size: 1803 bytes --]

On Fri, 22 Sep 2017 12:57:21 +0200
Alexis Ballier <aballier@gentoo.org> wrote:

> On Fri, 22 Sep 2017 06:07:18 +0200
> Michał Górny <mgorny@gentoo.org> wrote:
> 
> > W dniu czw, 21.09.2017 o godzinie 15∶41 -0700, użytkownik Matt Turner
> > napisał:  
> > > On Thu, Sep 21, 2017 at 2:25 PM, Michał Górny <mgorny@gentoo.org>
> > > wrote:    
> > > > Given that sandbox is utterly broken by design, I don't really
> > > > want to put too much effort in trying to make it a little better.
> > > > I'd rather put the minimal effort required to make it
> > > > not-much-worse.    
> > > 
> > > You said in your initial email that you weren't an expert in its
> > > internals, but here you say it's broken by design. Why do you think
> > > that?
> > >     
> > 
> > Because it uses LD_PRELOAD which is a huge hack and which causes
> > guaranteed issues we can't really fix. All we can do is disable it for
> > emacs, for compiler-rt and I'm afraid this list will grow because
> > overriding random library functions is never a good idea.
> >   
> 
> I think we're all ears for a better solution. There are probably much
> better ways to do sandboxing these days than 15 years ago.
> 
> LD_PRELOAD does not work with static binaries. Hence the non
> portable ptrace stuff. Hence bugs. Etc. The point is, that's the
> best we have now.

Some other distros try harder to isolate build environment either
through chroot and/or private mount/user/network namespace that
contains only explicitly specified files in build environment.

That would require more cooperation from package manager to fetch
list of all visible depends.

Don't know if drop-in relacement could be implemented for sandbox
that way. I like clear sandbox error reporting.

-- 

  Sergei

[-- Attachment #2: Цифровая подпись OpenPGP --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

Re: [gentoo-dev] Reviving the Sandbox project

[Alexis Ballier]

On Fri, 22 Sep 2017 12:38:54 +0100
Sergei Trofimovich <slyfox@gentoo.org> wrote:

> On Fri, 22 Sep 2017 12:57:21 +0200
> Alexis Ballier <aballier@gentoo.org> wrote:
> 
> > On Fri, 22 Sep 2017 06:07:18 +0200
> > Michał Górny <mgorny@gentoo.org> wrote:
> >   
> > > W dniu czw, 21.09.2017 o godzinie 15∶41 -0700, użytkownik Matt
> > > Turner napisał:    
> > > > On Thu, Sep 21, 2017 at 2:25 PM, Michał Górny
> > > > <mgorny@gentoo.org> wrote:      
> > > > > Given that sandbox is utterly broken by design, I don't really
> > > > > want to put too much effort in trying to make it a little
> > > > > better. I'd rather put the minimal effort required to make it
> > > > > not-much-worse.      
> > > > 
> > > > You said in your initial email that you weren't an expert in its
> > > > internals, but here you say it's broken by design. Why do you
> > > > think that?
> > > >       
> > > 
> > > Because it uses LD_PRELOAD which is a huge hack and which causes
> > > guaranteed issues we can't really fix. All we can do is disable
> > > it for emacs, for compiler-rt and I'm afraid this list will grow
> > > because overriding random library functions is never a good idea.
> > >     
> > 
> > I think we're all ears for a better solution. There are probably
> > much better ways to do sandboxing these days than 15 years ago.
> > 
> > LD_PRELOAD does not work with static binaries. Hence the non
> > portable ptrace stuff. Hence bugs. Etc. The point is, that's the
> > best we have now.  
> 
> Some other distros try harder to isolate build environment either
> through chroot and/or private mount/user/network namespace that
> contains only explicitly specified files in build environment.
> 
> That would require more cooperation from package manager to fetch
> list of all visible depends.
> 
> Don't know if drop-in relacement could be implemented for sandbox
> that way. I like clear sandbox error reporting.


We definitely do need a kind of drop-in replacement since PMS
mandates some parts of the sandbox API (addwrite/addpredict & co for
instance)

Re: [gentoo-dev] Reviving the Sandbox project

[Rich Freeman]

On Fri, Sep 22, 2017 at 7:38 AM, Sergei Trofimovich <slyfox@gentoo.org> wrote:
>
> Some other distros try harder to isolate build environment either
> through chroot and/or private mount/user/network namespace that
> contains only explicitly specified files in build environment.
>
> That would require more cooperation from package manager to fetch
> list of all visible depends.
>

I definitely think this is something that would be VERY useful to
have, because it makes build-time dependency issues almost impossible.

However, you don't need that complete solution just to have a sandbox.
You could simply create a container with the entire contents of the
main filesystem, but read-only, with the exception of the build area.
That would replicate the functionality of the current sandbox and
would be easier than building out just the files specified in the
dependencies.

The main issue I see with making it dependency aware is performance.
You would need to walk all the dependencies and their run-time
dependencies, and the system set, and then individually bind-mount the
files that belong to them read-only into the container.  That isn't
necessarily difficult, but I imagine that it would be slow.  Walking
the dependencies obviously already happens during resolution so that
could probably be cached.  You could also cache the list of files for
the system set, and you could even maintain a snapshot of these that
is used as the base of the container (somebody would need to work out
the savings of doing this vs the cost of updating it every time a
system set package changes).

However, the main thing I wanted to point out is that you don't need a
dependency-aware solution to just replace the existing sandbox.

> I like clear sandbox error reporting.

As far as error reporting goes, you would get kernel-level errors like
attempting to write to a read-only bind mount, or trying to read a
file that doesn't exist.  To a portage dev it should be pretty obvious
what is going on.  You could add canned text to the portage error
output at the end.  I'm not sure how visible the problems would be to
portage except to the degree that the build system makes them visible
- it would just see make terminate with a non-zero return.

I would think that containers would be almost completely bug-free
though.  The only thing I could see as an issue is some build system
that relies on IPC with the host, network access, etc.  Namespaces
themselves are very robust, and the kernel already looks at every
process as belonging to a set of namespaces even in the default case
where you only have one set of namespaces on the system, so they don't
involve different code paths/etc.

-- 
Rich

Re: [gentoo-dev] Reviving the Sandbox project

[James McMechan]

On Fri, Sep 22, 2017 at 5:27 AM, Rich Freeman <rich0@gentoo.org> wrote: 
>On Fri, Sep 22, 2017 at 7:38 AM, Sergei Trofimovich <slyfox@gentoo.org> wrote:
>>
>> Some other distros try harder to isolate build environment either
>> through chroot and/or private mount/user/network namespace that
>> contains only explicitly specified files in build environment.
>>
>> That would require more cooperation from package manager to fetch
>> list of all visible depends.
>>
>
>I definitely think this is something that would be VERY useful to
>have, because it makes build-time dependency issues almost impossible.
>
>However, you don't need that complete solution just to have a sandbox.
>You could simply create a container with the entire contents of the
>main filesystem, but read-only, with the exception of the build area.
>That would replicate the functionality of the current sandbox and
>would be easier than building out just the files specified in the
>dependencies.
>
>The main issue I see with making it dependency aware is performance.
>You would need to walk all the dependencies and their run-time
>dependencies, and the system set, and then individually bind-mount the
>files that belong to them read-only into the container.  That isn't
>necessarily difficult, but I imagine that it would be slow.  Walking
>the dependencies obviously already happens during resolution so that
>could probably be cached.  You could also cache the list of files for
>the system set, and you could even maintain a snapshot of these that
>is used as the base of the container (somebody would need to work out
>the savings of doing this vs the cost of updating it every time a
>system set package changes).
>
>However, the main thing I wanted to point out is that you don't need a
>dependency-aware solution to just replace the existing sandbox.
>
>> I like clear sandbox error reporting.
>
>As far as error reporting goes, you would get kernel-level errors like
>attempting to write to a read-only bind mount, or trying to read a
>file that doesn't exist.  To a portage dev it should be pretty obvious
>what is going on.  You could add canned text to the portage error
>output at the end.  I'm not sure how visible the problems would be to
>portage except to the degree that the build system makes them visible
>- it would just see make terminate with a non-zero return.
>
>I would think that containers would be almost completely bug-free
>though.  The only thing I could see as an issue is some build system
>that relies on IPC with the host, network access, etc.  Namespaces
>themselves are very robust, and the kernel already looks at every
>process as belonging to a set of namespaces even in the default case
>where you only have one set of namespaces on the system, so they don't
>involve different code paths/etc.
>
>-- 
>Rich

Another possibility, could be to have the sandbox be an overlayfs
not of the build directory but of the entire system starting at "/" and stick
that into the container, only CLONE_NEWNS looks to be needed.

A container with all writes going to the upper layer of the overlay could be
safe against even #RM -RF / and other disasters, while still tracking what
is happening during the build.

Should performance be too much of a problem having a bind mount of
the build directory on top of the overlay should give normal performance to
everything that is obeying good practice.

After the build completes the directory that was mounted as upper could
be scanned to find any wayward writes that had occurred...

This would not require LD_PRELOAD or ptrace eliminating most of the
"high magic" currently used.

Just yesterday I had a lot of ptrace failures while trying something odd
with ROOT= and custom USE

Jim McMechan

Re: [gentoo-dev] Reviving the Sandbox project

[Brian Dolbec]

On Fri, 22 Sep 2017 15:06:49 +0000
James McMechan <james_mcmechan@hotmail.com> wrote:

> On Fri, Sep 22, 2017 at 5:27 AM, Rich Freeman <rich0@gentoo.org>
> wrote: 
> >On Fri, Sep 22, 2017 at 7:38 AM, Sergei Trofimovich
> ><slyfox@gentoo.org> wrote:  
> >>
> >> Some other distros try harder to isolate build environment either
> >> through chroot and/or private mount/user/network namespace that
> >> contains only explicitly specified files in build environment.
> >>
> >> That would require more cooperation from package manager to fetch
> >> list of all visible depends.
> >>  
> >
> >I definitely think this is something that would be VERY useful to
> >have, because it makes build-time dependency issues almost
> >impossible.
> >
> >However, you don't need that complete solution just to have a
> >sandbox. You could simply create a container with the entire
> >contents of the main filesystem, but read-only, with the exception
> >of the build area. That would replicate the functionality of the
> >current sandbox and would be easier than building out just the files
> >specified in the dependencies.
> >
> >The main issue I see with making it dependency aware is performance.
> >You would need to walk all the dependencies and their run-time
> >dependencies, and the system set, and then individually bind-mount
> >the files that belong to them read-only into the container.  That
> >isn't necessarily difficult, but I imagine that it would be slow.
> >Walking the dependencies obviously already happens during resolution
> >so that could probably be cached.  You could also cache the list of
> >files for the system set, and you could even maintain a snapshot of
> >these that is used as the base of the container (somebody would need
> >to work out the savings of doing this vs the cost of updating it
> >every time a system set package changes).
> >
> >However, the main thing I wanted to point out is that you don't need
> >a dependency-aware solution to just replace the existing sandbox.
> >  
> >> I like clear sandbox error reporting.  
> >
> >As far as error reporting goes, you would get kernel-level errors
> >like attempting to write to a read-only bind mount, or trying to
> >read a file that doesn't exist.  To a portage dev it should be
> >pretty obvious what is going on.  You could add canned text to the
> >portage error output at the end.  I'm not sure how visible the
> >problems would be to portage except to the degree that the build
> >system makes them visible
> >- it would just see make terminate with a non-zero return.
> >
> >I would think that containers would be almost completely bug-free
> >though.  The only thing I could see as an issue is some build system
> >that relies on IPC with the host, network access, etc.  Namespaces
> >themselves are very robust, and the kernel already looks at every
> >process as belonging to a set of namespaces even in the default case
> >where you only have one set of namespaces on the system, so they
> >don't involve different code paths/etc.
> >
> >-- 
> >Rich  
> 
> Another possibility, could be to have the sandbox be an overlayfs
> not of the build directory but of the entire system starting at "/"
> and stick that into the container, only CLONE_NEWNS looks to be
> needed.
> 
> A container with all writes going to the upper layer of the overlay
> could be safe against even #RM -RF / and other disasters, while still
> tracking what is happening during the build.
> 
> Should performance be too much of a problem having a bind mount of
> the build directory on top of the overlay should give normal
> performance to everything that is obeying good practice.
> 
> After the build completes the directory that was mounted as upper
> could be scanned to find any wayward writes that had occurred...
> 
> This would not require LD_PRELOAD or ptrace eliminating most of the
> "high magic" currently used.
> 
> Just yesterday I had a lot of ptrace failures while trying something
> odd with ROOT= and custom USE
> 
> Jim McMechan

I kinda like this idea, It looks to me to have pretty much all the
benefits of a sandbox and nearly none of the drawbacks.

Sounds like it should get more research into this idea, figure out it's
limitations and if there are any caveats.

-- 
Brian Dolbec <dolsen>

Re: [gentoo-dev] Reviving the Sandbox project

[Patrick McLean]

On 2017-09-22 10:03 AM, Brian Dolbec wrote:
> On Fri, 22 Sep 2017 15:06:49 +0000
> James McMechan <james_mcmechan@hotmail.com> wrote:
> 
>> On Fri, Sep 22, 2017 at 5:27 AM, Rich Freeman <rich0@gentoo.org>
>> wrote: 
>>> On Fri, Sep 22, 2017 at 7:38 AM, Sergei Trofimovich
>>> <slyfox@gentoo.org> wrote:  
>>>>
>>>> Some other distros try harder to isolate build environment either
>>>> through chroot and/or private mount/user/network namespace that
>>>> contains only explicitly specified files in build environment.
>>>>
>>>> That would require more cooperation from package manager to fetch
>>>> list of all visible depends.
>>>>  
>>>
>>> I definitely think this is something that would be VERY useful to
>>> have, because it makes build-time dependency issues almost
>>> impossible.
>>>
>>> However, you don't need that complete solution just to have a
>>> sandbox. You could simply create a container with the entire
>>> contents of the main filesystem, but read-only, with the exception
>>> of the build area. That would replicate the functionality of the
>>> current sandbox and would be easier than building out just the files
>>> specified in the dependencies.
>>>
>>> The main issue I see with making it dependency aware is performance.
>>> You would need to walk all the dependencies and their run-time
>>> dependencies, and the system set, and then individually bind-mount
>>> the files that belong to them read-only into the container.  That
>>> isn't necessarily difficult, but I imagine that it would be slow.
>>> Walking the dependencies obviously already happens during resolution
>>> so that could probably be cached.  You could also cache the list of
>>> files for the system set, and you could even maintain a snapshot of
>>> these that is used as the base of the container (somebody would need
>>> to work out the savings of doing this vs the cost of updating it
>>> every time a system set package changes).
>>>
>>> However, the main thing I wanted to point out is that you don't need
>>> a dependency-aware solution to just replace the existing sandbox.
>>>  
>>>> I like clear sandbox error reporting.  
>>>
>>> As far as error reporting goes, you would get kernel-level errors
>>> like attempting to write to a read-only bind mount, or trying to
>>> read a file that doesn't exist.  To a portage dev it should be
>>> pretty obvious what is going on.  You could add canned text to the
>>> portage error output at the end.  I'm not sure how visible the
>>> problems would be to portage except to the degree that the build
>>> system makes them visible
>>> - it would just see make terminate with a non-zero return.
>>>
>>> I would think that containers would be almost completely bug-free
>>> though.  The only thing I could see as an issue is some build system
>>> that relies on IPC with the host, network access, etc.  Namespaces
>>> themselves are very robust, and the kernel already looks at every
>>> process as belonging to a set of namespaces even in the default case
>>> where you only have one set of namespaces on the system, so they
>>> don't involve different code paths/etc.
>>>
>>> -- 
>>> Rich  
>>
>> Another possibility, could be to have the sandbox be an overlayfs
>> not of the build directory but of the entire system starting at "/"
>> and stick that into the container, only CLONE_NEWNS looks to be
>> needed.
>>
>> A container with all writes going to the upper layer of the overlay
>> could be safe against even #RM -RF / and other disasters, while still
>> tracking what is happening during the build.
>>
>> Should performance be too much of a problem having a bind mount of
>> the build directory on top of the overlay should give normal
>> performance to everything that is obeying good practice.
>>
>> After the build completes the directory that was mounted as upper
>> could be scanned to find any wayward writes that had occurred...
>>
>> This would not require LD_PRELOAD or ptrace eliminating most of the
>> "high magic" currently used.
>>
>> Just yesterday I had a lot of ptrace failures while trying something
>> odd with ROOT= and custom USE
>>
>> Jim McMechan
> 
> I kinda like this idea, It looks to me to have pretty much all the
> benefits of a sandbox and nearly none of the drawbacks.
> 
> Sounds like it should get more research into this idea, figure out it's
> limitations and if there are any caveats.
> 

I haven't had time as of yet to look in to it, but theoretically one
could use seccomp with bpf to make a more robust sandbox, though you
might not get as nice error reporting as we have now.

Also apparmor could theoretically be used to make a nice sandbox, but I
suspect that could come with other implications.

Re: [gentoo-dev] Reviving the Sandbox project

[Michał Górny]

W dniu pią, 22.09.2017 o godzinie 12∶57 +0200, użytkownik Alexis Ballier
napisał:
> On Fri, 22 Sep 2017 06:07:18 +0200
> Michał Górny <mgorny@gentoo.org> wrote:
> 
> > W dniu czw, 21.09.2017 o godzinie 15∶41 -0700, użytkownik Matt Turner
> > napisał:
> > > On Thu, Sep 21, 2017 at 2:25 PM, Michał Górny <mgorny@gentoo.org>
> > > wrote:  
> > > > Given that sandbox is utterly broken by design, I don't really
> > > > want to put too much effort in trying to make it a little better.
> > > > I'd rather put the minimal effort required to make it
> > > > not-much-worse.  
> > > 
> > > You said in your initial email that you weren't an expert in its
> > > internals, but here you say it's broken by design. Why do you think
> > > that?
> > >   
> > 
> > Because it uses LD_PRELOAD which is a huge hack and which causes
> > guaranteed issues we can't really fix. All we can do is disable it for
> > emacs, for compiler-rt and I'm afraid this list will grow because
> > overriding random library functions is never a good idea.
> > 
> 
> I think we're all ears for a better solution. There are probably much
> better ways to do sandboxing these days than 15 years ago.
> 
> LD_PRELOAD does not work with static binaries. Hence the non
> portable ptrace stuff. Hence bugs. Etc. The point is, that's the
> best we have now.
> 

I know of two obvious alternatives: ptrace and filesystem layer (e.g.
FUSE).

For the former, there's sydbox. I'm going to look into integrating it
into Portage when I have more time.

For the latter, I have writing one in TODO. But I'm not sure when I'll
have enough time to do work on it.

-- 
Best regards,
Michał Górny

Re: [gentoo-dev] Reviving the Sandbox project

[Alexis Ballier]

On Fri, 22 Sep 2017 17:20:23 +0200
Michał Górny <mgorny@gentoo.org> wrote:

> W dniu pią, 22.09.2017 o godzinie 12∶57 +0200, użytkownik Alexis
> Ballier napisał:
> > On Fri, 22 Sep 2017 06:07:18 +0200
> > Michał Górny <mgorny@gentoo.org> wrote:
> >   
> > > W dniu czw, 21.09.2017 o godzinie 15∶41 -0700, użytkownik Matt
> > > Turner napisał:  
> > > > On Thu, Sep 21, 2017 at 2:25 PM, Michał Górny
> > > > <mgorny@gentoo.org> wrote:    
> > > > > Given that sandbox is utterly broken by design, I don't really
> > > > > want to put too much effort in trying to make it a little
> > > > > better. I'd rather put the minimal effort required to make it
> > > > > not-much-worse.    
> > > > 
> > > > You said in your initial email that you weren't an expert in its
> > > > internals, but here you say it's broken by design. Why do you
> > > > think that?
> > > >     
> > > 
> > > Because it uses LD_PRELOAD which is a huge hack and which causes
> > > guaranteed issues we can't really fix. All we can do is disable
> > > it for emacs, for compiler-rt and I'm afraid this list will grow
> > > because overriding random library functions is never a good idea.
> > >   
> > 
> > I think we're all ears for a better solution. There are probably
> > much better ways to do sandboxing these days than 15 years ago.
> > 
> > LD_PRELOAD does not work with static binaries. Hence the non
> > portable ptrace stuff. Hence bugs. Etc. The point is, that's the
> > best we have now.
> >   
> 
> I know of two obvious alternatives: ptrace and filesystem layer (e.g.
> FUSE).
> 
> For the former, there's sydbox. I'm going to look into integrating it
> into Portage when I have more time.

From: https://github.com/alip/pinktrace/blob/master/configure.ac
case "$host_cpu" in
i[[3456]]86|pentium)
x86?64*|amd64)
ia64)
powerpc64*)
powerpc*)
arm*)
 [add support for those arches]
*)
	AC_MSG_RESULT([NO!])
	AC_MSG_ERROR([Architecture $host_cpu is not supported by
pinktrace]) ;;

sandbox keywords:
2.11-r5:0: ~alpha ~amd64 ~arm ~arm64 ~hppa ~ia64 ~m68k ~mips ~ppc
 ~ppc64 ~s390 ~sh ~sparc ~sparc-fbsd ~x86 ~x86-fbsd


Good luck adding the missing bits!


> For the latter, I have writing one in TODO. But I'm not sure when I'll
> have enough time to do work on it.

Not sure how that would work, but you'll likely need some kind of
chroot/container since you don't want to trust a random binary ran as
root to respect environment variables.

Alexis.

Re: [gentoo-dev] Reviving the Sandbox project

[Michał Górny]

W dniu pią, 22.09.2017 o godzinie 19∶15 +0200, użytkownik Alexis Ballier
napisał:
> On Fri, 22 Sep 2017 17:20:23 +0200
> Michał Górny <mgorny@gentoo.org> wrote:
> 
> > W dniu pią, 22.09.2017 o godzinie 12∶57 +0200, użytkownik Alexis
> > Ballier napisał:
> > > On Fri, 22 Sep 2017 06:07:18 +0200
> > > Michał Górny <mgorny@gentoo.org> wrote:
> > >   
> > > > W dniu czw, 21.09.2017 o godzinie 15∶41 -0700, użytkownik Matt
> > > > Turner napisał:  
> > > > > On Thu, Sep 21, 2017 at 2:25 PM, Michał Górny
> > > > > <mgorny@gentoo.org> wrote:    
> > > > > > Given that sandbox is utterly broken by design, I don't really
> > > > > > want to put too much effort in trying to make it a little
> > > > > > better. I'd rather put the minimal effort required to make it
> > > > > > not-much-worse.    
> > > > > 
> > > > > You said in your initial email that you weren't an expert in its
> > > > > internals, but here you say it's broken by design. Why do you
> > > > > think that?
> > > > >     
> > > > 
> > > > Because it uses LD_PRELOAD which is a huge hack and which causes
> > > > guaranteed issues we can't really fix. All we can do is disable
> > > > it for emacs, for compiler-rt and I'm afraid this list will grow
> > > > because overriding random library functions is never a good idea.
> > > >   
> > > 
> > > I think we're all ears for a better solution. There are probably
> > > much better ways to do sandboxing these days than 15 years ago.
> > > 
> > > LD_PRELOAD does not work with static binaries. Hence the non
> > > portable ptrace stuff. Hence bugs. Etc. The point is, that's the
> > > best we have now.
> > >   
> > 
> > I know of two obvious alternatives: ptrace and filesystem layer (e.g.
> > FUSE).
> > 
> > For the former, there's sydbox. I'm going to look into integrating it
> > into Portage when I have more time.
> 
> From: https://github.com/alip/pinktrace/blob/master/configure.ac
> case "$host_cpu" in
> i[[3456]]86|pentium)
> x86?64*|amd64)
> ia64)
> powerpc64*)
> powerpc*)
> arm*)
>  [add support for those arches]
> *)
> 	AC_MSG_RESULT([NO!])
> 	AC_MSG_ERROR([Architecture $host_cpu is not supported by
> pinktrace]) ;;
> 
> sandbox keywords:
> 2.11-r5:0: ~alpha ~amd64 ~arm ~arm64 ~hppa ~ia64 ~m68k ~mips ~ppc
>  ~ppc64 ~s390 ~sh ~sparc ~sparc-fbsd ~x86 ~x86-fbsd
> 
> 
> Good luck adding the missing bits!
> 
> 
> > For the latter, I have writing one in TODO. But I'm not sure when I'll
> > have enough time to do work on it.
> 
> Not sure how that would work, but you'll likely need some kind of
> chroot/container since you don't want to trust a random binary ran as
> root to respect environment variables.
> 

Environment variables? What for?

-- 
Best regards,
Michał Górny

Re: [gentoo-dev] Reviving the Sandbox project

[Alexis Ballier]

On Fri, 22 Sep 2017 19:39:16 +0200
Michał Górny <mgorny@gentoo.org> wrote:

> W dniu pią, 22.09.2017 o godzinie 19∶15 +0200, użytkownik Alexis
> Ballier napisał:
> > On Fri, 22 Sep 2017 17:20:23 +0200
> > Michał Górny <mgorny@gentoo.org> wrote:
> >   
> > > W dniu pią, 22.09.2017 o godzinie 12∶57 +0200, użytkownik Alexis
> > > Ballier napisał:  
> > > > On Fri, 22 Sep 2017 06:07:18 +0200
> > > > Michał Górny <mgorny@gentoo.org> wrote:
> > > >     
> > > > > W dniu czw, 21.09.2017 o godzinie 15∶41 -0700, użytkownik Matt
> > > > > Turner napisał:    
> > > > > > On Thu, Sep 21, 2017 at 2:25 PM, Michał Górny
> > > > > > <mgorny@gentoo.org> wrote:      
> > > > > > > Given that sandbox is utterly broken by design, I don't
> > > > > > > really want to put too much effort in trying to make it a
> > > > > > > little better. I'd rather put the minimal effort required
> > > > > > > to make it not-much-worse.      
> > > > > > 
> > > > > > You said in your initial email that you weren't an expert
> > > > > > in its internals, but here you say it's broken by design.
> > > > > > Why do you think that?
> > > > > >       
> > > > > 
> > > > > Because it uses LD_PRELOAD which is a huge hack and which
> > > > > causes guaranteed issues we can't really fix. All we can do
> > > > > is disable it for emacs, for compiler-rt and I'm afraid this
> > > > > list will grow because overriding random library functions is
> > > > > never a good idea. 
> > > > 
> > > > I think we're all ears for a better solution. There are probably
> > > > much better ways to do sandboxing these days than 15 years ago.
> > > > 
> > > > LD_PRELOAD does not work with static binaries. Hence the non
> > > > portable ptrace stuff. Hence bugs. Etc. The point is, that's the
> > > > best we have now.
> > > >     
> > > 
> > > I know of two obvious alternatives: ptrace and filesystem layer
> > > (e.g. FUSE).
> > > 
> > > For the former, there's sydbox. I'm going to look into
> > > integrating it into Portage when I have more time.  
> > 
> > From: https://github.com/alip/pinktrace/blob/master/configure.ac
> > case "$host_cpu" in
> > i[[3456]]86|pentium)
> > x86?64*|amd64)
> > ia64)
> > powerpc64*)
> > powerpc*)
> > arm*)
> >  [add support for those arches]
> > *)
> > 	AC_MSG_RESULT([NO!])
> > 	AC_MSG_ERROR([Architecture $host_cpu is not supported by
> > pinktrace]) ;;
> > 
> > sandbox keywords:
> > 2.11-r5:0: ~alpha ~amd64 ~arm ~arm64 ~hppa ~ia64 ~m68k ~mips ~ppc
> >  ~ppc64 ~s390 ~sh ~sparc ~sparc-fbsd ~x86 ~x86-fbsd
> > 
> > 
> > Good luck adding the missing bits!
> > 
> >   
> > > For the latter, I have writing one in TODO. But I'm not sure when
> > > I'll have enough time to do work on it.  
> > 
> > Not sure how that would work, but you'll likely need some kind of
> > chroot/container since you don't want to trust a random binary ran
> > as root to respect environment variables.
> >   
> 
> Environment variables? What for?
> 

I don't know :)
I have no idea how you would provide a virtual FS that would be the
only thing ever seen by all processes spawned.

Re: [gentoo-dev] Reviving the Sandbox project

[Michał Górny]

W dniu pią, 22.09.2017 o godzinie 20∶31 +0200, użytkownik Alexis Ballier
napisał:
> On Fri, 22 Sep 2017 19:39:16 +0200
> Michał Górny <mgorny@gentoo.org> wrote:
> 
> > W dniu pią, 22.09.2017 o godzinie 19∶15 +0200, użytkownik Alexis
> > Ballier napisał:
> > > On Fri, 22 Sep 2017 17:20:23 +0200
> > > Michał Górny <mgorny@gentoo.org> wrote:
> > >   
> > > > W dniu pią, 22.09.2017 o godzinie 12∶57 +0200, użytkownik Alexis
> > > > Ballier napisał:  
> > > > > On Fri, 22 Sep 2017 06:07:18 +0200
> > > > > Michał Górny <mgorny@gentoo.org> wrote:
> > > > >     
> > > > > > W dniu czw, 21.09.2017 o godzinie 15∶41 -0700, użytkownik Matt
> > > > > > Turner napisał:    
> > > > > > > On Thu, Sep 21, 2017 at 2:25 PM, Michał Górny
> > > > > > > <mgorny@gentoo.org> wrote:      
> > > > > > > > Given that sandbox is utterly broken by design, I don't
> > > > > > > > really want to put too much effort in trying to make it a
> > > > > > > > little better. I'd rather put the minimal effort required
> > > > > > > > to make it not-much-worse.      
> > > > > > > 
> > > > > > > You said in your initial email that you weren't an expert
> > > > > > > in its internals, but here you say it's broken by design.
> > > > > > > Why do you think that?
> > > > > > >       
> > > > > > 
> > > > > > Because it uses LD_PRELOAD which is a huge hack and which
> > > > > > causes guaranteed issues we can't really fix. All we can do
> > > > > > is disable it for emacs, for compiler-rt and I'm afraid this
> > > > > > list will grow because overriding random library functions is
> > > > > > never a good idea. 
> > > > > 
> > > > > I think we're all ears for a better solution. There are probably
> > > > > much better ways to do sandboxing these days than 15 years ago.
> > > > > 
> > > > > LD_PRELOAD does not work with static binaries. Hence the non
> > > > > portable ptrace stuff. Hence bugs. Etc. The point is, that's the
> > > > > best we have now.
> > > > >     
> > > > 
> > > > I know of two obvious alternatives: ptrace and filesystem layer
> > > > (e.g. FUSE).
> > > > 
> > > > For the former, there's sydbox. I'm going to look into
> > > > integrating it into Portage when I have more time.  
> > > 
> > > From: https://github.com/alip/pinktrace/blob/master/configure.ac
> > > case "$host_cpu" in
> > > i[[3456]]86|pentium)
> > > x86?64*|amd64)
> > > ia64)
> > > powerpc64*)
> > > powerpc*)
> > > arm*)
> > >  [add support for those arches]
> > > *)
> > > 	AC_MSG_RESULT([NO!])
> > > 	AC_MSG_ERROR([Architecture $host_cpu is not supported by
> > > pinktrace]) ;;
> > > 
> > > sandbox keywords:
> > > 2.11-r5:0: ~alpha ~amd64 ~arm ~arm64 ~hppa ~ia64 ~m68k ~mips ~ppc
> > >  ~ppc64 ~s390 ~sh ~sparc ~sparc-fbsd ~x86 ~x86-fbsd
> > > 
> > > 
> > > Good luck adding the missing bits!
> > > 
> > >   
> > > > For the latter, I have writing one in TODO. But I'm not sure when
> > > > I'll have enough time to do work on it.  
> > > 
> > > Not sure how that would work, but you'll likely need some kind of
> > > chroot/container since you don't want to trust a random binary ran
> > > as root to respect environment variables.
> > >   
> > 
> > Environment variables? What for?
> > 
> 
> I don't know :)
> I have no idea how you would provide a virtual FS that would be the
> only thing ever seen by all processes spawned.
> 

Using chroot(), obviously.

-- 
Best regards,
Michał Górny

Re: [gentoo-dev] Reviving the Sandbox project

[Patrick McLean]

On 2017-09-21 02:07 PM, Mart Raudsepp wrote:
> Ühel kenal päeval, N, 21.09.2017 kell 22:54, kirjutas Michał Górny:
>> W dniu czw, 21.09.2017 o godzinie 23∶33 +0300, użytkownik Mart
>> Raudsepp
>> napisał:
>>> Ühel kenal päeval, N, 21.09.2017 kell 21:56, kirjutas Michał Górny:
>>>> Hi, everyone.
>>>>
>>>> TL;DR: I'm looking for new people to resume work on sandbox, and
>>>> I
>>>> will
>>>> most likely branch it at v2.10 and start keep-alive work on top
>>>> of
>>>> that.
>>>>
>>>>
>>>> The state of our sandbox is not very well. Gentoo is currently
>>>> using
>>>> the old v2.10 with local patches. v2.11 is p.masked for a long
>>>> time
>>>> because of multiple unsolved bugs. The git repository also
>>>> includes
>>>> v2.12 tag that has not even been packaged for Gentoo. From what
>>>> I've
>>>> been able to establish, the bugs causing v2.11 mask are still
>>>> present
>>>> in git head.
>>>>
>>>> To add to this, the only person maintaining the code (vapier) has
>>>> not
>>>> touched it since March, and AFAIA he's not responding to any
>>>> contact
>>>> attempts from within Gentoo. Given this and the importance of
>>>> sandbox
>>>> to
>>>> Gentoo at the moment, I think it's reasonable to presume he's MIA
>>>> and start looking for volunteers to join the effort.
>>>>
>>>> I have already added myself to the project page [1] and I'm going
>>>> to
>>>> try
>>>> to put some effort into improving the state of things. However,
>>>> I'm
>>>> not
>>>> really an expert in the high magic used in sandbox. If anyone is
>>>> interested in helping out, feel free to add yourself as well.
>>>>
>>>>
>>>> The above considered, I don't think I'm really going to be able
>>>> to
>>>> solve
>>>> the problems introduced by v2.11. If nobody has a better idea,
>>>> I'm
>>>> going
>>>> to branch sandbox at v2.10 and look into backporting whatever's
>>>> feasible
>>>> and resuming development in hotfix mode on top of that.
>>>
>>> Do you have a handy list of problems with v2.11?
>>> Perhaps all is well for Gentoo usages after the more ptrace-happy
>>> fallbacking commit (apparently was needed by ChromeOS) is reverted
>>> if
>>> ptrace-path problems don't get fixes?
>>>
>>
>> There are two bugs listed in p.mask reason. There's a lot of bugs on
>> Bugzilla but I don't know which versions they affect.
> 
> 
> #580726 comes from the ptrace thing I mentioned. I identified the
> commit that makes it fallback to ptrace for firefox case, while it
> seemed to work fine before without fallback, and then there are issues
> in that ptrace codepath that might have always been there, but they
> just didn't get hit due to ptrace fallback being much rarer before
> this. I believe the commit hash is mentioned on the bug.
> 

I have been running sandbox-2.11 for quite awhile on my workstation, the
only problem I have actually encountered is the issue with Firefox. That
issue seems to have gone away in firefox-55 though, so I am not sure
that it is going to be relevant for much longer.

> #578582 seems to be just patrick being special and refusing to provide
> any information about the bug that no-one else hits.
> 
> Maybe if we revert that more easy ptrace fallback stuff for now, the
> rest in sandbox git master is fine (if my opendir fix gets applied
> that's only patched in via ebuild right now)?
> 
> Mart
> 

Re: [gentoo-dev] Reviving the Sandbox project

[R0b0t1]

On Thu, Sep 21, 2017 at 2:56 PM, Michał Górny <mgorny@gentoo.org> wrote:
> [1]:https://wiki.gentoo.org/wiki/Project:Sandbox
>

I think I understand, in principle, why a sandbox could be useful, but
would it not be more productive to follow up with projects which do
unexpected things to ask that they not do those things?

In the sense that Portage can in its entirely be isolated in various
ways (user groups, containers, virtual machines, etc) I am not sure
adding another layer is the most expedient option, especially if it is
hard to maintain.

I once saw Java developers talking about introducing changes to an
enterprise program by not modifying the source, but keeping the source
as is, and then maintaining a set of reflection-based patches that
would modify the program after it was loaded but before it was run.
This did not make sense to me, and it seems a lot like what is being
done with the sandbox.

In some cases that can make sense, I suppose. I am not a very smart
man, so I would not know the necessary burden of proof.

Respectfully,
     R0b0t1

Re: [gentoo-dev] Reviving the Sandbox project

[Michael Orlitzky]

On 09/22/2017 05:51 PM, R0b0t1 wrote:
> On Thu, Sep 21, 2017 at 2:56 PM, Michał Górny <mgorny@gentoo.org> wrote:
>> [1]:https://wiki.gentoo.org/wiki/Project:Sandbox
>>
> 
> I think I understand, in principle, why a sandbox could be useful, but
> would it not be more productive to follow up with projects which do
> unexpected things to ask that they not do those things?
> 

The sandbox isn't a security feature, it's more of a QA tool. How do you
*know* when the upstream project does something wrong? See, for example,

  https://bugs.gentoo.org/599706

The sandbox doesn't catch something, and the upstream project dropped
DESTDIR from its build system. The result? /usr/bin is now owned by the
"nagios" user. Of course the upstream build system shouldn't be making
/usr/bin owned by nagios, but it would take you a good long time to
notice it without the sandbox.

Re: [gentoo-dev] Reviving the Sandbox project

[Alec Warner]

[-- Attachment #1: Type: text/plain, Size: 2097 bytes --]

On Fri, Sep 22, 2017 at 5:51 PM, R0b0t1 <r030t1@gmail.com> wrote:

> On Thu, Sep 21, 2017 at 2:56 PM, Michał Górny <mgorny@gentoo.org> wrote:
> > [1]:https://wiki.gentoo.org/wiki/Project:Sandbox
> >
>
> I think I understand, in principle, why a sandbox could be useful, but
> would it not be more productive to follow up with projects which do
> unexpected things to ask that they not do those things?
>

So step one is figuring out what those things are. So the LD_PRELOAD
sandbox isn't designed to be a "security boundary" (its trivially
defeat-able[1]). Instead its designed to be a fairly straightforward
detector of 'anomalous' behavior. It works by intercepting file-operations
and comparing them against a whitelist.

You can't tell people do stop doing unexpected things if you don't know
their software is doing unexpected things.

[1] So defeatable in fact that ebuilds have an API to modify the boundaries
of the sandbox and even if the enforcement was stronger (e.g. via
seccomp-bpf) there is still the idea that ebuilds can rather arbitrarily
alter the sandbox boundaries...so nothing really prevents application code
from also altering the boundaries in the current design; I suspect fixing
this would be fairly tricky without some major changes.

-A


> In the sense that Portage can in its entirely be isolated in various
> ways (user groups, containers, virtual machines, etc) I am not sure
> adding another layer is the most expedient option, especially if it is
> hard to maintain.
>
> I once saw Java developers talking about introducing changes to an
> enterprise program by not modifying the source, but keeping the source
> as is, and then maintaining a set of reflection-based patches that
> would modify the program after it was loaded but before it was run.
> This did not make sense to me, and it seems a lot like what is being
> done with the sandbox.
>
> In some cases that can make sense, I suppose. I am not a very smart
> man, so I would not know the necessary burden of proof.
>
> Respectfully,
>      R0b0t1
>
>

[-- Attachment #2: Type: text/html, Size: 2834 bytes --]

Re: [gentoo-dev] Reviving the Sandbox project

[R0b0t1]

On Fri, Sep 22, 2017 at 5:01 PM, Michael Orlitzky <mjo@gentoo.org> wrote:
> On 09/22/2017 05:51 PM, R0b0t1 wrote:
>> On Thu, Sep 21, 2017 at 2:56 PM, Michał Górny <mgorny@gentoo.org> wrote:
>>> [1]:https://wiki.gentoo.org/wiki/Project:Sandbox
>>>
>>
>> I think I understand, in principle, why a sandbox could be useful, but
>> would it not be more productive to follow up with projects which do
>> unexpected things to ask that they not do those things?
>>
>
> The sandbox isn't a security feature, it's more of a QA tool. How do you
> *know* when the upstream project does something wrong? See, for example,
>

I was aware of this part. I suggested sandboxing mechanisms which were
security related not for security purposes, but due to the fact that
their original design goals makes them more comprehensive. As a bonus,
they already exist.


On Fri, Sep 22, 2017 at 5:05 PM, Alec Warner <antarus@gentoo.org> wrote:
>
>
> On Fri, Sep 22, 2017 at 5:51 PM, R0b0t1 <r030t1@gmail.com> wrote:
>>
>> On Thu, Sep 21, 2017 at 2:56 PM, Michał Górny <mgorny@gentoo.org> wrote:
>> > [1]:https://wiki.gentoo.org/wiki/Project:Sandbox
>> >
>>
>> I think I understand, in principle, why a sandbox could be useful, but
>> would it not be more productive to follow up with projects which do
>> unexpected things to ask that they not do those things?
>
>
> So step one is figuring out what those things are. So the LD_PRELOAD sandbox
> isn't designed to be a "security boundary" (its trivially defeat-able[1]).
> Instead its designed to be a fairly straightforward detector of 'anomalous'
> behavior. It works by intercepting file-operations and comparing them
> against a whitelist.
>
> You can't tell people do stop doing unexpected things if you don't know
> their software is doing unexpected things.
>

Right, I suppose a sandboxing mechanism is the best way to detect
those changes. Is it necessary for it to be implemented with something
like ptrace or ld tricks? Like above, I ask because other mechanisms
are more comprehensive.

The mechanism used to implement containers, in particular, is
extremely interesting because it does (more or less) exactly what is
wanted. Look for the CLONE_NEW* options in `man 2 clone`. It is
possible containers are the best interface to this syscall, but I am
not sure how to evaluate them in the context they would be used.

Respectfully,
     R0b0t1