public inbox for gentoo-dev@lists.gentoo.org
 help / color / mirror / Atom feed
* [gentoo-dev] mbox -- looks sort of interesting
@ 2014-02-11  0:36 Jason A. Donenfeld
  2014-02-11  6:29 ` Michael Haubenwallner
                   ` (3 more replies)
  0 siblings, 4 replies; 22+ messages in thread
From: Jason A. Donenfeld @ 2014-02-11  0:36 UTC (permalink / raw
  To: gentoo-dev

Hey folks,

Late night clicking-while-drooling, I came across something a few
minutes ago that mildly piqued my interest -- mbox
<http://pdos.csail.mit.edu/mbox/>. It's a sandbox that uses a
combination of ptrace and seccomp bpf; neither ours nor exherbo's uses
both of these together. The killer feature, for us, that's motivating
me to write to this list, is that it creates a "shadow file system",
and then has the option to commit the changes of that file system to
the real file system, piece by piece, when the process is done. It
made me think of some discussions we had at FOSDEM about Portage
evolution and whatnot. I haven't looked at this tool past an initial
glance, but it does look like interesting food for thought.

Jason

-- 
Jason A. Donenfeld
Gentoo Linux Security & Infrastructure
zx2c4@gentoo.org
www.zx2c4.com


^ permalink raw reply	[flat|nested] 22+ messages in thread

* Re: [gentoo-dev] mbox -- looks sort of interesting
  2014-02-11  0:36 [gentoo-dev] mbox -- looks sort of interesting Jason A. Donenfeld
@ 2014-02-11  6:29 ` Michael Haubenwallner
  2014-02-11  6:56 ` [gentoo-dev] " Michael Palimaka
                   ` (2 subsequent siblings)
  3 siblings, 0 replies; 22+ messages in thread
From: Michael Haubenwallner @ 2014-02-11  6:29 UTC (permalink / raw
  To: gentoo-dev


On 02/11/14 01:36, Jason A. Donenfeld wrote:
> Hey folks,
> 
> Late night clicking-while-drooling, I came across something a few
> minutes ago that mildly piqued my interest -- mbox
> <http://pdos.csail.mit.edu/mbox/>. It's a sandbox that uses a
> combination of ptrace and seccomp bpf; neither ours nor exherbo's uses
> both of these together. The killer feature, for us, that's motivating
> me to write to this list, is that it creates a "shadow file system",
> and then has the option to commit the changes of that file system to
> the real file system, piece by piece, when the process is done. It
> made me think of some discussions we had at FOSDEM about Portage
> evolution and whatnot. I haven't looked at this tool past an initial
> glance, but it does look like interesting food for thought.

Sounds interesting, especially the "without special privileges" bit...

/haubi/


^ permalink raw reply	[flat|nested] 22+ messages in thread

* [gentoo-dev] Re: mbox -- looks sort of interesting
  2014-02-11  0:36 [gentoo-dev] mbox -- looks sort of interesting Jason A. Donenfeld
  2014-02-11  6:29 ` Michael Haubenwallner
@ 2014-02-11  6:56 ` Michael Palimaka
  2014-02-11 12:34   ` Rich Freeman
  2014-02-11 12:49 ` [gentoo-dev] " justin
  2014-02-11 21:39 ` Wulf C. Krueger
  3 siblings, 1 reply; 22+ messages in thread
From: Michael Palimaka @ 2014-02-11  6:56 UTC (permalink / raw
  To: gentoo-dev

On 02/11/2014 11:36 AM, Jason A. Donenfeld wrote:
> Hey folks,
> 
> Late night clicking-while-drooling, I came across something a few
> minutes ago that mildly piqued my interest -- mbox
> <http://pdos.csail.mit.edu/mbox/>. It's a sandbox that uses a
> combination of ptrace and seccomp bpf; neither ours nor exherbo's uses
> both of these together. The killer feature, for us, that's motivating
> me to write to this list, is that it creates a "shadow file system",
> and then has the option to commit the changes of that file system to
> the real file system, piece by piece, when the process is done. It
> made me think of some discussions we had at FOSDEM about Portage
> evolution and whatnot. I haven't looked at this tool past an initial
> glance, but it does look like interesting food for thought.
> 
> Jason
> 

Looks interesting. It reminds me somewhat of autodep[1].

[1]: http://soc.dev.gentoo.org/~bay/autodep/intro.html



^ permalink raw reply	[flat|nested] 22+ messages in thread

* Re: [gentoo-dev] Re: mbox -- looks sort of interesting
  2014-02-11  6:56 ` [gentoo-dev] " Michael Palimaka
@ 2014-02-11 12:34   ` Rich Freeman
  2014-02-11 12:39     ` Michael Palimaka
  0 siblings, 1 reply; 22+ messages in thread
From: Rich Freeman @ 2014-02-11 12:34 UTC (permalink / raw
  To: gentoo-dev

On Tue, Feb 11, 2014 at 1:56 AM, Michael Palimaka <kensington@gentoo.org> wrote:
>
> Looks interesting. It reminds me somewhat of autodep[1].
>

Interesting - does this work?  I don't see it in portage.

One of those ideas I've always wanted to implement is to create a
portage hook/patch that looks at the dependencies for the package
being built and configures sandbox to block read-access to anything
that wasn't explicitly declared.  Sandbox works for read-access as
well as write-access, though in /etc/sandbox.d/00default read-access
is enabled everywhere by default.

And, yes, it could be configured to allow access to @system...

Rich


^ permalink raw reply	[flat|nested] 22+ messages in thread

* [gentoo-dev] Re: mbox -- looks sort of interesting
  2014-02-11 12:34   ` Rich Freeman
@ 2014-02-11 12:39     ` Michael Palimaka
  2014-02-11 14:03       ` Rich Freeman
  0 siblings, 1 reply; 22+ messages in thread
From: Michael Palimaka @ 2014-02-11 12:39 UTC (permalink / raw
  To: gentoo-dev

On 02/11/2014 11:34 PM, Rich Freeman wrote:
> On Tue, Feb 11, 2014 at 1:56 AM, Michael Palimaka <kensington@gentoo.org> wrote:
>>
>> Looks interesting. It reminds me somewhat of autodep[1].
>>
> 
> Interesting - does this work?  I don't see it in portage.
It used to work pretty well, but the bundled portage version doesn't
support EAPI 5. I previously made an attempt to merge a newer version of
portage in, but I was not successful.

> One of those ideas I've always wanted to implement is to create a
> portage hook/patch that looks at the dependencies for the package
> being built and configures sandbox to block read-access to anything
> that wasn't explicitly declared.  Sandbox works for read-access as
> well as write-access, though in /etc/sandbox.d/00default read-access
> is enabled everywhere by default.
> 
> And, yes, it could be configured to allow access to @system...
That's pretty much what emerge_strict does.



^ permalink raw reply	[flat|nested] 22+ messages in thread

* Re: [gentoo-dev] mbox -- looks sort of interesting
  2014-02-11  0:36 [gentoo-dev] mbox -- looks sort of interesting Jason A. Donenfeld
  2014-02-11  6:29 ` Michael Haubenwallner
  2014-02-11  6:56 ` [gentoo-dev] " Michael Palimaka
@ 2014-02-11 12:49 ` justin
  2014-02-11 21:39 ` Wulf C. Krueger
  3 siblings, 0 replies; 22+ messages in thread
From: justin @ 2014-02-11 12:49 UTC (permalink / raw
  To: gentoo-dev

[-- Attachment #1: Type: text/plain, Size: 1253 bytes --]

On 11/02/14 01:36, Jason A. Donenfeld wrote:
> Hey folks,
> 
> Late night clicking-while-drooling, I came across something a few
> minutes ago that mildly piqued my interest -- mbox
> <http://pdos.csail.mit.edu/mbox/>. It's a sandbox that uses a
> combination of ptrace and seccomp bpf; neither ours nor exherbo's uses
> both of these together. The killer feature, for us, that's motivating
> me to write to this list, is that it creates a "shadow file system",
> and then has the option to commit the changes of that file system to
> the real file system, piece by piece, when the process is done. It
> made me think of some discussions we had at FOSDEM about Portage
> evolution and whatnot. I haven't looked at this tool past an initial
> glance, but it does look like interesting food for thought.
> 
> Jason
> 

At FOSDEM I have seen this interesting talk[1,2] on a similar subject.
PRoot[3] would be similar to mbox. But CARE[4] might be great to
reproduce build problems on user machines.

justin

1 https://fosdem.org/2014/schedule/event/syscall/
2
http://ftp.belnet.be/FOSDEM/2014/H2215_Ferrer/Saturday/Software_engineering_tools_based_on_syscall_instrumentation.webm
3 http://proot.me/
4 http://reproducible.io/


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 901 bytes --]

^ permalink raw reply	[flat|nested] 22+ messages in thread

* Re: [gentoo-dev] Re: mbox -- looks sort of interesting
  2014-02-11 12:39     ` Michael Palimaka
@ 2014-02-11 14:03       ` Rich Freeman
  2014-02-11 14:36         ` Michael Palimaka
  0 siblings, 1 reply; 22+ messages in thread
From: Rich Freeman @ 2014-02-11 14:03 UTC (permalink / raw
  To: gentoo-dev

On Tue, Feb 11, 2014 at 7:39 AM, Michael Palimaka <kensington@gentoo.org> wrote:
> On 02/11/2014 11:34 PM, Rich Freeman wrote:
>
>> One of those ideas I've always wanted to implement is to create a
>> portage hook/patch that looks at the dependencies for the package
>> being built and configures sandbox to block read-access to anything
>> that wasn't explicitly declared.  Sandbox works for read-access as
>> well as write-access, though in /etc/sandbox.d/00default read-access
>> is enabled everywhere by default.
>>
>> And, yes, it could be configured to allow access to @system...
> That's pretty much what emerge_strict does.

What is emerge_strict?  The Google is failing me here...

Rich


^ permalink raw reply	[flat|nested] 22+ messages in thread

* [gentoo-dev] Re: mbox -- looks sort of interesting
  2014-02-11 14:03       ` Rich Freeman
@ 2014-02-11 14:36         ` Michael Palimaka
  2014-02-12  5:56           ` Brian Dolbec
  0 siblings, 1 reply; 22+ messages in thread
From: Michael Palimaka @ 2014-02-11 14:36 UTC (permalink / raw
  To: gentoo-dev

On 02/12/2014 01:03 AM, Rich Freeman wrote:
> On Tue, Feb 11, 2014 at 7:39 AM, Michael Palimaka <kensington@gentoo.org> wrote:
>> On 02/11/2014 11:34 PM, Rich Freeman wrote:
>>
>>> One of those ideas I've always wanted to implement is to create a
>>> portage hook/patch that looks at the dependencies for the package
>>> being built and configures sandbox to block read-access to anything
>>> that wasn't explicitly declared.  Sandbox works for read-access as
>>> well as write-access, though in /etc/sandbox.d/00default read-access
>>> is enabled everywhere by default.
>>>
>>> And, yes, it could be configured to allow access to @system...
>> That's pretty much what emerge_strict does.
> 
> What is emerge_strict?  The Google is failing me here...
> 
> Rich
> 
> 
Sorry, I should have clarified. It's provided by autodep, extending the
dependency analysis by denying access to any files not part of the
specified dependencies and @system.



^ permalink raw reply	[flat|nested] 22+ messages in thread

* Re: [gentoo-dev] mbox -- looks sort of interesting
  2014-02-11  0:36 [gentoo-dev] mbox -- looks sort of interesting Jason A. Donenfeld
                   ` (2 preceding siblings ...)
  2014-02-11 12:49 ` [gentoo-dev] " justin
@ 2014-02-11 21:39 ` Wulf C. Krueger
  2014-02-11 23:13   ` Jason A. Donenfeld
  3 siblings, 1 reply; 22+ messages in thread
From: Wulf C. Krueger @ 2014-02-11 21:39 UTC (permalink / raw
  To: gentoo-dev

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hello Jason,

On 11.02.2014 01:36, Jason A. Donenfeld wrote:
> It's a sandbox that uses a combination of ptrace and seccomp bpf; 
> neither ours nor exherbo's uses both of these together.

Actually, sydbox, Exherbo's sandbox *does* use both together.

- -- 
Best regards, Wulf
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.22 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iEYEARECAAYFAlL6mKUACgkQnuVXRcSi+5olDwCfRDoP9f2zfM1GndKcG1rkNWZR
I9YAn2Rwdb40m0vnL0FIdyN3v/J3Ka7I
=ZbOm
-----END PGP SIGNATURE-----


^ permalink raw reply	[flat|nested] 22+ messages in thread

* Re: [gentoo-dev] mbox -- looks sort of interesting
  2014-02-11 21:39 ` Wulf C. Krueger
@ 2014-02-11 23:13   ` Jason A. Donenfeld
  0 siblings, 0 replies; 22+ messages in thread
From: Jason A. Donenfeld @ 2014-02-11 23:13 UTC (permalink / raw
  To: gentoo-dev

On Tue, Feb 11, 2014 at 10:39 PM, Wulf C. Krueger <wk@mailstation.de> wrote:
> On 11.02.2014 01:36, Jason A. Donenfeld wrote:
>> It's a sandbox that uses a combination of ptrace and seccomp bpf;
>> neither ours nor exherbo's uses both of these together.
>
> Actually, sydbox, Exherbo's sandbox *does* use both together.

I didn't know sydbox made use of bpf. That's really cool. I'll have to
take another look.


^ permalink raw reply	[flat|nested] 22+ messages in thread

* Re: [gentoo-dev] Re: mbox -- looks sort of interesting
  2014-02-11 14:36         ` Michael Palimaka
@ 2014-02-12  5:56           ` Brian Dolbec
  2014-02-12 11:45             ` Michael Palimaka
  0 siblings, 1 reply; 22+ messages in thread
From: Brian Dolbec @ 2014-02-12  5:56 UTC (permalink / raw
  To: gentoo-dev

On Wed, 12 Feb 2014 01:36:01 +1100
Michael Palimaka <kensington@gentoo.org> wrote:

> On 02/12/2014 01:03 AM, Rich Freeman wrote:
> > On Tue, Feb 11, 2014 at 7:39 AM, Michael Palimaka
> > <kensington@gentoo.org> wrote:
> >> On 02/11/2014 11:34 PM, Rich Freeman wrote:
> >>
> >>> One of those ideas I've always wanted to implement is to create a
> >>> portage hook/patch that looks at the dependencies for the package
> >>> being built and configures sandbox to block read-access to
> >>> anything that wasn't explicitly declared.  Sandbox works for
> >>> read-access as well as write-access, though
> >>> in /etc/sandbox.d/00default read-access is enabled everywhere by
> >>> default.
> >>>
> >>> And, yes, it could be configured to allow access to @system...
> >> That's pretty much what emerge_strict does.
> > 
> > What is emerge_strict?  The Google is failing me here...
> > 
> > Rich
> > 
> > 
> Sorry, I should have clarified. It's provided by autodep, extending
> the dependency analysis by denying access to any files not part of the
> specified dependencies and @system.
> 
> 

There was a gentoo gsoc project a few years ago that did exactly this
for doing dep checks on ebuilds.  There was also one for determining
deps automatically.

Is this the project mentioned? ^^^

-- 
Brian Dolbec <dolsen>



^ permalink raw reply	[flat|nested] 22+ messages in thread

* [gentoo-dev] Re: mbox -- looks sort of interesting
  2014-02-12  5:56           ` Brian Dolbec
@ 2014-02-12 11:45             ` Michael Palimaka
  2014-02-13  7:36               ` Александр Берсенев
  0 siblings, 1 reply; 22+ messages in thread
From: Michael Palimaka @ 2014-02-12 11:45 UTC (permalink / raw
  To: gentoo-dev

On 02/12/2014 04:56 PM, Brian Dolbec wrote:
> On Wed, 12 Feb 2014 01:36:01 +1100
> Michael Palimaka <kensington@gentoo.org> wrote:
> 
>> On 02/12/2014 01:03 AM, Rich Freeman wrote:
>>> On Tue, Feb 11, 2014 at 7:39 AM, Michael Palimaka
>>> <kensington@gentoo.org> wrote:
>>>> On 02/11/2014 11:34 PM, Rich Freeman wrote:
>>>>
>>>>> One of those ideas I've always wanted to implement is to create a
>>>>> portage hook/patch that looks at the dependencies for the package
>>>>> being built and configures sandbox to block read-access to
>>>>> anything that wasn't explicitly declared.  Sandbox works for
>>>>> read-access as well as write-access, though
>>>>> in /etc/sandbox.d/00default read-access is enabled everywhere by
>>>>> default.
>>>>>
>>>>> And, yes, it could be configured to allow access to @system...
>>>> That's pretty much what emerge_strict does.
>>>
>>> What is emerge_strict?  The Google is failing me here...
>>>
>>> Rich
>>>
>>>
>> Sorry, I should have clarified. It's provided by autodep, extending
>> the dependency analysis by denying access to any files not part of the
>> specified dependencies and @system.
>>
>>
> 
> There was a gentoo gsoc project a few years ago that did exactly this
> for doing dep checks on ebuilds.  There was also one for determining
> deps automatically.
> 
> Is this the project mentioned? ^^^
> 

Should be, autodep was GSoC 2011.



^ permalink raw reply	[flat|nested] 22+ messages in thread

* Re: [gentoo-dev] Re: mbox -- looks sort of interesting
  2014-02-12 11:45             ` Michael Palimaka
@ 2014-02-13  7:36               ` Александр Берсенев
  2014-02-13 10:11                 ` Michael Palimaka
  0 siblings, 1 reply; 22+ messages in thread
From: Александр Берсенев @ 2014-02-13  7:36 UTC (permalink / raw
  To: gentoo-dev@lists.gentoo.org

[-- Attachment #1: Type: text/plain, Size: 1642 bytes --]

Hi, It was my project. The portage changed a lot since that time, I can try
to renew it, if it's still used.


2014-02-12 17:45 GMT+06:00 Michael Palimaka <kensington@gentoo.org>:

> On 02/12/2014 04:56 PM, Brian Dolbec wrote:
> > On Wed, 12 Feb 2014 01:36:01 +1100
> > Michael Palimaka <kensington@gentoo.org> wrote:
> >
> >> On 02/12/2014 01:03 AM, Rich Freeman wrote:
> >>> On Tue, Feb 11, 2014 at 7:39 AM, Michael Palimaka
> >>> <kensington@gentoo.org> wrote:
> >>>> On 02/11/2014 11:34 PM, Rich Freeman wrote:
> >>>>
> >>>>> One of those ideas I've always wanted to implement is to create a
> >>>>> portage hook/patch that looks at the dependencies for the package
> >>>>> being built and configures sandbox to block read-access to
> >>>>> anything that wasn't explicitly declared.  Sandbox works for
> >>>>> read-access as well as write-access, though
> >>>>> in /etc/sandbox.d/00default read-access is enabled everywhere by
> >>>>> default.
> >>>>>
> >>>>> And, yes, it could be configured to allow access to @system...
> >>>> That's pretty much what emerge_strict does.
> >>>
> >>> What is emerge_strict?  The Google is failing me here...
> >>>
> >>> Rich
> >>>
> >>>
> >> Sorry, I should have clarified. It's provided by autodep, extending
> >> the dependency analysis by denying access to any files not part of the
> >> specified dependencies and @system.
> >>
> >>
> >
> > There was a gentoo gsoc project a few years ago that did exactly this
> > for doing dep checks on ebuilds.  There was also one for determining
> > deps automatically.
> >
> > Is this the project mentioned? ^^^
> >
>
> Should be, autodep was GSoC 2011.
>
>
>

[-- Attachment #2: Type: text/html, Size: 2545 bytes --]

^ permalink raw reply	[flat|nested] 22+ messages in thread

* [gentoo-dev] Re: mbox -- looks sort of interesting
  2014-02-13  7:36               ` Александр Берсенев
@ 2014-02-13 10:11                 ` Michael Palimaka
  2014-02-13 16:07                   ` Brian Dolbec
  0 siblings, 1 reply; 22+ messages in thread
From: Michael Palimaka @ 2014-02-13 10:11 UTC (permalink / raw
  To: gentoo-dev

On 02/13/2014 06:36 PM, Александр Берсенев wrote:
> Hi, It was my project. The portage changed a lot since that time, I can
> try to renew it, if it's still used.

I used to use it a lot until it stopped working because of not
understanding EAPI 5. I think others would find it useful, but I don't
think many people are aware of it.




^ permalink raw reply	[flat|nested] 22+ messages in thread

* Re: [gentoo-dev] Re: mbox -- looks sort of interesting
  2014-02-13 10:11                 ` Michael Palimaka
@ 2014-02-13 16:07                   ` Brian Dolbec
  2014-02-13 17:00                     ` Rich Freeman
  0 siblings, 1 reply; 22+ messages in thread
From: Brian Dolbec @ 2014-02-13 16:07 UTC (permalink / raw
  To: gentoo-dev

On Thu, 13 Feb 2014 21:11:36 +1100
Michael Palimaka <kensington@gentoo.org> wrote:

> On 02/13/2014 06:36 PM, Александр Берсенев wrote:
> > Hi, It was my project. The portage changed a lot since that time, I
> > can try to renew it, if it's still used.
> 
> I used to use it a lot until it stopped working because of not
> understanding EAPI 5. I think others would find it useful, but I don't
> think many people are aware of it.
> 
> 
> 

Yes, if you can please work on updating it.  Please contact us on the
gentoo-portage-dev mail list or irc #gentoo-portage for changes to
portage that will help keep it working in the future.  I started
development of a public_api branch long ago just for having a stable
API for apps to use.  Perhaps some of what you need can be integrated
there.

-- 
Brian Dolbec <dolsen>



^ permalink raw reply	[flat|nested] 22+ messages in thread

* Re: [gentoo-dev] Re: mbox -- looks sort of interesting
  2014-02-13 16:07                   ` Brian Dolbec
@ 2014-02-13 17:00                     ` Rich Freeman
  2014-02-14  7:28                       ` Александр Берсенев
  0 siblings, 1 reply; 22+ messages in thread
From: Rich Freeman @ 2014-02-13 17:00 UTC (permalink / raw
  To: gentoo-dev

On Thu, Feb 13, 2014 at 11:07 AM, Brian Dolbec <dolsen@gentoo.org> wrote:
> Yes, if you can please work on updating it.  Please contact us on the
> gentoo-portage-dev mail list or irc #gentoo-portage for changes to
> portage that will help keep it working in the future.  I started
> development of a public_api branch long ago just for having a stable
> API for apps to use.  Perhaps some of what you need can be integrated
> there.

Seems like a valuable tool.  It would be best if it could either use
portage APIs where available, or if effort could be directed at
incorporating the necessary features into portage or its APIs so that
you're not stuck maintaining a fork.  I'm sure the portage team will
help where they can.

Rich


^ permalink raw reply	[flat|nested] 22+ messages in thread

* Re: [gentoo-dev] Re: mbox -- looks sort of interesting
  2014-02-13 17:00                     ` Rich Freeman
@ 2014-02-14  7:28                       ` Александр Берсенев
  2014-02-14  7:50                         ` Greg Turner
  0 siblings, 1 reply; 22+ messages in thread
From: Александр Берсенев @ 2014-02-14  7:28 UTC (permalink / raw
  To: gentoo-dev@lists.gentoo.org

[-- Attachment #1: Type: text/plain, Size: 850 bytes --]

Ok, I'll work on it.


2014-02-13 23:00 GMT+06:00 Rich Freeman <rich0@gentoo.org>:

> On Thu, Feb 13, 2014 at 11:07 AM, Brian Dolbec <dolsen@gentoo.org> wrote:
> > Yes, if you can please work on updating it.  Please contact us on the
> > gentoo-portage-dev mail list or irc #gentoo-portage for changes to
> > portage that will help keep it working in the future.  I started
> > development of a public_api branch long ago just for having a stable
> > API for apps to use.  Perhaps some of what you need can be integrated
> > there.
>
> Seems like a valuable tool.  It would be best if it could either use
> portage APIs where available, or if effort could be directed at
> incorporating the necessary features into portage or its APIs so that
> you're not stuck maintaining a fork.  I'm sure the portage team will
> help where they can.
>
> Rich
>
>

[-- Attachment #2: Type: text/html, Size: 1354 bytes --]

^ permalink raw reply	[flat|nested] 22+ messages in thread

* Re: [gentoo-dev] Re: mbox -- looks sort of interesting
  2014-02-14  7:28                       ` Александр Берсенев
@ 2014-02-14  7:50                         ` Greg Turner
  2014-02-14  8:08                           ` Александр Берсенев
  0 siblings, 1 reply; 22+ messages in thread
From: Greg Turner @ 2014-02-14  7:50 UTC (permalink / raw
  To: gentoo-dev

Holy crap, that looks awesome!  How does one pronounce your name, Александр?

On Thu, Feb 13, 2014 at 11:28 PM, Александр Берсенев <bay@hackerdom.ru> wrote:
> Ok, I'll work on it.
>
>
> 2014-02-13 23:00 GMT+06:00 Rich Freeman <rich0@gentoo.org>:
>
>> On Thu, Feb 13, 2014 at 11:07 AM, Brian Dolbec <dolsen@gentoo.org> wrote:
>> > Yes, if you can please work on updating it.  Please contact us on the
>> > gentoo-portage-dev mail list or irc #gentoo-portage for changes to
>> > portage that will help keep it working in the future.  I started
>> > development of a public_api branch long ago just for having a stable
>> > API for apps to use.  Perhaps some of what you need can be integrated
>> > there.
>>
>> Seems like a valuable tool.  It would be best if it could either use
>> portage APIs where available, or if effort could be directed at
>> incorporating the necessary features into portage or its APIs so that
>> you're not stuck maintaining a fork.  I'm sure the portage team will
>> help where they can.
>>
>> Rich
>>
>


^ permalink raw reply	[flat|nested] 22+ messages in thread

* Re: [gentoo-dev] Re: mbox -- looks sort of interesting
  2014-02-14  7:50                         ` Greg Turner
@ 2014-02-14  8:08                           ` Александр Берсенев
  2014-02-22  9:50                             ` Александр Берсенев
  0 siblings, 1 reply; 22+ messages in thread
From: Александр Берсенев @ 2014-02-14  8:08 UTC (permalink / raw
  To: gentoo-dev@lists.gentoo.org

[-- Attachment #1: Type: text/plain, Size: 1218 bytes --]

Aleksandr


2014-02-14 13:50 GMT+06:00 Greg Turner <gmt@malth.us>:

> Holy crap, that looks awesome!  How does one pronounce your name,
> Александр?
>
> On Thu, Feb 13, 2014 at 11:28 PM, Александр Берсенев <bay@hackerdom.ru>
> wrote:
> > Ok, I'll work on it.
> >
> >
> > 2014-02-13 23:00 GMT+06:00 Rich Freeman <rich0@gentoo.org>:
> >
> >> On Thu, Feb 13, 2014 at 11:07 AM, Brian Dolbec <dolsen@gentoo.org>
> wrote:
> >> > Yes, if you can please work on updating it.  Please contact us on the
> >> > gentoo-portage-dev mail list or irc #gentoo-portage for changes to
> >> > portage that will help keep it working in the future.  I started
> >> > development of a public_api branch long ago just for having a stable
> >> > API for apps to use.  Perhaps some of what you need can be integrated
> >> > there.
> >>
> >> Seems like a valuable tool.  It would be best if it could either use
> >> portage APIs where available, or if effort could be directed at
> >> incorporating the necessary features into portage or its APIs so that
> >> you're not stuck maintaining a fork.  I'm sure the portage team will
> >> help where they can.
> >>
> >> Rich
> >>
> >
>
>

[-- Attachment #2: Type: text/html, Size: 1878 bytes --]

^ permalink raw reply	[flat|nested] 22+ messages in thread

* Re: [gentoo-dev] Re: mbox -- looks sort of interesting
  2014-02-14  8:08                           ` Александр Берсенев
@ 2014-02-22  9:50                             ` Александр Берсенев
  2014-02-25 18:28                               ` Michael Palimaka
  0 siblings, 1 reply; 22+ messages in thread
From: Александр Берсенев @ 2014-02-22  9:50 UTC (permalink / raw
  To: gentoo-dev@lists.gentoo.org

[-- Attachment #1: Type: text/plain, Size: 3914 bytes --]

Hello,

I've updated the autodep and testing it now.

I've fixed a problem with the integration of autodep in the newest version
of Portage, but I think I did it in a hackish way. I want to ask if it is
possble to solve it better.

First, some words about the architecture of autodep. Autodep consists of
two parts:
1) A library, injected with LD_PRELOAD, which intercepting glibc functions
like open, read, write, etc. The library connects using unix socket with a
server part and asks this server about each file operation. The server can
say to permit or to block an operation.
2) The server part, which logs requests from clients and answers them.

The integration with the portage is made, using two hooks on
build(EbuildBuild.py):
 build = EbuildExecuter(background=self.background, pkg=pkg,
  scheduler=scheduler, settings=settings)

 build.addStartListener(self._build_start)
 build.addExitListener(self._build_stop)

 self._start_task(build, self._build_exit)

The _build_start creates the server thread, and passes a unix-socket path
using LOG_SOCKET environment variable.

 self.logserver.start()

 # Copy socket path to LOG_SOCKET environment variable
 env=self.settings.configdict["pkg"]
 env['LOG_SOCKET'] = self.logserver.socket_name

The  _build_stop stops the server and removes LOG_SOCKET environment
variable.
 env=self.settings.configdict["pkg"]
 if 'LOG_SOCKET' in env:
  del env['LOG_SOCKET']

 events=self.logserver.stop()
 self.logserver=None

If LOG_SOCKET env var is set then the new processes(like ebuild.sh <stage>)
are started with my library injected using LD_PRELOAD trick.

The problem: on newer portage versions packages are merged in a subprocess
to allow the Scheduler run in main thread while files are moved or copied
asynchronously(file _MergeProcess.py). This subprocess is created before
the _build_stop is called, so it still has a copy of settings with
LOG_SOCKET set. When _build_stop is called the server stops and merging
fails because the unix-socket is closed.

To solve this I added this code before forking in _MergeProcess.py:
 # Delete LOG_SOCKET from environment
 env=self.settings.configdict["pkg"]
 if 'LOG_SOCKET' in env:
  del env['LOG_SOCKET']

But I think this can be done better. Do you know a better way?

P.S. The old variant of integration patch is available here:
http://git.overlays.gentoo.org/gitweb/?p=proj/autodep.git;a=blob;f=integration_with_portage.patch;h=ff8f487306e85cbc6ba9e69be6fe613e99b39b53;hb=HEAD

Best,
Alexander Bersenev

2014-02-14 14:08 GMT+06:00 Александр Берсенев <bay@hackerdom.ru>:

> Aleksandr
>
>
> 2014-02-14 13:50 GMT+06:00 Greg Turner <gmt@malth.us>:
>
> Holy crap, that looks awesome!  How does one pronounce your name,
>> Александр?
>>
>> On Thu, Feb 13, 2014 at 11:28 PM, Александр Берсенев <bay@hackerdom.ru>
>> wrote:
>> > Ok, I'll work on it.
>> >
>> >
>> > 2014-02-13 23:00 GMT+06:00 Rich Freeman <rich0@gentoo.org>:
>> >
>> >> On Thu, Feb 13, 2014 at 11:07 AM, Brian Dolbec <dolsen@gentoo.org>
>> wrote:
>> >> > Yes, if you can please work on updating it.  Please contact us on the
>> >> > gentoo-portage-dev mail list or irc #gentoo-portage for changes to
>> >> > portage that will help keep it working in the future.  I started
>> >> > development of a public_api branch long ago just for having a stable
>> >> > API for apps to use.  Perhaps some of what you need can be integrated
>> >> > there.
>> >>
>> >> Seems like a valuable tool.  It would be best if it could either use
>> >> portage APIs where available, or if effort could be directed at
>> >> incorporating the necessary features into portage or its APIs so that
>> >> you're not stuck maintaining a fork.  I'm sure the portage team will
>> >> help where they can.
>> >>
>> >> Rich
>> >>
>> >
>>
>>
>

[-- Attachment #2: Type: text/html, Size: 6185 bytes --]

^ permalink raw reply	[flat|nested] 22+ messages in thread

* [gentoo-dev] Re: mbox -- looks sort of interesting
  2014-02-22  9:50                             ` Александр Берсенев
@ 2014-02-25 18:28                               ` Michael Palimaka
  2014-02-25 19:22                                 ` Александр Берсенев
  0 siblings, 1 reply; 22+ messages in thread
From: Michael Palimaka @ 2014-02-25 18:28 UTC (permalink / raw
  To: gentoo-dev

On 02/22/2014 08:50 PM, Александр Берсенев wrote:
> Hello,
> 
> I've updated the autodep and testing it now.

Thanks for taking the time to do this!




^ permalink raw reply	[flat|nested] 22+ messages in thread

* Re: [gentoo-dev] Re: mbox -- looks sort of interesting
  2014-02-25 18:28                               ` Michael Palimaka
@ 2014-02-25 19:22                                 ` Александр Берсенев
  0 siblings, 0 replies; 22+ messages in thread
From: Александр Берсенев @ 2014-02-25 19:22 UTC (permalink / raw
  To: gentoo-dev@lists.gentoo.org

[-- Attachment #1: Type: text/plain, Size: 419 bytes --]

You are welcome!

While testing I found strange race-condition bug in portage, I've filed it
here: https://bugs.gentoo.org/show_bug.cgi?id=502428


2014-02-26 0:28 GMT+06:00 Michael Palimaka <kensington@gentoo.org>:

> On 02/22/2014 08:50 PM, Александр Берсенев wrote:
> > Hello,
> >
> > I've updated the autodep and testing it now.
>
> Thanks for taking the time to do this!
>
>
>
>

[-- Attachment #2: Type: text/html, Size: 858 bytes --]

^ permalink raw reply	[flat|nested] 22+ messages in thread

end of thread, other threads:[~2014-02-25 19:23 UTC | newest]

Thread overview: 22+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2014-02-11  0:36 [gentoo-dev] mbox -- looks sort of interesting Jason A. Donenfeld
2014-02-11  6:29 ` Michael Haubenwallner
2014-02-11  6:56 ` [gentoo-dev] " Michael Palimaka
2014-02-11 12:34   ` Rich Freeman
2014-02-11 12:39     ` Michael Palimaka
2014-02-11 14:03       ` Rich Freeman
2014-02-11 14:36         ` Michael Palimaka
2014-02-12  5:56           ` Brian Dolbec
2014-02-12 11:45             ` Michael Palimaka
2014-02-13  7:36               ` Александр Берсенев
2014-02-13 10:11                 ` Michael Palimaka
2014-02-13 16:07                   ` Brian Dolbec
2014-02-13 17:00                     ` Rich Freeman
2014-02-14  7:28                       ` Александр Берсенев
2014-02-14  7:50                         ` Greg Turner
2014-02-14  8:08                           ` Александр Берсенев
2014-02-22  9:50                             ` Александр Берсенев
2014-02-25 18:28                               ` Michael Palimaka
2014-02-25 19:22                                 ` Александр Берсенев
2014-02-11 12:49 ` [gentoo-dev] " justin
2014-02-11 21:39 ` Wulf C. Krueger
2014-02-11 23:13   ` Jason A. Donenfeld

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox