* [gentoo-portage-dev] xattr wrapper for install, bug #465000
@ 2014-01-26 22:53 Anthony G. Basile
2014-01-27 14:02 ` vivo75
2014-01-31 4:43 ` Mike Frysinger
0 siblings, 2 replies; 8+ messages in thread
From: Anthony G. Basile @ 2014-01-26 22:53 UTC (permalink / raw
To: gentoo-portage-dev
Hi everyone,
A while back, I wrote a python wrapper for install to preserve xattrs.
Its installed in LIBDIR/portage/bin/install.py. It is *painfully* slow.
For a package like moodle with 16650 .php files, none of which
probably need any xattr's set, it takes about 30 mins to install. I
rewrote the wrapper in C. Replacing the python wrapper with the C
wrapper, the same example reduces from about 30 mins to 2 mins.
Mike and I did some back and forth about how best to write it. The
latest version is pretty much done at
https://bugs.gentoo.org/show_bug.cgi?id=465000#c56
We need to get that integrated into portage. 1) I'm not 100% sure how
to do that. 2) We may want to install it at /usr/bin/install-xattr
because I'm sure it will be useful for more than just portage.
Comments?
--
Anthony G. Basile, Ph. D.
Chair of Information Technology
D'Youville College
Buffalo, NY 14201
(716) 829-8197
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [gentoo-portage-dev] xattr wrapper for install, bug #465000
2014-01-26 22:53 [gentoo-portage-dev] xattr wrapper for install, bug #465000 Anthony G. Basile
@ 2014-01-27 14:02 ` vivo75
2014-01-28 4:36 ` Mike Frysinger
2014-01-29 16:33 ` Anthony G. Basile
2014-01-31 4:43 ` Mike Frysinger
1 sibling, 2 replies; 8+ messages in thread
From: vivo75 @ 2014-01-27 14:02 UTC (permalink / raw
To: gentoo-portage-dev
On 01/26/14 23:53, Anthony G. Basile wrote:
> Hi everyone,
>
> A while back, I wrote a python wrapper for install to preserve xattrs.
> Its installed in LIBDIR/portage/bin/install.py. It is *painfully*
> slow. For a package like moodle with 16650 .php files, none of which
> probably need any xattr's set, it takes about 30 mins to install. I
> rewrote the wrapper in C. Replacing the python wrapper with the C
> wrapper, the same example reduces from about 30 mins to 2 mins.
>
> Mike and I did some back and forth about how best to write it. The
> latest version is pretty much done at
>
> https://bugs.gentoo.org/show_bug.cgi?id=465000#c56
>
> We need to get that integrated into portage. 1) I'm not 100% sure how
> to do that. 2) We may want to install it at /usr/bin/install-xattr
> because I'm sure it will be useful for more than just portage.
>
> Comments?
>
patch install from coreutils (and then upstream changes) is not an option?
they already support selinux contexts
anyway install-xattr could be useful and /usr/bin would be a good option
IMHO
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [gentoo-portage-dev] xattr wrapper for install, bug #465000
2014-01-27 14:02 ` vivo75
@ 2014-01-28 4:36 ` Mike Frysinger
2014-01-29 16:33 ` Anthony G. Basile
1 sibling, 0 replies; 8+ messages in thread
From: Mike Frysinger @ 2014-01-28 4:36 UTC (permalink / raw
To: gentoo-portage-dev
[-- Attachment #1: Type: text/plain, Size: 266 bytes --]
On Monday, January 27, 2014 15:02:30 vivo75@gmail.com wrote:
> patch install from coreutils (and then upstream changes) is not an option?
that route is being pursued independently. we already have a wrapper and will
have one for the foreseeable future.
-mike
[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 836 bytes --]
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [gentoo-portage-dev] xattr wrapper for install, bug #465000
2014-01-27 14:02 ` vivo75
2014-01-28 4:36 ` Mike Frysinger
@ 2014-01-29 16:33 ` Anthony G. Basile
2014-01-29 23:50 ` Francesco R.
1 sibling, 1 reply; 8+ messages in thread
From: Anthony G. Basile @ 2014-01-29 16:33 UTC (permalink / raw
To: gentoo-portage-dev
On 01/27/2014 09:02 AM, vivo75@gmail.com wrote:
> On 01/26/14 23:53, Anthony G. Basile wrote:
>> Hi everyone,
>>
>> A while back, I wrote a python wrapper for install to preserve xattrs.
>> Its installed in LIBDIR/portage/bin/install.py. It is *painfully*
>> slow. For a package like moodle with 16650 .php files, none of which
>> probably need any xattr's set, it takes about 30 mins to install. I
>> rewrote the wrapper in C. Replacing the python wrapper with the C
>> wrapper, the same example reduces from about 30 mins to 2 mins.
>>
>> Mike and I did some back and forth about how best to write it. The
>> latest version is pretty much done at
>>
>> https://bugs.gentoo.org/show_bug.cgi?id=465000#c56
>>
>> We need to get that integrated into portage. 1) I'm not 100% sure how
>> to do that. 2) We may want to install it at /usr/bin/install-xattr
>> because I'm sure it will be useful for more than just portage.
>>
>> Comments?
>>
> patch install from coreutils (and then upstream changes) is not an option?
> they already support selinux contexts
>
> anyway install-xattr could be useful and /usr/bin would be a good option
> IMHO
>
Been there and I even had a patch ready. Upstream answer was '\0'. The
only people who engaged the discussion were gentoo devs.
Would patching coreutils have been the better approach? Some people
might argue install and cp and mv etc should just copy contents to keep
these utilities as simple as possible. Although, as you say, install
can copy selinux contexts, and cp can copy xattr attributes. So what's
the problem with extending installs functionality to include arbitrary
xattr attributes?
Anyhow, seeing as upstream is uninterested, I prefer this wrapper to
maintaining a local patch against coreutils.
--
Anthony G. Basile, Ph. D.
Chair of Information Technology
D'Youville College
Buffalo, NY 14201
(716) 829-8197
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [gentoo-portage-dev] xattr wrapper for install, bug #465000
2014-01-29 16:33 ` Anthony G. Basile
@ 2014-01-29 23:50 ` Francesco R.
0 siblings, 0 replies; 8+ messages in thread
From: Francesco R. @ 2014-01-29 23:50 UTC (permalink / raw
To: gentoo-portage-dev
Il 29/01/2014 17:33, Anthony G. Basile ha scritto:
> On 01/27/2014 09:02 AM, vivo75@gmail.com wrote:
>> On 01/26/14 23:53, Anthony G. Basile wrote:
>>> Hi everyone,
>>>
>>> A while back, I wrote a python wrapper for install to preserve xattrs.
>>> Its installed in LIBDIR/portage/bin/install.py. It is *painfully*
>>> slow. For a package like moodle with 16650 .php files, none of which
>>> probably need any xattr's set, it takes about 30 mins to install. I
>>> rewrote the wrapper in C. Replacing the python wrapper with the C
>>> wrapper, the same example reduces from about 30 mins to 2 mins.
>>>
>>> Mike and I did some back and forth about how best to write it. The
>>> latest version is pretty much done at
>>>
>>> https://bugs.gentoo.org/show_bug.cgi?id=465000#c56
>>>
>>> We need to get that integrated into portage. 1) I'm not 100% sure how
>>> to do that. 2) We may want to install it at /usr/bin/install-xattr
>>> because I'm sure it will be useful for more than just portage.
>>>
>>> Comments?
>>>
>> patch install from coreutils (and then upstream changes) is not an
>> option?
>> they already support selinux contexts
>>
>> anyway install-xattr could be useful and /usr/bin would be a good option
>> IMHO
>>
>
> Been there and I even had a patch ready. Upstream answer was '\0'.
> The only people who engaged the discussion were gentoo devs.
>
> Would patching coreutils have been the better approach? Some people
> might argue install and cp and mv etc should just copy contents to
> keep these utilities as simple as possible. Although, as you say,
> install can copy selinux contexts, and cp can copy xattr attributes.
> So what's the problem with extending installs functionality to include
> arbitrary xattr attributes?
>
> Anyhow, seeing as upstream is uninterested, I prefer this wrapper to
> maintaining a local patch against coreutils.
>
>
thanks for answering, so yes the wrapper seem the only sensible thing,
good luck, since there are no other questions in 3 days the plan must be
perfect ;)
on a totally unrelated and ipotetical note kde is discussing a metadata
engine that use xattr instead of databases, so this could become
important even for desktops
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [gentoo-portage-dev] xattr wrapper for install, bug #465000
2014-01-26 22:53 [gentoo-portage-dev] xattr wrapper for install, bug #465000 Anthony G. Basile
2014-01-27 14:02 ` vivo75
@ 2014-01-31 4:43 ` Mike Frysinger
2014-01-31 4:53 ` Brian Dolbec
1 sibling, 1 reply; 8+ messages in thread
From: Mike Frysinger @ 2014-01-31 4:43 UTC (permalink / raw
To: gentoo-portage-dev
[-- Attachment #1: Type: text/plain, Size: 661 bytes --]
On Sunday, January 26, 2014 17:53:44 Anthony G. Basile wrote:
> 1) I'm not 100% sure how to do that.
we'll have to (re-)introduce support for building C files. shouldn't be too
hard to start a new directory to hold all the C code. and a simple Makefile
to build them all. i think the tbz2tool.c was the only C program we had
before and it was compiled via the ebuild (which we should avoid).
> 2) We may want to install it at /usr/bin/install-xattr
> because I'm sure it will be useful for more than just portage.
if that's the case, then i don't think it belongs in portage. make it a sep
package that portage will just depend on.
-mike
[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 836 bytes --]
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [gentoo-portage-dev] xattr wrapper for install, bug #465000
2014-01-31 4:43 ` Mike Frysinger
@ 2014-01-31 4:53 ` Brian Dolbec
2014-01-31 19:55 ` Greg Turner
0 siblings, 1 reply; 8+ messages in thread
From: Brian Dolbec @ 2014-01-31 4:53 UTC (permalink / raw
To: gentoo-portage-dev
On Thu, 30 Jan 2014 23:43:53 -0500
Mike Frysinger <vapier@gentoo.org> wrote:
> On Sunday, January 26, 2014 17:53:44 Anthony G. Basile wrote:
> > 1) I'm not 100% sure how to do that.
>
> we'll have to (re-)introduce support for building C files. shouldn't
> be too hard to start a new directory to hold all the C code. and a
> simple Makefile to build them all. i think the tbz2tool.c was the
> only C program we had before and it was compiled via the ebuild
> (which we should avoid).
>
> > 2) We may want to install it at /usr/bin/install-xattr
> > because I'm sure it will be useful for more than just portage.
>
> if that's the case, then i don't think it belongs in portage. make
> it a sep package that portage will just depend on.
> -mike
Yes, I agree, make it a separate pkg. That way it is useful for other
pkgs and utilities. I am doing the same for several other small
utility libs to avoid copy/pasting the same code over and over again.
Besides, portage will have the python code as an emergency backup.
--
Brian Dolbec <dolsen>
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [gentoo-portage-dev] xattr wrapper for install, bug #465000
2014-01-31 4:53 ` Brian Dolbec
@ 2014-01-31 19:55 ` Greg Turner
0 siblings, 0 replies; 8+ messages in thread
From: Greg Turner @ 2014-01-31 19:55 UTC (permalink / raw
To: gentoo-portage-dev
On Thu, Jan 30, 2014 at 8:53 PM, Brian Dolbec <dolsen@gentoo.org> wrote:
> On Thu, 30 Jan 2014 23:43:53 -0500 Mike Frysinger <vapier@gentoo.org> wrote:
>> On Sunday, January 26, 2014 17:53:44 Anthony G. Basile wrote:
[snip]
>> > 2) We may want to install it at /usr/bin/install-xattr
>> > because I'm sure it will be useful for more than just portage.
>>
>> if that's the case, then i don't think it belongs in portage. make
>> it a sep package that portage will just depend on.
>> -mike
>
> Yes, I agree, make it a separate pkg. That way it is useful for other
> pkgs and utilities. I am doing the same for several other small
> utility libs to avoid copy/pasting the same code over and over again.
I took a crack at this:
https://github.com/gmt/xattr-install
https://github.com/gmt/portage
https://github.com/gmt/xattr-overlay
more details are in bug 465000
-gmt
^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2014-01-31 19:55 UTC | newest]
Thread overview: 8+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2014-01-26 22:53 [gentoo-portage-dev] xattr wrapper for install, bug #465000 Anthony G. Basile
2014-01-27 14:02 ` vivo75
2014-01-28 4:36 ` Mike Frysinger
2014-01-29 16:33 ` Anthony G. Basile
2014-01-29 23:50 ` Francesco R.
2014-01-31 4:43 ` Mike Frysinger
2014-01-31 4:53 ` Brian Dolbec
2014-01-31 19:55 ` Greg Turner
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox