[gentoo-dev] Re: Files owned by multiple slots

[gentoo-dev] Re: Files owned by multiple slots

[Sven Schwyn]

Marijn wrote:
>> Is it feasable to extend Portage to allow a file to be owned by  
>> several
>> slots and remove it only once the last slot is uninstalled (aka: once
>> the ebuild is completely uninstalled)?
>
> Do we have any guarantees that the file you want to share will be  
> compatible
> with all gems that will possibly use it?

Only executable wrappers are subject to sharing and those are  
generated by RubyGems, not written by an author. Unless RubyGems  
changes it's policy (and I don't see any signs for this), it's safe to  
assume that the wrapper will work for all gem versions.

Marijn wrote:
> You could split the executable off into its own package to avoid this
> duplication. On the other hand, these file collisions between  
> different versions
> of the same gem seem to be an upstream problem; how exactly does  
> RubyGems handle
> this?

It doesn't have to. The generated code is downwards compatible and  
thus an update will simply overwrite the wrapper. (In most cases, the  
new wrapper code is identical.)

Marijn wrote:
> Hmm, so you aren't merely claiming compatibility (like I asked about  
> above) but
> identicalness (is that even a word?). Is that really realistic?

Check the wrappers out: Even executables of different gems have only  
minor differences. Here's an example with rake (make for Ruby) and  
passenger-install-apache2-module (the mod_passenger installer):

> --- /Gentoo/usr/bin/passenger-install-apache2-module	2009-05-06  
> 21:45:46 +0200
> +++ /Gentoo/usr/bin/rake	2009-05-06 21:46:01 +0200
> @@ -2,7 +2,7 @@
>  #
>  # This file was generated by RubyGems.
>  #
> -# The application 'passenger' is installed as part of a gem, and
> +# The application 'rake' is installed as part of a gem, and
>  # this file is here to facilitate running it.
>  #
>
> @@ -15,5 +15,5 @@
>    ARGV.shift
>  end
>
> -gem 'passenger', version
> -load 'passenger-install-apache2-module'
> +gem 'rake', version
> +load 'rake'


Marijn wrote:
>> - All outdated ebuilds must be kept due to gem dependencies to  
>> outdated versions.
>
>
> I'm sure we can check this before gems are added and we can  
> periodically
> automatically remove old gems. Everything could also be kept in an  
> overlay so as
> not to pollute the tree.

Of course this can be done. The question is whether it's worth the  
hassle. After all, executables from gems are the only issue Portage  
has with RubyGems and vice versa that I'm aware of. Btw: It would take  
at least two overlays - the second one for Gentoo Prefix.

One followup about gem2ebuild: AFAIK gems don't track external  
dependencies. (Existing tools such as gem2rpm only convert inter-gem  
dependencies.) Therefore, automatically generated ebuilds are just as  
iffy as installing the gem directly. This can only be prevented if  
someone adds the dependencies either manually or with a template. And  
makes sure, that they are still correct on every release of the gem.  
It takes a lot of manpower to do this.

Marijn wrote:
> Haskell and Perl (and many more probably) each have repositories of
> libraries/applications plus some sort of package manager to install  
> and keep
> track of them. Is there a general solution? Is it possible to  
> redirect calls to
> these external package managers to the main package manager through  
> some
> interface (a bit like pkgkit)? What about eix support?

I wouldn't be happy about such a solution. This "assimilation" sounds  
like trouble and the current co-existence works quite well in most  
cases.

-sven

[gentoo-dev] Re: Files owned by multiple slots

[Sven Schwyn]

Gilles wrote:
> so this wrapper could be installed in a separate eselect sort of
> package ?
How exactly can this be done? If a gem creates five executables, would  
this mean that this gem comes in six ebuilds?
> If those kind of wrappers are generic enough, it could even be a  
> single
> eselect package and gems would enable (symlink to wrapper)  
> themselves at
> postinst. I'm an eselect n00b but this all sound like something an
> eselect module could do.

The wrappers can't be boiled down to one unified wrapper.
Cheers, -sven

Re: [gentoo-dev] Re: Files owned by multiple slots

[Gilles Dartiguelongue]

Le mardi 12 mai 2009 à 22:55 +0200, Sven Schwyn a écrit :
> Gilles wrote:
> > so this wrapper could be installed in a separate eselect sort of
> > package ?
> How exactly can this be done? If a gem creates five executables, would  
> this mean that this gem comes in six ebuilds?

well given the next answer, it sounds like that's what I'm proposing.
I release this might create a packaging overhead.

> > If those kind of wrappers are generic enough, it could even be a  
> > single
> > eselect package and gems would enable (symlink to wrapper)  
> > themselves at
> > postinst. I'm an eselect n00b but this all sound like something an
> > eselect module could do.
> 
> The wrappers can't be boiled down to one unified wrapper.
> Cheers, -sven
> 
> 
-- 
Gilles Dartiguelongue <eva@gentoo.org>
Gentoo

[gentoo-dev] Re: Files owned by multiple slots

[Sven]

Gilles Dartiguelongue <eva <at> gentoo.org> writes:
> > > so this wrapper could be installed in a separate eselect sort of
> > > package ?
> > How exactly can this be done? If a gem creates five executables, would  
> > this mean that this gem comes in six ebuilds?
> 
> well given the next answer, it sounds like that's what I'm proposing.
> I release this might create a packaging overhead.

I'll check the docs to learn about how this could be done.

Yet meanwhile I'd love to get some feedback on my original proposition: Is it
feasable and acceptable to expand Portage by the possibility to declare a file
shared between slots and only delete it once the last slot is uninstalled?

Thanks for some feedback on this!   

-sven

[gentoo-dev] Re: Files owned by multiple slots

[Duncan]

Sven <svoop@delirium.ch> posted loom.20090513T143352-787@post.gmane.org,
excerpted below, on  Wed, 13 May 2009 14:34:20 +0000:

> Yet meanwhile I'd love to get some feedback on my original proposition:
> Is it feasable and acceptable to expand Portage by the possibility to
> declare a file shared between slots and only delete it once the last
> slot is uninstalled?

FWIW, that'd not be a portage issue per se, but an EAPI issue, since it 
would have to do with the ebuilds and the other PMs (package managers, 
Gentoo has three, portage, paludis, pkgcore, and anything that an ebuild 
can depend on must be in an approved EAPI, approval being conditioned on 
implementation in all three PMs) would need to deal with it also.

Granted, whether it can be added to an EAPI spec is to some degree 
dependent on whether portage (as well as the other PMs) can feasibly 
support it, but you'd need to go thru the EAPI process to do it, not 
simply add it to portage.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

[gentoo-dev] Re: Files owned by multiple slots

[Sven]

Duncan <1i5t5.duncan <at> cox.net> writes:
> FWIW, that'd not be a portage issue per se, but an EAPI issue, since it

I see, very interesting! Sounds like a lengthy process, but never mind. Do you
know where EAPI changes are discussed? (Google and the Bugtracker didn't yield a
useful reply.)

Cheers, -sven

Re: [gentoo-dev] Re: Files owned by multiple slots

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 523 bytes --]

On Thu, 14 May 2009 18:34:29 +0000 (UTC)
Sven <svoop@delirium.ch> wrote:
> Duncan <1i5t5.duncan <at> cox.net> writes:
> > FWIW, that'd not be a portage issue per se, but an EAPI issue,
> > since it
> 
> I see, very interesting! Sounds like a lengthy process, but never
> mind. Do you know where EAPI changes are discussed? (Google and the
> Bugtracker didn't yield a useful reply.)

The best way is to make a carefully thought out proposal, mail it to
this list and it'll get picked up.

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 197 bytes --]

[gentoo-dev] Re: Files owned by multiple slots

[Duncan]

Sven <svoop@delirium.ch> posted loom.20090514T182756-964@post.gmane.org,
excerpted below, on  Thu, 14 May 2009 18:34:29 +0000:

> Duncan <1i5t5.duncan <at> cox.net> writes:
>> FWIW, that'd not be a portage issue per se, but an EAPI issue, since it
> 
> I see, very interesting! Sounds like a lengthy process, but never mind.
> Do you know where EAPI changes are discussed? (Google and the Bugtracker
> didn't yield a useful reply.)

Here?  (See the current GLEP55 thread, among others.)  Also the council 
list and meetings, since they must approve EAPIs before they are allowed 
in the tree.

Also see the various PMS (package management spec) threads, and the 
app-doc/pms package, for the current EAPI documentation.  From the 
package metadata, the homepage is:

http://www.gentoo.org/proj/en/qa/pms.xml

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

Re: [gentoo-dev] Re: Files owned by multiple slots

[Petteri Räty]

[-- Attachment #1: Type: text/plain, Size: 947 bytes --]

Sven wrote:
> Gilles Dartiguelongue <eva <at> gentoo.org> writes:
>>>> so this wrapper could be installed in a separate eselect sort of
>>>> package ?
>>> How exactly can this be done? If a gem creates five executables, would  
>>> this mean that this gem comes in six ebuilds?
>> well given the next answer, it sounds like that's what I'm proposing.
>> I release this might create a packaging overhead.
> 
> I'll check the docs to learn about how this could be done.
> 
> Yet meanwhile I'd love to get some feedback on my original proposition: Is it
> feasable and acceptable to expand Portage by the possibility to declare a file
> shared between slots and only delete it once the last slot is uninstalled?
> 
> Thanks for some feedback on this!   
> 
> -sven
> 
> 

This can also be accomplish by a shared dependency package so is there a
particular benefit for extending EAPI to support this?

Regards,
Petteri




[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 261 bytes --]

[gentoo-dev] Re: Files owned by multiple slots

[Sven]

> This can also be accomplish by a shared dependency package so is there a
> particular benefit for extending EAPI to support this?

If you look at it from a Gentoo-only perspecitve, there's probably no benefit.
But it would allow Gentoo to work together nicely with RubyGems.

So (apart from feasability) the question is: What's Gentoo's approach towards
other lib managers such as RubyGems?

(1)
All gems should be installed from ebuilds only.

(2)
If an ebuild requires a gem, it has to be installed from the corresponding
ebuild. For all other gems, Gentoo leaves the choice to the user and tries to
work together as well as possible with RubyGems.

Currently, reality is closer to (2) and I honestly don't believe it makes much
sense to build the infrastructure to convert all gems to ebuilds automatically
and timely enough for (1) to come true. If the delay were only hours, Ruby devs
would rather use the RubyGems manager than waiting for the ebuild to appear.

-sven

[gentoo-dev] Re: Files owned by multiple slots

[Sven]

> This can also be accomplish by a shared dependency package so is there a
> particular benefit for extending EAPI to support this?

If you look at it from a Gentoo-only perspecitve, there's probably no benefit.
But it would allow Gentoo to work together nicely with RubyGems.

So (apart from feasability) the question is: What's Gentoo's approach towards
other lib managers such as RubyGems?

(1)
All gems should be installed from ebuilds only.

(2)
If an ebuild requires a gem, it has to be installed from the corresponding
ebuild. For all other gems, Gentoo leaves the choice to the user and tries to
work together as well as possible with RubyGems.

Currently, reality is closer to (2) and I honestly don't believe it makes much
sense to build the infrastructure to convert all gems to ebuilds automatically
and timely enough for (1) to come true. If the delay were only hours, Ruby devs
would rather use the RubyGems manager than waiting for the ebuild to appear.

-sven

Re: [gentoo-dev] Re: Files owned by multiple slots

[Hans de Graaff]

[-- Attachment #1: Type: text/plain, Size: 1627 bytes --]

On Tue, 2009-06-02 at 17:43 +0000, Sven wrote:

> (1)
> All gems should be installed from ebuilds only.
> 
> (2)
> If an ebuild requires a gem, it has to be installed from the corresponding
> ebuild. For all other gems, Gentoo leaves the choice to the user and tries to
> work together as well as possible with RubyGems.
> 
> Currently, reality is closer to (2) and I honestly don't believe it makes much
> sense to build the infrastructure to convert all gems to ebuilds automatically
> and timely enough for (1) to come true. If the delay were only hours, Ruby devs
> would rather use the RubyGems manager than waiting for the ebuild to appear.

My personal preference is still to follow [1] as much as possible,
because it allows me to handle all dependencies of a particular
application within a single system. At work we have a private overlay
which contains meta-ebuilds for our web applications, and I can use that
to set up a blank server for one of our web apps with one emerge
command, or update existing servers to new dependencies.

On the other hand I also don't believe that we should provide all 1500+
gems as ebuilds, either through a manual or automatic process. Right now
our rule of thumb is to add gems when they are popular (e.g. rails),
when they provide an application that has been requested, or when they
are needed as dependencies for either one of those.

We also have a Summer of Code project to break down the monolithic
default gem installation into Gentoo phases so that we can have more
fine-grained control over the build, test, and installation process.

Hans

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 198 bytes --]