[gentoo-osx] Re: [RFC] Separate alt-prefix repo for base-system packages.

[gentoo-osx] Re: [RFC] Separate alt-prefix repo for base-system packages.

[Grobian]

Replying to the gentoo-osx mailing list here to include the people 
actually being called.  For the sake of those people that haven't read 
this yet, I retain the lengthy quotes.

Kito wrote:
> 
> On Aug 27, 2005, at 4:32 PM, Grobian wrote:
> 
>> Kito wrote:
>>> First, I'll apologize for this being a brief brain-splurt...
>>>
>>>     As the new portage grows near(savior) with support for multiple 
>>> PORTDIR, added to the increasing confusion/kludginess of the current 
>>> collision-protect mess, why do we not start a repo for the 
>>> base-system package ebuilds(bash,perl,python,readline,etc.) that 
>>> gives a proper gentoo environment in an alt-prefix?
>>
>> I can't really think of a counter argument...
>>
>>> In the interim it can simply be an overlay, using a portage snapshot, 
>>> giving us free reign to do what is needed to get the important things 
>>> working without having to worry about Apple provided files. Then with 
>>> some simple PATH mangling(think finks /sw/init) we can start making 
>>> use of it now and actually be a step ahead of the game.
>>
>> I'm lovin' this idea.  kito++;  Completely fits into my idea of a 
>> 'pilot'.
>>
>>>  Doing this outside the main tree has IMHO worked quite well for 
>>> g/fbsd project, and allowed them to get their base-system in order 
>>> before they had to go mucking up the tree with hacks that may or may 
>>> not be permanent, and is also the reason they aren't hated quite as 
>>> much as us by the other projects...
>>
>> We can't revert that.  Diego is getting some things done for us, we 
>> would need too.  So we can simply fly on his wings every now and then. 
>> Good.  Getting life when we actually have something?  Is it possible 
>> to be against that?
>>
>> So, from a managing point of view, I throw in a few things here:
>> - who is in charge/takes the lead in setting something up
>> - what are the concrete steps to take
> 
> First off, creating the repo. Path of least resistance would be adding 
> an overlay in gentoo-src/gentoo-projects/darwin/macos, but thats not 
> really the best way IMHO... Slickest way to stay as in sync as possible 
> with the main tree would need to be investigated & established...maybe 
> svn would be a candidate, not sure on that yet.

At best rely on existing architecture, with known tools.  I'm still 
frustrated that I haven't been able to install svn normally.  Where does 
Diego have his preliminary BSD stuff?

> Do the initial import of the minimum required packages from the main 
> tree, which shouldn't be too hard, basically a stage1 (see a 
> packages.build file in one of the linux profiles for instance) give or 
> take some things.
> 
> Create a new profile(s), which should probably be a complete tear down, 
> Finn Thain has had some great suggestions in this area. FINN! Care to 
> jump in here?

^^^ Finn ^^^ :D

> The user interface would need to be hashed out as well of course. How do 
> you install/bootstrap it? Where is the local configuration data stored? 
> This is an area that I think would be acceptable to take some Mac OS 
> specific indulgences, such as plists for the main config data(prefix 
> info, search paths, etc), pkgs/dmgs to bootstrap/install, and I also 
> think we should abuse the umbrella Framework mechanism when feasible.

Wow, using plists would be a first start on getting portage widely 
accepted because it includes the buzz word XML.  LOL.  On a serious 
note, I think Apple has shown XML can work somehow.  At least it has an 
open character, and it's great when you can 'script' your Keynote 
presentation by just doing string manipulation in a big XML file.
So I would say, let's try to use this horrible XML on a pilot base for 
small configuration files.  Maybe we should do it better than Apple at 
some point because their XML does not always make use of the tree 
structure of XML.  For XML I would say: only deal with it if it looks 
appropriate for the case and it is relatively easy to extract the data 
(which is often very flat, as in the .plist files).
Let's indeed make it a 'native' application for OSX users.  Native in 
the sense of how it installs and looks like.
I may give file locations a thought today, but maybe I should know the 
Framework stuff a bit better first.  Can we install the whole Gentoo 
stuff in a Framework? or is it better to just use a shortest path 
algorithm and end with /gentoo?  Actually the user should be able to 
select a disk to install to during install...

>> - who will participate into this pilot system
> 
> You can obviously count me in =)
I'd like to be part of this pilot too!

> Since we will be wanting to abuse the new hotness in portage as it 
> becomes available, the portage team will have to be involved at least a 
> little, probably whether they like it or not :p But this should bear 
> some fruit that would further portage as well IMHO, not just Mac OS.

ferringb was in our channel two days ago with a small question I could 
help him out with on rsync.  If I got it well he was related with the 
OSX project from the start to get portage going.  Maybe he's the one to 
be in it this time too?

> Some random broad philosophy/design goals that I think should be stated...:
> 
>   The repo should never ever never ever EVER rely on anything it doesn't 
> know how to supply itself with, whether that be a tool, a library, 
> knowledge of a filesystem, a host, a protocol, whatever. It doesn't 
> matter how it gets it, it just needs to know at least one way to get it. 
> This implies of course proper implementation of ferringbs BDEPENDS idea.

so, you mean if there is something (a filesystem) portage hasn't 
installed, then we should create the proper handles to deal with the OS 
provided one?  As in create a compatibility tool for "fdisk.HFS+".  I'm 
a bit clueless on how exactly you want to achieve what you want.  Maybe 
I don't understand correctly what you want exactly too.

>   Although we want this for ppc-macos at the moment, it should not be 
> specific to either of those things, ppc, or macos. Adhering to this rule 
> assumes a lot...again the BDEPENDS issue, and keeping as close to the 
> main tree as possible, with those things in place, and done 
> properly(i.e. what it REALLY takes to bootstrap an 
> {x86,amd_64,ppc,ppc64,mips,sh,whatever}-{linux,*bsd,darwin,macos,whatever} 
> toolchain) , you have a sane cross-compile ready repo, that is pretty 
> damn portable.

That would be really great.

>   Binary packages have to work. Thats a fun one. But all this done 
> properly, should allow for at least a little more consistency than the 
> current situation. I'm still sold on using xar[1] for this despite the 
> rather heavy deps (they are deps I would want in most any environment 
> anyway - libcurl,ssl,libxml,zlib), and it just fits the bill too well 
> imho, support for most standard arch specific data such bsd flags, ACLs, 
> resource forks
> ,.etc as well an excellent TOC structure that is perfect for storing 
> environment settings and package metadata.

Again XML.  If you do it XML, you have to do it all XML, something Apple 
apparently understood.  It appears we will have the blessings if we use 
XML, so I think we should.  In the end we can always dump all that XML 
into MonetDB/XQuery to have extremely fast querying over all the files, 
tree based.  I think it would seriously be the first project to use 
XQuery and XML for it's configuration.  However, if you come to think of 
it, it is a tree, an extensible tree, and might be a much more a logical 
choice than it appears to be.

>   Avoid package.provided or anything of its likeness like the plague. 
> This repo needs to describe a toolchain from the ground up, regardless 
> of the host. "What CAN it build and how?", not "What WON'T the host OS 
> let me do".

Uhm, yes please!

>   Keep the number of ebuilds to a bare minimum. We can't get too carried 
> away with maintaining a complete tree, or we risk drifting too far 
> downstream and the zealots pushing Darwin/Mac OS support out of the main 
> tree entirely. That would be bad. This can't go in the direction of a 
> fork, just an experimental branch that will be merged back in at some 
> point.

Ok, this requires identification of the main packages we need.  I think 
along the lines of Python, Perl, bash and such.  We should come up with 
a (preliminary|complete)? list.

>> - glasjnost and perestroika please!
>>
>> I think it's important to have a fairly focussed pilot shared by just 
>> a very few devs to figure out how to get it set up and deal with it.
> 
> I agree, the less noise, the more work will get done. Politics entering 
> even a little will kill any hope of progress and momentum.

It appears we're on the same wave length here ;)

> 
> --Kito
> 
> [1] http://www.opendarwin.org/projects/xar/

-- 
Fabian Groffen
Gentoo for Mac OS X
-- 
gentoo-osx@gentoo.org mailing list

[gentoo-osx] Re: [RFC] Separate alt-prefix repo for base-system packages.

[Finn Thain]

On Mon, 29 Aug 2005, Grobian wrote:

> Kito wrote:
> > 
> > On Aug 27, 2005, at 4:32 PM, Grobian wrote:
> > 

[snip]

> 
> > Do the initial import of the minimum required packages from the main 
> > tree, which shouldn't be too hard, basically a stage1 (see a 
> > packages.build file in one of the linux profiles for instance) give or 
> > take some things.
> > 
> > Create a new profile(s), which should probably be a complete tear 
> > down, Finn Thain has had some great suggestions in this area. FINN! 
> > Care to jump in here?
> 
> ^^^ Finn ^^^ :D

Sure. The idea was this profile inheriance tree:


                                 .-> ppc-od
                 .-> ppc-darwin <
                /                `-> ppc-macos
default-darwin <
                \                .-> x86-od
                 `-> x86-darwin <
                                 `-> x86-macos


The purpose being, "progressive" and "collision-protect" are best given 
their own profiles in order to avoid having ebuilds test for the 
"collision-protect" feature.

So, I was advocating that an "upstream darwin" profile, {x86,ppc}-darwin, 
should be used for anything that didn't bow down to mac os. Hence, the 
ppc-macos profile would be taken to mean "behave as a secondary package 
manager: play nice with mac os, and test the mac os package database for 
dependencies, and use the mac os installer to install them if they are 
available in a .pkg".

At the moment, ppc-macos would need the collision-protect feature, but 
once we have a feature for prefixed installs, that should be used instead 
(by default, at least).

> > The user interface would need to be hashed out as well of course. How 
> > do you install/bootstrap it?

It would be nice to have ebuilds that could invoke the Darwin Build 
Scripts (and merge the result on a ROOT).

http://www.opendarwin.org/projects/darwinbuild/releases/

Given such ebuilds, surely catalyst can bootstrap it.

> > Where is the local configuration data stored? This is an area that I 
> > think would be acceptable to take some Mac OS specific indulgences, 
> > such as plists for the main config data(prefix info, search paths, 
> > etc), pkgs/dmgs to bootstrap/install, and I also think we should abuse 
> > the umbrella Framework mechanism when feasible.
> 
> Wow, using plists would be a first start on getting portage widely 
> accepted because it includes the buzz word XML.  LOL.  On a serious 
> note, I think Apple has shown XML can work somehow.  At least it has an 
> open character, and it's great when you can 'script' your Keynote 
> presentation by just doing string manipulation in a big XML file.
>
> So I would say, let's try to use this horrible XML on a pilot base for 
> small configuration files.  Maybe we should do it better than Apple at 
> some point because their XML does not always make use of the tree 
> structure of XML.  For XML I would say: only deal with it if it looks 
> appropriate for the case and it is relatively easy to extract the data 
> (which is often very flat, as in the .plist files).

I reckon XML is important, though perhaps not in the way you describe. As 
I see it, where ever portage is deployed as a secondary package manager, 
it needs to consult the primary one. That means that there needs to be a 
standard protocol for one package manager to query another.

In the mac os case, we can write a shim to talk XML to portage.

(BTW, a similar protocol, but allowing one package manager to _update_ 
another could be used to make a universal binary installer. But it would 
need rpm, apt-get, portage etc to all implement a standard XML protocol 
and API. But this is only really interesting in a closed source context. 
And I've digressed. Mentioning XML just reminded me.)

> Let's indeed make it a 'native' application for OSX users.  Native in 
> the sense of how it installs and looks like. I may give file locations a 
> thought today, but maybe I should know the Framework stuff a bit better 
> first.  Can we install the whole Gentoo stuff in a Framework? or is it 
> better to just use a shortest path algorithm and end with /gentoo?  
> Actually the user should be able to select a disk to install to during 
> install...

I reckon get it working (with an "upstream darwin" profile) in a chroot 
stage first (which could double as a boot disk).

[snip]

> > Some random broad philosophy/design goals that I think should be 
> > stated...:
> > 
> >   The repo should never ever never ever EVER rely on anything it 
> > doesn't know how to supply itself with, whether that be a tool, a 
> > library, knowledge of a filesystem, a host, a protocol, whatever. It 
> > doesn't matter how it gets it, it just needs to know at least one way 
> > to get it. This implies of course proper implementation of ferringbs 
> > BDEPENDS idea.
>
> so, you mean if there is something (a filesystem) portage hasn't 
> installed, then we should create the proper handles to deal with the OS 
> provided one?  As in create a compatibility tool for "fdisk.HFS+".  I'm 
> a bit clueless on how exactly you want to achieve what you want.  Maybe 
> I don't understand correctly what you want exactly too.

This is how I would handle that case:

If a program (say fsck_hfs) is available upstream, build it with Dawin 
Build, merge it with portage (I expect an eclass for darwin build is 
required, and of course an ebuild for diskdev_cmds.)

If a package (say macext2fs) is available as a .pkg or .dmg, download it 
with portage, and install it with mac os installer(8). Again, an eclass to 
handle these would be useful (this is probably how they are handled 
already?)

> >   Although we want this for ppc-macos at the moment, it should not be 
> > specific to either of those things, ppc, or macos. Adhering to this 
> > rule assumes a lot...again the BDEPENDS issue, and keeping as close to 
> > the main tree as possible, with those things in place, and done 
> > properly(i.e. what it REALLY takes to bootstrap an 
> > {x86,amd_64,ppc,ppc64,mips,sh,whatever}-{linux,*bsd,darwin,macos,whatever} 
> > toolchain) , you have a sane cross-compile ready repo, that is pretty 
> > damn portable.

Darwin is not self-hosting (out of the box). You need a carefully 
contrived a build environment to make it so, and this is what the Darwin 
Build Scripts do: install that build environment, and automate the builds.

Darwin build uses plists to describe all the packages in a mac os release, 
along with build dependencies, and so forth. These build dependencies are 
confined to the darwin build chroot, as I understand it. Portage wouldn't 
have to know about them, I think it just has to do the merge of the build 
product and install the runtime dependencies thereof (of course).

> That would be really great.
> 
> >   Binary packages have to work. Thats a fun one. But all this done 
> > properly, should allow for at least a little more consistency than the 
> > current situation. I'm still sold on using xar[1] for this despite the 
> > rather heavy deps (they are deps I would want in most any environment 
> > anyway - libcurl,ssl,libxml,zlib), and it just fits the bill too well 
> > imho, support for most standard arch specific data such bsd flags, 
> > ACLs, resource forks ,.etc as well an excellent TOC structure that is 
> > perfect for storing environment settings and package metadata.
> 
> Again XML.  If you do it XML, you have to do it all XML, something Apple 
> apparently understood.  It appears we will have the blessings if we use 
> XML, so I think we should.  In the end we can always dump all that XML 
> into MonetDB/XQuery to have extremely fast querying over all the files, 
> tree based. I think it would seriously be the first project to use 
> XQuery and XML for it's configuration.  However, if you come to think of 
> it, it is a tree, an extensible tree, and might be a much more a logical 
> choice than it appears to be.

Why not use one of the open source .pkg tools to generate binary packages? 
Kito tells me he has already been able to unpack .pkg format and install 
it via portage.

> >   Avoid package.provided or anything of its likeness like the plague. 
> > This repo needs to describe a toolchain from the ground up, regardless 
> > of the host. "What CAN it build and how?", not "What WON'T the host OS 
> > let me do".
> 
> Uhm, yes please!

Hear, hear!

package.provided must die. In the long run, it is not sustainable to 
attempt to use profiles to describe a moving target. If you do that, you 
end up with not one profile for 10.4, but profiles for 10.4, 10.4.1 and 
10.4.2 because, for example, 10.4.2 fixed a bug in apple's xyz package 
that forced apples xyz to be masked in the 10.4.1 profile.

And then you still have the problem that the user may actually be running 
10.4.6, thanks to a recent Software Update, while the latest profile is at 
10.4.4, thanks to an old emerge --sync, all while /etc/make.profile is 
still a symlink to the 10.4.1 profile, thanks to the user neglecting to 
re-point it since installation!

> >   Keep the number of ebuilds to a bare minimum. We can't get too 
> > carried away with maintaining a complete tree, or we risk drifting too 
> > far downstream and the zealots pushing Darwin/Mac OS support out of 
> > the main tree entirely. That would be bad. This can't go in the 
> > direction of a fork, just an experimental branch that will be merged 
> > back in at some point.

IMHO, this sounds like a "gentoo-darwin" sub-project to gentoo-alt, 
along-side os x and bsd. It isn't really a fork except in as much as the 
profile arrangement doesn't really accomodate work on darwin proper (but 
then the profile arrangemnet is flawed anyway: it only exists this way 
because of the package.provided crutch).

-f
-- 
gentoo-osx@gentoo.org mailing list

Re: [gentoo-osx] Re: [RFC] Separate alt-prefix repo for base-system packages.

[Kito]

On Aug 29, 2005, at 5:30 AM, Finn Thain wrote:

>
>
> On Mon, 29 Aug 2005, Grobian wrote:
>
>> Kito wrote:
>>>
>>> On Aug 27, 2005, at 4:32 PM, Grobian wrote:
>
> At the moment, ppc-macos would need the collision-protect feature, but
> once we have a feature for prefixed installs, that should be used  
> instead
> (by default, at least).
>
>>> The user interface would need to be hashed out as well of course.  
>>> How
>>> do you install/bootstrap it?
>
> It would be nice to have ebuilds that could invoke the Darwin Build
> Scripts (and merge the result on a ROOT).

Yeap. already planned on using this to build a libSystem, etc.

>
> http://www.opendarwin.org/projects/darwinbuild/releases/
>
> Given such ebuilds, surely catalyst can bootstrap it.
>
>>> Where is the local configuration data stored? This is an area that I
>>> think would be acceptable to take some Mac OS specific indulgences,
>>> such as plists for the main config data(prefix info, search paths,
>>> etc), pkgs/dmgs to bootstrap/install, and I also think we should  
>>> abuse
>>> the umbrella Framework mechanism when feasible.
>>
>> Wow, using plists would be a first start on getting portage widely
>> accepted because it includes the buzz word XML.  LOL.  On a serious
>> note, I think Apple has shown XML can work somehow.  At least it  
>> has an
>> open character, and it's great when you can 'script' your Keynote
>> presentation by just doing string manipulation in a big XML file.
>>
>> So I would say, let's try to use this horrible XML on a pilot base  
>> for
>> small configuration files.  Maybe we should do it better than  
>> Apple at
>> some point because their XML does not always make use of the tree
>> structure of XML.  For XML I would say: only deal with it if it looks
>> appropriate for the case and it is relatively easy to extract the  
>> data
>> (which is often very flat, as in the .plist files).
>
> I reckon XML is important, though perhaps not in the way you  
> describe. As
> I see it, where ever portage is deployed as a secondary package  
> manager,
> it needs to consult the primary one. That means that there needs to  
> be a
> standard protocol for one package manager to query another.
>

I'm not sure I agree. I think this gets too close to a  
package.provided situation, portage will never know enough about  
another systems packages to map their functionality to its own. I  
think its preferable to let portage handle what it knows first hand  
before trying to force it data from a foreign host.

>
>> Let's indeed make it a 'native' application for OSX users.  Native in
>> the sense of how it installs and looks like. I may give file  
>> locations a
>> thought today, but maybe I should know the Framework stuff a bit  
>> better
>> first.  Can we install the whole Gentoo stuff in a Framework? or  
>> is it
>> better to just use a shortest path algorithm and end with /gentoo?
>> Actually the user should be able to select a disk to install to  
>> during
>> install...
>
> I reckon get it working (with an "upstream darwin" profile) in a  
> chroot
> stage first (which could double as a boot disk).
>

I want to start a lot smaller than that first. Think stage1. You  
can't boot a stage1, its a just the corelibs and a toolchain pretty  
much.



>>>   The repo should never ever never ever EVER rely on anything it
>>> doesn't know how to supply itself with, whether that be a tool, a
>>> library, knowledge of a filesystem, a host, a protocol, whatever. It
>>> doesn't matter how it gets it, it just needs to know at least one  
>>> way
>>> to get it. This implies of course proper implementation of ferringbs
>>> BDEPENDS idea.
>>
>> so, you mean if there is something (a filesystem) portage hasn't
>> installed, then we should create the proper handles to deal with  
>> the OS
>> provided one?  As in create a compatibility tool for "fdisk.HFS 
>> +".  I'm
>> a bit clueless on how exactly you want to achieve what you want.   
>> Maybe
>> I don't understand correctly what you want exactly too.
>
> This is how I would handle that case:
>
> If a program (say fsck_hfs) is available upstream, build it with Dawin
> Build, merge it with portage (I expect an eclass for darwin build is
> required, and of course an ebuild for diskdev_cmds.)

About what I was thinking...

>>
>>>   Binary packages have to work. Thats a fun one. But all this done
>>> properly, should allow for at least a little more consistency  
>>> than the
>>> current situation. I'm still sold on using xar[1] for this  
>>> despite the
>>> rather heavy deps (they are deps I would want in most any  
>>> environment
>>> anyway - libcurl,ssl,libxml,zlib), and it just fits the bill too  
>>> well
>>> imho, support for most standard arch specific data such bsd flags,
>>> ACLs, resource forks ,.etc as well an excellent TOC structure  
>>> that is
>>> perfect for storing environment settings and package metadata.
>>
>> Again XML.  If you do it XML, you have to do it all XML, something  
>> Apple
>> apparently understood.  It appears we will have the blessings if  
>> we use
>> XML, so I think we should.  In the end we can always dump all that  
>> XML
>> into MonetDB/XQuery to have extremely fast querying over all the  
>> files,
>> tree based. I think it would seriously be the first project to use
>> XQuery and XML for it's configuration.  However, if you come to  
>> think of
>> it, it is a tree, an extensible tree, and might be a much more a  
>> logical
>> choice than it appears to be.
>
> Why not use one of the open source .pkg tools to generate binary  
> packages?
> Kito tells me he has already been able to unpack .pkg format and  
> install
> it via portage.

Thats just to get extract files...I'm talking about binary packages  
that portage can use. I don't like the current tbz2 hack.

I have no interest personally in producing packages with a  
proprietary format for portage. Be a nice feature for OS X users and  
devs sure, but thats more like an add-on bells-and-whistles type  
feature... patches accepted ;)

>
>>>   Avoid package.provided or anything of its likeness like the  
>>> plague.
>>> This repo needs to describe a toolchain from the ground up,  
>>> regardless
>>> of the host. "What CAN it build and how?", not "What WON'T the  
>>> host OS
>>> let me do".
>>
>> Uhm, yes please!
>
> Hear, hear!
>
>>>   Keep the number of ebuilds to a bare minimum. We can't get too
>>> carried away with maintaining a complete tree, or we risk  
>>> drifting too
>>> far downstream and the zealots pushing Darwin/Mac OS support out of
>>> the main tree entirely. That would be bad. This can't go in the
>>> direction of a fork, just an experimental branch that will be merged
>>> back in at some point.
>
> IMHO, this sounds like a "gentoo-darwin" sub-project to gentoo-alt,
> along-side os x and bsd. It isn't really a fork except in as much  
> as the
> profile arrangement doesn't really accomodate work on darwin proper  
> (but
> then the profile arrangemnet is flawed anyway: it only exists this way
> because of the package.provided crutch).

I was looking at it more as a place to develop some new portage  
features...Gentoo/Darwin has always been lurking, this is more in the  
area of just getting offsets working.

>
> -f
> -- 
> gentoo-osx@gentoo.org mailing list
>

-- 
gentoo-osx@gentoo.org mailing list

Re: [gentoo-osx] Re: [RFC] Separate alt-prefix repo for base-system packages.

[Finn Thain]

> On Aug 29, 2005, at 5:30 AM, Finn Thain wrote:
> 
> >I reckon XML is important, though perhaps not in the way you describe. 
> >As I see it, where ever portage is deployed as a secondary package 
> >manager, it needs to consult the primary one. That means that there 
> >needs to be a standard protocol for one package manager to query 
> >another.

(I should have mentioned, you don't really need XML for this, you can 
generate /etc/package.provided by a wrapper around emerge. Then you can 
mask untested stuff in it using a single ppc-macos profile and avoid the 
bogus package.provided files in all the present macos profiles.)

> I'm not sure I agree. I think this gets too close to a package.provided 
> situation, portage will never know enough about another systems packages 
> to map their functionality to its own. I think its preferable to let 
> portage handle what it knows first hand before trying to force it data 
> from a foreign host.

I'm not proposing that one "injects" non-identical packages under the same 
names. Actually, I have been against that since the beginning.

I was thinking of something like, at run time, query the vendor package 
manager and use the result to populate the tree with packages like 
vendor-apple/sys-devel/xcode-1.5, vendor-sun/app-arch/cpio-x.y.z for 
example (please substitute sgi, bsd-ports, redhat or debian etc if you are 
hostile to any of my examples).

Apple's XCode is closed source, and sun's cpio is now open. The former 
requires an ebuild to invoke installer(8), the latter requires an ebuild 
to build it from source. No-one is lying to portage here.

And, if sys-apps/bsd-awk-x.y.z builds the same thing that apple ships, it 
can provide vendor-apple/sys-apps/bsd-awk.

Also, the ebuilds for both vendor-apple/sys-apps/bsd-awk and 
sys-apps/bsd-awk should provide virtual/awk. So, when arbitrary ebuild foo 
wants generic awk (doesn't care about gnu extensions), it can depend on 
that virtual (unless virtuals are to be deprecated, in which case foo 
somehow has to depend on any vendor, including gentoo).

> >IMHO, this sounds like a "gentoo-darwin" sub-project to gentoo-alt, 
> >along-side os x and bsd. It isn't really a fork except in as much as 
> >the profile arrangement doesn't really accomodate work on darwin proper 
> >(but then the profile arrangemnet is flawed anyway: it only exists this 
> >way because of the package.provided crutch).
> 
> I was looking at it more as a place to develop some new portage 
> features...Gentoo/Darwin has always been lurking, this is more in the 
> area of just getting offsets working.

OK, I see what you are getting at now. That was something that I failed to 
infer from the email you forwarded to the list. Most of what I said in 
reply isn't very relevant to that. Excepting that, if you can leverage 
existing packages, prefixed installs are much more useful -- having a 
complete set of deps installed on a prefix is not much better than a 
stage3 chroot with your home directory bind mounted below it.

-f
-- 
gentoo-osx@gentoo.org mailing list

Re: [gentoo-osx] Re: [RFC] Separate alt-prefix repo for base-system packages.

[Brian Harring]

[-- Attachment #1: Type: text/plain, Size: 3769 bytes --]

Sidenote, no need to cc, just joined the ml for the discussion in the 
meantime since people occasionally forget to cc (I know I do).

On Tue, Aug 30, 2005 at 01:09:49PM +1000, Finn Thain wrote:
> > I'm not sure I agree. I think this gets too close to a package.provided 
> > situation, portage will never know enough about another systems packages 
> > to map their functionality to its own. I think its preferable to let 
> > portage handle what it knows first hand before trying to force it data 
> > from a foreign host.
> 
> I'm not proposing that one "injects" non-identical packages under the same 
> names. Actually, I have been against that since the beginning.
> 
> I was thinking of something like, at run time, query the vendor package 
> manager and use the result to populate the tree with packages like 
> vendor-apple/sys-devel/xcode-1.5, vendor-sun/app-arch/cpio-x.y.z for 
> example (please substitute sgi, bsd-ports, redhat or debian etc if you are 
> hostile to any of my examples).
> 
> Apple's XCode is closed source, and sun's cpio is now open. The former 
> requires an ebuild to invoke installer(8), the latter requires an ebuild 
> to build it from source. No-one is lying to portage here.
> 
> And, if sys-apps/bsd-awk-x.y.z builds the same thing that apple ships, it 
> can provide vendor-apple/sys-apps/bsd-awk.
> 
> Also, the ebuilds for both vendor-apple/sys-apps/bsd-awk and 
> sys-apps/bsd-awk should provide virtual/awk. So, when arbitrary ebuild foo 
> wants generic awk (doesn't care about gnu extensions), it can depend on 
> that virtual (unless virtuals are to be deprecated, in which case foo 
> somehow has to depend on any vendor, including gentoo).
The rewrite's domain's abstraction (additionally the goal of binding 
the resolver to the domain, and being able to do inter-domain 
resolving) would allow for this, but I *really* don't think it'll work 
well.

Reasoning is, how do you know that pkg xyz is actually the package 
you're after?  The expanded restrictions subsystem, specifically 
ability to depends based on contents restrictions (I want the pkg that 
owns file abc essentially) gives basic ability for this, but it 
doesn't cover the abi angle.

What you're proposing could sort of be hacked together to pull off 
strictly for src compiles, probably with a good collection of 
impossible to quash annoying bugs.  Doing it for binaries is a helluva 
lot harder though. :)

> > >IMHO, this sounds like a "gentoo-darwin" sub-project to gentoo-alt, 
> > >along-side os x and bsd. It isn't really a fork except in as much as 
> > >the profile arrangement doesn't really accomodate work on darwin proper 
> > >(but then the profile arrangemnet is flawed anyway: it only exists this 
> > >way because of the package.provided crutch).
> > 
> > I was looking at it more as a place to develop some new portage 
> > features...Gentoo/Darwin has always been lurking, this is more in the 
> > area of just getting offsets working.
> 
> OK, I see what you are getting at now. That was something that I failed to 
> infer from the email you forwarded to the list. Most of what I said in 
> reply isn't very relevant to that. Excepting that, if you can leverage 
> existing packages, prefixed installs are much more useful -- having a 
> complete set of deps installed on a prefix is not much better than a 
> stage3 chroot with your home directory bind mounted below it.

The rewrite's general core is intended to allow for alt 
formats/repos/whatever jammed into it; that said, making seperate 
formats play nice with each other (unless they can natively) isn't 
something I think is incredibly easy to pull off, as mentioned above.

Thoughts?
~harring

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-osx] Re: [RFC] Separate alt-prefix repo for base-system packages.

[Finn Thain]

On Mon, 29 Aug 2005, Brian Harring wrote:

> The rewrite's domain's abstraction (additionally the goal of binding the 
> resolver to the domain, and being able to do inter-domain resolving) 
> would allow for this, but I *really* don't think it'll work well.
>
> Reasoning is, how do you know that pkg xyz is actually the package 
> you're after?

Not sure I understand the problem... in glep 37, a virtual is replaced 
with a meta-package having a list of depends alternatives. If you add the 
vendor package to that list, and then make the vendor packages 
"package.preferred" where portage is a secondary package manager, surely 
there is no question that package xyz is the one you're after?

(I'm assuming all repos share the same namespace WRT package names... if 
cpv's do not have to be unique accross repos, other solutions might be 
possible...)

> The expanded restrictions subsystem, specifically ability to depends 
> based on contents restrictions (I want the pkg that owns file abc 
> essentially) gives basic ability for this

I don't understand why portage would go looking for deps in the 
filesystem. If configure can't find them, does it help that portage can?

The last I read about this was here...
http://article.gmane.org/gmane.linux.gentoo.devel/28154
I guess there is more too it?

> but it doesn't cover the abi angle.

I hadn't considered the ABI problem. In the short term I think a synthetic 
package.provided is okay, since most multilib machines cater to the 
ABI-ignorant.

But, generating package.provided with a wrapper is a hack, so long term, a 
vendor package manager (or a shim on top of it) would have to state what 
ABI was used for each package, when queried. Knowing the ABIs of the 
vendor packages, and knowing the native ABI of the target domain (?), the 
resolver could probably do the right thing...

> What you're proposing could sort of be hacked together to pull off 
> strictly for src compiles, probably with a good collection of impossible 
> to quash annoying bugs.  Doing it for binaries is a helluva lot harder 
> though. :)

I think these are the bugs which have been the gentoo-osx team's burden 
already... brave souls that they are :-)

> The rewrite's general core is intended to allow for alt 
> formats/repos/whatever jammed into it; that said, making seperate 
> formats play nice with each other (unless they can natively) isn't 
> something I think is incredibly easy to pull off, as mentioned above.
>
> Thoughts?

I probably need a better handle on the constraints of the relations 
between domains, repos, prefixes... anyway, here is my naive partial 
config. This config is premature, of course, but it might help me 
understand what is and is not possible in the rewrite if you would kindly 
shoot some holes in it ;-)

Notes: I've used a hypothetical feature for collecting information about 
vendor packages -- presumably the apple domain also needs an ephemeral vdb 
to hold it (like package.provided). I don't know if this vendor domain 
needs a repo. I guess the main repo would have to mask broken/testing 
vendor deps as it sees fit, including those from other domains...

-f


[vdb]
type = repo
class = portage.installed_pkg.repository
path = 

[rsync repo]
type = repo 
class = portage.ebuild.repository
path = /opt/gentoo/usr/portage

[ppc-macos]
type = config
ACCEPT_KEYWORDS = "ppc-macos"

[default domain]
type = domain
root = "/opt/gentoo"
repositories = 'rsync repo' vdb
config = ppc-macos

[vendor-apple]
type = config
FEATURES = "query-apple-packages"
ACCEPT_KEYWORDS = "ppc-darwin"

[apple domain]
type = domain
root = "/"
config = vendor-apple



-- 
gentoo-osx@gentoo.org mailing list

Re: [gentoo-osx] Re: [RFC] Separate alt-prefix repo for base-system packages.

[Brian Harring]

[-- Attachment #1: Type: text/plain, Size: 4522 bytes --]

On Tue, Aug 30, 2005 at 09:33:36PM +1000, Finn Thain wrote:
> On Mon, 29 Aug 2005, Brian Harring wrote:
> 
<inserting the relevant quote>
> > > I was thinking of something like, at run time, query the vendor package
> > > manager and use the result to populate the tree with packages like
> > > vendor-apple/sys-devel/xcode-1.5, vendor-sun/app-arch/cpio-x.y.z for
> > > example (please substitute sgi, bsd-ports, redhat or debian etc if 
> > > you are hostile to any of my examples).
> >
> > The rewrite's domain's abstraction (additionally the goal of binding the 
> > resolver to the domain, and being able to do inter-domain resolving) 
> > would allow for this, but I *really* don't think it'll work well.
> >
> > Reasoning is, how do you know that pkg xyz is actually the package 
> > you're after?
Re-inserted the quote to clarify what I'm talking about; mapping 
another pkg managers db into our own requires either a lot of human 
intervention, or some dodgy rules that somewhat manage it, with bugs.

> Not sure I understand the problem... in glep 37, a virtual is replaced 
> with a meta-package having a list of depends alternatives. If you add the 
> vendor package to that list, and then make the vendor packages 
> "package.preferred" where portage is a secondary package manager, surely 
> there is no question that package xyz is the one you're after?
> 
> (I'm assuming all repos share the same namespace WRT package names... if 
> cpv's do not have to be unique accross repos, other solutions might be 
> possible...)
If you're managing a list of provided packages, my points don't 
matter; points were strictly in reference to trying to have portage 
query another pkg manager (primary or otherwise) :)

> > The expanded restrictions subsystem, specifically ability to depends 
> > based on contents restrictions (I want the pkg that owns file abc 
> > essentially) gives basic ability for this
> 
> I don't understand why portage would go looking for deps in the 
> filesystem. If configure can't find them, does it help that portage can?
Quote above is in reference to the (potentially incorrectly perceived) 
idea of directly querying another pkg manager.

> The last I read about this was here...
> http://article.gmane.org/gmane.linux.gentoo.devel/28154
> I guess there is more too it?
Haubi's glep/patches were moreso for pulling off shifted presets, 
relevant to the shifted preset goal's y'all have, but not pertinent to 
portion of the discussion involving attempting to query apple's 
installed db (which is the proposal you put forth) :)

> But, generating package.provided with a wrapper is a hack, so long term, a 
> vendor package manager (or a shim on top of it)
My question/concerns raised have been regarding the potential 
shim/wrapper.  How do you wrap their pkg manager's namespace into 
portages namespace?  To do so you need to either manage a list of 
mappings yourself, or come up with rules (which will have exceptions).

That's what I'm getting at; people don't support interop between dpkg 
and rpm (fex) due to the fact the namespaces are different (although 
it's possible to do so if you account for abi).

> I probably need a better handle on the constraints of the relations 
> between domains, repos, prefixes... anyway, here is my naive partial 
> config. This config is premature, of course, but it might help me 
> understand what is and is not possible in the rewrite if you would kindly 
> shoot some holes in it ;-)
> 
> Notes: I've used a hypothetical feature for collecting information about 
> vendor packages -- presumably the apple domain also needs an ephemeral vdb 
> to hold it (like package.provided). I don't know if this vendor domain 
> needs a repo. I guess the main repo would have to mask broken/testing 
> vendor deps as it sees fit, including those from other domains...
Offhand, what you're proposing (querying the vendor installed pkg db) 
is actually a readonly vdb, if it were implemented.
It's an installed package database you can query (use for solving 
deps), but cannot modify.

> [vendor-apple]
> type = config
> FEATURES = "query-apple-packages"
> ACCEPT_KEYWORDS = "ppc-darwin"
Definitions not totally right as mentioned (doesn't matter, just being 
nitpicky :), but the crux of my concern is embodied inthe 
FEATURES="query-apple-packages".

Mainly, how?  How to query the vendor db, and map that back into 
portage package namespace?

~harring

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-osx] Re: [RFC] Separate alt-prefix repo for base-system packages.

[Finn Thain]

On Tue, 30 Aug 2005, Brian Harring wrote:

> On Tue, Aug 30, 2005 at 09:33:36PM +1000, Finn Thain wrote:
> > On Mon, 29 Aug 2005, Brian Harring wrote:
> > 
> <inserting the relevant quote>
> > > > I was thinking of something like, at run time, query the vendor 
> > > > package manager and use the result to populate the tree with 
> > > > packages like vendor-apple/sys-devel/xcode-1.5, 
> > > > vendor-sun/app-arch/cpio-x.y.z for example (please substitute sgi, 
> > > > bsd-ports, redhat or debian etc if you are hostile to any of my 
> > > > examples).
> > >
> > > The rewrite's domain's abstraction (additionally the goal of binding 
> > > the resolver to the domain, and being able to do inter-domain 
> > > resolving) would allow for this, but I *really* don't think it'll 
> > > work well.
> > >
> > > Reasoning is, how do you know that pkg xyz is actually the package 
> > > you're after?
> Re-inserted the quote to clarify what I'm talking about; mapping another 
> pkg managers db into our own requires either a lot of human 
> intervention, or some dodgy rules that somewhat manage it, with bugs.

OK, I see what you mean. You're asking, how does portage know that vendor 
package xyz is the portage package abc?

Short answer is package.mask, meta-packages and name mapping.

A particular vendor package version is a known-good dep, as tested by 
devs, otherwise it is masked. E.g. package.mask says
>vendor-sun/app-arch/cpio-x.y.z if no higher version has been tested. In
mac os, automated updates mean that most of the time, there will be some 
vendor packages that the tree hasn't been tested against. These have to be 
masked until the user does emerge sync.

Another part of the problem is solved by meta-packages. If the vendor dep 
is still masked after emerge sync, it can't be used, and the next most 
"package.preferred" dep will be built and installed straight from the tree 
(on a prefix). Once the vendor dep is unmasked, depclean will remove it 
again.

Virtuals/meta-packages already permit a bit of slack. The packages that 
provide a virtual are not the same, of course, so any ebuild using a 
virtual/meta-package as a dep is not going to be too fussy (e.g. needs awk 
but not gnu extensions). vendor packages are a good fit here.

> > Not sure I understand the problem... in glep 37, a virtual is replaced 
> > with a meta-package having a list of depends alternatives. If you add 
> > the vendor package to that list, and then make the vendor packages 
> > "package.preferred" where portage is a secondary package manager, 
> > surely there is no question that package xyz is the one you're after?
> > 
> > (I'm assuming all repos share the same namespace WRT package names... 
> > if cpv's do not have to be unique accross repos, other solutions might 
> > be possible...)
> If you're managing a list of provided packages, my points don't matter; 
> points were strictly in reference to trying to have portage query 
> another pkg manager (primary or otherwise) :)

BTW, do repos share a namespace? Presented with the same cpv in several 
repos, is portage's behaviour defined yet?

> > > The expanded restrictions subsystem, specifically ability to depends 
> > > based on contents restrictions (I want the pkg that owns file abc 
> > > essentially) gives basic ability for this
> > 
> > I don't understand why portage would go looking for deps in the 
> > filesystem. If configure can't find them, does it help that portage 
> > can?
> Quote above is in reference to the (potentially incorrectly perceived) 
> idea of directly querying another pkg manager.
> 
> > The last I read about this was here... 
> > http://article.gmane.org/gmane.linux.gentoo.devel/28154 I guess there 
> > is more too it?
> Haubi's glep/patches were moreso for pulling off shifted presets, 
> relevant to the shifted preset goal's y'all have, but not pertinent to 
> portion of the discussion involving attempting to query apple's 
> installed db (which is the proposal you put forth) :)
> 
> > But, generating package.provided with a wrapper is a hack, so long 
> > term, a vendor package manager (or a shim on top of it)
> My question/concerns raised have been regarding the potential 
> shim/wrapper.  How do you wrap their pkg manager's namespace into 
> portages namespace?  To do so you need to either manage a list of 
> mappings yourself, or come up with rules (which will have exceptions).

My feeling is that the burden of managing the mappings is better than the 
burden of managing one package.provided for mac os 10.3, alongside another 
for 10.4, etc. (If I'm wrong about that, then this exercise is pointless.)

In some cases the mapping is trivial: take the closed source XCode 
package. I would have an ebuild called vendor-apple/sys-devel/xcode-1.5 to 
install the DeveloperTools.pkg file, from a distfile taken from apple's 
cd. So this ebuild would contain the forward mapping, by storing the 
vendor's name in it's env. (Same applies to solaris pkgadd and irix 
swinst...)

So the same ebuild also embodies the reverse mapping, and portage can map 
a vendor's package-version to the ebuild itself, and populate a vdb with 
the ebuild's PV. IOW, the ebuilds themselves provide the 1:M mapping of 
vendor name to portage names.

Several vendor packages are sometimes required to make up one of portage's 
deps (e.g. vendor-apple/x11-base/xfree would comprise both of X11SDK.pkg 
and X11User.pkg). That ebuild would install two vendor packages and 
contain two vendor names (basically an N:1 mapping).

Now, what if there is no ebuild to match a vendor package? No mapping. Dep 
has to come from the tree instead. Not a big deal for a secondary package 
manager.

But what if you want dev-tcltk/tcllib to satisfy an explicit dep on 
vendor-apple/dev-tcltk/tcllib? You can't. But I can't see this being a 
problem. (Though, you could have a metapackage for tcllib and 
package.prefer dev-tcltk/tcllib in your profile.)

I would say that explicit deps on vendor packages can be avoided whereever 
said package is open source and we have an ebuild to build it.

Of the open source stuff, those packages that are built from vendor 
sources (or patches), are not "vendor packages" in the sense I've used the 
term, that is, they don't get installed by a foriegn package manager, and 
they don't live under /vendor-*. They are just regular ebuilds.

> That's what I'm getting at; people don't support interop between dpkg 
> and rpm (fex) due to the fact the namespaces are different (although 
> it's possible to do so if you account for abi).
> 
> > I probably need a better handle on the constraints of the relations 
> > between domains, repos, prefixes... anyway, here is my naive partial 
> > config. This config is premature, of course, but it might help me 
> > understand what is and is not possible in the rewrite if you would 
> > kindly shoot some holes in it ;-)
> > 
> > Notes: I've used a hypothetical feature for collecting information 
> > about vendor packages -- presumably the apple domain also needs an 
> > ephemeral vdb to hold it (like package.provided). I don't know if this 
> > vendor domain needs a repo. I guess the main repo would have to mask 
> > broken/testing vendor deps as it sees fit, including those from other 
> > domains...
> Offhand, what you're proposing (querying the vendor installed pkg db) is 
> actually a readonly vdb, if it were implemented. It's an installed 
> package database you can query (use for solving deps), but cannot 
> modify.
> 
> > [vendor-apple] type = config FEATURES = "query-apple-packages" 
> > ACCEPT_KEYWORDS = "ppc-darwin"
> Definitions not totally right as mentioned (doesn't matter, just being 
> nitpicky :), but the crux of my concern is embodied inthe 
> FEATURES="query-apple-packages".
> 
> Mainly, how?  How to query the vendor db, and map that back into portage 
> package namespace?

Did I read something about the rewrite being modular? Could the shim/query 
take the form of a portage plugin that implements the query-apple-packages 
feature? Obviously, if implemented the way I descibed above, it would need 
to be intimate with certain ebuilds' environments.

-f

> 
> ~harring
> 
-- 
gentoo-osx@gentoo.org mailing list

Re: [gentoo-osx] Re: [RFC] Separate alt-prefix repo for base-system packages.

[Brian Harring]

[-- Attachment #1: Type: text/plain, Size: 2572 bytes --]

On Wed, Aug 31, 2005 at 01:32:04AM +1000, Finn Thain wrote:
> > > > Reasoning is, how do you know that pkg xyz is actually the package 
> > > > you're after?
> > Re-inserted the quote to clarify what I'm talking about; mapping another 
> > pkg managers db into our own requires either a lot of human 
> > intervention, or some dodgy rules that somewhat manage it, with bugs.
> 
> OK, I see what you mean. You're asking, how does portage know that vendor 
> package xyz is the portage package abc?
> 
> Short answer is package.mask, meta-packages and name mapping.
> 
> A particular vendor package version is a known-good dep, as tested by 
> devs, otherwise it is masked. E.g. package.mask says
> >vendor-sun/app-arch/cpio-x.y.z if no higher version has been tested. In
> mac os, automated updates mean that most of the time, there will be some 
> vendor packages that the tree hasn't been tested against. These have to be 
> masked until the user does emerge sync.

Alright, so I'm just being a tool 'coz I thought you were talking 
about dynamic mapping (vs dev managed mappings).  Nevermind me :)

> BTW, do repos share a namespace? Presented with the same cpv in several 
> repos, is portage's behaviour defined yet?
repo's have their own *total* namespace now; an overlay + repo is 
different though since an overlay is slaved to a repo.

<=2.1 basically lacks any true support for N repos; you can have a 
portdir(+overlays), a vdb, and a bintree.  Rewrite has no such 
restriction built into it.

> My feeling is that the burden of managing the mappings is better than the 
> burden of managing one package.provided for mac os 10.3, alongside another 
> for 10.4, etc. (If I'm wrong about that, then this exercise is pointless.)
Actually, I agree; it's cleaner then just autoassuming stuff is there.

> Did I read something about the rewrite being modular? Could the shim/query 
> take the form of a portage plugin that implements the query-apple-packages 
> feature? Obviously, if implemented the way I descibed above, it would need 
> to be intimate with certain ebuilds' environments.

Well, considering I'm seriously considering when/if rewrite is 
released, it's released as two packages; portage-core, and 
portage-ebuild... yes.  Very modular.

There pretty much is one point of required entry into the code; 
getting the config obj- from there it loads the code it needs, 
instantiating objects on the fly.  Aside from the entry point/config 
obj, everything else is intended to be configurable.

~harring

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-osx] Re: [RFC] Separate alt-prefix repo for base-system packages.

[Finn Thain]

On Tue, 30 Aug 2005, Brian Harring wrote:

> On Wed, Aug 31, 2005 at 01:32:04AM +1000, Finn Thain wrote:
>
> > My feeling is that the burden of managing the mappings is better than 
> > the burden of managing one package.provided for mac os 10.3, alongside 
> > another for 10.4, etc. (If I'm wrong about that, then this exercise is 
> > pointless.)
> Actually, I agree; it's cleaner then just autoassuming stuff is there.

Thing is, if you undertake to track another package manager in this way, 
you must expect the worst from the vendor. So the question is, just how 
robust are such mappings are in the face of upstream mayhem?

Here are some scenarios, where I think the present package.provided method 
seems to work better,

1. Vendor changes package manager API.
   - user will have no vendor deps until devs update the shim

2. Vendor renames a package.
   - user will lose a vendor dep until devs add a new mapping

3. Vendor combines two packages.
   - see 2.

4. Vendor bumps a package version.
   - see 2.

And here are some scenarios where the proposed method of mapping & masking 
is better:

5. Vendor splits a package.
   - existing method loses if any part is no longer installed by default
   - new method works, given vendor bumps package version

6. Vendor drops a package entirely or renders it broken as a dep.
   - existing method loses. e.g. if this happened in mac os 10.3.9, all 
     10.3 users would have suffered
   - new method works, given vendor bumps package version

7. Vendor fixes a previously broken package
   - existing method loses. e.g. if this happened in mac os 10.3.9, no 
     10.3 users benefit, because many still do not have the fix
   - new method works, given vendor bumps package version

8. Vendor adds a package already in the tree.
   - old method loses if the two packages are not equivalent
   - new method wins if a virtual/metapackage can be used to mean "close 
     enough"

9. Vendor adds a package not already in the tree.
   - old method loses because deprecated profiles (e.g. mac os 10.2) will 
     never get the benefit of that package
   - new method wins because a new mapping is all that is required

As I described it, a mapping is a full mapping from vendor PV to portage 
PV. So a new mapping is needed when the vendor revs the package (point 4 
above).

Now, if we could also have an ebuild without a version number, like 
/opt/gentoo/usr/portage/vendor-apple/sys-devel/xcode/xcode.ebuild, it 
could just map a vendor name to a portage name. The synthetic vendor 
package that shows up in the vdb would get the vendor's full version. In 
this case, the devs' work is lessened because a new mapping wouldn't have 
to be added every time the vendor revs a package. Instead, package.mask 
would provide QA.

-f
-- 
gentoo-osx@gentoo.org mailing list

Re: [gentoo-osx] Re: [RFC] Separate alt-prefix repo for base-system packages.

[Kito]

On Aug 29, 2005, at 10:26 PM, Brian Harring wrote:
>
> On Tue, Aug 30, 2005 at 01:09:49PM +1000, Finn Thain wrote:
>>>
>>> I was looking at it more as a place to develop some new portage
>>> features...Gentoo/Darwin has always been lurking, this is more in  
>>> the
>>> area of just getting offsets working.
>> Excepting that, if you can leverage
>> existing packages, prefixed installs are much more useful -- having a
>> complete set of deps installed on a prefix is not much better than a
>> stage3 chroot with your home directory bind mounted below it.

Maybe so, but we can't have one without the other. First get packages  
to install in a prefix, then work up from there. The issue of  
leveraging existing packages is currently handled by  
package.provided, which we all know doesn't really serve our  
purposes, but I see no reason work couldn't be done in parallel on  
these issues...if you have an ebuild repo  talking to and  
understanding a vendor repo such as apples Receipts or whatever, that  
will work wether the packages get installed in a prefix or not,  
likewise if we have packages that can be installed to an alt-prefix,  
they should work regardless of how portage is resolving the deps.

> The rewrite's general core is intended to allow for alt
> formats/repos/whatever jammed into it; that said, making seperate
> formats play nice with each other (unless they can natively) isn't
> something I think is incredibly easy to pull off, as mentioned above.

Right, this would be a great feature, but I look at this as multi- 
level deps, which should come later IMHO. My goal for having a branch  
of some base packages is to hash out the namespace and all the other  
issues of portage managing a flat set of deps under an offset root.  
Once that is functional, making the offset repo talk to another repo,  
regardless of vendor, host, location, etc could be looked at.

I know that its good to get a solid design before running off and  
writing a bunch of code, but it seems to me the portage rewrite has  
been thought out sufficiently to allow for this type of feature  
expansion in the future, without limiting what we can do right now.

--Kito

-- 
gentoo-osx@gentoo.org mailing list

Re: [gentoo-osx] Re: [RFC] Separate alt-prefix repo for base-system packages.

[Lina Pezzella]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1


On Aug 30, 2005, at 11:01 AM, Kito wrote:

> Maybe so, but we can't have one without the other. First get  
> packages to install in a prefix, then work up from there.

Amen. If we start really big here, we're just going to end up talking  
about this for months and never get anything done. Perhaps we could  
agree to work on getting packages to install in a prefix firstly and  
develop a procedure for making that happen. I see the rest of the  
issues in this thread as separate, even though they are related.

> The issue of leveraging existing packages is currently handled by  
> package.provided, which we all know doesn't really serve our  
> purposes, but I see no reason work couldn't be done in parallel on  
> these issues...

Exactly. Since the solution to this problem seems to be the crux of  
the discussion here, perhaps we could split this thread and continue  
the discussion about alternatives to package.provided in a new thread  
while working out a procedure for installing to a prefix simultaneously.

- --Lina Pezzella
Ebuild & Porting Co-Lead
Gentoo for OS X

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.2 (Darwin)

iD8DBQFDGejjNJ9STR9DbYERAsFlAJwJHdtzCo6BRFIZGfWkkjv8gT2t1wCbBWcX
/k91HnML1Hw66Ijj6itlmsU=
=p6es
-----END PGP SIGNATURE-----
-- 
gentoo-osx@gentoo.org mailing list

[gentoo-osx] Re: [RFC] Separate alt-prefix repo for base-system packages.

[Kito]

On Aug 29, 2005, at 2:07 AM, Brian Harring wrote:

> Kindly cc me on responses- I'm not on the osx mailing list anymore due
> to too much fricking mail per day :)
>
>>> From: kito
>>
>
>
>>> In the interim it can simply be an overlay, using a portage
>>> snapshot, giving us free reign to do what is needed to get the
>>> important things working without having to worry about Apple
>>> provided files. Then with some simple PATH mangling(think finks /sw/
>>> init) we can start making use of it now and actually be a step
>>> ahead of the game.
>
> You're not stating how the path mangling will occur, without
> addressing this the future merging of this proposed tree and the
> mainline tree is questionable.

Fink handles this using a patched bash with a special set of init  
scripts [1]. We would probably have to do something similar, but a  
lot more flexible and dynamic of course, we have to support a lot  
more archs. Even the current profile setup should offer some of the  
functionality required to achieve this.

>
>
>>> Doing this outside the main tree has IMHO worked quite well for g/
>>> fbsd project, and allowed them to get their base-system in order
>>> before they had to go mucking up the tree with hacks that may or
>>> may not be permanent, and is also the reason they aren't hated
>>> quite as much as us by the other projects...
>
> Different angles though; their goal was to nail down a base, and
> integrate the changes *back* into the tree.

Same goal. As I said in my other email, getting too far away from  
main tree must be avoided.

>
> Why am I implying this would basically result in a fork of the tree?
> Their modifications were patches, tweaks to the existing ebuilds such
> that they played nice with *bsd; yes, bit more complex then that, but
> roughly the jist.
>
> Swiveling the offset isn't just swiveling root; any alt offset is
> going to require making the ebuild aware of the offset.  How?

My current thoughts on this is to store all the package metadata in  
Xarchives, probably do it as a portage feature. So we use something  
like the ICANINSTALLTO var (gotta find a better name for that, ugh)  
and each offset filesystem structure is stored in the archive TOC,  
with the optional actual binary package as well if enabled in  
FEATURES, along with all the other typical data stored in /var/db/ 
pkg, almost like a tiered bom(5) file + a lot more.

Initially in practice we would probably ONLY swivel /, with a flat  
set of dependencies, and then work up from there.

If we have accurate *DEPENDS, we just have to make sure portage has  
sufficient knowledge of how to supply them for every offset.

>
> Likely introduction of a new feature/var- how is this going to be
> built up in a seperate repo, then brought back and merged into the
> tree?

EAPI comes to mind, if this is done right, we start establishing an  
ebuild API version that supports offsets.

>
> Mind you I'm not stating that for osx stuff it should be installed  
> to / ;
> I'm just trying to point out that the issue of having the offset
> dynamic isn't really addressed, all y'all can do if you don't address
> that issue is build up a repo of ebuilds that have a different set of
> hardcodings, effectively a fork that cannot be merged with the tree
> till a proper solution for swivelling the offset is created.

I'm definitely not interested in a bunch of hacked ebuilds with a  
hardcoded prefix. The whole goal would be doing it in a manner where  
no matter how/when a stable portage supports it, we would be ready,  
and hopefully helped in ushering it along.

>
> The nasty details of it can be found in the -dev thread from a few
> months back re: ICANINSTALLTO ; personally, it's something I'd like
> but not something willing to push atm, due to the fact already have
> enough people screaming that I'm trying to shove more work onto devs.

We would be the devs to not mind some more work =) Instead of an 8  
month long ML thread, we can use this as a sandbox and a constant  
running proof-of-concept. Should also be a good place to get the UI  
side of the portage rewrite hashed out, as emerge and repoman would  
definitely need some serious tweaking for this.

>
> Either way, you could split out the bases, but the point re: how to
> have those bases used by other packages (whether via modification of
> ebuilds, or portage) needs to be addressed, and imo someone should
> have a pretty good idea of the dynamic offset bit, so that the repo
> can be merged back in down the line. Yes, overblowing it a bit re: the
> packages that would depend on the base, but it's possible and likely
> will be raised by others if you do this.

Agreed, and I don't think you are overblowing the need to do it  
right, which is why i CC'd you, I'd like to have the portage people  
involved as much as possible.

--Kito

[1] http://cvs.sourceforge.net/viewcvs.py/fink/base-files/init.sh.in? 
rev=1.19&view=markup
-- 
gentoo-osx@gentoo.org mailing list