* [gentoo-dev] DRAFT GLEP: MULTI-REPO @ 2005-12-17 8:17 Andrew Muraco 2005-12-17 8:19 ` Andrew Muraco 2005-12-17 23:15 ` [gentoo-dev] draft glep: multi-repo Brian Harring 0 siblings, 2 replies; 9+ messages in thread From: Andrew Muraco @ 2005-12-17 8:17 UTC (permalink / raw To: gentoo-dev [-- Attachment #1: Type: text/plain, Size: 413 bytes --] Attached is a draft of a glep for formalizing multiple-repository support This is far from ideal in many ways, but i'm too tired and I drank too much caffine to be sane. Also, i have no clue how to use docutils so i just tried my best. (** is my way of doing another level of bullets, please let me know how to properly do this) Comments, objections, anything consructive is welcome. Thanks, Andrew Muraco [-- Attachment #2: glep-multi-repo-draft.txt --] [-- Type: text/plain, Size: 4460 bytes --] GLEP: XX Title: Multiple Repository Support in Portage Version: $Revision: 1.0 $ Author: Andrew Muraco <tuxp3@leetworks.com> Last-Modified: $Date: 2005/12/17 03:13:10 $ Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 17-Dec-2005 Post-History: 17-Dec-2005 Abstract ======== To implement a functional and expandable method for Portage to support multiple repositories. Motivation ========== Multiple Repository support is needed, this GLEP is to address this need. Specification ============= Portage will make use of two (2) ways to address repositories: * A User-defined name, which is likely to be used as a convinance in most situations - this will be referred to as REPO_NAME in this GLEP * A hard-coded repository-id which will be found in the repository tree at: metadata/repo_id - this will be referred to as REPO_ID in this GLEP Both names will contain no spaces, and only standard characters [TODO: references] Repositories ------------ Each repository will contain: * the repo name in metadata/repo_id * repo information such as maintainer of the repo, notes on who hosts it, etc will be contained metadata/repo_info * unique packages.mask which will only apply to ebuilds within that specific repo. The REPO_ID must match the name that will be used for rsync Therefore, rsync://MyServer.tdl/REPO_ID/ /etc/portage/* ------------- In order to provide users with the current set of options and extend them so they can be customized to each repository, the structure of /etc/portage will remain similar with these changes: * /etc/portage/REPO_NAME/* will be the location of repository-specific portage files. * /etc/portage/ will continue to function over all repos ** ex) =sys-devel/gcc-4 -* in /etc/portage/package.keywords would use the latest gcc-4 regardless of what tree it comes from. The following new files will be added to /etc/portage: * /etc/portage/repositories.perfer - will contain each REPO_NAME in order of preferance, higher is more perfered. (Each REPO_NAME will be on a seperate line) ** In the absence of this file portage should use repositories in alphabetical order. * /etc/portage/REPO_NAME/repository.id - contains the specific REPO_ID which REPO_NAME applies to. */etc/portage/REPO_NAME/repository.conf - will contain any repository-specific options, which can include, but is not limited to, FEATURES="" C[XX]FLAGS="". ** This will also include a new variable; OPTIONS="" of which is similar to FEATURES, but modifies the way portage will handle that specific repository. A few examples of options which could be useful: *** EXCLUDESYNC - Prevents portage from doing a sync on this repo. *** EXCLUDEUPDATE - Prevents portage from using ebuilds in this repo as updates for packages which currently reside in a different repo. *** EXCLUSIVEUPDATE - forces any update to any package which is from this repository to a newer version which resides inside of this repo. *** et al. All of the repository rsync URIs will be stored in /etc/make.conf SYNC="rsync://myfavoriterepo.org/myportage \ rsync://rsync.namerica.gentoo.org/gentoo-portage" The Tree: /usr/portage -> /var/repositories/REPO_ID/ ---------------------- The repository tree will need to be moved, each repository will have its own folder: /var/repositories/REPO_ID/. For compatibility reasons, /usr/portage will be treated as /var/repositories/gentoo-portage Ebuilds ------- Ebuilds will now be able to have dependencies based on packages from specific repositories. * DEP Atoms now support the following format: =REPO_ID:SLOTNUM:CAT/EBUILD-X.Y.Z ** Ex1) >=MyRepo:2:sys-devel/gcc-4.0 ** Ex2) <gentoo-portage::kde-base/kdelibs-3.5 ** Ex3 ::foo/bar Dependency atoms that lack the new format (::) or do not have a REPO_ID will then just use any ebuild which fulfills the requirements. /var/db/pkg/ ------------ Along with other information about the building of packages, the specific repo which provided the ebuild will be recorded. /var/db/pkg/cat/package/REPOSITORY will contain REPO_ID If no REPOSITORY file is found, then gentoo-portage is assumed. Backwards Compatibility ======================= /usr/portage will be treated as /var/repositories/gentoo-portage so it would be possible to function with no changes after the upgrade. Packages in /var/db that lack REPOSITORY will be assumed to be gentoo-portage. References ========== [TODO] Copyright ========= This document has been placed in the public domain. ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [gentoo-dev] DRAFT GLEP: MULTI-REPO 2005-12-17 8:17 [gentoo-dev] DRAFT GLEP: MULTI-REPO Andrew Muraco @ 2005-12-17 8:19 ` Andrew Muraco 2005-12-17 23:15 ` [gentoo-dev] draft glep: multi-repo Brian Harring 1 sibling, 0 replies; 9+ messages in thread From: Andrew Muraco @ 2005-12-17 8:19 UTC (permalink / raw To: gentoo-dev I apologize for the caps in the subject. Andrew Muraco -- gentoo-dev@gentoo.org mailing list ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [gentoo-dev] draft glep: multi-repo 2005-12-17 8:17 [gentoo-dev] DRAFT GLEP: MULTI-REPO Andrew Muraco 2005-12-17 8:19 ` Andrew Muraco @ 2005-12-17 23:15 ` Brian Harring 2005-12-17 23:25 ` Ciaran McCreesh 1 sibling, 1 reply; 9+ messages in thread From: Brian Harring @ 2005-12-17 23:15 UTC (permalink / raw To: gentoo-dev [-- Attachment #1: Type: text/plain, Size: 11317 bytes --] Pardon bluntness; don't mean offense, just specifically picking the hell out of this proposal to make a point (lucky you) :) On Sat, Dec 17, 2005 at 03:17:44AM -0500, Andrew Muraco wrote: > Attached is a draft of a glep for formalizing multiple-repository support I appreciate trying to chip in, but frankly this glep needs a lot more thought put into it. Further, I _really_ do not see the point of glepping this either. Puking up proposals due to folks making noise is a waste of time- don't document/propose just because folks are making noise- do it for large scale changes, or conflict, not because someone requires a glep/spec before they're willing to listen to the _developers_ of a project about how to integrate a new feature into _their_ project. > This is far from ideal in many ways, but i'm too tired and I drank too > much caffine to be sane. > > Comments, objections, anything consructive is welcome. Inlined... > Abstract > ======== > To implement a functional and expandable method for Portage to support multiple repositories. > > Motivation > ========== > Multiple Repository support is needed, this GLEP is to address this need. define multiple repository. We _have_ multi repo already (binpkg and portdir, let alone overlays). > Specification > ============= > > Portage will make use of two (2) ways to address repositories: > * A User-defined name, which is likely to be used as a convinance in most situations - this will be referred to as REPO_NAME in this GLEP > * A hard-coded repository-id which will be found in the repository tree at: metadata/repo_id - this will be referred to as REPO_ID in this GLEP > Both names will contain no spaces, and only standard characters [TODO: references] Portage externally will use user defined, internally it will do it's own thing. > Repositories > ------------ > > Each repository will contain: > * the repo name in metadata/repo_id > * repo information such as maintainer of the repo, notes on who hosts it, etc will be contained metadata/repo_info > * unique packages.mask which will only apply to ebuilds within that specific repo. > > The REPO_ID must match the name that will be used for rsync > Therefore, rsync://MyServer.tdl/REPO_ID/ No. It's arbitrary, and invalid to assume rsync is the only sync uri that's going to be used- this isn't even getting into _remote_ repos. *ANY* unique id tagged into a repo is just that, a magic constant in it's metadata. Just that. No mandates about SYNC, file layout, etc, will fly that bind to the metadata id. > /etc/portage/* > ------------- > > In order to provide users with the current set of options and extend them so they can be customized to each repository, the structure of /etc/portage > will remain similar with these changes: > * /etc/portage/REPO_NAME/* will be the location of repository-specific portage files. > * /etc/portage/ will continue to function over all repos > ** ex) =sys-devel/gcc-4 -* in /etc/portage/package.keywords would use the latest gcc-4 regardless of what tree it comes from. > > The following new files will be added to /etc/portage: > * /etc/portage/repositories.perfer - will contain each REPO_NAME in order of preferance, higher is more perfered. (Each REPO_NAME will be on a seperate line) yuck. This is a bit of a waste for a single file. > ** In the absence of this file portage should use repositories in alphabetical order. directory returned ordering, not alpha- no ordering == set of repositories, thus don't try to induce a fallback ordering. > * /etc/portage/REPO_NAME/repository.id - contains the specific REPO_ID which REPO_NAME applies to. This is going to induce more slowdown for any config instantiation- more directories/files to scan. Iow, python -c 'import portage' (which instantiates a config obj), is going to get slower, which will piss off the slower system folk even further (hell, even with 1.5ghz and decent IO it still is a 10s import for me uncached). > */etc/portage/REPO_NAME/repository.conf - will contain any repository-specific options, which can include, but is not limited to, FEATURES="" C[XX]FLAGS="". > ** This will also include a new variable; OPTIONS="" of which is similar to FEATURES, but modifies the way portage will handle that specific repository. > A few examples of options which could be useful: This seems a bit arbitrary. > *** EXCLUDESYNC - Prevents portage from doing a sync on this repo. And how are you going to specify the sync method for that repo? > *** EXCLUDEUPDATE - Prevents portage from using ebuilds in this repo as updates for packages which currently reside in a different repo. > *** EXCLUSIVEUPDATE - forces any update to any package which is from this repository to a newer version which resides inside of this repo. And this is implemented how? Why is it specifying resolver directives as repo attributes? When is this forced -U going to be triggered? Sync? Next random emerge call? How is this going to be bound to the resolver, let alone the question of how build configuration data is going to be bound to the actual build? yes, leading question there. :) Where in this proposal, is it extending similar capabilities to binpkgs? > *** et al. > > All of the repository rsync URIs will be stored in /etc/make.conf > SYNC="rsync://myfavoriterepo.org/myportage \ rsync://rsync.namerica.gentoo.org/gentoo-portage" No. No no no no no.... at least this explains the metadata ID being part of SYNC comment thought :) > The Tree: /usr/portage -> /var/repositories/REPO_ID/ > ---------------------- > The repository tree will need to be moved, each repository will have its own folder: /var/repositories/REPO_ID/. This directly castrates the user's ability to store the tree wherever they want, loss of that long standing capability is an issue. No go there. > For compatibility reasons, /usr/portage will be treated as /var/repositories/gentoo-portage Building hacks into portage isn't going to fly; no special cases. The need for this is a sign that the forced FHS compliant path (instead of letting the user do whatever they want) is a bad idea. This beast can be accomplished without sacrificing our existing flexibility. > Ebuilds > ------- > > Ebuilds will now be able to have dependencies based on packages from specific repositories. > > * DEP Atoms now support the following format: =REPO_ID:SLOTNUM:CAT/EBUILD-X.Y.Z > ** Ex1) >=MyRepo:2:sys-devel/gcc-4.0 > ** Ex2) <gentoo-portage::kde-base/kdelibs-3.5 > ** Ex3 ::foo/bar > Dependency atoms that lack the new format (::) or do not have a REPO_ID will then just use any ebuild which fulfills the requirements. Backwards compatibility? Protection for portage version that see this and aren't capable of handling it? Why are you proposing the use/slot additions be changed so slot is prefixed rather then postfixed? > Backwards Compatibility > ======================= > /usr/portage will be treated as /var/repositories/gentoo-portage so it would be possible to function with no changes after the upgrade. This is a bit short... see above. lot more to it, especially since you're trying to use stable. Glep doesn't provide any way for knowing what repo's are active... nor what repo's are overlaying each other, nor extending the capabilties to anything beyond ebuild trees. Bintree? VDB? (yes, extending slaving capabilities to vdb has uses). The shortcomings I'm pointing at are historical- you're trying to shoehorn into the existing portage configuration, thus I'm pointing out the failings of it :) Basically... this glep is exactly opposite of what I'm after, and what I've already written in saviour. Saviour's configuration *currently* is ini based, although that isn't a requirement (pluggable config parser is there, although marienz is working on making my initial prototype not suck). Make.conf support will be there (backwards compatibility); just will require a config parser written to convert make.conf format into the internal config definitions. The existing stable configuration is inherintly single master repo set, single configuration, single domain, single root, single sync. There's no groupping in make.conf (nor aux files), thus it's pretty fricking hard to try and make it groupped. Further, no control over the individual objects- no way to specify that a repo is remote fex. All the make.conf support would do for saviour crap is translate it into a single domain, single repo + overlays, etc. If folks need more power (seperate caches, seperate syncs for repos, N domains/roots...), they need to use a more powerful config format. Via ini, you have repo definitions, defining the repo class to use, location (if applicable), host (if applicable), cache reference (cache instance to use, defined via it's own configuration item), slaving, multiplexing together, package.* specifications (state the filepath)... etc. Plus, user label (section label for that repo definition). The framework in place allows for a helluva lot more crazy stuff (your own repo classes, say a strict R->L union of repos), I'm just rambling off the obvious targets. The configuration parser/handler actually addresses all of this crap on it's own without forcing us to introduce hacks to try and shoehorn N repo's into the current single repo config. Old doc of details/intentions, http://dev.gentoo.org/~ferringb/portage/3.0/dev-notes/framework/config.txt . It's generalized and flexible- prior to screaming off with his head for the ini usage, please read through http://dev.gentoo.org/~ferringb/portage/3.0/dev-notes ; there are a _lot_ of good reasons, jotting down of intention, layout/design of portage 3, etc. Do the reading prior to the screaming ;) Either way... here's how it *should* proceed from where I'm sitting. Introduce full PORTDIR capabilities to all repos available- vdb, binpkg, portdir/portdir_overlay. That right there is a major request people have been pushing for, and in doing so the framework gets hammered on/debugged. Introducing resolver level constraints (match strictly from this repo) requires atom extension among other things, and introduces it's own class of problems. Introduce the capabilities while treating existing repos as a single repo set, then extend it with what's needed for true stand alone repos. Either way, while discussion of standalone repos is good, I'm going to kindly remind y'all that glep42 just needs to pass in a repo id- it's not blocked by anything but minor api changes to portageq so that the portage devs aren't forced to fix someone elses proposal down the line (in the process breaking crap for users when we do so). True stand alone repository capabilities aren't required/bound to glep42, all that's required out of glep42 is that the syncing repo id be used now (even if it may seem superfluous). Iow, news *should* work regardless if it's an overlay or a stand alone repo. Please keep that in mind. ~harring [-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --] ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [gentoo-dev] draft glep: multi-repo 2005-12-17 23:15 ` [gentoo-dev] draft glep: multi-repo Brian Harring @ 2005-12-17 23:25 ` Ciaran McCreesh 2005-12-18 0:14 ` Brian Harring 0 siblings, 1 reply; 9+ messages in thread From: Ciaran McCreesh @ 2005-12-17 23:25 UTC (permalink / raw To: gentoo-dev [-- Attachment #1: Type: text/plain, Size: 754 bytes --] On Sat, 17 Dec 2005 15:15:48 -0800 Brian Harring <ferringb@gentoo.org> wrote: | True stand alone repository capabilities aren't required/bound to | glep42, all that's required out of glep42 is that the syncing repo id | be used now (even if it may seem superfluous). Iow, news *should* | work regardless if it's an overlay or a stand alone repo. Not quite true. I also need to know how to handle Display-If-Installed and Display-If-Profile headers with multiple repositories. Hence why I'd prefer to leave it as "something that can easily be implemented in the future"... -- Ciaran McCreesh : Gentoo Developer (I can kill you with my brain) Mail : ciaranm at gentoo.org Web : http://dev.gentoo.org/~ciaranm [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 189 bytes --] ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [gentoo-dev] draft glep: multi-repo 2005-12-17 23:25 ` Ciaran McCreesh @ 2005-12-18 0:14 ` Brian Harring 2005-12-18 0:24 ` Ciaran McCreesh 0 siblings, 1 reply; 9+ messages in thread From: Brian Harring @ 2005-12-18 0:14 UTC (permalink / raw To: gentoo-dev [-- Attachment #1: Type: text/plain, Size: 945 bytes --] On Sat, Dec 17, 2005 at 11:25:58PM +0000, Ciaran McCreesh wrote: > On Sat, 17 Dec 2005 15:15:48 -0800 Brian Harring <ferringb@gentoo.org> > wrote: > | True stand alone repository capabilities aren't required/bound to > | glep42, all that's required out of glep42 is that the syncing repo id > | be used now (even if it may seem superfluous). Iow, news *should* > | work regardless if it's an overlay or a stand alone repo. > > Not quite true. I also need to know how to handle Display-If-Installed > and Display-If-Profile headers with multiple repositories. Hence why > I'd prefer to leave it as "something that can easily be implemented in > the future"... News is repo specific. What I stated in the glep42 thread email covers this already; zmedico already adress Display-If-Profile (namely, a simple portageq extension). What remaining straw men are there for ignoring the portage developers requests? ~harring [-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --] ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [gentoo-dev] draft glep: multi-repo 2005-12-18 0:14 ` Brian Harring @ 2005-12-18 0:24 ` Ciaran McCreesh 2005-12-18 1:01 ` Brian Harring 0 siblings, 1 reply; 9+ messages in thread From: Ciaran McCreesh @ 2005-12-18 0:24 UTC (permalink / raw To: gentoo-dev [-- Attachment #1: Type: text/plain, Size: 644 bytes --] On Sat, 17 Dec 2005 16:14:15 -0800 Brian Harring <ferringb@gentoo.org> wrote: | What remaining straw men are there for ignoring the portage | developers requests? Asking you to specify how multiple repositories will work before I try to extend the GLEP to support multiple repositories is hardly a straw man argument. *shrug* But if you prefer, I'll change the GLEP to support multiple repositories the way I'd like to see them done rather than the way you'd like. -- Ciaran McCreesh : Gentoo Developer (I can kill you with my brain) Mail : ciaranm at gentoo.org Web : http://dev.gentoo.org/~ciaranm [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 189 bytes --] ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [gentoo-dev] draft glep: multi-repo 2005-12-18 0:24 ` Ciaran McCreesh @ 2005-12-18 1:01 ` Brian Harring 2005-12-18 1:08 ` Ciaran McCreesh 0 siblings, 1 reply; 9+ messages in thread From: Brian Harring @ 2005-12-18 1:01 UTC (permalink / raw To: gentoo-dev [-- Attachment #1: Type: text/plain, Size: 2222 bytes --] On Sun, Dec 18, 2005 at 12:24:48AM +0000, Ciaran McCreesh wrote: > On Sat, 17 Dec 2005 16:14:15 -0800 Brian Harring <ferringb@gentoo.org> > wrote: > | What remaining straw men are there for ignoring the portage > | developers requests? > > Asking you to specify how multiple repositories will work before I try > to extend the GLEP to support multiple repositories is hardly a straw > man argument. Do you need to know how every bit of a car works to drive it? No. We're simply asking you to tag in a repo_id to your calls to portage. It's bloody simple, just need to know what you explicitly need from a news client standpoint. Stop being a stubborn mule, and realize that you're trying to shove a feature into portage that *we* have to maintain, including your built in design flaw for N repos. If you can't accept our criticism and suggestions for your glep, tough cookies, we're the ones stuck maintaining it, and the users are the one stuck when we expand portage functionality (thus breaking your implementation). > *shrug* But if you prefer, I'll change the GLEP to > support multiple repositories the way I'd like to see them done rather > than the way you'd like. Ciaranm, cut the word games and stupid threats. If you can't work with others, and actually understand that they may not agree with your views (let alone be willing to have you dump a mess on them), I suggest you leave the distro and go work on your social skills. Propose the glep however you want. As long as the glep is around, I'm going to do the same you do- point out the flaws in it. If you're unwilling to even nail down what is needed so that all parties are happy, that's fine- I would expect the council to realize the glep (heavily affecting the portage group, since server and client side mods fall on our head) needs to be worked out further and thus reject it. Or, you stop debating the fact portage group might have a say in this, and start discussing what needs to be done so progress is made. Balls in your court, you know we view it as unacceptable right now, if you're not willing to work with us there isn't much that can be done. ~harring [-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --] ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [gentoo-dev] draft glep: multi-repo 2005-12-18 1:01 ` Brian Harring @ 2005-12-18 1:08 ` Ciaran McCreesh 2005-12-18 1:31 ` Brian Harring 0 siblings, 1 reply; 9+ messages in thread From: Ciaran McCreesh @ 2005-12-18 1:08 UTC (permalink / raw To: gentoo-dev [-- Attachment #1: Type: text/plain, Size: 422 bytes --] On Sat, 17 Dec 2005 17:01:59 -0800 Brian Harring <ferringb@gentoo.org> wrote: | Do you need to know how every bit of a car works to drive it? No. No, but I do need to know whether it has manual or automatic gears, and where the steering wheel is. -- Ciaran McCreesh : Gentoo Developer (I can kill you with my brain) Mail : ciaranm at gentoo.org Web : http://dev.gentoo.org/~ciaranm [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 189 bytes --] ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [gentoo-dev] draft glep: multi-repo 2005-12-18 1:08 ` Ciaran McCreesh @ 2005-12-18 1:31 ` Brian Harring 0 siblings, 0 replies; 9+ messages in thread From: Brian Harring @ 2005-12-18 1:31 UTC (permalink / raw To: gentoo-dev [-- Attachment #1: Type: text/plain, Size: 812 bytes --] On Sun, Dec 18, 2005 at 01:08:31AM +0000, Ciaran McCreesh wrote: > On Sat, 17 Dec 2005 17:01:59 -0800 Brian Harring <ferringb@gentoo.org> > wrote: > | Do you need to know how every bit of a car works to drive it? No. > > No, but I do need to know whether it has manual or automatic gears, and > where the steering wheel is. Stop spamming the list with idiot snarky responses, and discuss the issues at hand. Bluntly, others may back down from your shit but I'm not going to. A half baked proposal that is *broken* from the start being dumped on the mess that is portage is not acceptable. If you're not going to fix your glep, state so, stop wasting others time. Just take it to the council and have them ixnay it on the same issues people have continually pointed out. ~harring [-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --] ^ permalink raw reply [flat|nested] 9+ messages in thread
end of thread, other threads:[~2005-12-18 1:35 UTC | newest] Thread overview: 9+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2005-12-17 8:17 [gentoo-dev] DRAFT GLEP: MULTI-REPO Andrew Muraco 2005-12-17 8:19 ` Andrew Muraco 2005-12-17 23:15 ` [gentoo-dev] draft glep: multi-repo Brian Harring 2005-12-17 23:25 ` Ciaran McCreesh 2005-12-18 0:14 ` Brian Harring 2005-12-18 0:24 ` Ciaran McCreesh 2005-12-18 1:01 ` Brian Harring 2005-12-18 1:08 ` Ciaran McCreesh 2005-12-18 1:31 ` Brian Harring
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox