[gentoo-dev] For starters.. requirements of the portage API

[gentoo-dev] For starters.. requirements of the portage API

[Chris White]

[-- Attachment #1: Type: text/plain, Size: 1789 bytes --]

Ok, since it was asked of by jstubbs, time to get some stuff rolling as
to API requirements.

1) repository can be more than just a standard abiding set of files.
Maybe a custom repository, or a relational database.

2) access to an internal download library built into portage (not wget
or some such).  This would not only allow us to do basic downloading,
but also address new protocols that come later on.

3) ability to access certain information regarding package environment
(CFLAGS, USE, etc.), which when trying to get this information, has been
stated as "unclean" in case the methods change later on (ie. pulling
from /var/pkg/db/category/package/USE I was told was a bad idea for some
reason).  If portage can do it cleanly internally, an api should be able
to tap into that and give us something more solid to work with.  This
should also help in getting rid of those pesky "Did my package get
compiled with this use flag" requests.

4) a searching API to look for certain packages.  This sort of goes hand
in  hand with #1, as it gives the ability for someone to get mysql
querying to acccess a mysql repository.

5) Better info on SRC_URI's, including total size, downloaded size,
md5's, blah blah,etc etc.  Somewhat goes hand in hand with #4.

6) An output api which would help in supporting gui displaying as well
as ncurses or anything else where one chooses that they want to put
stuff.  Also, I'd like for a localization support feature in here
(localized strings basically).  This would help in portage localization.

That's all that's on my mind for now.  Please let me know if I need to
be clearer with some of my ideas.  Please feel free to put your 2
$country_coin_denomiation_value in to make this another meaningful
discussion.

[-- Attachment #2: このメッセージにはデジタル署名された部分があります --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

[gentoo-dev] Re: For starters.. requirements of the portage API

[Patrick Lauer]

[-- Attachment #1: Type: text/plain, Size: 3464 bytes --]

On Sun, 2005-03-06 at 14:27 +0900, Chris White wrote:
> Ok, since it was asked of by jstubbs, time to get some stuff rolling as
> to API requirements.
Very good idea.
I'm looking at it especially from a "tinderbox" (automated building)
point of view

- generic get/set access to all config options
We need to be able to generate a pre-defined environment for building

- interface to the depgraph
For certain operations, including build parallelization, we need a local
copy of all dependency information. Also, repoman might be able to be a
wrapper for this interface?

- access to ebuild.sh functions, e.g. tell portage "unpack and patch
foobar-1.2 for me"

> 1) repository can be more than just a standard abiding set of files.
> Maybe a custom repository, or a relational database.
That's backend and independent of the "outside" API

> 2) access to an internal download library built into portage (not wget
> or some such).  This would not only allow us to do basic downloading,
> but also address new protocols that come later on.
Excellent idea. afaik this idea is being implemented as "transports"?

> 3) ability to access certain information regarding package environment
> (CFLAGS, USE, etc.), which when trying to get this information, has been
> stated as "unclean" in case the methods change later on (ie. pulling
> from /var/pkg/db/category/package/USE I was told was a bad idea for some
> reason).
Does that differ from my idea for a generic config interface?

>   If portage can do it cleanly internally, an api should be able
> to tap into that and give us something more solid to work with.  This
> should also help in getting rid of those pesky "Did my package get
> compiled with this use flag" requests.
> 
> 4) a searching API to look for certain packages.  This sort of goes hand
> in  hand with #1, as it gives the ability for someone to get mysql
> querying to acccess a mysql repository.
I'd like to keep those two things separated.
From a user p.o.v. the backend (flat files, db, pigeons) should be
transparent. I think searching based on atoms, descriptions etc. would
be best, the SQL interface might be documented separately ...

> 5) Better info on SRC_URI's, including total size, downloaded size,
> md5's, blah blah,etc etc.  Somewhat goes hand in hand with #4.
> 
> 6) An output api which would help in supporting gui displaying as well
> as ncurses or anything else where one chooses that they want to put
> stuff.  Also, I'd like for a localization support feature in here
> (localized strings basically).  This would help in portage localization.
erm ...
that's not what the API does.
The API only gives you access to portage internals. Your GUI gets
wrapped around that so that any GUI uses the same communication
primitives.

Basically all your GUI needs to do should be:
import portage
blah=portage #blah is an instance of portage
blah.init()  # initialize that instance

then work with that.


> That's all that's on my mind for now.  Please let me know if I need to
> be clearer with some of my ideas.  Please feel free to put your 2
> $country_coin_denomiation_value in to make this another meaningful
> discussion.
I think you should get the frontend and the backend separated.
My GUI doesn't care about your SQL backend. Your SQL backend doesn't
care about my OpenGL GUI. 
Otherwise I think your first rough draft looks good.

wkr,
Patrick

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

[gentoo-dev] Re: For starters.. requirements of the portage API

[Patrick Lauer]

[-- Attachment #1: Type: text/plain, Size: 2077 bytes --]

On Sun, 2005-03-06 at 12:50 +0100, Fredrik Klasson wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> Chris White wrote:
> | 2) access to an internal download library built into portage (not wget
> | or some such).  This would not only allow us to do basic downloading,
> | but also address new protocols that come later on.
> uhm, just a radnom thought struck me, why re-invent the wheel? What I'm
> thinking is that, why not simply provide some Download Abstraction Layer
> (DAL) or something like that, ie, "downloadThisFileForMe(filename);"
I think that was the idea.
You tell portage "use that download wrapper" in a config file, then just
invoke "portage.fetch("URI")

> that /[cw]ould/ invoke
> ~  wget (why not? ;),
> ~  scp (maybe a bad idea),
> ~  or someother nifty file fecthing tool 
For that you might want some extra information.
E.g. if you had a diffserver (binary diffs between versions) portage has
the metadata to decide if the full file or only the diff should be
fetched. A wrapper would have to generate that metadata.
Therefore a portage-internal fetch mechanism should work better.

> (maybe even some bittorent
> client - woulnd't it be nicer to make use of p2p to fetch files at
> (hopefully) full download speed while reducing the load on the mirrors?).
Maybe bittorrent is not optimal for many small files, but there are some
simple yet effective ways of "enhancing" the file fetching.

> IMO, writing a new download library seems to me like asking for one more
> part that can be subject to Murphys law (if it can break it will),
> though, one could argue that some kind of DAL would be a great place for
> "interface bugs" (or whatever to call bugs that reside between one pice
> of software communitating with another).
Right, but you'd get a lot of extra functionality. Right now, implementing 
a new fetch mechanism is quite awkward. Having portage support for that 
would seriously help all those mechanisms.

> just my 2 öre YAFYGI ;)
How much is that in real money? ;-)

hth,
Patrick

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

[gentoo-dev] Re: For starters.. requirements of the portage API

[Jason Stubbs]

On Sunday 06 March 2005 21:09, Patrick Lauer wrote:
> On Sun, 2005-03-06 at 12:50 +0100, Fredrik Klasson wrote:
> > -----BEGIN PGP SIGNED MESSAGE-----
> > Hash: SHA1
> >
> > Chris White wrote:
> > | 2) access to an internal download library built into portage (not wget
> > | or some such).  This would not only allow us to do basic downloading,
> > | but also address new protocols that come later on.
> >
> > uhm, just a radnom thought struck me, why re-invent the wheel? What I'm
> > thinking is that, why not simply provide some Download Abstraction Layer
> > (DAL) or something like that, ie, "downloadThisFileForMe(filename);"
>
> I think that was the idea.
> You tell portage "use that download wrapper" in a config file, then just
> invoke "portage.fetch("URI")

I think URI is even taking it too far. Even fetching of individual files 
doesn't seem to be required. All that's really needed is to tell portage to 
fetch the required files for any particular package.

If you want to be able to download individual files for use outside of portage 
then you'll need to know where they are going and whatever else. If you want 
all that, you can grab the URI from SRC_URI and do it manually.

Regards,
Jason Stubbs
--
gentoo-dev@gentoo.org mailing list