Re: [gentoo-dev] [soc] Python bindings for Paludis

[Ciaran McCreesh <ciaranm@ciaranm.org>]

On Sat, 24 Mar 2007 20:25:45 +0100
Luca Barbato <lu_zero@gentoo.org> wrote:
> Ciaran McCreesh wrote:
> > Assuming you mean piotr, who is not pioto... The difference is,
> > piotr's proposal is possible and doable within the timeframe,
> > whereas lu_zero's sounds nice if you don't know anything about any
> > of the package managers in question and can't be delivered within
> > three months.
> 
> I'd like to know your opinion about which are the pitfalls and the
> issues since you are surely more informed than me on paludis and, to a
> large degree, on portage internals.
> 
> I assumed that for a foundation and a non exaustive converage the
> summer would be more than enough.

If you're wanting to do a very simple API supporting approximately the
following, you're ok:

* Fetching a list of package versions that match a particular
dependency atom
* Fetching a list of available packages
* Simple metadata queries upon a particular package
* Fetching the contents of a particular package

If you're wanting to make a powerful API that lets people solve real
world problems, you're in for an awful lot of trouble.

The problem is this... Although Paludis, Pkgcore and Portage solve the
same ultimate problem, they do it in extremely different ways.
Internally and from a public API perspective, there's very little in
common between the three.

Portage is by and large procedural and messy. It's basically an
incoherent bunch of routines to do particular things. It doesn't
generalise well, and things you'd expect to be similar aren't (e.g.
you'd think finding out something about a package in VDB would be
the same as finding out something about a package in the tree, but
that would be far too easy...).

Paludis is basically what you'd expect from a highly OO, resource
managed language like C++. The problem is, a generalised API would end
up hiding nearly all of the flexibility and functionality.

You also can't wrap Paludis in any programming language that doesn't do
resource management of some kind (preferably fully controlled, but
since only C++ offers that, garbage collected works too). Writing a
common middle layer in C and then writing language extensions on top of
that isn't doable -- the common middle layer would have to be C++,
since you can't write Ruby extensions in Python or suchlike...

Pkgcore is closer to being AO than OO, largely because of programming
language differences. Again, a generalised API would mask flexibility
and functionality. You'd have a hard time getting callbacks to
generalise cleanly.

Design issues aside, there're also problems conceptually. The three
package managers have very different ideas of certain key concepts like
repositories, packages, the general operating environment (or domain)
and version metadata. You'd have to come up with a whole new conceptual
model that can handle all three paradigms, and you'd have to do it in
such a way that you don't kill the performance techniques (delayed and
batch loading, effectively) used by Paludis and Pkgcore.

So it's down to a question of scope. Are you trying to make an API to
do a few very basic queries, or are you trying to make an API powerful
enough to, say, make a graphical front end? The former is doable and
useless, the latter is a massive project.

Now, what you *could* do is implement a portageq-style tool with more
functionality. You'd still have conceptual issues (Paludis doesn't
particularly like giving you global configuration information, for
example -- simple things like querying whether a USE flag is enabled
need an associated c/p-v::r), but they wouldn't be as bad. Such a tool
would be slow, of limited use and easily doable within the available
time.

> I'm more interested in a solid base than a complete and exaustive
> wrapper =)

Which is the problem... The base is extremely different for all three.

-- 
Ciaran McCreesh

-- 
gentoo-dev@gentoo.org mailing list