Re: [gentoo-portage-dev] Re: Re: portage-ng roadmap?

Re: [gentoo-portage-dev] Re: Re: portage-ng roadmap?

[Paul de Vrieze]

>> You're right, they probably don't. My system will be quite modular and I image
>> a new tree, maybe I'll implement it in the initial release. One concept I'm
thinking about is something I call xpack (where xbuild is the extension of
ebuilds that should be guaranteed to work with the new parser). This
basically
>> is a text based packaging format (like tar, but actually diffeable), that
would contain all parts needed for an ebuild (source is optional). Such a
file
>> would make things a lot easier to manage and to download on demand.
>
> Totally agree. You can also force a validating commit with cvs : any commit
would have to pass QA test (parsing) to be really commited. But, please
consider not using rsync anymore. It's too slow for 85 000 files :(
> as previously discuss, subversion would be great. The best way is maybe the
debian way : Put all xbuilds in a single tar file that would downloaded when
"emerge update" (example).

Well it should be easy to have a different source of tree information. I like
subversion (I maintain the ebuilds), but it does need to stand up to the load
that the developers and users put on it. Subversion is also rather hard to
mirror so we might want to look to other solutions or have mirrors only mirror
the head revision not the below revisions ;-).

>
>> While my main focus will not be on ondemand downloading of xpacks (ebuilds is
>> pointless) it should be fairly trivial to generate metadata files for the
packages contained, maybe even on a category level. The transfer size could
still be sizeable though. I believe that introduction of xpack files
containing everything needed for an ebuild (except the manifest and
metadata.xml) allready reduces the amount of files in the tree enormously.
It
>> also helps in being able to easilly remove unused patches ;-).
>
> I like this new portage :)
> So these xbuilds are not needed on end-users computers. They are only needed
to generate the dependance tree.

Well think of xpack files as .tar files but text based for nonbinary content.
If metadata would be generated from the xbuild files (optional) then that
metadata would suffice except for the actual compiling/mergin of packages.

>
> If you need any help on this, please email me.

I'll remember you and surely will ask your help at the point where there's
something you can do ;-)

Paul

--
Paul de Vrieze
Researcher
Mail: pauldv@cs.kun.nl
Homepage: http://www.devrieze.net



--
Paul de Vrieze
Researcher
Mail: pauldv@cs.kun.nl
Homepage: http://www.devrieze.net


--
gentoo-portage-dev@gentoo.org mailing list

[gentoo-portage-dev] portage-ng roadmap?

[Adrian Gschwend]

Hi all,

I follow this list for some time now, mainly because I am interested in
portage-ng. I did read the docs about portag-ng at gentoo.org and I searched
some other sites for more detailed information, also about the prototypes
mentioned in the docs.

Unfortunately I couldn't find much about it, so I decided to ask here about
the status. How is portage-ng getting along? Is there a roadmap somewhere?
What needs to be done?

cu

Adrian




--
gentoo-portage-dev@gentoo.org mailing list

Re: [gentoo-portage-dev] portage-ng roadmap?

[Marius Mauch]

[-- Attachment #1: Type: text/plain, Size: 553 bytes --]

On 06/28/04  Adrian Gschwend wrote:

> Unfortunately I couldn't find much about it, so I decided to ask here
> about the status. How is portage-ng getting along? Is there a roadmap
> somewhere? What needs to be done?

It's not getting along at all, we're concentrating our efforts on the
current portage. Right now I consider portage-ng a dead project.

Marius

-- 
Public Key at http://www.genone.de/info/gpg-key.pub

In the beginning, there was nothing. And God said, 'Let there be
Light.' And there was still nothing, but you could see a bit better.

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

[gentoo-portage-dev] Re: portage-ng roadmap?

[Adrian Gschwend]

On Mon, 28 Jun 2004 16:08:20 +0200, Marius Mauch wrote:

>It's not getting along at all, we're concentrating our efforts on the
>current portage. Right now I consider portage-ng a dead project.

ok I propose that the one who can update the pages should do that :-)

Now maybe I should write why I asked for it: I would like to use portage on
another platform. I am admin of an open source project for OS/2 (no, not dead
yet) and we have our unix layer as well. So far some guys did their own way
or portage, unfortunately not a very flexible one. At work I use Gentoo
nowadays and portage is a great tool so I thought migrating our ports to
portage would be nice.

What about the limitations in portage that are listed on the portage-ng page?
Do you think it should be possible to get it to work on other platforms than
Linux within a reasonable time? Where could I run into problems? Are there
still people working on portage for Fink or BSD?

cu

Adrian





--
gentoo-portage-dev@gentoo.org mailing list

Re: [gentoo-portage-dev] Re: portage-ng roadmap?

[Hasan Khalil]

There are, absolutely, people working on portage for alternative 
platforms. I am one of four developers in training, shooting to work on 
gentoo-macos. pvdabeel is the gentoo-macos god currently, so he can fill 
you in with all the details.

I'm wondering the same, actually (regarding all the portage limitations 
listed on the portage-ng page). What's going on, then?

	-Hasan

Adrian Gschwend wrote:


> What about the limitations in portage that are listed on the portage-ng page?
> Do you think it should be possible to get it to work on other platforms than
> Linux within a reasonable time? Where could I run into problems? Are there
> still people working on portage for Fink or BSD?
> 
> cu
> 
> Adrian
> 
> 
> 
> 
> 
> --
> gentoo-portage-dev@gentoo.org mailing list
> 

--
gentoo-portage-dev@gentoo.org mailing list

Re: [gentoo-portage-dev] Re: portage-ng roadmap?

[George Shapovalov]

Well, what's going on is not that its "legally dead". It got its start and we 
threw a bunch of comments on the "design". However there does not seem to be 
much push behind it at the moment. I suspect there may be only one dev 
working on it (pvdabeel?), if at all. However there is clearly an effort to 
clean-up and enhance the [oroginal] portage under way.

So, if there is enough interest I guess portage-ng may be "restarted", but it 
will take a lot of *sustained* interest to do so.
(And no, I am not the dev working on it nor did I do much work on it :), I 
just followed the progress for some time).

George

On Tuesday 29 June 2004 11:44, Hasan Khalil wrote:
> There are, absolutely, people working on portage for alternative
> platforms. I am one of four developers in training, shooting to work on
> gentoo-macos. pvdabeel is the gentoo-macos god currently, so he can fill
> you in with all the details.
>
> I'm wondering the same, actually (regarding all the portage limitations
> listed on the portage-ng page). What's going on, then?



--
gentoo-portage-dev@gentoo.org mailing list

[gentoo-portage-dev] Re: Re: portage-ng roadmap?

[Adrian Gschwend]

On Tue, 29 Jun 2004 14:15:18 -0700, George Shapovalov wrote:

>Well, what's going on is not that its "legally dead". It got its start and we 
>threw a bunch of comments on the "design". However there does not seem to be 
>much push behind it at the moment. I suspect there may be only one dev 
>working on it (pvdabeel?), if at all. However there is clearly an effort to 
>clean-up and enhance the [oroginal] portage under way.

ok after all I care less about a portage-ng but about something I can use on
non-Linux platforms :-) Is pvdabeel also reading this list?

cu

Adrian




--
gentoo-portage-dev@gentoo.org mailing list

Re: [gentoo-portage-dev] Re: Re: portage-ng roadmap?

[John Nilsson]

[-- Attachment #1: Type: text/plain, Size: 331 bytes --]

What are the problem you want to solve?

What features of gentoo seems to be a good solution?

-John 

On Wed, 2004-06-30 at 11:03, Adrian Gschwend wrote:
> ok after all I care less about a portage-ng but about something I can use on
> non-Linux platforms :-) Is pvdabeel also reading this list?
> 
> cu
> 
> Adrian


[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-portage-dev] Re: Re: portage-ng roadmap?

[Paul de Vrieze]

[-- Attachment #1: Type: text/plain, Size: 1309 bytes --]

On Wednesday 30 June 2004 11:03, Adrian Gschwend wrote:
> On Tue, 29 Jun 2004 14:15:18 -0700, George Shapovalov wrote:
> >Well, what's going on is not that its "legally dead". It got its start and
> > we threw a bunch of comments on the "design". However there does not seem
> > to be much push behind it at the moment. I suspect there may be only one
> > dev working on it (pvdabeel?), if at all. However there is clearly an
> > effort to clean-up and enhance the [oroginal] portage under way.
>
> ok after all I care less about a portage-ng but about something I can use
> on non-Linux platforms :-) Is pvdabeel also reading this list?

I don't know. Currently I'm working on my own effort on a portage rewrite. It 
should follow most of the ideas that have been floating around for ng, but I 
cannot give any time on delivery of a useable system. The only thing that I 
have really working well now is searching (emerge -s) which is a lot faster. 
Further I've developed an ebuild parser in c++ which still has issues. If 
there is a lot of interest I might put it in gentoo cvs or release a tarbal. 
However as I'm going away in 8 days I think that will happen only in a month.

Paul

-- 
Paul de Vrieze
Gentoo Developer
Mail: pauldv@gentoo.org
Homepage: http://www.devrieze.net

[-- Attachment #2: signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-portage-dev] Re: Re: portage-ng roadmap?

[Brian Harring]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

> If there is a lot of interest I might put it in gentoo cvs or release 
> a tarbal.
> However as I'm going away in 8 days I think that will happen only in a 
> month.

Wait for an entire month?  Best release a snapshot prior to then :)
I'd be curious what you have so far, even if it's in a non-functional 
state.
~brian
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.4 (Darwin)

iD8DBQFA4zN4vdBxRoA3VU0RAtSSAJ0W14y3RgCRJGgDIUSY3cEG/8j1zwCfTmGX
xgjj75MmppwVRvd7n3obCu4=
=tV1H
-----END PGP SIGNATURE-----


--
gentoo-portage-dev@gentoo.org mailing list

Re: [gentoo-portage-dev] Re: Re: portage-ng roadmap?

[Nathaniel McCallum]

On Wed, 2004-06-30 at 16:41 -0500, Brian Harring wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> > If there is a lot of interest I might put it in gentoo cvs or release 
> > a tarbal.
> > However as I'm going away in 8 days I think that will happen only in a 
> > month.
> 
> Wait for an entire month?  Best release a snapshot prior to then :)
> I'd be curious what you have so far, even if it's in a non-functional 
> state.

As would I.

Nathaniel


--
gentoo-portage-dev@gentoo.org mailing list

Re: [gentoo-portage-dev] Re: Re: portage-ng roadmap?

[Paul de Vrieze]

[-- Attachment #1: Type: text/plain, Size: 837 bytes --]

On Thursday 01 July 2004 00:23, Nathaniel McCallum wrote:
> On Wed, 2004-06-30 at 16:41 -0500, Brian Harring wrote:
> > -----BEGIN PGP SIGNED MESSAGE-----
> > Hash: SHA1
> >
> > > If there is a lot of interest I might put it in gentoo cvs or
> > > release a tarbal.
> > > However as I'm going away in 8 days I think that will happen only
> > > in a month.
> >
> > Wait for an entire month?  Best release a snapshot prior to then :)
> > I'd be curious what you have so far, even if it's in a non-functional
> > state.
>
> As would I.
>
> Nathaniel
>

Ok, I'll make sure that I'll release what I have until now. Note that it 
is not yet functional in the sense that there is no interface outside of 
python.

Paul

-- 
Paul de Vrieze
Gentoo Developer
Mail: pauldv@gentoo.org
Homepage: http://www.devrieze.net

[-- Attachment #2: signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-portage-dev] Re: Re: portage-ng roadmap?

[Philippe Lafoucrière]

> I don't know. Currently I'm working on my own effort on a portage rewrite. It 
> should follow most of the ideas that have been floating around for ng, but I 
> cannot give any time on delivery of a useable system. The only thing that I 
> have really working well now is searching (emerge -s) which is a lot faster. 
> Further I've developed an ebuild parser in c++ which still has issues. If 
> there is a lot of interest I might put it in gentoo cvs or release a tarbal. 
> However as I'm going away in 8 days I think that will happen only in a month.


need help ?
I'd like to work on the cache part, since I'm getting sic of "emerge rsync".

--
gentoo-portage-dev@gentoo.org mailing list

Re: [gentoo-portage-dev] Re: Re: portage-ng roadmap?

[Michael Kohl]

[-- Attachment #1: Type: text/plain, Size: 628 bytes --]

On Mon, 12 Jul 2004 11:08:16 +0200
Philippe Lafoucrière <lafou@wanadoo.fr> wrote:

> I'd like to work on the cache part, since I'm getting sic of "emerge
> rsync".

2.0.51 is changing the cache format. I don't know yet if this makes
"emerge sync" faster, but if you're eager to try it out you can unmask
it in your /etc/portage/package.unmask.

Reading carpaski's mail about _pre13 might be of interest for you:

http://thread.gmane.org/gmane.linux.gentoo.devel/19521

Regards,
Michael

-- 
www.cargal.org 
GnuPG-key-ID: 0x90CA09E3
Jabber-ID: citizen428 [at] cargal [dot] org
Registered Linux User #278726

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-portage-dev] Re: Re: portage-ng roadmap?

[Brian Harring]

[-- Attachment #1: Type: text/plain, Size: 1020 bytes --]

On Mon, 2004-07-12 at 04:17, Michael Kohl wrote:
> On Mon, 12 Jul 2004 11:08:16 +0200
> Philippe Lafoucrière <lafou@wanadoo.fr> wrote:
> 
> > I'd like to work on the cache part, since I'm getting sic of "emerge
> > rsync".
> 
> 2.0.51 is changing the cache format. I don't know yet if this makes
> "emerge sync" faster, but if you're eager to try it out you can unmask
> it in your /etc/portage/package.unmask.
The change is due to the addition of provide's to the cache; that
doesn't affect speed, although .51 is a bit faster since a couple of
issues w/ the cache handling code have been corrected.

Main one being the addition of __cache(Array|Key) to portage_db_template
storing the last 3 keys pulled for that category, correcting an issue w/
aux_get where it went to disk *way* more then was needed.  At some point
I'd think the __cache(Array|Key) addition should be moved up above the
portage_db layer, since w/ right now it makes addition of a memcached
based cache semi-tricky/ugly.
~brian

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-portage-dev] Re: Re: portage-ng roadmap?

[Paul de Vrieze]

>> I don't know. Currently I'm working on my own effort on a portage rewrite.
>> It
>> should follow most of the ideas that have been floating around for ng, but I
>> cannot give any time on delivery of a useable system. The only thing that I
>> have really working well now is searching (emerge -s) which is a lot faster.
>> Further I've developed an ebuild parser in c++ which still has issues. If
>> there is a lot of interest I might put it in gentoo cvs or release a tarbal.
>> However as I'm going away in 8 days I think that will happen only in a
>> month.
>
>
> need help ?
> I'd like to work on the cache part, since I'm getting sic of "emerge rsync".

A big change in what I have, which is not nearly workeable yet (and I won't
work on it at all until august) is that it tries to use lazy evaluation
techniques combined with in memory caching. When you open an ebuild object you
can do with it whatever you want but the actual file is only read at the
moment you request the value of a property other than the version. Similarly
the packages in a category are only requested on demand. Searching (emerge -s
like) uses a smart strategy using the glob function ;-). The cache
precalculation should not be necessary for the full rewrite.

Paul

--
Paul de Vrieze
Gentoo Developer
Mail: pauldv@cs.kun.nl
Homepage: http://www.devrieze.net


--
gentoo-portage-dev@gentoo.org mailing list

Re: [gentoo-portage-dev] Re: Re: portage-ng roadmap?

[Philippe Lafoucrière]

>>need help ?
>>I'd like to work on the cache part, since I'm getting sic of "emerge rsync".
> 
> 
> A big change in what I have, which is not nearly workeable yet (and I won't
> work on it at all until august) is that it tries to use lazy evaluation
> techniques combined with in memory caching. When you open an ebuild object you
> can do with it whatever you want but the actual file is only read at the
> moment you request the value of a property other than the version. Similarly
> the packages in a category are only requested on demand. Searching (emerge -s
> like) uses a smart strategy using the glob function ;-). The cache
> precalculation should not be necessary for the full rewrite.


Paul,

do you think end-users need all ebuilds ? I have a laptop, and fetching 
the whole portage tree doesn't make any sence, since I emerge only a few 
apps. On the other hand, I'd like to fetch a "pre-computed" cache or 
world file, and fetch only ebuilds when necessary (emerging a package 
for example). Those "emerge rsync" are way too long, and can be 
drasticly reducted using a different way of thinking.

Of course, the whole portage tree could be installed, as it is in FreeBSD ;)

--
gentoo-portage-dev@gentoo.org mailing list

Re: [gentoo-portage-dev] Re: Re: portage-ng roadmap?

[Paul de Vrieze]

> do you think end-users need all ebuilds ? I have a laptop, and fetching
> the whole portage tree doesn't make any sence, since I emerge only a few
> apps. On the other hand, I'd like to fetch a "pre-computed" cache or
> world file, and fetch only ebuilds when necessary (emerging a package
> for example). Those "emerge rsync" are way too long, and can be
> drasticly reducted using a different way of thinking.
>
> Of course, the whole portage tree could be installed, as it is in FreeBSD ;)

You're right, they probably don't. My system will be quite modular and I image
a new tree, maybe I'll implement it in the initial release. One concept I'm
thinking about is something I call xpack (where xbuild is the extension of
ebuilds that should be guaranteed to work with the new parser). This basically
is a text based packaging format (like tar, but actually diffeable), that
would contain all parts needed for an ebuild (source is optional). Such a file
would make things a lot easier to manage and to download on demand.

While my main focus will not be on ondemand downloading of xpacks (ebuilds is
pointless) it should be fairly trivial to generate metadata files for the
packages contained, maybe even on a category level. The transfer size could
still be sizeable though. I believe that introduction of xpack files
containing everything needed for an ebuild (except the manifest and
metadata.xml) allready reduces the amount of files in the tree enormously. It
also helps in being able to easilly remove unused patches ;-).

Paul

--
Paul de Vrieze
Gentoo Developer
Mail: pauldv@gentoo.org
Homepage: http://www.devrieze.net


--
gentoo-portage-dev@gentoo.org mailing list

Re: [gentoo-portage-dev] Re: Re: portage-ng roadmap?

[Philippe Lafoucrière]

> You're right, they probably don't. My system will be quite modular and I image
> a new tree, maybe I'll implement it in the initial release. One concept I'm
> thinking about is something I call xpack (where xbuild is the extension of
> ebuilds that should be guaranteed to work with the new parser). This basically
> is a text based packaging format (like tar, but actually diffeable), that
> would contain all parts needed for an ebuild (source is optional). Such a file
> would make things a lot easier to manage and to download on demand.

Totally agree. You can also force a validating commit with cvs : any 
commit would have to pass QA test (parsing) to be really commited.
But, please consider not using rsync anymore. It's too slow for 85 000 
files :(
as previously discuss, subversion would be great. The best way is maybe 
the debian way : Put all xbuilds in a single tar file that would 
downloaded when "emerge update" (example).

> While my main focus will not be on ondemand downloading of xpacks (ebuilds is
> pointless) it should be fairly trivial to generate metadata files for the
> packages contained, maybe even on a category level. The transfer size could
> still be sizeable though. I believe that introduction of xpack files
> containing everything needed for an ebuild (except the manifest and
> metadata.xml) allready reduces the amount of files in the tree enormously. It
> also helps in being able to easilly remove unused patches ;-).

I like this new portage :)
So these xbuilds are not needed on end-users computers. They are only 
needed to generate the dependance tree.

If you need any help on this, please email me.

Philippe

--
gentoo-portage-dev@gentoo.org mailing list

Re: [gentoo-portage-dev] Re: Re: portage-ng roadmap?

[Pieter Van den Abeele]

On 30 Jun 2004, at 11:03, Adrian Gschwend wrote:

> On Tue, 29 Jun 2004 14:15:18 -0700, George Shapovalov wrote:
>
>> Well, what's going on is not that its "legally dead". It got its 
>> start and we
>> threw a bunch of comments on the "design". However there does not 
>> seem to be
>> much push behind it at the moment. I suspect there may be only one dev
>> working on it (pvdabeel?), if at all. However there is clearly an 
>> effort to
>> clean-up and enhance the [oroginal] portage under way.
>
> ok after all I care less about a portage-ng but about something I can 
> use on
> non-Linux platforms :-) Is pvdabeel also reading this list?

Yep, sorry. I was at WWDC, back now.

Current portage uses some old algorithms, and features get continuously 
added. The portage team itself is working on cleaning up the code; 
There is jstubbs portage modularization efforts, last snapshot I looked 
at got all the global variables at the beginning of the file, ... 
Portage-ng comes in where old portage stops: revisiting the core 
algorithms. I am working on a new core that leads to better 
expressiveness, better performance and shorter, more intuitive, code. 
This obviously leads to better non-Linux support, but that is not the 
only reason for doing this.

The reason I'm doing this alone atm, is because I'm doing this as a MCS 
thesis.

Best regards,

Pieter Van den Abeele

> cu
>
> Adrian
>
>
>
>
> --
> gentoo-portage-dev@gentoo.org mailing list
>


--
gentoo-portage-dev@gentoo.org mailing list

Re: [gentoo-portage-dev] Re: Re: portage-ng roadmap?

[gentoo]

unsubscribe gentoo-portage-dev


--
gentoo-portage-dev@gentoo.org mailing list

[gentoo-portage-dev] Re: Re: portage-ng roadmap?

[Adrian Gschwend]

On Tue, 29 Jun 2004 13:44:41 -0500, Hasan Khalil wrote:

>There are, absolutely, people working on portage for alternative 
>platforms. I am one of four developers in training, shooting to work on 
>gentoo-macos. pvdabeel is the gentoo-macos god currently, so he can fill 
>you in with all the details.

Do you have a dicussion list somewhere? I am quite sure I will run into
similar problems that you might have :-)

cu

Adrian




--
gentoo-portage-dev@gentoo.org mailing list

Re: [gentoo-portage-dev] Re: Re: portage-ng roadmap?

[Hasan Khalil]

Yes. Check the list of gentoo mailing lists.

http://www.gentoo.org/main/en/lists.xml

It's been fairly inactive for the past year or so since I've joined (has 
it really been that long?).

	-Hasan

Adrian Gschwend wrote:

> Do you have a dicussion list somewhere? I am quite sure I will run into
> similar problems that you might have :-)

--
gentoo-portage-dev@gentoo.org mailing list