[gnap-dev] [RFC] Summer of Code Proposal: GNAP Love

[gnap-dev] [RFC] Summer of Code Proposal: GNAP Love

[Philipp Riegger]

Good morning,

I have some ideas for Summer of Code which i'd like to discuss with you.

I'm mostly interested in making GNAP usable and my proposal includes
lots of tasks in GNAP and catalyst, of which i'd like to finish as many
as possible and do some research on the rest.

catalyst tasks:

• Squashfs snapshot support
I don't like it, that it takes so much time to unpack a new snapshot and
it takes so much disk space to store it. Plan is: distribute tree
snapshots as squashfs images, mount them directly into the workdir,
use /mnt/distfiles and /mnt/packages instead of snapshot subdirectories.

• (Squashfs seedstage support)
Maybe the same is possible for stages? A future plan for this could be
to mount the squashfs image directly and use unionfs for the real work,
but i have no idea how deleting files in such a setup works. Research is
necessary here.

• uclibc-cross-compiling support
Cross compiling is hard, but it should be possible to build arch-uclibc
from arch. Research, what needs to be done combined with the
implementation, if possible. This would also make GNAP more flexible.

• Documentation
In an email from some days ago i read, that documentation is planed for
after the release. I could support this and proofread it, since i need
the knowledge anyway.

• (Code cleanups)
I write this everywhere, but since i need a basic understanding, i have
to read (parts of) the source and maybe i find something worth
improving.

• Cross compiling research
This is the continuation of the uclibc stuff with some plans on what
could be done how.

• Non-root builds research
Wouldn't it be nice if being root was not necessary for using catalyst?
I would like to look into the possibilities there and what is needed to
make it reality.


GNAP tasks:

• Code/Tree hosting, VCS
I've done some work last year and that does only exist as some patches,
since there is no central repository for GNAP development. First i'd
like to establish one and move all existing resources there.

• GNAP 2.1
There are some known and easy to fix bugs in GNAP 2.0. There have also
been some code cleanups. The last thing is, that the current GNAP is not
really usable, since the tree snapshot is so old, that most of the
distfiles are not fetchable from mirrors or other places. So i'd like to
make a new release, be it in the tree or in an overlay.

• Reimplementation in python research
Catalyst is written in python, maybe we could make better use of it if
we reimplemented some parts of GNAP in python? This is just a research
item, i'd like to look into it and form some statement about it. I'll
probably not do it.

• uclibc-cross-compiling support
This depends on catalyst.

• (Cross-compiling support)
My last years project. My thoughts/plans about that should be discussed
with the community and put on the GNAP development website.

• GNAP releases, long term support
This is just an idea i have. Think of something like this: With each
gentoo release, we take the release snapshot, form our reduced GNAP
snapshot (only supported profiles, unsupported USE flags masked,
unneeded ebuilds (X, desktop stuff) not in the tree) and support it for
some time security wise (like the releng team does during release). This
is more of a research topic, with informing what needs to be done,
writing of scripts, maybe doing a proof of concept (maintaining a
snapshot for some weeks to see how it works).


Thanks for reading this, any comments are welcome, constructive,
destructive, "this is not necessary" and "i like to see this one", and
most of all "i/we've been working on this with the following results".
This is not about occupying me, this is about helping.

I set the Reply-To to gnap-dev, you're welcome to join there, it's
really really low traffic at the moment.

Philipp

-- 
gnap-dev@lists.gentoo.org mailing list

Re: [gnap-dev] [RFC] Summer of Code Proposal: GNAP Love

[josé Alberto Suárez López]

hi :)

as we talked before...

El mié, 26-03-2008 a las 13:57 +0100, Philipp Riegger escribió:
> Good morning,
> 
> I have some ideas for Summer of Code which i'd like to discuss with you.

new ideas ever welcome

> I'm mostly interested in making GNAP usable and my proposal includes
> lots of tasks in GNAP and catalyst, of which i'd like to finish as many
> as possible and do some research on the rest.
> 
> catalyst tasks:
> 
> • Squashfs snapshot support
> I don't like it, that it takes so much time to unpack a new snapshot and
> it takes so much disk space to store it. Plan is: distribute tree
> snapshots as squashfs images, mount them directly into the workdir,
> use /mnt/distfiles and /mnt/packages instead of snapshot subdirectories.

i like this can be usefull, the problem is how mcuh espace you need to
redistribute all sources we need (not only gnap-core srcs, i mean
extensions srcs too)

> • (Squashfs seedstage support)
> Maybe the same is possible for stages? A future plan for this could be
> to mount the squashfs image directly and use unionfs for the real work,
> but i have no idea how deleting files in such a setup works. Research is
> necessary here.

i dont understand it well

> • uclibc-cross-compiling support
> Cross compiling is hard, but it should be possible to build arch-uclibc
> from arch. Research, what needs to be done combined with the
> implementation, if possible. This would also make GNAP more flexible.

this will be one of the biggest features so as you know is hard.

> • Documentation
> In an email from some days ago i read, that documentation is planed for
> after the release. I could support this and proofread it, since i need
> the knowledge anyway.

you mean gnap doc or catalyst doc?

> • (Code cleanups)
> I write this everywhere, but since i need a basic understanding, i have
> to read (parts of) the source and maybe i find something worth
> improving.

ever welcome :)

> • Cross compiling research
> This is the continuation of the uclibc stuff with some plans on what
> could be done how.
> 
> • Non-root builds research
> Wouldn't it be nice if being root was not necessary for using catalyst?
> I would like to look into the possibilities there and what is needed to
> make it reality.

i think that is hard, you dont need root only for catalyst, other GNAP
operations need to be root.

> GNAP tasks:
> 
> • Code/Tree hosting, VCS
> I've done some work last year and that does only exist as some patches,
> since there is no central repository for GNAP development. First i'd
> like to establish one and move all existing resources there.

yes, what about your google-code testing?

> • GNAP 2.1
> There are some known and easy to fix bugs in GNAP 2.0. There have also
> been some code cleanups. The last thing is, that the current GNAP is not
> really usable, since the tree snapshot is so old, that most of the
> distfiles are not fetchable from mirrors or other places. So i'd like to
> make a new release, be it in the tree or in an overlay.

you are right, but first we need and updated tree

> • Reimplementation in python research
> Catalyst is written in python, maybe we could make better use of it if
> we reimplemented some parts of GNAP in python? This is just a research
> item, i'd like to look into it and form some statement about it. I'll
> probably not do it.

i prefer to keep bash

> • uclibc-cross-compiling support
> This depends on catalyst.
> 
> • (Cross-compiling support)
> My last years project. My thoughts/plans about that should be discussed
> with the community and put on the GNAP development website.
> 
> • GNAP releases, long term support
> This is just an idea i have. Think of something like this: With each
> gentoo release, we take the release snapshot, form our reduced GNAP
> snapshot (only supported profiles, unsupported USE flags masked,
> unneeded ebuilds (X, desktop stuff) not in the tree) and support it for
> some time security wise (like the releng team does during release). This
> is more of a research topic, with informing what needs to be done,
> writing of scripts, maybe doing a proof of concept (maintaining a
> snapshot for some weeks to see how it works).


--
gnap-dev@lists.gentoo.org mailing list

Re: [gnap-dev] [RFC] Summer of Code Proposal: GNAP Love

[Philipp Riegger]

On Wed, 2008-03-26 at 14:19 +0100, josé Alberto Suárez López wrote:
> El mié, 26-03-2008 a las 13:57 +0100, Philipp Riegger escribió:
> > I'm mostly interested in making GNAP usable and my proposal includes
> > lots of tasks in GNAP and catalyst, of which i'd like to finish as many
> > as possible and do some research on the rest.
> > 
> > catalyst tasks:
> > 
> > • Squashfs snapshot support
> > I don't like it, that it takes so much time to unpack a new snapshot and
> > it takes so much disk space to store it. Plan is: distribute tree
> > snapshots as squashfs images, mount them directly into the workdir,
> > use /mnt/distfiles and /mnt/packages instead of snapshot subdirectories.
> 
> i like this can be usefull, the problem is how mcuh espace you need to
> redistribute all sources we need (not only gnap-core srcs, i mean
> extensions srcs too)

This is all about catalyst. Catalyst takes a tree snapshot (/usr/portage
as .tar.bz2), unpacks it somewhere and bind mounts it to its workdir.
I'd like to get rid of this step, since it takes lots of time and wastes
lots of disk space. I'm interested, what catalyst devs will say about
it.

> > • (Squashfs seedstage support)
> > Maybe the same is possible for stages? A future plan for this could be
> > to mount the squashfs image directly and use unionfs for the real work,
> > but i have no idea how deleting files in such a setup works. Research is
> > necessary here.
> 
> i dont understand it well

Same as above. Has nothing to do with GNAP, only catalyst. Imagine you
want to build a stage2. You rsync stage1 to a workdir and build stage2
there. Therefore you have to unpack the stage1 somewhere and store it
there. I want to get rid of this.

> > • uclibc-cross-compiling support
> > Cross compiling is hard, but it should be possible to build arch-uclibc
> > from arch. Research, what needs to be done combined with the
> > implementation, if possible. This would also make GNAP more flexible.
> 
> this will be one of the biggest features so as you know is hard.

I know, but i already know where to start, since i alredy got in touch
with it last year. I just don't know, how many other problems there will
be on the way.

> > • Documentation
> > In an email from some days ago i read, that documentation is planed for
> > after the release. I could support this and proofread it, since i need
> > the knowledge anyway.
> 
> you mean gnap doc or catalyst doc?

Catalyst, actually, but this could be extended to both.

> > • Cross compiling research
> > • Non-root builds research
> 
> i think that is hard, you dont need root only for catalyst, other GNAP
> operations need to be root.

Why, exactly? There are reasons like "mounting stuff", "creating /dev
files" and others, but maybe we can do that without root...

> > GNAP tasks:
> > 
> > • Code/Tree hosting, VCS
> > I've done some work last year and that does only exist as some patches,
> > since there is no central repository for GNAP development. First i'd
> > like to establish one and move all existing resources there.
> 
> yes, what about your google-code testing?

I did not look into it, yet. If i slend half the week, testing it, i
don't have an application for SoC. :-)

> > • GNAP 2.1
> > There are some known and easy to fix bugs in GNAP 2.0. There have also
> > been some code cleanups. The last thing is, that the current GNAP is not
> > really usable, since the tree snapshot is so old, that most of the
> > distfiles are not fetchable from mirrors or other places. So i'd like to
> > make a new release, be it in the tree or in an overlay.
> 
> you are right, but first we need and updated tree

Yes, you're right. Maybe we could use the old tree, only fix security
problems and ship that. Maybe we could package.mask it and use a
development tree, that is only tested for building. Maybe some GNAP user
(there have been some last year) is willing to share his tree and we can
build on that.

I'd like to look into an easy, mostly automated way of creating the
tree, like a bunch of scripts and some documentation. If this is the
only thing i get done, it was worth it. If i only get it started, then
we are closer to being usable than we have before.

> > • Reimplementation in python research
> > Catalyst is written in python, maybe we could make better use of it if
> > we reimplemented some parts of GNAP in python? This is just a research
> > item, i'd like to look into it and form some statement about it. I'll
> > probably not do it.
> 
> i prefer to keep bash

Again, i'll have to look into it. It's only research. There were some
things i did not like in gnap, but i saw no other way to do it in such a
low level language as bash. If i can do the scripts with half or one
third of the lines of code in python, i would do that. If gnap and
catalyst would merge in a way, that catalyst provides the tools and gnap
only very very simple scripts and resources, that would be perfect, but
gnap should not handle overlays, configurations and stuff, it should
just pass them to catalyst. Much easier to do there, i think. But again,
that are my current thoughts.

Philipp

-- 
gnap-dev@lists.gentoo.org mailing list

Re: [gnap-dev] [RFC] Summer of Code Proposal: GNAP Love

[Philipp Riegger]

On Wed, 2008-03-26 at 13:57 +0100, Philipp Riegger wrote:
> Good morning,
> 
> I have some ideas for Summer of Code which i'd like to discuss with you.
> 
[...]
> Philipp
> 

No time for being a student this year. I applied to be a mentor, so if
they want me i'll be working with you. Some of you. Maybe. Somehow.

Anyway, i'd like to get a few of the points i mentioned done anyway, so
you will probably hear from me.

Philipp

-- 
gnap-dev@lists.gentoo.org mailing list