[gentoo-soc] Looking for a mentor for a Portage SOC Project

[gentoo-soc] Looking for a mentor for a Portage SOC Project

[Colleen Josephson]

[-- Attachment #1: Type: text/plain, Size: 1776 bytes --]

I'm interested in expanding the Portage "tags" idea listed on the wiki.
To make it SOC-worthy, it obviously needs to be expanded.

I have thought of a few possibilities:
-Integrate eix into Portage as a replacement to search
-Add a Debian-apt style autocomplete
-Add a flag to force the removal of config files when removing a package, if
desired
-Add a feature that lists which commands were installed after merging a
package
e.g. after installing net-tools successfully, emerge would say:
"Commands in net-tools: arp, hostname, ifconfig, ipmaddr, iptunnel,
mii-tool, nameif, netstat, plipconfig, rarp, route and slattach"
For packages that install something in /etc/init.d, it would list how to
start it: e.g. "To start service, run '/etc/init.d/servicename start'"
It would also be nice to have the ability to look this up to see the
commands associated with an already installed package:
e.g. "emerge --list-commands packagename" or something similar.

I am trying to determine whether or not these features are enough to fill a
summer, or if I need more.
There are 14 weeks in SOC, 13 weeks of coding if I leave the last week for
polishing things.
I'm thinking: 3 weeks for eix integration, 1-2 weeks for tags, 1 week for
the --remove-config flag, 2-3 weeks for the autocomplete, and 4 weeks for
the "commands installed" feature.
It looks like I need 1 more "agenda item" to fill the summer. Any
suggestions?

Do these timeframes sound reasonable? I have 3-4 years of coding experience,
with 2 years of Python.

I'm in need of a mentor ASAP so I can get feedback while writing up my
proposal, and have somebody point me to a good bugfix to do.

--
Colleen Josephson
Dept. of Electrical Engineering and Computer Science
Massachusetts Institute of Technology
Class of 2013

[-- Attachment #2: Type: text/html, Size: 1956 bytes --]

Re: [gentoo-soc] Looking for a mentor for a Portage SOC Project

[Jeremy Olexa]

 On Wed, 30 Mar 2011 15:22:47 -0500, Colleen Josephson wrote:
> Im interested in expanding the Portage "tags" idea listed on the 
> wiki.
> To make it SOC-worthy, it obviously needs to be expanded.
>
> I have thought of a few possibilities:
> -Integrate eix into Portage as a replacement to search
>  -Add a Debian-apt style autocomplete
> -Add a flag to force the removal of config files when removing a
> package, if desired
> -Add a feature that lists which commands were installed after merging
> a package
> e.g. after installing net-tools successfully, emerge would say:
>  "Commands in net-tools: arp, hostname, ifconfig, ipmaddr, iptunnel,
> mii-tool, nameif, netstat, plipconfig, rarp, route and slattach"
> For packages that install something in /etc/init.d, it would list how
> to start it: e.g. "To start service, run /etc/init.d/servicename
> start"
>  It would also be nice to have the ability to look this up to see the
> commands associated with an already installed package:
> e.g. "emerge --list-commands packagename" or something similar.
>
> I am trying to determine whether or not these features are enough to
> fill a summer, or if I need more.
>  There are 14 weeks in SOC, 13 weeks of coding if I leave the last
> week for polishing things.
> Im thinking: 3 weeks for eix integration, 1-2 weeks for tags, 1 week
> for the --remove-config flag, 2-3 weeks for the autocomplete, and 4
> weeks for the "commands installed" feature.
>  It looks like I need 1 more "agenda item" to fill the summer. Any
> suggestions?

 Hello, thanks for your interest in GSOC. I don't want to shoot you down 
 right away, but I think you are seriously over estimating the amount of 
 time needed and you'll find this proposal to be too easy. For example, I 
 just did you "4 weeks" of "commands installed feature" in 30 seconds. :)

 % cat /etc/portage/bashrc
 if [[ $EBUILD_PHASE == postinst ]]; then
         ewarn "BINARIES: "
         ewarn $(qlist ${PN} | grep bin)
 fi

 % sudo emerge -av1 less

 These are the packages that would be merged, in order:

 Calculating dependencies... done!
 [binary   R   ] sys-apps/less-436  USE="unicode"

 Total: 1 package (1 reinstall, 1 binary), Size of downloads: 0 kB

 Would you like to merge these packages? [Yes/No]
>>> Emerging binary (1 of 1) sys-apps/less-436
>>> Installing (1 of 1) sys-apps/less-436
>>> Jobs: 1 of 1 complete                           Load avg: 0.34, 
>>> 0.46, 0.64

  * Messages for package sys-apps/less-436:

  * BINARIES:
  * /usr/bin/code2color /usr/bin/lesspipe /usr/bin/lessecho 
 /usr/bin/lesspipe.sh /usr/bin/lesskey /usr/bin/less
>>> Auto-cleaning packages...

>>> No outdated packages were found on your system.

  * GNU info directory index is up-to-date.

 Granted, that is in bash, but still. On that note, feel free to expand 
 your ideas. :)
 -Jeremy

>
> Do these timeframes sound reasonable? I have 3-4 years of coding
> experience, with 2 years of Python.
>
> Im in need of a mentor ASAP so I can get feedback while writing up my
> proposal, and have somebody point me to a good bugfix to do.
>
> --
> Colleen Josephson
> Dept. of Electrical Engineering and Computer Science
> Massachusetts Institute of Technology
> Class of 2013

Re: [gentoo-soc] Looking for a mentor for a Portage SOC Project

[Colleen Josephson]

[-- Attachment #1: Type: text/plain, Size: 4107 bytes --]

Thanks Jeremy,

I suppose part of the timeline is me figuring out the inner workings of
Portage and how to work with the existing code,
Do you have any ideas for expansions that would be more significant? Also,
any comments on the timelines for the rest of the ideas?

Perhaps I could merge the work with Portage into the ebuild generator
project--I have often wished for the ability to take source I have
downloaded and easily work it into a local Portage layover.

Perhaps half the summer on Portage features and half on the e-build
generator would be more reasonable?


On Wed, Mar 30, 2011 at 3:57 PM, Jeremy Olexa <darkside@gentoo.org> wrote:

> On Wed, 30 Mar 2011 15:22:47 -0500, Colleen Josephson wrote:
>
>> Im interested in expanding the Portage "tags" idea listed on the wiki.
>> To make it SOC-worthy, it obviously needs to be expanded.
>>
>> I have thought of a few possibilities:
>> -Integrate eix into Portage as a replacement to search
>>  -Add a Debian-apt style autocomplete
>> -Add a flag to force the removal of config files when removing a
>> package, if desired
>> -Add a feature that lists which commands were installed after merging
>> a package
>> e.g. after installing net-tools successfully, emerge would say:
>>  "Commands in net-tools: arp, hostname, ifconfig, ipmaddr, iptunnel,
>> mii-tool, nameif, netstat, plipconfig, rarp, route and slattach"
>> For packages that install something in /etc/init.d, it would list how
>> to start it: e.g. "To start service, run /etc/init.d/servicename
>> start"
>>  It would also be nice to have the ability to look this up to see the
>> commands associated with an already installed package:
>> e.g. "emerge --list-commands packagename" or something similar.
>>
>> I am trying to determine whether or not these features are enough to
>> fill a summer, or if I need more.
>>  There are 14 weeks in SOC, 13 weeks of coding if I leave the last
>> week for polishing things.
>> Im thinking: 3 weeks for eix integration, 1-2 weeks for tags, 1 week
>> for the --remove-config flag, 2-3 weeks for the autocomplete, and 4
>> weeks for the "commands installed" feature.
>>  It looks like I need 1 more "agenda item" to fill the summer. Any
>> suggestions?
>>
>
> Hello, thanks for your interest in GSOC. I don't want to shoot you down
> right away, but I think you are seriously over estimating the amount of time
> needed and you'll find this proposal to be too easy. For example, I just did
> you "4 weeks" of "commands installed feature" in 30 seconds. :)
>
> % cat /etc/portage/bashrc
> if [[ $EBUILD_PHASE == postinst ]]; then
>        ewarn "BINARIES: "
>        ewarn $(qlist ${PN} | grep bin)
> fi
>
> % sudo emerge -av1 less
>
> These are the packages that would be merged, in order:
>
> Calculating dependencies... done!
> [binary   R   ] sys-apps/less-436  USE="unicode"
>
> Total: 1 package (1 reinstall, 1 binary), Size of downloads: 0 kB
>
> Would you like to merge these packages? [Yes/No]
>
>> Emerging binary (1 of 1) sys-apps/less-436
>>>> Installing (1 of 1) sys-apps/less-436
>>>> Jobs: 1 of 1 complete                           Load avg: 0.34, 0.46,
>>>> 0.64
>>>>
>>>
>  * Messages for package sys-apps/less-436:
>
>  * BINARIES:
>  * /usr/bin/code2color /usr/bin/lesspipe /usr/bin/lessecho
> /usr/bin/lesspipe.sh /usr/bin/lesskey /usr/bin/less
>
>> Auto-cleaning packages...
>>>>
>>>
>  No outdated packages were found on your system.
>>>>
>>>
>  * GNU info directory index is up-to-date.
>
> Granted, that is in bash, but still. On that note, feel free to expand your
> ideas. :)
> -Jeremy
>
>
>
>> Do these timeframes sound reasonable? I have 3-4 years of coding
>> experience, with 2 years of Python.
>>
>> Im in need of a mentor ASAP so I can get feedback while writing up my
>> proposal, and have somebody point me to a good bugfix to do.
>>
>> --
>> Colleen Josephson
>> Dept. of Electrical Engineering and Computer Science
>> Massachusetts Institute of Technology
>> Class of 2013
>>
>
>
>


-- 
Colleen Josephson
Dept. of Electrical Engineering and Computer Science
Massachusetts Institute of Technology
Class of 2013

[-- Attachment #2: Type: text/html, Size: 6405 bytes --]

Re: [gentoo-soc] Looking for a mentor for a Portage SOC Project

[Donnie Berkholz]

[-- Attachment #1: Type: text/plain, Size: 2036 bytes --]

On 15:22 Wed 30 Mar     , Colleen Josephson wrote:
> I'm interested in expanding the Portage "tags" idea listed on the wiki.
> To make it SOC-worthy, it obviously needs to be expanded.
> 
> I have thought of a few possibilities:
> -Integrate eix into Portage as a replacement to search

This sounds interesting, particularly if you could get portage to use an 
eix-style cache to speed up dependency calculations too.

> -Add a Debian-apt style autocomplete

I think we have some bash-completion that does this already.

emerge www<tab><tab> gives:

: $;emerge www-
www-apache/   www-client/   www-plugins/  
www-apps/     www-misc/     www-servers/  

completing to www-client/ gives:

: $;emerge www-client/
amaya                firefox-bin          pybugz
arora                galeon               rekonq
chromium             htmlview             seamonkey
chromium-bin         httrack              seamonkey-bin
ck4up                icecat               surf
conkeror             jd                   surfraw
dillo                jumanji              uget
downman              links                uzbl
elinks               luakit               w3m
epiphany             lynx                 w3mir
epiphany-extensions  midori               w3mmee
fetch                netrik               
firefox              opera   

> -Add a flag to force the removal of config files when removing a package, if
> desired

I think this will be really really small, like the other bit about 
listing installed commands/daemons. Both could probably be done in a 
morning.

You could consider coupling this with other ideas, such as the "SCM 
snapshot management infrastructure" that already has a suggested mentor, 
which would also help with your other question about finding a mentor. 
Another nice benefit of that idea is that nobody else has yet expressed 
interest in applying for it.

-- 
Thanks,
Donnie

Donnie Berkholz
Admin, Summer of Code
Gentoo Linux
Blog: http://dberkholz.com

[-- Attachment #2: Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-soc] Looking for a mentor for a Portage SOC Project

[Colleen Josephson]

[-- Attachment #1: Type: text/plain, Size: 2560 bytes --]

On autocomplete: do I have to change a setting somewhere for autocompletion?
I've never knew about this!

The SCM integration sounds interesting. Combining my portage ideas with the
ebuild generator/portage GUI frontend is also another possibility (though
I've seen other mailings about those).

On Wed, Mar 30, 2011 at 4:34 PM, Donnie Berkholz <dberkholz@gentoo.org>wrote:

> On 15:22 Wed 30 Mar     , Colleen Josephson wrote:
> > I'm interested in expanding the Portage "tags" idea listed on the wiki.
> > To make it SOC-worthy, it obviously needs to be expanded.
> >
> > I have thought of a few possibilities:
> > -Integrate eix into Portage as a replacement to search
>
> This sounds interesting, particularly if you could get portage to use an
> eix-style cache to speed up dependency calculations too.
>
> > -Add a Debian-apt style autocomplete
>
> I think we have some bash-completion that does this already.
>
> emerge www<tab><tab> gives:
>
> : $;emerge www-
> www-apache/   www-client/   www-plugins/
> www-apps/     www-misc/     www-servers/
>
> completing to www-client/ gives:
>
> : $;emerge www-client/
> amaya                firefox-bin          pybugz
> arora                galeon               rekonq
> chromium             htmlview             seamonkey
> chromium-bin         httrack              seamonkey-bin
> ck4up                icecat               surf
> conkeror             jd                   surfraw
> dillo                jumanji              uget
> downman              links                uzbl
> elinks               luakit               w3m
> epiphany             lynx                 w3mir
> epiphany-extensions  midori               w3mmee
> fetch                netrik
> firefox              opera
>
> > -Add a flag to force the removal of config files when removing a package,
> if
> > desired
>
> I think this will be really really small, like the other bit about
> listing installed commands/daemons. Both could probably be done in a
> morning.
>
> You could consider coupling this with other ideas, such as the "SCM
> snapshot management infrastructure" that already has a suggested mentor,
> which would also help with your other question about finding a mentor.
> Another nice benefit of that idea is that nobody else has yet expressed
> interest in applying for it.
>
> --
> Thanks,
> Donnie
>
> Donnie Berkholz
> Admin, Summer of Code
> Gentoo Linux
> Blog: http://dberkholz.com
>



-- 
Colleen Josephson
Dept. of Electrical Engineering and Computer Science
Massachusetts Institute of Technology
Class of 2013

[-- Attachment #2: Type: text/html, Size: 3257 bytes --]

Re: [gentoo-soc] Looking for a mentor for a Portage SOC Project

[Donnie Berkholz]

[-- Attachment #1: Type: text/plain, Size: 657 bytes --]

On 17:26 Wed 30 Mar     , Colleen Josephson wrote:
> On autocomplete: do I have to change a setting somewhere for autocompletion?
> I've never knew about this!

Check your `eselect bashcomp` settings. I think there's one for emerge 
or portage that you need to enable.

> The SCM integration sounds interesting. Combining my portage ideas with the
> ebuild generator/portage GUI frontend is also another possibility (though
> I've seen other mailings about those).

Sounds good.

By the way: http://www.html-faq.com/etiquette/?toppost

-- 
Thanks,
Donnie

Donnie Berkholz
Admin, Summer of Code
Gentoo Linux
Blog: http://dberkholz.com

[-- Attachment #2: Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-soc] Looking for a mentor for a Portage SOC Project

[Brian Dolbec]

[-- Attachment #1: Type: text/plain, Size: 3330 bytes --]

On Wed, 2011-03-30 at 15:22 -0500, Colleen Josephson wrote:
> I'm interested in expanding the Portage "tags" idea listed on the
> wiki.
> To make it SOC-worthy, it obviously needs to be expanded.
> 
> I have thought of a few possibilities:
> -Integrate eix into Portage as a replacement to search

I saw the comments on irc, not likey to happen.

> -Add a Debian-apt style autocomplete
> -Add a flag to force the removal of config files when removing a
> package, if desired

fairly small feature addition, not much time needed to complete, I
think.

> -Add a feature that lists which commands were installed after merging
> a package
> e.g. after installing net-tools successfully, emerge would say:
> "Commands in net-tools: arp, hostname, ifconfig, ipmaddr, iptunnel,
> mii-tool, nameif, netstat, plipconfig, rarp, route and slattach"
> For packages that install something in /etc/init.d, it would list how
> to start it: e.g. "To start service, run '/etc/init.d/servicename
> start'"
> It would also be nice to have the ability to look this up to see the
> commands associated with an already installed package:
> e.g. "emerge --list-commands packagename" or something similar.

Well, the equery command in gentoolkit already does this in the files
submodule.  It also has several builtin filters for the type of file
your looking for.  In this case it would be:

	equery files -f cmd $package


And since it is coded in python and most of equery uses portage for it's
information.  This would be a trivial addition to portage.  But to be
honest, I think this would be putting too many options to emerge.
Emerge is already hard pressed for letters to use for short options.
That and for this task, it would be slower than equery is now, emerge
has more things to initialize and parse than a basic portage import and
data query.


> 
> I am trying to determine whether or not these features are enough to
> fill a summer, or if I need more.
> There are 14 weeks in SOC, 13 weeks of coding if I leave the last week
> for polishing things.
> I'm thinking: 3 weeks for eix integration, 1-2 weeks for tags, 1 week
> for the --remove-config flag, 2-3 weeks for the autocomplete, and 4
> weeks for the "commands installed" feature.
> It looks like I need 1 more "agenda item" to fill the summer. Any
> suggestions?

If your looking for more portage related projects, have you looked at
the backend restructure for porthole,  project idea?

http://www.gentoo.org/proj/en/userrel/soc/ideas.xml#doc_chap2_sect8

It will involve coding up a new backend system for porthole so it can
use a new portage public api for it's data needs as well as be able to
run merges from within the imported portage code.  Once that is working,
it will then continue to create an interface to pkgcore, so it can be
used as an alternate PM.

> 
> Do these timeframes sound reasonable? I have 3-4 years of coding
> experience, with 2 years of Python. 
> 
> I'm in need of a mentor ASAP so I can get feedback while writing up my
> proposal, and have somebody point me to a good bugfix to do.
> 
> --
> Colleen Josephson
> Dept. of Electrical Engineering and Computer Science
> Massachusetts Institute of Technology
> Class of 2013


-- 
Brian Dolbec <brian.dolbec@gmail.com>

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 490 bytes --]

Re: [gentoo-soc] Looking for a mentor for a Portage SOC Project

[Zac Medico]

On 03/30/2011 01:22 PM, Colleen Josephson wrote:
> -Integrate eix into Portage as a replacement to search

Due to poor performance of emerge --search, this is a recurring theme.
For the record, my recommendation is to extend egencache with an option
to concatenate all of the entries from metadata/cache/ into a single
file that can be quickly searched by tools like eix and emerge --search
(among many others). Having all the metadata cached in a single file
makes for fast searching because it minimizes disk seeks. If we enable
generation of this type of single-file-metadata-cache on the master
rsync mirror, then tools like eix and emerge --search can use that
directly instead of needing to generate a separate cache after each sync.
-- 
Thanks,
Zac

Re: [gentoo-soc] Looking for a mentor for a Portage SOC Project

[Colleen Josephson]

> For the record, my recommendation is to extend egencache with an option
> to concatenate all of the entries from metadata/cache/ into a single
> file that can be quickly searched by tools like eix and emerge --search
> (among many others).

How substantial of a task do you think this would be?

Also, do you have any advice on approaching adding tags to Portage?



-- 
Colleen Josephson
Dept. of Electrical Engineering and Computer Science
Massachusetts Institute of Technology
Class of 2013

Re: [gentoo-soc] Looking for a mentor for a Portage SOC Project

[Zac Medico]

On 04/02/2011 08:30 PM, Colleen Josephson wrote:
>> For the record, my recommendation is to extend egencache with an option
>> to concatenate all of the entries from metadata/cache/ into a single
>> file that can be quickly searched by tools like eix and emerge --search
>> (among many others).
> 
> How substantial of a task do you think this would be?

Implementing the cache would be fairly simple, but it would require a
good amount of feedback from the community in order to ensure widespread
adoption among the various search tools that are available. It would be
good idea to propose it as a GLEP [1]. Part of the project could be to
implement support for the new cache in multiple search tools.

Since the cache is going to be large in size relative to other files
that are distributed in the rsync tree, we may want to compress it in
order to save some space (gzip seems suitable). Also, we may want to
consider excluding metadata that typically isn't useful for the types of
search tools that we're targeting. For example, we could probably save
lots of space by excluding things like *DEPEND and SRC_URI, since these
variables typically aren't used for search tools.

> Also, do you have any advice on approaching adding tags to Portage?

This would also require a good amount of feedback from the community,
and would make another good topic for a GLEP. The tags themselves could
be stored either in metadata.xml files or in a new ebuild metadata
variable (similar to DESCRIPTION, HOMEPAGE, etc). You could add support
to egencache to generate a central index. For example, egencache uses
local USE descriptions from all the metadata.xml files to generate a
central index that's located at profiles/use.local.desc.

[1] http://www.gentoo.org/proj/en/glep/
-- 
Thanks,
Zac